2024-04-24T23:55:24.278 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T23:55:24.278 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi --verbose --archive /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230/orig.config.yaml 2024-04-24T23:55:24.338 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T23:55:24.435 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7672230' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8528 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 122/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi042.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHzZSUDoSn0SEiwW3XVlhGDc79UQwRAjy2PlvKenau5lW0eR+3ndkkRnl7pxdJYAKgPtYq2718cT1yC2G3It3+o= smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwGSIxW8EyyY1jHfe62Qav65/Dy3hj1p+qIiBY/JqmY6aFiq+LOkEP9et96nPr5TauOhBQwxaiaqtNj+L6HvUk= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMxUsav4S7sr+lhIyKK9VByJmzCHpRIdYbH8hxivCss0qCQ+weMjpj/7+lrvX27db56RqzwG2t16ZYF0aSysLrE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T23:55:24.435 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa; will attempt to use it 2024-04-24T23:55:24.436 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d2c45d977142e024a59b68a8f1dce8f0b9840551/qa/tasks 2024-04-24T23:55:24.436 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T23:55:24.438 INFO:teuthology.task.internal:Checking packages... 2024-04-24T23:55:24.462 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-24T23:55:24.462 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T23:55:24.462 INFO:teuthology.packaging:ref: None 2024-04-24T23:55:24.462 INFO:teuthology.packaging:tag: None 2024-04-24T23:55:24.462 INFO:teuthology.packaging:branch: squid 2024-04-24T23:55:24.462 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-24T23:55:24.463 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-24T23:55:24.768 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2445.gd2c45d97 2024-04-24T23:55:24.770 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T23:55:24.777 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T23:55:24.778 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T23:55:24.792 INFO:teuthology.task.internal:Saving configuration 2024-04-24T23:55:24.823 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T23:55:24.829 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T23:55:24.854 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi042.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:48:15.269827', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHzZSUDoSn0SEiwW3XVlhGDc79UQwRAjy2PlvKenau5lW0eR+3ndkkRnl7pxdJYAKgPtYq2718cT1yC2G3It3+o='} 2024-04-24T23:55:24.876 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:48:15.271356', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBwGSIxW8EyyY1jHfe62Qav65/Dy3hj1p+qIiBY/JqmY6aFiq+LOkEP9et96nPr5TauOhBQwxaiaqtNj+L6HvUk='} 2024-04-24T23:55:24.899 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 23:48:15.268021', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMxUsav4S7sr+lhIyKK9VByJmzCHpRIdYbH8hxivCss0qCQ+weMjpj/7+lrvX27db56RqzwG2t16ZYF0aSysLrE='} 2024-04-24T23:55:24.899 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T23:55:24.906 INFO:teuthology.task.internal:roles: ubuntu@smithi042.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-24T23:55:24.906 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-24T23:55:24.906 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-24T23:55:24.906 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T23:55:25.008 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f8512e0a160>, signals=[15]) 2024-04-24T23:55:25.009 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T23:55:25.015 INFO:teuthology.task.internal:Opening connections... 2024-04-24T23:55:25.016 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi042.front.sepia.ceph.com 2024-04-24T23:55:25.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:55:25.095 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2024-04-24T23:55:25.096 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:55:25.180 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-04-24T23:55:25.181 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:55:25.261 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T23:55:25.268 DEBUG:teuthology.orchestra.run.smithi042:> uname -m 2024-04-24T23:55:25.287 INFO:teuthology.orchestra.run.smithi042.stdout:x86_64 2024-04-24T23:55:25.287 DEBUG:teuthology.orchestra.run.smithi042:> cat /etc/os-release 2024-04-24T23:55:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:NAME="CentOS Stream" 2024-04-24T23:55:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION="9" 2024-04-24T23:55:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:ID="centos" 2024-04-24T23:55:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:55:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:VERSION_ID="9" 2024-04-24T23:55:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:55:25.346 INFO:teuthology.orchestra.run.smithi042.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:55:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:ANSI_COLOR="0;31" 2024-04-24T23:55:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:55:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:55:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:55:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:55:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:55:25.347 INFO:teuthology.orchestra.run.smithi042.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:55:25.348 INFO:teuthology.lock.ops:Updating smithi042.front.sepia.ceph.com on lock server 2024-04-24T23:55:25.372 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2024-04-24T23:55:25.390 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2024-04-24T23:55:25.390 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2024-04-24T23:55:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2024-04-24T23:55:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2024-04-24T23:55:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2024-04-24T23:55:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:55:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2024-04-24T23:55:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:55:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:55:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2024-04-24T23:55:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:55:25.451 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:55:25.452 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:55:25.452 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:55:25.452 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:55:25.452 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:55:25.452 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2024-04-24T23:55:25.472 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-04-24T23:55:25.491 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-04-24T23:55:25.491 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-04-24T23:55:25.547 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-04-24T23:55:25.547 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-04-24T23:55:25.547 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-04-24T23:55:25.547 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-04-24T23:55:25.547 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-04-24T23:55:25.548 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-04-24T23:55:25.548 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-24T23:55:25.548 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-04-24T23:55:25.548 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-04-24T23:55:25.548 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-24T23:55:25.548 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-04-24T23:55:25.548 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T23:55:25.548 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-24T23:55:25.548 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-24T23:55:25.548 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-04-24T23:55:25.572 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T23:55:25.583 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T23:55:25.589 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T23:55:25.589 DEBUG:teuthology.orchestra.run.smithi042:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:55:25.592 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:55:25.594 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-04-24T23:55:25.610 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T23:55:25.616 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T23:55:25.617 DEBUG:teuthology.orchestra.run.smithi042:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:55:25.649 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:55:25.652 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-04-24T23:55:25.711 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T23:55:25.786 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T23:55:25.799 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-04-24T23:55:25.799 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-24T23:55:25.799 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-24T23:55:25.800 DEBUG:teuthology.orchestra.run.smithi042:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:55:25.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:55:25.818 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-24T23:55:25.995 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T23:55:25.996 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-24T23:55:25.996 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-04-24T23:55:26.014 INFO:teuthology.orchestra.run.smithi042.stdout:5.14.0-437.el9.x86_64 2024-04-24T23:55:26.014 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T23:55:26.014 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-24T23:55:26.015 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum upgrade -y linux-firmware 2024-04-24T23:55:28.766 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-04-24T23:55:30.542 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-04-24T23:55:36.110 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 3.4 MB/s | 6.2 MB 00:01 2024-04-24T23:55:37.569 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-04-24T23:55:38.648 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 22 MB/s | 21 MB 00:00 2024-04-24T23:55:44.012 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-24T23:55:46.080 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-24T23:55:46.081 INFO:teuthology.orchestra.run.smithi042.stdout:Nothing to do. 2024-04-24T23:55:46.081 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-24T23:55:46.186 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T23:55:46.186 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-24T23:55:46.403 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-24T23:55:46.403 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-24T23:55:46.405 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi042.front.sepia.ceph.com to complete... 2024-04-24T23:55:58.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-24 23:55:58 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-24T23:55:58.748 INFO:teuthology.orchestra.run.smithi042.stderr:FINISHED --2024-04-24 23:55:58-- 2024-04-24T23:55:58.749 INFO:teuthology.orchestra.run.smithi042.stderr:Total wall clock time: 12s 2024-04-24T23:55:58.749 INFO:teuthology.orchestra.run.smithi042.stderr:Downloaded: 1 files, 341M in 12s (28.5 MB/s) 2024-04-24T23:55:58.753 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-24T23:55:58.753 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-24T23:56:00.932 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying... ######################################## 2024-04-24T23:56:01.044 INFO:teuthology.orchestra.run.smithi042.stdout:Preparing... ######################################## 2024-04-24T23:56:01.150 INFO:teuthology.orchestra.run.smithi042.stdout:Updating / installing... 2024-04-24T23:56:09.440 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-24T23:56:49.958 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi042.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-24T23:56:49.961 DEBUG:teuthology.orchestra.run.smithi042:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-24T23:56:50.052 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-24T23:56:50.052 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-24T23:56:50.052 INFO:teuthology.orchestra.run.smithi042.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-24T23:56:50.052 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-24T23:56:50.053 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-24T23:56:50.054 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-24T23:56:50.055 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-24T23:56:50.056 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-24T23:56:50.057 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-24T23:56:50.058 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-24T23:56:50.059 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-24T23:56:50.060 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-24T23:56:50.061 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-24T23:56:50.062 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-24T23:56:50.063 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-24T23:56:50.064 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-24T23:56:50.065 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-24T23:56:50.066 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-24T23:56:50.067 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-24T23:56:50.068 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-24T23:56:50.068 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-24T23:56:50.068 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-24T23:56:50.068 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-24T23:56:50.068 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-24T23:56:50.068 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-24T23:56:50.068 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-24T23:56:50.068 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-24T23:56:50.068 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-24T23:56:50.068 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-24T23:56:50.069 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-24T23:56:50.070 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-24T23:56:50.071 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-24T23:56:50.072 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-24T23:56:50.073 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-24T23:56:50.074 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-24T23:56:50.075 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-24T23:56:50.076 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-24T23:56:50.077 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-24T23:56:50.078 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-24T23:56:50.079 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-24T23:56:50.080 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-24T23:56:50.081 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-24T23:56:50.082 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-24T23:56:50.083 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-24T23:56:50.084 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-24T23:56:50.085 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-24T23:56:50.086 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-24T23:56:50.087 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-24T23:56:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-24T23:56:50.089 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-24T23:56:50.090 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-24T23:56:50.091 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-24T23:56:50.092 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-24T23:56:50.093 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-24T23:56:50.094 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-24T23:56:50.095 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-24T23:56:50.096 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-24T23:56:50.097 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-24T23:56:50.098 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-24T23:56:50.099 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-24T23:56:50.100 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-24T23:56:50.101 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-24T23:56:50.102 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-24T23:56:50.103 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-24T23:56:50.104 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-24T23:56:50.105 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-24T23:56:50.106 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-24T23:56:50.107 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-24T23:56:50.108 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-24T23:56:50.109 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-24T23:56:50.110 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-24T23:56:50.111 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-24T23:56:50.112 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-24T23:56:50.113 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-24T23:56:50.114 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-24T23:56:50.115 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-24T23:56:50.116 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-24T23:56:50.117 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-24T23:56:50.118 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-24T23:56:50.119 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-24T23:56:50.120 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-24T23:56:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-24T23:56:50.122 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-24T23:56:50.123 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-24T23:56:50.124 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-24T23:56:50.125 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-24T23:56:50.126 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-24T23:56:50.127 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-24T23:56:50.128 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-24T23:56:50.129 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-24T23:56:50.130 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-24T23:56:50.131 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-24T23:56:50.132 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-24T23:56:50.133 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-24T23:56:50.134 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-24T23:56:50.135 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-24T23:56:50.136 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-24T23:56:50.137 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-24T23:56:50.138 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-24T23:56:50.139 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-24T23:56:50.140 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-24T23:56:50.141 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-24T23:56:50.142 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-24T23:56:50.143 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-24T23:56:50.144 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-24T23:56:50.145 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-24T23:56:50.146 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-24T23:56:50.147 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-24T23:56:50.148 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-24T23:56:50.149 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-24T23:56:50.150 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-24T23:56:50.151 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-24T23:56:50.152 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-24T23:56:50.153 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-24T23:56:50.154 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-24T23:56:50.155 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-24T23:56:50.156 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-24T23:56:50.157 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-24T23:56:50.158 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-24T23:56:50.159 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-24T23:56:50.160 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-24T23:56:50.161 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-24T23:56:50.162 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-24T23:56:50.163 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-24T23:56:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-24T23:56:50.165 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-24T23:56:50.166 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-24T23:56:50.167 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-24T23:56:50.167 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-24T23:56:50.167 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-24T23:56:50.167 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-24T23:56:50.169 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-24T23:56:50.170 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-24T23:56:50.171 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-24T23:56:50.172 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-24T23:56:50.173 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-24T23:56:50.174 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-24T23:56:50.175 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-24T23:56:50.176 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-24T23:56:50.177 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-24T23:56:50.178 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-24T23:56:50.179 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-24T23:56:50.180 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-24T23:56:50.181 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-24T23:56:50.182 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-24T23:56:50.183 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-24T23:56:50.184 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-24T23:56:50.185 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-24T23:56:50.186 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-24T23:56:50.187 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-24T23:56:50.188 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-24T23:56:50.188 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-24T23:56:50.188 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-24T23:56:50.188 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-24T23:56:50.188 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-24T23:56:50.188 INFO:teuthology.orchestra.run.smithi042.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T23:56:50.192 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-24T23:56:50.192 DEBUG:teuthology.orchestra.run.smithi042:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-24T23:56:50.215 INFO:teuthology.orchestra.run.smithi042.stdout:postinstall scriptlet (using /bin/sh): 2024-04-24T23:56:50.215 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout:for file in vmlinuz System.map config; do 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: fi 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout:done 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-24T23:56:50.216 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-04-24T23:56:50.217 INFO:teuthology.orchestra.run.smithi042.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-24T23:56:50.217 INFO:teuthology.orchestra.run.smithi042.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-24T23:56:50.217 INFO:teuthology.orchestra.run.smithi042.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-24T23:56:50.217 INFO:teuthology.orchestra.run.smithi042.stdout:fi 2024-04-24T23:56:50.218 DEBUG:teuthology.orchestra.run.smithi042:> sudo rpm -qi grub2-tools 2024-04-24T23:56:50.290 INFO:teuthology.orchestra.run.smithi042.stdout:Name : grub2-tools 2024-04-24T23:56:50.290 INFO:teuthology.orchestra.run.smithi042.stdout:Epoch : 1 2024-04-24T23:56:50.290 INFO:teuthology.orchestra.run.smithi042.stdout:Version : 2.06 2024-04-24T23:56:50.290 INFO:teuthology.orchestra.run.smithi042.stdout:Release : 68.el9 2024-04-24T23:56:50.290 INFO:teuthology.orchestra.run.smithi042.stdout:Architecture: x86_64 2024-04-24T23:56:50.290 INFO:teuthology.orchestra.run.smithi042.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-24T23:56:50.290 INFO:teuthology.orchestra.run.smithi042.stdout:Group : Unspecified 2024-04-24T23:56:50.290 INFO:teuthology.orchestra.run.smithi042.stdout:Size : 8264254 2024-04-24T23:56:50.290 INFO:teuthology.orchestra.run.smithi042.stdout:License : GPLv3+ 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:Packager : builder@centos.org 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:Vendor : CentOS 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:Summary : Support tools for GRUB. 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:Description : 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:hardware devices. 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-24T23:56:50.291 INFO:teuthology.orchestra.run.smithi042.stdout:This subpackage provides tools for support of all platforms. 2024-04-24T23:56:50.292 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-24T23:56:50.292 INFO:teuthology.task.kernel:Updating grub on smithi042 to boot 6.9.0-rc3-g806ec36b2112 2024-04-24T23:56:50.293 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-24T23:56:50.892 INFO:teuthology.orchestra.run.smithi042.stderr:Generating grub configuration file ... 2024-04-24T23:56:52.288 INFO:teuthology.orchestra.run.smithi042.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-24T23:56:52.322 INFO:teuthology.orchestra.run.smithi042.stderr:done 2024-04-24T23:56:52.324 DEBUG:teuthology.orchestra.run.smithi042:> mktemp 2024-04-24T23:56:52.364 INFO:teuthology.orchestra.run.smithi042.stdout:/tmp/tmp.Q6cyFo6bV1 2024-04-24T23:56:52.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Q6cyFo6bV1 2024-04-24T23:56:52.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0666 /tmp/tmp.Q6cyFo6bV1 2024-04-24T23:56:52.578 DEBUG:teuthology.orchestra.remote:smithi042:/tmp/tmp.Q6cyFo6bV1 is 6KB 2024-04-24T23:56:52.629 DEBUG:teuthology.orchestra.run.smithi042:> rm -fr /tmp/tmp.Q6cyFo6bV1 2024-04-24T23:56:52.645 DEBUG:teuthology.orchestra.run.smithi042:> sudo /bin/ls /boot/loader/entries || true 2024-04-24T23:56:52.707 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-24T23:56:52.708 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-24T23:56:52.708 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-24T23:56:52.708 INFO:teuthology.orchestra.run.smithi042.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-24T23:56:52.709 DEBUG:teuthology.orchestra.run.smithi042:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-24T23:56:52.882 DEBUG:teuthology.orchestra.run.smithi042:> sudo shutdown -r now 2024-04-24T23:57:22.885 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-24T23:57:22.886 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-24T23:57:22.886 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:57:41.282 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-24T23:57:50.283 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-24T23:57:50.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:57:53.342 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-24T23:58:05.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-24T23:58:05.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:58:23.774 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.42 2024-04-24T23:58:38.775 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-24T23:58:38.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T23:58:39.194 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-04-24T23:58:39.857 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-24T23:58:39.858 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-24T23:58:39.858 DEBUG:teuthology.orchestra.run.smithi042:> uname -r 2024-04-24T23:58:39.898 INFO:teuthology.orchestra.run.smithi042.stdout:6.9.0-rc3-g806ec36b2112 2024-04-24T23:58:39.898 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-24T23:58:39.898 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-24T23:58:39.901 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-24T23:58:40.902 DEBUG:teuthology.parallel:result is None 2024-04-24T23:58:40.903 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T23:58:40.914 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T23:58:40.914 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:58:40.920 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:58:40.923 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T23:58:40.956 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T23:58:40.963 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T23:58:41.000 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T23:58:41.001 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:58:41.006 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:58:41.008 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T23:58:41.060 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T23:58:41.067 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T23:58:41.067 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:58:41.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:41.089 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T23:58:41.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:41.106 DEBUG:teuthology.orchestra.run.smithi042:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:58:41.111 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:58:41.132 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T23:58:41.171 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:41.182 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:41.190 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:41.195 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:41.242 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:41.293 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T23:58:41.297 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T23:58:41.305 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T23:58:41.305 DEBUG:teuthology.orchestra.run.smithi042:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:58:41.310 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:58:41.312 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T23:58:41.369 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T23:58:41.379 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T23:58:41.380 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:58:41.384 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:58:41.386 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T23:58:41.421 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:41.536 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:41.635 DEBUG:teuthology.orchestra.run.smithi042:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:41.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:41.798 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-24T23:58:41.798 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:58:41.886 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:41.930 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:41.982 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:42.066 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:42.107 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-24T23:58:42.107 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:58:42.173 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:42.215 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T23:58:42.287 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:42.335 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T23:58:42.372 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-24T23:58:42.372 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T23:58:42.438 DEBUG:teuthology.orchestra.run.smithi042:> sudo service rsyslog restart 2024-04-24T23:58:42.443 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2024-04-24T23:58:42.445 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-04-24T23:58:42.479 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:58:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:58:42.538 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T23:58:43.049 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T23:58:43.057 INFO:teuthology.task.internal:Starting timer... 2024-04-24T23:58:43.057 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T23:58:43.069 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T23:58:43.077 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-24T23:58:43.078 DEBUG:teuthology.orchestra.run.smithi042:> sudo service auditd rotate 2024-04-24T23:58:43.204 INFO:teuthology.orchestra.run.smithi042.stdout:Rotating logs: 2024-04-24T23:58:43.208 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2024-04-24T23:58:43.267 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2024-04-24T23:58:43.268 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-04-24T23:58:43.359 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-04-24T23:58:43.361 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T23:58:43.361 DEBUG:teuthology.orchestra.run.smithi042:> /usr/sbin/getenforce 2024-04-24T23:58:43.418 INFO:teuthology.orchestra.run.smithi042.stdout:Permissive 2024-04-24T23:58:43.418 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2024-04-24T23:58:43.440 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2024-04-24T23:58:43.440 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-04-24T23:58:43.475 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-04-24T23:58:43.475 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi042.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive'} 2024-04-24T23:58:43.475 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:58:43.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:43.560 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:58:43.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:43.586 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-24T23:58:43.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T23:58:43.613 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T23:58:43.613 DEBUG:teuthology.orchestra.run.smithi042:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:58:43.687 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:58:43.717 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-04-24T23:58:43.742 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T23:58:43.753 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-24T23:58:43.758 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-24T23:58:43.925 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T23:58:43.946 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T23:58:43.948 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi042.front.sepia.ceph.com,smithi106.front.sepia.ceph.com,smithi136.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T00:04:00.538 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi042.front.sepia.ceph.com'), Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com')] 2024-04-25T00:04:00.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-25T00:04:00.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi042.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:04:00.651 DEBUG:teuthology.orchestra.run.smithi042:> true 2024-04-25T00:04:00.815 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi042.front.sepia.ceph.com' 2024-04-25T00:04:00.816 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T00:04:00.816 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:04:00.895 DEBUG:teuthology.orchestra.run.smithi106:> true 2024-04-25T00:04:00.975 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2024-04-25T00:04:00.975 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-25T00:04:00.976 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T00:04:01.057 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-04-25T00:04:01.142 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-25T00:04:01.143 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T00:04:01.277 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T00:04:01.277 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:04:01.277 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:04:01.285 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:04:01.285 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:04:01.288 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T00:04:01.288 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T00:04:01.317 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:04:01.318 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:04:01.331 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:04:01.332 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:04:01.356 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2024-04-25T00:04:01.356 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-04-25T00:04:01.366 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-04-25T00:04:01.367 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2024-04-25T00:04:01.377 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T00:04:01.379 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:04:01.380 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:04:01.392 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:04:01.394 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:04:01.421 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T00:04:01.443 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:04:01.447 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:04:01.447 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-04-25T00:04:01.447 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.448 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.448 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.448 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.464 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:04:01.469 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:04:01.469 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-25T00:04:01.469 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.469 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.469 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.469 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.500 INFO:teuthology.orchestra.run.smithi042.stderr:sudo: ntpd: command not found 2024-04-25T00:04:01.534 INFO:teuthology.orchestra.run.smithi042.stdout:506 Cannot talk to daemon 2024-04-25T00:04:01.578 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T00:04:01.621 INFO:teuthology.orchestra.run.smithi042.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T00:04:01.772 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-04-25T00:04:01.781 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T00:04:01.781 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-04-25T00:04:01.781 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.782 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.782 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.782 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T00:04:01.783 INFO:teuthology.run_tasks:Running task install... 2024-04-25T00:04:01.800 DEBUG:teuthology.task.install:project ceph 2024-04-25T00:04:01.800 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-25T00:04:01.800 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-25T00:04:01.800 INFO:teuthology.task.install:Using flavor: default 2024-04-25T00:04:01.806 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T00:04:01.806 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T00:04:01.807 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:04:01.807 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:04:01.809 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:04:01.809 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:04:01.810 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-25T00:04:01.810 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:04:01.953 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:04:01.953 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:04:01.960 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:04:01.960 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:04:02.035 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-25T00:04:02.035 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-25T00:04:02.072 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:04:02.072 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:04:02.072 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:04:02.133 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:04:02.134 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T00:04:02.134 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:04:02.149 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T00:04:02.150 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T00:04:02.150 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T00:04:02.162 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:04:02.162 DEBUG:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:04:02.166 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:04:02.166 DEBUG:teuthology.orchestra.run.smithi042:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:04:02.176 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T00:04:02.176 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T00:04:02.229 DEBUG:teuthology.orchestra.run.smithi106:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:04:02.242 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:04:02.301 DEBUG:teuthology.orchestra.run.smithi042:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T00:04:02.305 DEBUG:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:04:02.315 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:04:02.369 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2024-04-25T00:04:02.372 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum clean all 2024-04-25T00:04:02.378 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-04-25T00:04:02.380 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-04-25T00:04:02.419 DEBUG:teuthology.orchestra.run.smithi042:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T00:04:02.503 INFO:teuthology.orchestra.run.smithi042.stdout:check_obsoletes = 1 2024-04-25T00:04:02.508 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum clean all 2024-04-25T00:04:02.596 INFO:teuthology.orchestra.run.smithi106.stdout:48 files removed 2024-04-25T00:04:02.602 INFO:teuthology.orchestra.run.smithi136.stdout:48 files removed 2024-04-25T00:04:02.623 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:04:02.629 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:04:02.834 INFO:teuthology.orchestra.run.smithi042.stdout:48 files removed 2024-04-25T00:04:02.875 DEBUG:teuthology.orchestra.run.smithi042:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T00:04:03.650 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 110 kB/s | 83 kB 00:00 2024-04-25T00:04:03.672 INFO:teuthology.orchestra.run.smithi106.stdout:ceph packages for x86_64 105 kB/s | 83 kB 00:00 2024-04-25T00:04:04.300 INFO:teuthology.orchestra.run.smithi042.stdout:ceph packages for x86_64 86 kB/s | 83 kB 00:00 2024-04-25T00:04:04.314 INFO:teuthology.orchestra.run.smithi106.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-04-25T00:04:04.384 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-04-25T00:04:05.057 INFO:teuthology.orchestra.run.smithi106.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-04-25T00:04:05.068 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 2.9 kB/s | 1.9 kB 00:00 2024-04-25T00:04:05.200 INFO:teuthology.orchestra.run.smithi042.stdout:ceph noarch packages 14 kB/s | 11 kB 00:00 2024-04-25T00:04:05.801 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T00:04:06.075 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 8.2 MB/s | 8.0 MB 00:00 2024-04-25T00:04:06.218 INFO:teuthology.orchestra.run.smithi042.stdout:ceph source packages 2.0 kB/s | 1.9 kB 00:00 2024-04-25T00:04:07.677 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T00:04:08.141 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - BaseOS 4.4 MB/s | 8.0 MB 00:01 2024-04-25T00:04:10.242 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-25T00:04:11.268 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 19 MB 00:04 2024-04-25T00:04:11.853 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T00:04:14.703 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 9.5 kB/s | 16 kB 00:01 2024-04-25T00:04:14.951 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-25T00:04:15.520 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-04-25T00:04:16.762 INFO:teuthology.orchestra.run.smithi042.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-04-25T00:04:16.936 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-04-25T00:04:18.097 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-25T00:04:18.346 INFO:teuthology.orchestra.run.smithi042.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T00:04:19.246 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 4.7 MB/s | 21 MB 00:04 2024-04-25T00:04:23.147 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T00:04:24.363 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-04-25T00:04:24.713 INFO:teuthology.orchestra.run.smithi042.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-04-25T00:04:25.248 INFO:teuthology.orchestra.run.smithi106.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:04:25.249 INFO:teuthology.orchestra.run.smithi106.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:04:25.260 INFO:teuthology.orchestra.run.smithi106.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:04:25.261 INFO:teuthology.orchestra.run.smithi106.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:04:25.261 INFO:teuthology.orchestra.run.smithi106.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:04:25.262 INFO:teuthology.orchestra.run.smithi106.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:04:25.264 INFO:teuthology.orchestra.run.smithi106.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:04:25.266 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:04:25.267 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:04:25.328 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2024-04-25T00:04:25.344 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-25T00:04:25.344 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2024-04-25T00:04:25.344 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-25T00:04:25.344 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2024-04-25T00:04:25.344 INFO:teuthology.orchestra.run.smithi106.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:04:25.345 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:04:25.346 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:04:25.347 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:04:25.348 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:04:25.349 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:04:25.350 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:04:25.350 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:04:25.350 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:04:25.350 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:04:25.351 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:04:25.352 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:04:25.352 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:04:25.352 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:04:25.352 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:04:25.352 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:04:25.352 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:04:25.352 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:04:25.352 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:04:25.352 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:04:25.352 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:04:25.353 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:04:25.354 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:04:25.355 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:04:25.356 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout:Installing weak dependencies: 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2024-04-25T00:04:25.357 INFO:teuthology.orchestra.run.smithi106.stdout:======================================================================================= 2024-04-25T00:04:25.358 INFO:teuthology.orchestra.run.smithi106.stdout:Install 148 Packages 2024-04-25T00:04:25.358 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 4 Packages 2024-04-25T00:04:25.358 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:04:25.358 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 220 M 2024-04-25T00:04:25.362 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2024-04-25T00:04:26.489 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:04:26.490 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:04:26.500 INFO:teuthology.orchestra.run.smithi136.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:04:26.501 INFO:teuthology.orchestra.run.smithi136.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:04:26.502 INFO:teuthology.orchestra.run.smithi136.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:04:26.503 INFO:teuthology.orchestra.run.smithi136.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:04:26.504 INFO:teuthology.orchestra.run.smithi136.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:04:26.507 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:04:26.507 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:04:26.569 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-04-25T00:04:26.594 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-04-25T00:04:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-04-25T00:04:26.594 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-04-25T00:04:26.594 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:04:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:04:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:04:26.597 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:04:26.598 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:04:26.599 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:04:26.600 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:04:26.601 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:04:26.602 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:04:26.603 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:04:26.604 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout:Install 148 Packages 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 4 Packages 2024-04-25T00:04:26.605 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:04:26.606 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 220 M 2024-04-25T00:04:26.606 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-04-25T00:04:26.715 INFO:teuthology.orchestra.run.smithi106.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-04-25T00:04:27.215 INFO:teuthology.orchestra.run.smithi042.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:04:27.217 INFO:teuthology.orchestra.run.smithi042.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T00:04:27.238 INFO:teuthology.orchestra.run.smithi042.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T00:04:27.239 INFO:teuthology.orchestra.run.smithi042.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T00:04:27.240 INFO:teuthology.orchestra.run.smithi042.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T00:04:27.241 INFO:teuthology.orchestra.run.smithi042.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T00:04:27.244 INFO:teuthology.orchestra.run.smithi042.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T00:04:27.247 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T00:04:27.249 INFO:teuthology.orchestra.run.smithi042.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T00:04:27.325 INFO:teuthology.orchestra.run.smithi042.stdout:Dependencies resolved. 2024-04-25T00:04:27.340 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-04-25T00:04:27.340 INFO:teuthology.orchestra.run.smithi042.stdout: Package Arch Version Repository Size 2024-04-25T00:04:27.340 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-04-25T00:04:27.340 INFO:teuthology.orchestra.run.smithi042.stdout:Installing: 2024-04-25T00:04:27.340 INFO:teuthology.orchestra.run.smithi042.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T00:04:27.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-25T00:04:27.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-25T00:04:27.340 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 222 k 2024-04-25T00:04:27.341 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 27 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 18 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T00:04:27.342 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrading: 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T00:04:27.343 INFO:teuthology.orchestra.run.smithi042.stdout:Installing dependencies: 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T00:04:27.344 INFO:teuthology.orchestra.run.smithi042.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-25T00:04:27.345 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T00:04:27.346 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T00:04:27.347 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T00:04:27.348 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T00:04:27.349 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T00:04:27.350 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout:Installing weak dependencies: 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:04:27.351 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction Summary 2024-04-25T00:04:27.352 INFO:teuthology.orchestra.run.smithi042.stdout:======================================================================================= 2024-04-25T00:04:27.352 INFO:teuthology.orchestra.run.smithi042.stdout:Install 148 Packages 2024-04-25T00:04:27.352 INFO:teuthology.orchestra.run.smithi042.stdout:Upgrade 4 Packages 2024-04-25T00:04:27.352 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:04:27.352 INFO:teuthology.orchestra.run.smithi106.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.4 MB/s | 890 kB 00:00 2024-04-25T00:04:27.353 INFO:teuthology.orchestra.run.smithi042.stdout:Total download size: 220 M 2024-04-25T00:04:27.353 INFO:teuthology.orchestra.run.smithi042.stdout:Downloading Packages: 2024-04-25T00:04:27.607 INFO:teuthology.orchestra.run.smithi106.stdout:(3/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 3.8 MB/s | 5.4 MB 00:01 2024-04-25T00:04:27.866 INFO:teuthology.orchestra.run.smithi106.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 9.2 MB/s | 2.4 MB 00:00 2024-04-25T00:04:28.008 INFO:teuthology.orchestra.run.smithi136.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 10 kB/s | 6.6 kB 00:00 2024-04-25T00:04:28.082 INFO:teuthology.orchestra.run.smithi106.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-04-25T00:04:28.258 INFO:teuthology.orchestra.run.smithi106.stdout:(6/152): ceph-immutable-object-cache-19.0.0-244 159 kB/s | 145 kB 00:00 2024-04-25T00:04:28.548 INFO:teuthology.orchestra.run.smithi136.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.6 MB/s | 890 kB 00:00 2024-04-25T00:04:28.631 INFO:teuthology.orchestra.run.smithi136.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.7 MB/s | 145 kB 00:00 2024-04-25T00:04:28.633 INFO:teuthology.orchestra.run.smithi106.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 8.5 MB/s | 4.7 MB 00:00 2024-04-25T00:04:28.648 INFO:teuthology.orchestra.run.smithi042.stdout:(1/152): ceph-19.0.0-2445.gd2c45d97.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-25T00:04:28.833 INFO:teuthology.orchestra.run.smithi106.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 8.0 MB/s | 21 MB 00:02 2024-04-25T00:04:28.933 INFO:teuthology.orchestra.run.smithi106.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 252 kB/s | 25 kB 00:00 2024-04-25T00:04:29.273 INFO:teuthology.orchestra.run.smithi136.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 3.7 MB/s | 2.4 MB 00:00 2024-04-25T00:04:29.598 INFO:teuthology.orchestra.run.smithi042.stdout:(2/152): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x8 938 kB/s | 890 kB 00:00 2024-04-25T00:04:29.715 INFO:teuthology.orchestra.run.smithi042.stdout:(3/152): ceph-immutable-object-cache-19.0.0-244 1.2 MB/s | 145 kB 00:00 2024-04-25T00:04:29.748 INFO:teuthology.orchestra.run.smithi136.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-25T00:04:30.025 INFO:teuthology.orchestra.run.smithi106.stdout:(10/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 7.4 MB/s | 10 MB 00:01 2024-04-25T00:04:30.150 INFO:teuthology.orchestra.run.smithi106.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 1.7 MB/s | 222 kB 00:00 2024-04-25T00:04:30.259 INFO:teuthology.orchestra.run.smithi106.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 249 kB/s | 27 kB 00:00 2024-04-25T00:04:30.385 INFO:teuthology.orchestra.run.smithi106.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 254 kB/s | 32 kB 00:00 2024-04-25T00:04:30.626 INFO:teuthology.orchestra.run.smithi106.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.0 MB/s | 745 kB 00:00 2024-04-25T00:04:30.760 INFO:teuthology.orchestra.run.smithi106.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.2 MB/s | 163 kB 00:00 2024-04-25T00:04:30.885 INFO:teuthology.orchestra.run.smithi106.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.0 MB/s | 127 kB 00:00 2024-04-25T00:04:30.916 INFO:teuthology.orchestra.run.smithi042.stdout:(4/152): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86 2.0 MB/s | 2.4 MB 00:01 2024-04-25T00:04:30.940 INFO:teuthology.orchestra.run.smithi136.stdout:(6/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 3.9 MB/s | 4.7 MB 00:01 2024-04-25T00:04:31.027 INFO:teuthology.orchestra.run.smithi106.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.5 MB/s | 501 kB 00:00 2024-04-25T00:04:31.535 INFO:teuthology.orchestra.run.smithi042.stdout:(5/152): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86 2.4 MB/s | 1.5 MB 00:00 2024-04-25T00:04:32.627 INFO:teuthology.orchestra.run.smithi106.stdout:(18/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 3.8 MB/s | 17 MB 00:04 2024-04-25T00:04:32.665 INFO:teuthology.orchestra.run.smithi042.stdout:(6/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 1.2 MB/s | 5.4 MB 00:04 2024-04-25T00:04:32.694 INFO:teuthology.orchestra.run.smithi106.stdout:(19/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 3.2 MB/s | 5.3 MB 00:01 2024-04-25T00:04:32.711 INFO:teuthology.orchestra.run.smithi106.stdout:(20/152): python3-ceph-argparse-19.0.0-2445.gd2 541 kB/s | 45 kB 00:00 2024-04-25T00:04:32.811 INFO:teuthology.orchestra.run.smithi106.stdout:(21/152): python3-ceph-common-19.0.0-2445.gd2c4 1.1 MB/s | 133 kB 00:00 2024-04-25T00:04:32.911 INFO:teuthology.orchestra.run.smithi106.stdout:(22/152): python3-cephfs-19.0.0-2445.gd2c45d97. 808 kB/s | 162 kB 00:00 2024-04-25T00:04:33.020 INFO:teuthology.orchestra.run.smithi106.stdout:(23/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.5 MB/s | 321 kB 00:00 2024-04-25T00:04:33.112 INFO:teuthology.orchestra.run.smithi106.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 299 kB 00:00 2024-04-25T00:04:33.137 INFO:teuthology.orchestra.run.smithi106.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 851 kB/s | 99 kB 00:00 2024-04-25T00:04:33.157 INFO:teuthology.orchestra.run.smithi042.stdout:(7/152): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86 2.9 MB/s | 4.7 MB 00:01 2024-04-25T00:04:33.188 INFO:teuthology.orchestra.run.smithi106.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T00:04:33.257 INFO:teuthology.orchestra.run.smithi136.stdout:(7/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 3.6 MB/s | 21 MB 00:05 2024-04-25T00:04:33.297 INFO:teuthology.orchestra.run.smithi106.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 1.5 MB/s | 171 kB 00:00 2024-04-25T00:04:33.413 INFO:teuthology.orchestra.run.smithi106.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 272 kB/s | 32 kB 00:00 2024-04-25T00:04:33.539 INFO:teuthology.orchestra.run.smithi106.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.1 MB/s | 141 kB 00:00 2024-04-25T00:04:33.917 INFO:teuthology.orchestra.run.smithi136.stdout:(8/152): ceph-base-19.0.0-2445.gd2c45d97.el9.x8 839 kB/s | 5.4 MB 00:06 2024-04-25T00:04:34.015 INFO:teuthology.orchestra.run.smithi136.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 256 kB/s | 25 kB 00:00 2024-04-25T00:04:34.197 INFO:teuthology.orchestra.run.smithi106.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 2.9 MB/s | 3.1 MB 00:01 2024-04-25T00:04:34.523 INFO:teuthology.orchestra.run.smithi106.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 3.6 MB/s | 3.5 MB 00:00 2024-04-25T00:04:34.532 INFO:teuthology.orchestra.run.smithi136.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 4.7 MB/s | 17 MB 00:03 2024-04-25T00:04:34.615 INFO:teuthology.orchestra.run.smithi106.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.7 MB/s | 249 kB 00:00 2024-04-25T00:04:34.624 INFO:teuthology.orchestra.run.smithi136.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.4 MB/s | 222 kB 00:00 2024-04-25T00:04:34.683 INFO:teuthology.orchestra.run.smithi136.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 465 kB/s | 27 kB 00:00 2024-04-25T00:04:34.707 INFO:teuthology.orchestra.run.smithi106.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 539 kB/s | 49 kB 00:00 2024-04-25T00:04:34.749 INFO:teuthology.orchestra.run.smithi136.stdout:(13/152): libcephfs-devel-19.0.0-2445.gd2c45d97 477 kB/s | 32 kB 00:00 2024-04-25T00:04:34.799 INFO:teuthology.orchestra.run.smithi106.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 167 kB/s | 15 kB 00:00 2024-04-25T00:04:34.915 INFO:teuthology.orchestra.run.smithi106.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.2 MB/s | 264 kB 00:00 2024-04-25T00:04:34.975 INFO:teuthology.orchestra.run.smithi136.stdout:(14/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.2 MB/s | 745 kB 00:00 2024-04-25T00:04:35.067 INFO:teuthology.orchestra.run.smithi136.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.7 MB/s | 163 kB 00:00 2024-04-25T00:04:35.142 INFO:teuthology.orchestra.run.smithi136.stdout:(16/152): librados-devel-19.0.0-2445.gd2c45d97. 1.7 MB/s | 127 kB 00:00 2024-04-25T00:04:35.166 INFO:teuthology.orchestra.run.smithi106.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 3.0 MB/s | 762 kB 00:00 2024-04-25T00:04:35.258 INFO:teuthology.orchestra.run.smithi106.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 199 kB/s | 18 kB 00:00 2024-04-25T00:04:35.259 INFO:teuthology.orchestra.run.smithi136.stdout:(17/152): libradosstriper1-19.0.0-2445.gd2c45d9 4.2 MB/s | 501 kB 00:00 2024-04-25T00:04:35.399 INFO:teuthology.orchestra.run.smithi042.stdout:(8/152): ceph-common-19.0.0-2445.gd2c45d97.el9. 2.9 MB/s | 21 MB 00:07 2024-04-25T00:04:35.499 INFO:teuthology.orchestra.run.smithi106.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 332 kB/s | 80 kB 00:00 2024-04-25T00:04:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:(9/152): ceph-selinux-19.0.0-2445.gd2c45d97.el9 190 kB/s | 25 kB 00:00 2024-04-25T00:04:35.592 INFO:teuthology.orchestra.run.smithi106.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-04-25T00:04:35.650 INFO:teuthology.orchestra.run.smithi106.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-25T00:04:35.685 INFO:teuthology.orchestra.run.smithi136.stdout:(18/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 4.2 MB/s | 10 MB 00:02 2024-04-25T00:04:35.725 INFO:teuthology.orchestra.run.smithi106.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-25T00:04:35.776 INFO:teuthology.orchestra.run.smithi136.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 494 kB/s | 45 kB 00:00 2024-04-25T00:04:35.834 INFO:teuthology.orchestra.run.smithi106.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-25T00:04:35.835 INFO:teuthology.orchestra.run.smithi136.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 2.2 MB/s | 133 kB 00:00 2024-04-25T00:04:35.901 INFO:teuthology.orchestra.run.smithi106.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T00:04:35.902 INFO:teuthology.orchestra.run.smithi136.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 2.4 MB/s | 162 kB 00:00 2024-04-25T00:04:35.968 INFO:teuthology.orchestra.run.smithi106.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-25T00:04:36.002 INFO:teuthology.orchestra.run.smithi136.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 3.1 MB/s | 321 kB 00:00 2024-04-25T00:04:36.008 INFO:teuthology.orchestra.run.smithi042.stdout:(10/152): ceph-osd-19.0.0-2445.gd2c45d97.el9.x8 5.0 MB/s | 17 MB 00:03 2024-04-25T00:04:36.035 INFO:teuthology.orchestra.run.smithi106.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T00:04:36.108 INFO:teuthology.orchestra.run.smithi042.stdout:(11/152): cephfs-mirror-19.0.0-2445.gd2c45d97.e 2.2 MB/s | 222 kB 00:00 2024-04-25T00:04:36.110 INFO:teuthology.orchestra.run.smithi106.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-04-25T00:04:36.169 INFO:teuthology.orchestra.run.smithi136.stdout:(23/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 5.8 MB/s | 5.3 MB 00:00 2024-04-25T00:04:36.194 INFO:teuthology.orchestra.run.smithi136.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.5 MB/s | 299 kB 00:00 2024-04-25T00:04:36.200 INFO:teuthology.orchestra.run.smithi042.stdout:(12/152): cephfs-shell-19.0.0-2445.gd2c45d97.el 297 kB/s | 27 kB 00:00 2024-04-25T00:04:36.220 INFO:teuthology.orchestra.run.smithi136.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.9 MB/s | 99 kB 00:00 2024-04-25T00:04:36.245 INFO:teuthology.orchestra.run.smithi136.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-25T00:04:36.304 INFO:teuthology.orchestra.run.smithi136.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.9 MB/s | 171 kB 00:00 2024-04-25T00:04:36.354 INFO:teuthology.orchestra.run.smithi136.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 635 kB/s | 32 kB 00:00 2024-04-25T00:04:36.360 INFO:teuthology.orchestra.run.smithi106.stdout:(47/152): ceph-mgr-diskprediction-local-19.0.0- 3.4 MB/s | 7.4 MB 00:02 2024-04-25T00:04:36.367 INFO:teuthology.orchestra.run.smithi042.stdout:(13/152): ceph-radosgw-19.0.0-2445.gd2c45d97.el 3.2 MB/s | 10 MB 00:03 2024-04-25T00:04:36.404 INFO:teuthology.orchestra.run.smithi136.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 2.8 MB/s | 141 kB 00:00 2024-04-25T00:04:36.417 INFO:teuthology.orchestra.run.smithi042.stdout:(14/152): libcephfs-devel-19.0.0-2445.gd2c45d97 147 kB/s | 32 kB 00:00 2024-04-25T00:04:36.509 INFO:teuthology.orchestra.run.smithi042.stdout:(15/152): libcephsqlite-19.0.0-2445.gd2c45d97.e 1.8 MB/s | 163 kB 00:00 2024-04-25T00:04:36.535 INFO:teuthology.orchestra.run.smithi106.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 1.6 MB/s | 689 kB 00:00 2024-04-25T00:04:36.594 INFO:teuthology.orchestra.run.smithi106.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-25T00:04:36.610 INFO:teuthology.orchestra.run.smithi042.stdout:(16/152): libcephfs2-19.0.0-2445.gd2c45d97.el9. 3.0 MB/s | 745 kB 00:00 2024-04-25T00:04:36.635 INFO:teuthology.orchestra.run.smithi042.stdout:(17/152): librados-devel-19.0.0-2445.gd2c45d97. 1.0 MB/s | 127 kB 00:00 2024-04-25T00:04:36.646 INFO:teuthology.orchestra.run.smithi136.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 7.3 MB/s | 3.1 MB 00:00 2024-04-25T00:04:36.653 INFO:teuthology.orchestra.run.smithi106.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 368 kB/s | 21 kB 00:00 2024-04-25T00:04:36.753 INFO:teuthology.orchestra.run.smithi106.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 3.1 MB/s | 313 kB 00:00 2024-04-25T00:04:36.768 INFO:teuthology.orchestra.run.smithi042.stdout:(18/152): libradosstriper1-19.0.0-2445.gd2c45d9 3.1 MB/s | 501 kB 00:00 2024-04-25T00:04:36.787 INFO:teuthology.orchestra.run.smithi106.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 943 kB 00:00 2024-04-25T00:04:36.812 INFO:teuthology.orchestra.run.smithi106.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-04-25T00:04:36.879 INFO:teuthology.orchestra.run.smithi106.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-04-25T00:04:36.885 INFO:teuthology.orchestra.run.smithi042.stdout:(19/152): python3-ceph-argparse-19.0.0-2445.gd2 389 kB/s | 45 kB 00:00 2024-04-25T00:04:36.946 INFO:teuthology.orchestra.run.smithi106.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-04-25T00:04:36.986 INFO:teuthology.orchestra.run.smithi042.stdout:(20/152): python3-ceph-common-19.0.0-2445.gd2c4 1.3 MB/s | 133 kB 00:00 2024-04-25T00:04:37.005 INFO:teuthology.orchestra.run.smithi106.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-25T00:04:37.064 INFO:teuthology.orchestra.run.smithi106.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-04-25T00:04:37.080 INFO:teuthology.orchestra.run.smithi136.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 5.2 MB/s | 3.5 MB 00:00 2024-04-25T00:04:37.094 INFO:teuthology.orchestra.run.smithi042.stdout:(21/152): python3-cephfs-19.0.0-2445.gd2c45d97. 1.5 MB/s | 162 kB 00:00 2024-04-25T00:04:37.147 INFO:teuthology.orchestra.run.smithi136.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 3.6 MB/s | 249 kB 00:00 2024-04-25T00:04:37.222 INFO:teuthology.orchestra.run.smithi106.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-04-25T00:04:37.223 INFO:teuthology.orchestra.run.smithi136.stdout:(33/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 658 kB/s | 49 kB 00:00 2024-04-25T00:04:37.270 INFO:teuthology.orchestra.run.smithi042.stdout:(22/152): python3-rados-19.0.0-2445.gd2c45d97.e 1.8 MB/s | 321 kB 00:00 2024-04-25T00:04:37.289 INFO:teuthology.orchestra.run.smithi136.stdout:(34/152): ceph-prometheus-alerts-19.0.0-2445.gd 231 kB/s | 15 kB 00:00 2024-04-25T00:04:37.339 INFO:teuthology.orchestra.run.smithi106.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.4 MB/s | 3.0 MB 00:00 2024-04-25T00:04:37.373 INFO:teuthology.orchestra.run.smithi136.stdout:(35/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 3.1 MB/s | 264 kB 00:00 2024-04-25T00:04:37.381 INFO:teuthology.orchestra.run.smithi106.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-04-25T00:04:37.406 INFO:teuthology.orchestra.run.smithi106.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-04-25T00:04:37.440 INFO:teuthology.orchestra.run.smithi106.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-25T00:04:37.490 INFO:teuthology.orchestra.run.smithi106.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-25T00:04:37.503 INFO:teuthology.orchestra.run.smithi042.stdout:(23/152): librgw2-19.0.0-2445.gd2c45d97.el9.x86 6.1 MB/s | 5.3 MB 00:00 2024-04-25T00:04:37.506 INFO:teuthology.orchestra.run.smithi136.stdout:(36/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 5.6 MB/s | 762 kB 00:00 2024-04-25T00:04:37.515 INFO:teuthology.orchestra.run.smithi106.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-25T00:04:37.537 INFO:teuthology.orchestra.run.smithi042.stdout:(24/152): python3-rbd-19.0.0-2445.gd2c45d97.el9 1.1 MB/s | 299 kB 00:00 2024-04-25T00:04:37.549 INFO:teuthology.orchestra.run.smithi106.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-04-25T00:04:37.565 INFO:teuthology.orchestra.run.smithi136.stdout:(37/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 313 kB/s | 18 kB 00:00 2024-04-25T00:04:37.587 INFO:teuthology.orchestra.run.smithi042.stdout:(25/152): python3-rgw-19.0.0-2445.gd2c45d97.el9 1.2 MB/s | 99 kB 00:00 2024-04-25T00:04:37.621 INFO:teuthology.orchestra.run.smithi042.stdout:(26/152): rbd-fuse-19.0.0-2445.gd2c45d97.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-25T00:04:37.674 INFO:teuthology.orchestra.run.smithi136.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 742 kB/s | 80 kB 00:00 2024-04-25T00:04:37.705 INFO:teuthology.orchestra.run.smithi042.stdout:(27/152): rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86 2.0 MB/s | 171 kB 00:00 2024-04-25T00:04:37.766 INFO:teuthology.orchestra.run.smithi136.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 6.6 MB/s | 7.4 MB 00:01 2024-04-25T00:04:37.805 INFO:teuthology.orchestra.run.smithi042.stdout:(28/152): ceph-grafana-dashboards-19.0.0-2445.g 321 kB/s | 32 kB 00:00 2024-04-25T00:04:37.849 INFO:teuthology.orchestra.run.smithi136.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-04-25T00:04:37.889 INFO:teuthology.orchestra.run.smithi042.stdout:(29/152): ceph-mgr-cephadm-19.0.0-2445.gd2c45d9 1.7 MB/s | 141 kB 00:00 2024-04-25T00:04:37.891 INFO:teuthology.orchestra.run.smithi136.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T00:04:37.941 INFO:teuthology.orchestra.run.smithi136.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-25T00:04:37.984 INFO:teuthology.orchestra.run.smithi136.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-04-25T00:04:38.000 INFO:teuthology.orchestra.run.smithi136.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T00:04:38.022 INFO:teuthology.orchestra.run.smithi042.stdout:(30/152): rbd-mirror-19.0.0-2445.gd2c45d97.el9. 7.1 MB/s | 3.1 MB 00:00 2024-04-25T00:04:38.024 INFO:teuthology.orchestra.run.smithi106.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-04-25T00:04:38.025 INFO:teuthology.orchestra.run.smithi136.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T00:04:38.104 INFO:teuthology.orchestra.run.smithi136.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.2 MB/s | 125 kB 00:00 2024-04-25T00:04:38.127 INFO:teuthology.orchestra.run.smithi136.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-25T00:04:38.141 INFO:teuthology.orchestra.run.smithi106.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.6 MB/s | 4.7 MB 00:00 2024-04-25T00:04:38.233 INFO:teuthology.orchestra.run.smithi106.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-25T00:04:38.275 INFO:teuthology.orchestra.run.smithi106.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-04-25T00:04:38.294 INFO:teuthology.orchestra.run.smithi136.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.6 MB/s | 689 kB 00:00 2024-04-25T00:04:38.300 INFO:teuthology.orchestra.run.smithi106.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T00:04:38.394 INFO:teuthology.orchestra.run.smithi136.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 3.5 MB/s | 943 kB 00:00 2024-04-25T00:04:38.401 INFO:teuthology.orchestra.run.smithi106.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-25T00:04:38.419 INFO:teuthology.orchestra.run.smithi136.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 853 kB/s | 107 kB 00:00 2024-04-25T00:04:38.459 INFO:teuthology.orchestra.run.smithi106.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T00:04:38.470 INFO:teuthology.orchestra.run.smithi136.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 286 kB/s | 21 kB 00:00 2024-04-25T00:04:38.536 INFO:teuthology.orchestra.run.smithi136.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 492 kB/s | 33 kB 00:00 2024-04-25T00:04:38.562 INFO:teuthology.orchestra.run.smithi136.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-25T00:04:38.620 INFO:teuthology.orchestra.run.smithi136.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-04-25T00:04:38.671 INFO:teuthology.orchestra.run.smithi136.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-25T00:04:38.697 INFO:teuthology.orchestra.run.smithi042.stdout:(31/152): ceph-mgr-dashboard-19.0.0-2445.gd2c45 4.4 MB/s | 3.5 MB 00:00 2024-04-25T00:04:38.718 INFO:teuthology.orchestra.run.smithi106.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 13 MB/s | 5.8 MB 00:00 2024-04-25T00:04:38.754 INFO:teuthology.orchestra.run.smithi136.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-25T00:04:38.785 INFO:teuthology.orchestra.run.smithi106.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-25T00:04:38.789 INFO:teuthology.orchestra.run.smithi042.stdout:(32/152): ceph-mgr-modules-core-19.0.0-2445.gd2 2.7 MB/s | 249 kB 00:00 2024-04-25T00:04:38.821 INFO:teuthology.orchestra.run.smithi136.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 680 kB/s | 45 kB 00:00 2024-04-25T00:04:38.852 INFO:teuthology.orchestra.run.smithi106.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-25T00:04:38.877 INFO:teuthology.orchestra.run.smithi106.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-25T00:04:38.911 INFO:teuthology.orchestra.run.smithi106.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-04-25T00:04:38.914 INFO:teuthology.orchestra.run.smithi042.stdout:(33/152): ceph-mgr-diskprediction-local-19.0.0- 8.3 MB/s | 7.4 MB 00:00 2024-04-25T00:04:38.928 INFO:teuthology.orchestra.run.smithi106.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-04-25T00:04:38.957 INFO:teuthology.orchestra.run.smithi042.stdout:(34/152): ceph-mgr-rook-19.0.0-2445.gd2c45d97.e 297 kB/s | 49 kB 00:00 2024-04-25T00:04:38.970 INFO:teuthology.orchestra.run.smithi106.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T00:04:38.986 INFO:teuthology.orchestra.run.smithi106.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 592 kB/s | 35 kB 00:00 2024-04-25T00:04:38.988 INFO:teuthology.orchestra.run.smithi136.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-04-25T00:04:38.998 INFO:teuthology.orchestra.run.smithi042.stdout:(35/152): ceph-prometheus-alerts-19.0.0-2445.gd 185 kB/s | 15 kB 00:00 2024-04-25T00:04:39.045 INFO:teuthology.orchestra.run.smithi106.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-04-25T00:04:39.062 INFO:teuthology.orchestra.run.smithi042.stdout:(36/152): ceph-volume-19.0.0-2445.gd2c45d97.el9 2.5 MB/s | 264 kB 00:00 2024-04-25T00:04:39.097 INFO:teuthology.orchestra.run.smithi136.stdout:(59/152): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-04-25T00:04:39.104 INFO:teuthology.orchestra.run.smithi106.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-25T00:04:39.149 INFO:teuthology.orchestra.run.smithi042.stdout:(37/152): cephadm-19.0.0-2445.gd2c45d97.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-25T00:04:39.163 INFO:teuthology.orchestra.run.smithi136.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 383 kB/s | 25 kB 00:00 2024-04-25T00:04:39.171 INFO:teuthology.orchestra.run.smithi106.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-25T00:04:39.191 INFO:teuthology.orchestra.run.smithi042.stdout:(38/152): cephfs-top-19.0.0-2445.gd2c45d97.el9. 142 kB/s | 18 kB 00:00 2024-04-25T00:04:39.230 INFO:teuthology.orchestra.run.smithi106.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 42 kB 00:00 2024-04-25T00:04:39.258 INFO:teuthology.orchestra.run.smithi042.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 80 kB 00:00 2024-04-25T00:04:39.296 INFO:teuthology.orchestra.run.smithi106.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-25T00:04:39.322 INFO:teuthology.orchestra.run.smithi106.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-25T00:04:39.325 INFO:teuthology.orchestra.run.smithi042.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 6.0 MB/s | 803 kB 00:00 2024-04-25T00:04:39.355 INFO:teuthology.orchestra.run.smithi106.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-04-25T00:04:39.356 INFO:teuthology.orchestra.run.smithi136.stdout:(61/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-04-25T00:04:39.375 INFO:teuthology.orchestra.run.smithi042.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-25T00:04:39.389 INFO:teuthology.orchestra.run.smithi106.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-25T00:04:39.414 INFO:teuthology.orchestra.run.smithi106.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-04-25T00:04:39.422 INFO:teuthology.orchestra.run.smithi136.stdout:(62/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.4 MB/s | 3.0 MB 00:00 2024-04-25T00:04:39.426 INFO:teuthology.orchestra.run.smithi042.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-04-25T00:04:39.448 INFO:teuthology.orchestra.run.smithi106.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 916 kB/s | 54 kB 00:00 2024-04-25T00:04:39.493 INFO:teuthology.orchestra.run.smithi042.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T00:04:39.506 INFO:teuthology.orchestra.run.smithi136.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-25T00:04:39.507 INFO:teuthology.orchestra.run.smithi106.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 712 kB/s | 42 kB 00:00 2024-04-25T00:04:39.532 INFO:teuthology.orchestra.run.smithi136.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-25T00:04:39.560 INFO:teuthology.orchestra.run.smithi042.stdout:(44/152): python3-cryptography-36.0.1-4.el9.x86 6.7 MB/s | 1.2 MB 00:00 2024-04-25T00:04:39.565 INFO:teuthology.orchestra.run.smithi106.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-04-25T00:04:39.590 INFO:teuthology.orchestra.run.smithi136.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 423 kB/s | 35 kB 00:00 2024-04-25T00:04:39.602 INFO:teuthology.orchestra.run.smithi042.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-04-25T00:04:39.644 INFO:teuthology.orchestra.run.smithi042.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-04-25T00:04:39.674 INFO:teuthology.orchestra.run.smithi106.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-04-25T00:04:39.694 INFO:teuthology.orchestra.run.smithi042.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.3 MB/s | 215 kB 00:00 2024-04-25T00:04:39.733 INFO:teuthology.orchestra.run.smithi106.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 614 kB/s | 36 kB 00:00 2024-04-25T00:04:39.791 INFO:teuthology.orchestra.run.smithi106.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-04-25T00:04:39.807 INFO:teuthology.orchestra.run.smithi136.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-04-25T00:04:39.858 INFO:teuthology.orchestra.run.smithi106.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-25T00:04:39.949 INFO:teuthology.orchestra.run.smithi136.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-04-25T00:04:40.116 INFO:teuthology.orchestra.run.smithi136.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 5.2 MB/s | 1.6 MB 00:00 2024-04-25T00:04:40.208 INFO:teuthology.orchestra.run.smithi136.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 3.4 MB/s | 911 kB 00:00 2024-04-25T00:04:40.233 INFO:teuthology.orchestra.run.smithi136.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-25T00:04:40.292 INFO:teuthology.orchestra.run.smithi106.stdout:(97/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 4.3 MB/s | 48 MB 00:11 2024-04-25T00:04:40.317 INFO:teuthology.orchestra.run.smithi106.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 126 kB/s | 58 kB 00:00 2024-04-25T00:04:40.376 INFO:teuthology.orchestra.run.smithi106.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T00:04:40.428 INFO:teuthology.orchestra.run.smithi042.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 881 kB/s | 689 kB 00:00 2024-04-25T00:04:40.434 INFO:teuthology.orchestra.run.smithi106.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-04-25T00:04:40.458 INFO:teuthology.orchestra.run.smithi136.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.5 MB/s | 1.0 MB 00:00 2024-04-25T00:04:40.517 INFO:teuthology.orchestra.run.smithi136.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-25T00:04:40.544 INFO:teuthology.orchestra.run.smithi042.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-04-25T00:04:40.578 INFO:teuthology.orchestra.run.smithi042.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 716 kB/s | 107 kB 00:00 2024-04-25T00:04:40.585 INFO:teuthology.orchestra.run.smithi106.stdout:(101/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-04-25T00:04:40.610 INFO:teuthology.orchestra.run.smithi106.stdout:(102/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 295 kB/s | 94 kB 00:00 2024-04-25T00:04:40.635 INFO:teuthology.orchestra.run.smithi106.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 111 kB/s | 22 kB 00:00 2024-04-25T00:04:40.695 INFO:teuthology.orchestra.run.smithi042.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 144 kB/s | 21 kB 00:00 2024-04-25T00:04:40.702 INFO:teuthology.orchestra.run.smithi106.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 7.2 MB/s | 863 kB 00:00 2024-04-25T00:04:40.729 INFO:teuthology.orchestra.run.smithi042.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-04-25T00:04:40.734 INFO:teuthology.orchestra.run.smithi136.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-25T00:04:40.819 INFO:teuthology.orchestra.run.smithi106.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 167 kB/s | 19 kB 00:00 2024-04-25T00:04:40.821 INFO:teuthology.orchestra.run.smithi042.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 263 kB/s | 33 kB 00:00 2024-04-25T00:04:40.878 INFO:teuthology.orchestra.run.smithi106.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T00:04:40.911 INFO:teuthology.orchestra.run.smithi106.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-25T00:04:40.936 INFO:teuthology.orchestra.run.smithi106.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 443 kB/s | 26 kB 00:00 2024-04-25T00:04:40.946 INFO:teuthology.orchestra.run.smithi042.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 18 kB 00:00 2024-04-25T00:04:40.970 INFO:teuthology.orchestra.run.smithi106.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 831 kB/s | 49 kB 00:00 2024-04-25T00:04:40.984 INFO:teuthology.orchestra.run.smithi136.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 978 kB/s | 244 kB 00:00 2024-04-25T00:04:40.995 INFO:teuthology.orchestra.run.smithi106.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-25T00:04:41.029 INFO:teuthology.orchestra.run.smithi106.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-25T00:04:41.060 INFO:teuthology.orchestra.run.smithi136.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-25T00:04:41.080 INFO:teuthology.orchestra.run.smithi042.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 84 kB/s | 11 kB 00:00 2024-04-25T00:04:41.113 INFO:teuthology.orchestra.run.smithi106.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-25T00:04:41.118 INFO:teuthology.orchestra.run.smithi136.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 442 kB/s | 26 kB 00:00 2024-04-25T00:04:41.146 INFO:teuthology.orchestra.run.smithi106.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.4 MB/s | 838 kB 00:00 2024-04-25T00:04:41.172 INFO:teuthology.orchestra.run.smithi106.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 743 kB/s | 43 kB 00:00 2024-04-25T00:04:41.177 INFO:teuthology.orchestra.run.smithi136.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.3 MB/s | 248 kB 00:00 2024-04-25T00:04:41.197 INFO:teuthology.orchestra.run.smithi042.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-04-25T00:04:41.202 INFO:teuthology.orchestra.run.smithi136.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 229 kB/s | 19 kB 00:00 2024-04-25T00:04:41.205 INFO:teuthology.orchestra.run.smithi106.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 550 kB/s | 32 kB 00:00 2024-04-25T00:04:41.222 INFO:teuthology.orchestra.run.smithi106.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-25T00:04:41.269 INFO:teuthology.orchestra.run.smithi136.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-25T00:04:41.280 INFO:teuthology.orchestra.run.smithi042.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.4 MB/s | 3.0 MB 00:00 2024-04-25T00:04:41.289 INFO:teuthology.orchestra.run.smithi106.stdout:(117/152): libarrow-9.0.0-10.el9.x86_64.rpm 9.4 MB/s | 4.4 MB 00:00 2024-04-25T00:04:41.294 INFO:teuthology.orchestra.run.smithi136.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 378 kB/s | 35 kB 00:00 2024-04-25T00:04:41.314 INFO:teuthology.orchestra.run.smithi106.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-04-25T00:04:41.322 INFO:teuthology.orchestra.run.smithi042.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 364 kB/s | 45 kB 00:00 2024-04-25T00:04:41.348 INFO:teuthology.orchestra.run.smithi106.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-25T00:04:41.378 INFO:teuthology.orchestra.run.smithi136.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-04-25T00:04:41.381 INFO:teuthology.orchestra.run.smithi106.stdout:(120/152): python3-cherrypy-18.6.1-2.el9.noarch 2.2 MB/s | 358 kB 00:00 2024-04-25T00:04:41.399 INFO:teuthology.orchestra.run.smithi106.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 421 kB/s | 35 kB 00:00 2024-04-25T00:04:41.423 INFO:teuthology.orchestra.run.smithi106.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T00:04:41.437 INFO:teuthology.orchestra.run.smithi042.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.2 MB/s | 662 kB 00:00 2024-04-25T00:04:41.487 INFO:teuthology.orchestra.run.smithi136.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-25T00:04:41.489 INFO:teuthology.orchestra.run.smithi042.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.4 MB/s | 582 kB 00:00 2024-04-25T00:04:41.499 INFO:teuthology.orchestra.run.smithi106.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 17 MB/s | 2.0 MB 00:00 2024-04-25T00:04:41.516 INFO:teuthology.orchestra.run.smithi106.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-04-25T00:04:41.542 INFO:teuthology.orchestra.run.smithi106.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 1.0 MB/s | 144 kB 00:00 2024-04-25T00:04:41.565 INFO:teuthology.orchestra.run.smithi042.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 201 kB/s | 25 kB 00:00 2024-04-25T00:04:41.584 INFO:teuthology.orchestra.run.smithi106.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 209 kB/s | 18 kB 00:00 2024-04-25T00:04:41.609 INFO:teuthology.orchestra.run.smithi106.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 291 kB/s | 19 kB 00:00 2024-04-25T00:04:41.612 INFO:teuthology.orchestra.run.smithi136.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-04-25T00:04:41.615 INFO:teuthology.orchestra.run.smithi042.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 381 kB/s | 47 kB 00:00 2024-04-25T00:04:41.637 INFO:teuthology.orchestra.run.smithi136.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 514 kB/s | 77 kB 00:00 2024-04-25T00:04:41.642 INFO:teuthology.orchestra.run.smithi106.stdout:(128/152): python3-jaraco-collections-3.0.0-8.e 184 kB/s | 23 kB 00:00 2024-04-25T00:04:41.667 INFO:teuthology.orchestra.run.smithi106.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-04-25T00:04:41.690 INFO:teuthology.orchestra.run.smithi042.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.3 MB/s | 292 kB 00:00 2024-04-25T00:04:41.701 INFO:teuthology.orchestra.run.smithi106.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-25T00:04:41.704 INFO:teuthology.orchestra.run.smithi136.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 461 kB/s | 42 kB 00:00 2024-04-25T00:04:41.738 INFO:teuthology.orchestra.run.smithi136.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-25T00:04:41.741 INFO:teuthology.orchestra.run.smithi042.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-25T00:04:41.743 INFO:teuthology.orchestra.run.smithi106.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 617 kB/s | 46 kB 00:00 2024-04-25T00:04:41.768 INFO:teuthology.orchestra.run.smithi106.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 786 kB/s | 79 kB 00:00 2024-04-25T00:04:41.793 INFO:teuthology.orchestra.run.smithi106.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-04-25T00:04:41.819 INFO:teuthology.orchestra.run.smithi106.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-25T00:04:41.822 INFO:teuthology.orchestra.run.smithi136.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-25T00:04:41.841 INFO:teuthology.orchestra.run.smithi042.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 235 kB/s | 35 kB 00:00 2024-04-25T00:04:41.844 INFO:teuthology.orchestra.run.smithi106.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-04-25T00:04:41.847 INFO:teuthology.orchestra.run.smithi136.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-04-25T00:04:41.869 INFO:teuthology.orchestra.run.smithi106.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-25T00:04:41.894 INFO:teuthology.orchestra.run.smithi106.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-04-25T00:04:41.905 INFO:teuthology.orchestra.run.smithi136.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 612 kB/s | 51 kB 00:00 2024-04-25T00:04:41.920 INFO:teuthology.orchestra.run.smithi106.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 407 kB/s | 31 kB 00:00 2024-04-25T00:04:41.931 INFO:teuthology.orchestra.run.smithi136.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 643 kB/s | 54 kB 00:00 2024-04-25T00:04:41.998 INFO:teuthology.orchestra.run.smithi136.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 624 kB/s | 42 kB 00:00 2024-04-25T00:04:42.028 INFO:teuthology.orchestra.run.smithi106.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2024-04-25T00:04:42.053 INFO:teuthology.orchestra.run.smithi106.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 268 kB/s | 36 kB 00:00 2024-04-25T00:04:42.057 INFO:teuthology.orchestra.run.smithi136.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-04-25T00:04:42.079 INFO:teuthology.orchestra.run.smithi106.stdout:(141/152): python3-routes-2.5.1-5.el9.noarch.rp 901 kB/s | 188 kB 00:00 2024-04-25T00:04:42.104 INFO:teuthology.orchestra.run.smithi106.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 676 kB/s | 51 kB 00:00 2024-04-25T00:04:42.130 INFO:teuthology.orchestra.run.smithi106.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-25T00:04:42.148 INFO:teuthology.orchestra.run.smithi136.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.4 MB/s | 221 kB 00:00 2024-04-25T00:04:42.155 INFO:teuthology.orchestra.run.smithi106.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T00:04:42.189 INFO:teuthology.orchestra.run.smithi106.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-25T00:04:42.214 INFO:teuthology.orchestra.run.smithi106.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-25T00:04:42.239 INFO:teuthology.orchestra.run.smithi106.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-25T00:04:42.331 INFO:teuthology.orchestra.run.smithi106.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T00:04:42.332 INFO:teuthology.orchestra.run.smithi136.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 196 kB/s | 36 kB 00:00 2024-04-25T00:04:42.448 INFO:teuthology.orchestra.run.smithi106.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 12 MB/s | 1.3 MB 00:00 2024-04-25T00:04:42.474 INFO:teuthology.orchestra.run.smithi136.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 335 kB/s | 47 kB 00:00 2024-04-25T00:04:42.598 INFO:teuthology.orchestra.run.smithi106.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 16 MB/s | 2.4 MB 00:00 2024-04-25T00:04:42.791 INFO:teuthology.orchestra.run.smithi136.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-25T00:04:42.841 INFO:teuthology.orchestra.run.smithi136.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 831 kB/s | 305 kB 00:00 2024-04-25T00:04:42.916 INFO:teuthology.orchestra.run.smithi136.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-04-25T00:04:42.942 INFO:teuthology.orchestra.run.smithi136.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 384 kB/s | 58 kB 00:00 2024-04-25T00:04:43.009 INFO:teuthology.orchestra.run.smithi136.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 672 kB/s | 45 kB 00:00 2024-04-25T00:04:43.032 INFO:teuthology.orchestra.run.smithi106.stdout:(151/152): librados2-19.0.0-2445.gd2c45d97.el9. 4.1 MB/s | 3.4 MB 00:00 2024-04-25T00:04:43.042 INFO:teuthology.orchestra.run.smithi136.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 506 kB/s | 64 kB 00:00 2024-04-25T00:04:43.123 INFO:teuthology.orchestra.run.smithi106.stdout:(152/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 3.5 MB/s | 3.1 MB 00:00 2024-04-25T00:04:43.126 INFO:teuthology.orchestra.run.smithi136.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 191 kB/s | 22 kB 00:00 2024-04-25T00:04:43.126 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:04:43.127 INFO:teuthology.orchestra.run.smithi106.stdout:Total 12 MB/s | 220 MB 00:17 2024-04-25T00:04:43.317 INFO:teuthology.orchestra.run.smithi042.stdout:(66/152): ceph-test-19.0.0-2445.gd2c45d97.el9.x 6.2 MB/s | 48 MB 00:07 2024-04-25T00:04:43.393 INFO:teuthology.orchestra.run.smithi136.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.0 MB/s | 548 kB 00:00 2024-04-25T00:04:43.468 INFO:teuthology.orchestra.run.smithi136.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-04-25T00:04:43.519 INFO:teuthology.orchestra.run.smithi136.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 388 kB/s | 19 kB 00:00 2024-04-25T00:04:43.552 INFO:teuthology.orchestra.run.smithi136.stdout:(106/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.7 MB/s | 863 kB 00:00 2024-04-25T00:04:43.703 INFO:teuthology.orchestra.run.smithi136.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-04-25T00:04:43.728 INFO:teuthology.orchestra.run.smithi136.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 148 kB/s | 26 kB 00:00 2024-04-25T00:04:43.753 INFO:teuthology.orchestra.run.smithi136.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 966 kB/s | 49 kB 00:00 2024-04-25T00:04:43.813 INFO:teuthology.orchestra.run.smithi136.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-04-25T00:04:43.829 INFO:teuthology.orchestra.run.smithi136.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 669 kB/s | 67 kB 00:00 2024-04-25T00:04:43.921 INFO:teuthology.orchestra.run.smithi136.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 7.6 MB/s | 838 kB 00:00 2024-04-25T00:04:43.980 INFO:teuthology.orchestra.run.smithi136.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 746 kB/s | 43 kB 00:00 2024-04-25T00:04:44.013 INFO:teuthology.orchestra.run.smithi136.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 2.9 MB/s | 548 kB 00:00 2024-04-25T00:04:44.030 INFO:teuthology.orchestra.run.smithi136.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 643 kB/s | 32 kB 00:00 2024-04-25T00:04:44.072 INFO:teuthology.orchestra.run.smithi136.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-04-25T00:04:44.097 INFO:teuthology.orchestra.run.smithi136.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-04-25T00:04:44.131 INFO:teuthology.orchestra.run.smithi136.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-04-25T00:04:44.156 INFO:teuthology.orchestra.run.smithi136.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-04-25T00:04:44.189 INFO:teuthology.orchestra.run.smithi136.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 605 kB/s | 35 kB 00:00 2024-04-25T00:04:44.215 INFO:teuthology.orchestra.run.smithi136.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.4 MB/s | 206 kB 00:00 2024-04-25T00:04:44.225 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2024-04-25T00:04:44.265 INFO:teuthology.orchestra.run.smithi136.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-25T00:04:44.340 INFO:teuthology.orchestra.run.smithi136.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-25T00:04:44.366 INFO:teuthology.orchestra.run.smithi136.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 107 kB/s | 11 kB 00:00 2024-04-25T00:04:44.368 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2024-04-25T00:04:44.368 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2024-04-25T00:04:44.391 INFO:teuthology.orchestra.run.smithi136.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 351 kB/s | 18 kB 00:00 2024-04-25T00:04:44.424 INFO:teuthology.orchestra.run.smithi136.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 395 kB/s | 23 kB 00:00 2024-04-25T00:04:44.434 INFO:teuthology.orchestra.run.smithi042.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-04-25T00:04:44.441 INFO:teuthology.orchestra.run.smithi136.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-04-25T00:04:44.475 INFO:teuthology.orchestra.run.smithi136.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-04-25T00:04:44.517 INFO:teuthology.orchestra.run.smithi136.stdout:(129/152): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-25T00:04:44.534 INFO:teuthology.orchestra.run.smithi136.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 793 kB/s | 46 kB 00:00 2024-04-25T00:04:44.550 INFO:teuthology.orchestra.run.smithi042.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 1.7 MB/s | 4.7 MB 00:02 2024-04-25T00:04:44.568 INFO:teuthology.orchestra.run.smithi136.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T00:04:44.593 INFO:teuthology.orchestra.run.smithi136.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-04-25T00:04:44.635 INFO:teuthology.orchestra.run.smithi136.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-25T00:04:44.661 INFO:teuthology.orchestra.run.smithi136.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-04-25T00:04:44.676 INFO:teuthology.orchestra.run.smithi042.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 1.0 MB/s | 133 kB 00:00 2024-04-25T00:04:44.686 INFO:teuthology.orchestra.run.smithi136.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-25T00:04:44.718 INFO:teuthology.orchestra.run.smithi042.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 3.1 MB/s | 911 kB 00:00 2024-04-25T00:04:44.719 INFO:teuthology.orchestra.run.smithi136.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 482 kB/s | 28 kB 00:00 2024-04-25T00:04:44.744 INFO:teuthology.orchestra.run.smithi136.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 525 kB/s | 31 kB 00:00 2024-04-25T00:04:44.778 INFO:teuthology.orchestra.run.smithi136.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.1 MB/s | 188 kB 00:00 2024-04-25T00:04:44.803 INFO:teuthology.orchestra.run.smithi136.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T00:04:44.837 INFO:teuthology.orchestra.run.smithi136.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 613 kB/s | 36 kB 00:00 2024-04-25T00:04:44.863 INFO:teuthology.orchestra.run.smithi136.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 869 kB/s | 51 kB 00:00 2024-04-25T00:04:44.896 INFO:teuthology.orchestra.run.smithi136.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-04-25T00:04:44.921 INFO:teuthology.orchestra.run.smithi136.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-25T00:04:44.926 INFO:teuthology.orchestra.run.smithi042.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.9 MB/s | 1.0 MB 00:00 2024-04-25T00:04:44.963 INFO:teuthology.orchestra.run.smithi136.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-04-25T00:04:44.988 INFO:teuthology.orchestra.run.smithi136.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 299 kB/s | 20 kB 00:00 2024-04-25T00:04:45.030 INFO:teuthology.orchestra.run.smithi136.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-25T00:04:45.035 INFO:teuthology.orchestra.run.smithi042.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 816 kB/s | 87 kB 00:00 2024-04-25T00:04:45.081 INFO:teuthology.orchestra.run.smithi136.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-25T00:04:45.391 INFO:teuthology.orchestra.run.smithi042.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 8.1 MB/s | 5.8 MB 00:00 2024-04-25T00:04:45.439 INFO:teuthology.orchestra.run.smithi136.stdout:(148/152): librados2-19.0.0-2445.gd2c45d97.el9. 8.3 MB/s | 3.4 MB 00:00 2024-04-25T00:04:45.511 INFO:teuthology.orchestra.run.smithi042.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-04-25T00:04:45.515 INFO:teuthology.orchestra.run.smithi136.stdout:(149/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 7.2 MB/s | 3.1 MB 00:00 2024-04-25T00:04:45.607 INFO:teuthology.orchestra.run.smithi136.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 8.1 MB/s | 1.3 MB 00:00 2024-04-25T00:04:45.619 INFO:teuthology.orchestra.run.smithi042.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.3 MB/s | 248 kB 00:00 2024-04-25T00:04:45.715 INFO:teuthology.orchestra.run.smithi136.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-04-25T00:04:45.745 INFO:teuthology.orchestra.run.smithi042.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 205 kB/s | 26 kB 00:00 2024-04-25T00:04:45.878 INFO:teuthology.orchestra.run.smithi042.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-04-25T00:04:46.062 INFO:teuthology.orchestra.run.smithi042.stdout:(78/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 1.0 MB/s | 4.4 MB 00:04 2024-04-25T00:04:46.096 INFO:teuthology.orchestra.run.smithi042.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 797 kB/s | 172 kB 00:00 2024-04-25T00:04:46.125 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2024-04-25T00:04:46.126 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2024-04-25T00:04:46.213 INFO:teuthology.orchestra.run.smithi042.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 231 kB/s | 35 kB 00:00 2024-04-25T00:04:46.597 INFO:teuthology.orchestra.run.smithi042.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 655 kB/s | 250 kB 00:00 2024-04-25T00:04:46.738 INFO:teuthology.orchestra.run.smithi042.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.9 MB/s | 5.1 MB 00:00 2024-04-25T00:04:46.847 INFO:teuthology.orchestra.run.smithi042.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 720 kB/s | 77 kB 00:00 2024-04-25T00:04:47.006 INFO:teuthology.orchestra.run.smithi042.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 267 kB/s | 42 kB 00:00 2024-04-25T00:04:47.031 INFO:teuthology.orchestra.run.smithi042.stdout:(85/152): python3-oauthlib-3.1.1-5.el9.noarch.r 512 kB/s | 222 kB 00:00 2024-04-25T00:04:47.123 INFO:teuthology.orchestra.run.smithi042.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-04-25T00:04:47.240 INFO:teuthology.orchestra.run.smithi042.stdout:(87/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-25T00:04:47.471 INFO:teuthology.orchestra.run.smithi042.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 362 kB/s | 159 kB 00:00 2024-04-25T00:04:47.516 INFO:teuthology.orchestra.run.smithi042.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 185 kB/s | 51 kB 00:00 2024-04-25T00:04:47.557 INFO:teuthology.orchestra.run.smithi136.stdout:(152/152): ceph-test-19.0.0-2445.gd2c45d97.el9. 3.6 MB/s | 48 MB 00:13 2024-04-25T00:04:47.561 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:04:47.561 INFO:teuthology.orchestra.run.smithi136.stdout:Total 11 MB/s | 220 MB 00:20 2024-04-25T00:04:47.591 INFO:teuthology.orchestra.run.smithi042.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 450 kB/s | 54 kB 00:00 2024-04-25T00:04:47.717 INFO:teuthology.orchestra.run.smithi042.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 334 kB/s | 42 kB 00:00 2024-04-25T00:04:47.875 INFO:teuthology.orchestra.run.smithi042.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 280 kB/s | 44 kB 00:00 2024-04-25T00:04:48.285 INFO:teuthology.orchestra.run.smithi042.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 540 kB/s | 221 kB 00:00 2024-04-25T00:04:48.434 INFO:teuthology.orchestra.run.smithi042.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 242 kB/s | 36 kB 00:00 2024-04-25T00:04:48.593 INFO:teuthology.orchestra.run.smithi042.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 301 kB/s | 47 kB 00:00 2024-04-25T00:04:48.654 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-04-25T00:04:48.796 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-04-25T00:04:48.796 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-04-25T00:04:49.051 INFO:teuthology.orchestra.run.smithi042.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 666 kB/s | 305 kB 00:00 2024-04-25T00:04:49.318 INFO:teuthology.orchestra.run.smithi042.stdout:(97/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 9.0 MB/s | 16 MB 00:01 2024-04-25T00:04:49.360 INFO:teuthology.orchestra.run.smithi042.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 188 kB/s | 58 kB 00:00 2024-04-25T00:04:49.428 INFO:teuthology.orchestra.run.smithi042.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 871 kB/s | 94 kB 00:00 2024-04-25T00:04:49.553 INFO:teuthology.orchestra.run.smithi042.stdout:(100/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 359 kB/s | 45 kB 00:00 2024-04-25T00:04:49.595 INFO:teuthology.orchestra.run.smithi042.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 273 kB/s | 64 kB 00:00 2024-04-25T00:04:49.678 INFO:teuthology.orchestra.run.smithi042.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 179 kB/s | 22 kB 00:00 2024-04-25T00:04:49.737 INFO:teuthology.orchestra.run.smithi042.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-04-25T00:04:49.929 INFO:teuthology.orchestra.run.smithi042.stdout:(104/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-04-25T00:04:50.046 INFO:teuthology.orchestra.run.smithi042.stdout:(105/152): python3-babel-2.9.1-2.el9.noarch.rpm 1.2 MB/s | 6.0 MB 00:05 2024-04-25T00:04:50.088 INFO:teuthology.orchestra.run.smithi042.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 123 kB/s | 19 kB 00:00 2024-04-25T00:04:50.121 INFO:teuthology.orchestra.run.smithi042.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-25T00:04:50.164 INFO:teuthology.orchestra.run.smithi042.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 349 kB/s | 26 kB 00:00 2024-04-25T00:04:50.214 INFO:teuthology.orchestra.run.smithi042.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 533 kB/s | 49 kB 00:00 2024-04-25T00:04:50.239 INFO:teuthology.orchestra.run.smithi042.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 906 kB/s | 67 kB 00:00 2024-04-25T00:04:50.297 INFO:teuthology.orchestra.run.smithi042.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-25T00:04:50.415 INFO:teuthology.orchestra.run.smithi042.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 4.7 MB/s | 838 kB 00:00 2024-04-25T00:04:50.459 INFO:teuthology.orchestra.run.smithi042.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 3.3 MB/s | 548 kB 00:00 2024-04-25T00:04:50.501 INFO:teuthology.orchestra.run.smithi042.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 512 kB/s | 43 kB 00:00 2024-04-25T00:04:50.543 INFO:teuthology.orchestra.run.smithi042.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 389 kB/s | 32 kB 00:00 2024-04-25T00:04:50.554 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-04-25T00:04:50.554 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-04-25T00:04:50.585 INFO:teuthology.orchestra.run.smithi042.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 181 kB/s | 15 kB 00:00 2024-04-25T00:04:50.618 INFO:teuthology.orchestra.run.smithi042.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.3 MB/s | 172 kB 00:00 2024-04-25T00:04:50.676 INFO:teuthology.orchestra.run.smithi042.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 3.9 MB/s | 358 kB 00:00 2024-04-25T00:04:50.752 INFO:teuthology.orchestra.run.smithi042.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 472 kB/s | 35 kB 00:00 2024-04-25T00:04:50.786 INFO:teuthology.orchestra.run.smithi042.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2024-04-25T00:04:50.819 INFO:teuthology.orchestra.run.smithi042.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 3.0 MB/s | 206 kB 00:00 2024-04-25T00:04:50.878 INFO:teuthology.orchestra.run.smithi042.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-04-25T00:04:50.945 INFO:teuthology.orchestra.run.smithi042.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 162 kB/s | 11 kB 00:00 2024-04-25T00:04:51.012 INFO:teuthology.orchestra.run.smithi042.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 268 kB/s | 18 kB 00:00 2024-04-25T00:04:51.096 INFO:teuthology.orchestra.run.smithi042.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 280 kB/s | 23 kB 00:00 2024-04-25T00:04:51.163 INFO:teuthology.orchestra.run.smithi042.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 295 kB/s | 19 kB 00:00 2024-04-25T00:04:51.230 INFO:teuthology.orchestra.run.smithi042.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 298 kB/s | 20 kB 00:00 2024-04-25T00:04:51.380 INFO:teuthology.orchestra.run.smithi042.stdout:(128/152): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-04-25T00:04:51.447 INFO:teuthology.orchestra.run.smithi042.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 703 kB/s | 46 kB 00:00 2024-04-25T00:04:51.505 INFO:teuthology.orchestra.run.smithi042.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T00:04:51.556 INFO:teuthology.orchestra.run.smithi042.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 3.1 MB/s | 1.0 MB 00:00 2024-04-25T00:04:51.606 INFO:teuthology.orchestra.run.smithi042.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 579 kB/s | 58 kB 00:00 2024-04-25T00:04:51.665 INFO:teuthology.orchestra.run.smithi042.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 284 kB/s | 16 kB 00:00 2024-04-25T00:04:51.715 INFO:teuthology.orchestra.run.smithi042.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-04-25T00:04:51.749 INFO:teuthology.orchestra.run.smithi042.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-04-25T00:04:51.774 INFO:teuthology.orchestra.run.smithi042.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 488 kB/s | 28 kB 00:00 2024-04-25T00:04:51.799 INFO:teuthology.orchestra.run.smithi042.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 620 kB/s | 31 kB 00:00 2024-04-25T00:04:51.841 INFO:teuthology.orchestra.run.smithi042.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-04-25T00:04:51.892 INFO:teuthology.orchestra.run.smithi042.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-04-25T00:04:51.917 INFO:teuthology.orchestra.run.smithi042.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 477 kB/s | 36 kB 00:00 2024-04-25T00:04:51.942 INFO:teuthology.orchestra.run.smithi042.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T00:04:52.001 INFO:teuthology.orchestra.run.smithi042.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-25T00:04:52.026 INFO:teuthology.orchestra.run.smithi042.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-25T00:04:52.127 INFO:teuthology.orchestra.run.smithi042.stdout:(144/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.1 MB/s | 4.4 MB 00:02 2024-04-25T00:04:52.152 INFO:teuthology.orchestra.run.smithi042.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-04-25T00:04:52.194 INFO:teuthology.orchestra.run.smithi042.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.2 MB/s | 427 kB 00:00 2024-04-25T00:04:52.244 INFO:teuthology.orchestra.run.smithi042.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-04-25T00:04:52.272 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2024-04-25T00:04:52.378 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:04:52.424 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:04:52.477 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:04:52.512 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:04:52.537 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:04:52.578 INFO:teuthology.orchestra.run.smithi042.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-04-25T00:04:52.579 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:04:52.712 INFO:teuthology.orchestra.run.smithi042.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-04-25T00:04:52.887 INFO:teuthology.orchestra.run.smithi042.stdout:(150/152): librbd1-19.0.0-2445.gd2c45d97.el9.x8 4.9 MB/s | 3.1 MB 00:00 2024-04-25T00:04:52.914 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:04:52.945 INFO:teuthology.orchestra.run.smithi042.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 10 MB/s | 2.4 MB 00:00 2024-04-25T00:04:53.022 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:53.129 INFO:teuthology.orchestra.run.smithi042.stdout:(152/152): librados2-19.0.0-2445.gd2c45d97.el9. 3.6 MB/s | 3.4 MB 00:00 2024-04-25T00:04:53.135 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:04:53.136 INFO:teuthology.orchestra.run.smithi042.stdout:Total 8.5 MB/s | 220 MB 00:25 2024-04-25T00:04:53.144 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:53.173 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:53.264 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:53.293 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:04:53.446 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:04:53.507 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:04:53.535 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:53.622 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:53.645 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:04:53.707 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:04:53.737 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:04:53.755 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:04:53.946 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:04:53.978 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:04:54.086 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:04:54.184 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:04:54.220 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:04:54.255 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:04:54.329 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:04:54.371 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:04:54.415 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:04:54.447 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:04:54.475 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:04:54.537 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:04:54.589 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:04:54.620 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:04:54.663 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:04:54.741 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:04:54.774 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:04:54.817 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:04:54.846 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:04:54.916 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:04:54.969 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:04:55.044 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:04:55.179 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:04:55.230 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:04:55.789 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:04:56.031 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction check 2024-04-25T00:04:56.037 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:04:56.073 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:04:56.114 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:04:56.152 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:04:56.194 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:04:56.282 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction check succeeded. 2024-04-25T00:04:56.283 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction test 2024-04-25T00:04:56.348 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:04:56.414 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:04:56.469 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:04:56.525 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-04-25T00:04:56.541 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:04:56.572 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:04:56.609 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:04:56.626 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:04:56.655 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:04:56.665 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:04:56.691 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:04:56.734 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:04:56.777 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:04:56.810 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:04:56.913 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:04:56.957 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:04:56.961 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:04:56.986 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:04:56.987 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:04:57.029 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:04:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:04:57.124 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:04:57.154 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:04:57.194 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:04:57.228 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:04:57.252 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:04:57.327 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:04:57.361 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:04:57.455 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:04:57.479 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:57.485 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:04:57.513 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:04:57.547 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:04:57.586 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:04:57.600 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:04:57.617 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:04:57.630 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:57.648 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:04:57.682 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:04:57.716 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:04:57.722 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:04:57.751 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:04:57.757 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:04:57.791 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:04:57.896 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:04:57.898 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:04:57.939 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:04:57.948 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:04:57.976 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:57.978 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:04:58.026 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:04:58.087 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:04:58.125 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:04:58.127 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:04:58.172 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:04:58.173 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:04:58.195 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:04:58.218 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:04:58.322 INFO:teuthology.orchestra.run.smithi042.stdout:Transaction test succeeded. 2024-04-25T00:04:58.325 INFO:teuthology.orchestra.run.smithi042.stdout:Running transaction 2024-04-25T00:04:58.370 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:04:58.403 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:04:58.511 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:04:58.605 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:04:58.644 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:04:58.678 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:04:58.742 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:04:58.778 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:04:58.830 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:04:58.863 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:04:58.891 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:04:58.902 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:04:58.952 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:04:58.960 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:04:58.985 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:04:59.013 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:04:59.086 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:04:59.098 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:04:59.145 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:04:59.232 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:04:59.271 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:04:59.346 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:04:59.422 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:04:59.449 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:04:59.457 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:04:59.477 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:04:59.500 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:04:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:04:59.554 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:04:59.598 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:04:59.644 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:04:59.718 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:04:59.868 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:04:59.921 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:05:00.198 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:05:00.308 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:05:00.438 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:05:00.524 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:05:00.562 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:05:00.597 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:05:00.634 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:05:00.676 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:05:00.828 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:05:00.888 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:05:00.942 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:05:01.015 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:05:01.046 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:05:01.075 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:05:01.121 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:05:01.158 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:05:01.201 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:05:01.243 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:05:01.378 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:05:01.423 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:05:01.454 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:05:01.495 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:05:01.587 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:05:01.588 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:05:01.629 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:05:01.664 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:05:01.669 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:05:01.692 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:05:01.694 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:05:01.719 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:05:01.725 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:05:01.792 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:05:01.912 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:05:01.951 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:05:01.988 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:05:02.022 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:05:02.027 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:05:02.061 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:05:02.072 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:05:02.093 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:05:02.149 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:05:02.163 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:05:02.182 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:05:02.193 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:05:02.216 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:05:02.229 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:05:02.257 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:05:02.292 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:05:02.379 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:05:02.422 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:05:02.462 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:05:02.509 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:05:02.607 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:05:02.672 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:05:02.700 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:05:02.725 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:05:02.788 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:05:02.818 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:05:03.384 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:05:03.435 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:05:03.469 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:05:03.557 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:05:03.610 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:05:03.699 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:05:03.730 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:05:03.906 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:05:03.936 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:05:04.013 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:05:04.643 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:05:04.759 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:05:04.853 INFO:teuthology.orchestra.run.smithi042.stdout: Preparing : 1/1 2024-04-25T00:05:05.047 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T00:05:05.080 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:05:05.116 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T00:05:05.123 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:05:05.206 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T00:05:05.267 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T00:05:05.304 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T00:05:05.419 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T00:05:05.671 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:05:05.718 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:05:05.835 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:05:05.835 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:05:05.835 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:05:05.835 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:05.843 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T00:05:05.861 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:05:05.944 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:05:05.968 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:05:06.058 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:05:06.085 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:05:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:05:06.121 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:05:06.179 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:05:06.296 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:05:06.328 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:05:06.455 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:05:06.498 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:05:06.559 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T00:05:06.589 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:05:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:05:06.624 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T00:05:06.650 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:05:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:05:06.789 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:05:06.818 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:05:06.903 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 14/156 2024-04-25T00:05:06.952 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:05:07.018 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T00:05:07.130 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:05:07.160 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:05:07.231 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:05:07.253 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:05:07.322 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:05:07.386 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T00:05:07.592 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T00:05:07.723 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T00:05:07.769 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T00:05:07.828 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T00:05:07.922 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T00:05:07.981 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T00:05:08.041 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T00:05:08.085 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T00:05:08.142 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T00:05:08.265 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T00:05:08.350 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T00:05:08.386 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T00:05:08.441 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T00:05:08.593 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T00:05:08.641 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 31/156 2024-04-25T00:05:08.691 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 32/156 2024-04-25T00:05:08.723 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 33/156 2024-04-25T00:05:08.843 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T00:05:08.929 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T00:05:09.041 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T00:05:09.224 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T00:05:09.296 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T00:05:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:05:09.525 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:05:10.024 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T00:05:10.075 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:05:10.114 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:05:10.152 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T00:05:10.196 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T00:05:10.204 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:05:10.204 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:05:10.204 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:05:10.204 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:10.241 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:05:10.242 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T00:05:10.298 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 43/156 2024-04-25T00:05:10.361 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T00:05:10.604 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T00:05:10.674 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T00:05:10.757 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T00:05:10.860 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T00:05:10.916 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T00:05:10.957 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T00:05:11.024 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T00:05:11.069 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T00:05:11.142 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T00:05:11.192 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T00:05:11.391 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T00:05:11.453 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T00:05:11.489 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T00:05:11.549 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T00:05:11.701 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T00:05:11.748 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T00:05:11.795 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T00:05:11.829 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T00:05:11.863 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T00:05:12.018 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T00:05:12.235 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T00:05:12.278 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T00:05:12.317 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T00:05:12.372 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T00:05:12.436 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T00:05:12.501 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T00:05:12.550 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T00:05:12.604 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T00:05:12.653 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T00:05:12.732 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T00:05:12.785 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T00:05:12.965 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T00:05:13.036 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T00:05:13.074 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T00:05:13.139 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T00:05:13.278 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T00:05:13.356 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T00:05:14.334 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T00:05:14.388 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T00:05:14.446 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T00:05:14.579 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T00:05:14.651 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T00:05:14.772 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T00:05:14.839 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T00:05:15.099 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T00:05:15.136 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T00:05:15.234 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T00:05:16.084 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T00:05:16.229 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T00:05:17.199 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:05:17.199 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /sys 2024-04-25T00:05:17.199 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /proc 2024-04-25T00:05:17.199 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /mnt 2024-04-25T00:05:17.199 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /var/tmp 2024-04-25T00:05:17.199 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /home 2024-04-25T00:05:17.199 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /root 2024-04-25T00:05:17.199 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /tmp 2024-04-25T00:05:17.199 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:17.365 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:05:17.380 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:05:17.381 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:17.381 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:05:17.381 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:17.860 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T00:05:17.986 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T00:05:18.031 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T00:05:18.067 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T00:05:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T00:05:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:05:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 99/156 2024-04-25T00:05:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 100/156 2024-04-25T00:05:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T00:05:18.852 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:05:18.877 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:05:18.877 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:18.877 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:05:18.877 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:05:18.877 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:05:18.877 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:19.102 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:05:19.125 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:05:19.125 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:19.125 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:05:19.125 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:05:19.125 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:05:19.125 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T00:05:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:05:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 103/156 2024-04-25T00:05:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 104/156 2024-04-25T00:05:19.570 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:05:19.592 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:05:19.592 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:19.592 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:05:19.592 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:05:19.592 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:05:19.592 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:19.700 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:05:19.758 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:05:19.813 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:05:20.000 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:05:20.046 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:05:20.116 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:05:20.205 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:05:20.346 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:05:20.387 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:05:20.419 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:05:20.447 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:05:20.795 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:05:20.820 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:05:20.852 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:05:20.923 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:05:20.960 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:05:21.996 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:05:22.046 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:05:22.062 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:05:22.062 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-04-25T00:05:22.062 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-04-25T00:05:22.062 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-04-25T00:05:22.062 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-04-25T00:05:22.062 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-04-25T00:05:22.063 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-04-25T00:05:22.063 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-04-25T00:05:22.063 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:22.238 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:05:22.254 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:05:22.254 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:22.254 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:05:22.254 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:22.517 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:05:22.566 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:05:23.013 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:05:23.046 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:05:23.149 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:05:23.234 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 105/156 2024-04-25T00:05:23.287 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:05:23.310 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:05:23.340 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:05:23.363 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:05:23.363 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:23.363 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:05:23.363 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:05:23.363 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:05:23.363 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:23.407 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:05:23.418 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:05:23.456 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 106/156 2024-04-25T00:05:23.456 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T00:05:23.456 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T00:05:23.456 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:23.496 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:05:23.533 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:05:23.581 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:05:23.750 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:05:23.775 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:05:23.775 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:23.775 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:05:23.775 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:05:23.775 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:05:23.776 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:24.000 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:05:24.022 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:05:24.022 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:24.022 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:05:24.023 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:05:24.023 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:05:24.023 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:05:24.477 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:05:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:05:24.490 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:24.490 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:05:24.490 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:05:24.490 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:05:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:24.509 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:05:24.531 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:05:24.532 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:24.532 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:05:24.532 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:05:24.532 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:05:24.532 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:24.568 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:05:24.590 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:05:24.590 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:24.590 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:05:24.590 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:24.597 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:05:24.635 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:05:24.645 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:05:24.658 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:05:24.658 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:24.659 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:05:24.659 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:05:24.659 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:05:24.659 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:24.701 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:05:24.887 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:05:24.943 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:05:24.961 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:05:24.984 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:05:24.984 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:24.984 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:05:24.985 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:05:24.985 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:05:24.985 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:25.022 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:05:25.110 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:05:25.249 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:05:25.293 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:05:25.325 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:05:25.354 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:05:25.719 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:05:25.744 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:05:25.775 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:05:25.836 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:05:25.875 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:05:26.912 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:05:26.960 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:05:27.927 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:05:27.952 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:05:28.053 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:05:28.215 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:05:28.246 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:05:28.269 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:05:28.269 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:28.270 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:05:28.270 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:05:28.270 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:05:28.270 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:28.314 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:05:28.325 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:05:28.422 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:05:28.470 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:05:29.377 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:05:29.416 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:05:29.439 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:05:29.439 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:29.439 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:05:29.439 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:05:29.440 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:05:29.440 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:29.484 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:05:29.490 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:05:29.505 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:05:29.505 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:29.505 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:05:29.505 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:29.531 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:05:29.551 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:05:29.573 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:05:29.573 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:05:29.573 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:29.573 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:05:29.573 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:05:29.573 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:05:29.573 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:29.604 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:05:29.631 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:05:29.821 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:05:29.861 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:05:29.883 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:05:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:05:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:05:29.907 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:29.907 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:05:29.907 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:05:29.907 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:05:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:30.460 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:05:30.497 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:05:30.517 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:05:31.348 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:05:31.437 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:05:31.479 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:05:31.561 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:05:31.591 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:05:31.591 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:05:31.667 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:05:31.700 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:05:31.700 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:05:31.785 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:05:31.856 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:34.439 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:05:34.480 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:05:34.521 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:05:34.561 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:05:34.589 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:05:34.773 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:05:34.810 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:05:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:05:35.462 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:05:35.498 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:05:35.518 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:05:36.347 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:05:36.428 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:05:36.468 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:05:36.551 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:05:36.581 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:05:36.581 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:05:36.665 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:05:36.698 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:05:36.699 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:05:36.780 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:05:36.846 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:37.335 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 107/156 2024-04-25T00:05:37.335 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /sys 2024-04-25T00:05:37.335 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /proc 2024-04-25T00:05:37.335 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /mnt 2024-04-25T00:05:37.335 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /var/tmp 2024-04-25T00:05:37.335 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /home 2024-04-25T00:05:37.335 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /root 2024-04-25T00:05:37.335 INFO:teuthology.orchestra.run.smithi042.stdout:skipping the directory /tmp 2024-04-25T00:05:37.335 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:37.566 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:37.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:05:37.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:05:37.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:05:37.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:05:37.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:05:37.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:05:37.566 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:05:37.567 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:05:37.568 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:05:37.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:05:37.570 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:05:37.571 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:05:37.572 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:05:37.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:05:37.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:05:37.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:05:37.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:05:37.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:05:37.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:05:37.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:05:37.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:05:37.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:05:37.573 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:05:37.574 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:05:37.575 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:05:37.576 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:05:37.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:05:37.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:05:37.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:05:37.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:05:37.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:05:37.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:05:37.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:05:37.577 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:05:37.586 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:05:37.632 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 108/156 2024-04-25T00:05:37.632 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:37.632 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T00:05:37.632 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:39.357 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:05:39.441 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 109/156 2024-04-25T00:05:39.442 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:39.442 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T00:05:39.442 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:05:39.442 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T00:05:39.442 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:39.656 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:39.656 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:39.656 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:39.657 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.658 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:05:39.659 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:05:39.660 INFO:teuthology.orchestra.run.smithi106.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:05:39.661 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:05:39.662 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.663 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:05:39.664 INFO:teuthology.orchestra.run.smithi106.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:05:39.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:05:39.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:05:39.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:05:39.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:05:39.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:05:39.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:05:39.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:05:39.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:05:39.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:05:39.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:05:39.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.667 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:05:39.668 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:05:39.669 INFO:teuthology.orchestra.run.smithi106.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:05:39.670 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:05:39.670 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:05:39.670 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:05:39.670 INFO:teuthology.orchestra.run.smithi106.stdout: 2024-04-25T00:05:39.670 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2024-04-25T00:05:39.706 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:05:39.779 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 110/156 2024-04-25T00:05:39.779 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:39.779 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T00:05:39.779 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:05:39.780 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T00:05:39.780 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:40.072 DEBUG:teuthology.parallel:result is None 2024-04-25T00:05:40.291 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:05:40.368 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 111/156 2024-04-25T00:05:40.368 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:40.369 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T00:05:40.369 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:05:40.369 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T00:05:40.369 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:40.525 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T00:05:40.591 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T00:05:40.675 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T00:05:40.930 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T00:05:41.013 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T00:05:41.118 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T00:05:41.228 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:05:41.397 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:05:41.446 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T00:05:41.486 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T00:05:41.529 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T00:05:42.182 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:05:42.223 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 123/156 2024-04-25T00:05:42.277 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 124/156 2024-04-25T00:05:42.396 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 125/156 2024-04-25T00:05:42.425 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:42.425 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:05:42.425 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:05:42.425 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:05:42.425 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:05:42.425 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:05:42.425 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:05:42.426 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:05:42.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:05:42.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:05:42.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:05:42.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:05:42.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:05:42.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:05:42.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:05:42.427 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:05:42.428 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:05:42.428 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:05:42.428 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:05:42.428 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:05:42.428 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:05:42.434 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:05:42.442 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:05:42.442 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:05:42.442 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:05:42.443 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:05:42.444 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:05:42.445 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:05:42.445 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:05:42.445 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:05:42.445 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:05:42.445 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:05:42.445 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:05:42.445 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:05:42.446 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:05:42.447 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:05:42.448 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:05:42.449 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:05:42.450 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:05:42.450 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:05:42.450 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:05:42.450 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:05:42.450 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:05:42.450 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:05:42.450 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:05:42.450 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:05:42.451 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:05:42.452 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:05:42.453 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:05:42.453 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:05:42.453 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:05:42.453 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:05:43.799 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 126/156 2024-04-25T00:05:43.851 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:05:44.608 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:44.608 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:44.608 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.609 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.610 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:05:44.611 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:05:44.612 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:05:44.613 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:05:44.614 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:05:44.615 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:05:44.616 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:05:44.617 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:05:44.618 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:05:44.619 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:05:44.620 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:05:44.620 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:05:44.620 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-04-25T00:05:45.012 DEBUG:teuthology.parallel:result is None 2024-04-25T00:05:45.021 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 127/156 2024-04-25T00:05:45.052 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:05:45.206 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 128/156 2024-04-25T00:05:45.392 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 129/156 2024-04-25T00:05:45.429 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:05:45.514 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 130/156 2024-04-25T00:05:45.514 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:45.514 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T00:05:45.514 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:05:45.514 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T00:05:45.514 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:45.571 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:05:45.612 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 131/156 2024-04-25T00:05:45.757 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 132/156 2024-04-25T00:05:45.814 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 133/156 2024-04-25T00:05:46.865 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T00:05:46.933 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:05:47.019 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 135/156 2024-04-25T00:05:47.019 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:47.020 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T00:05:47.020 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:05:47.020 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T00:05:47.020 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:47.075 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:05:47.153 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 136/156 2024-04-25T00:05:47.153 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:47.153 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T00:05:47.153 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:47.234 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:05:47.316 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 137/156 2024-04-25T00:05:47.316 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:47.316 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T00:05:47.316 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:05:47.316 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T00:05:47.316 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:47.726 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:05:47.801 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 138/156 2024-04-25T00:05:47.801 INFO:teuthology.orchestra.run.smithi042.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T00:05:47.801 INFO:teuthology.orchestra.run.smithi042.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T00:05:47.801 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:05:47.801 INFO:teuthology.orchestra.run.smithi042.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T00:05:47.801 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:05:53.000 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 139/156 2024-04-25T00:05:53.050 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 140/156 2024-04-25T00:05:53.097 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 141/156 2024-04-25T00:05:53.131 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T00:05:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T00:05:53.467 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 144/156 2024-04-25T00:05:53.521 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T00:05:53.549 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 146/156 2024-04-25T00:05:54.507 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:05:54.556 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:05:54.615 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T00:05:56.014 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 148/156 2024-04-25T00:05:56.122 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T00:05:56.182 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T00:05:56.304 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T00:05:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T00:05:56.352 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:05:56.478 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T00:05:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:05:56.518 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:05:56.650 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T00:05:56.788 INFO:teuthology.orchestra.run.smithi042.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:57.083 INFO:teuthology.orchestra.run.smithi042.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:05:57.084 INFO:teuthology.orchestra.run.smithi042.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T00:05:57.084 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:06:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/156 2024-04-25T00:06:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/156 2024-04-25T00:06:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/156 2024-04-25T00:06:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/156 2024-04-25T00:06:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45 5/156 2024-04-25T00:06:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 6/156 2024-04-25T00:06:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/156 2024-04-25T00:06:02.651 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/156 2024-04-25T00:06:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/156 2024-04-25T00:06:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/156 2024-04-25T00:06:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/156 2024-04-25T00:06:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/156 2024-04-25T00:06:02.652 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 14/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_ 15/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 16/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 17/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_6 18/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86 19/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 20/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el 21/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9. 22/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_6 23/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 24/156 2024-04-25T00:06:02.653 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 25/156 2024-04-25T00:06:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/156 2024-04-25T00:06:02.654 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/156 2024-04-25T00:06:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 28/156 2024-04-25T00:06:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/156 2024-04-25T00:06:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97. 30/156 2024-04-25T00:06:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noa 31/156 2024-04-25T00:06:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.n 32/156 2024-04-25T00:06:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c 33/156 2024-04-25T00:06:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el 34/156 2024-04-25T00:06:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 35/156 2024-04-25T00:06:02.666 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.e 36/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 37/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 38/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 39/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T00:06:02.667 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T00:06:02.668 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T00:06:02.669 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T00:06:02.670 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T00:06:02.671 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T00:06:02.672 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T00:06:02.673 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T00:06:02.674 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T00:06:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T00:06:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T00:06:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T00:06:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 149/156 2024-04-25T00:06:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T00:06:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 151/156 2024-04-25T00:06:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T00:06:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T00:06:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T00:06:02.675 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T00:06:05.315 INFO:teuthology.orchestra.run.smithi042.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T00:06:05.315 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:06:05.315 INFO:teuthology.orchestra.run.smithi042.stdout:Upgraded: 2024-04-25T00:06:05.315 INFO:teuthology.orchestra.run.smithi042.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.315 INFO:teuthology.orchestra.run.smithi042.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.315 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T00:06:05.315 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T00:06:05.315 INFO:teuthology.orchestra.run.smithi042.stdout:Installed: 2024-04-25T00:06:05.315 INFO:teuthology.orchestra.run.smithi042.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T00:06:05.315 INFO:teuthology.orchestra.run.smithi042.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:06:05.316 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-shell-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: cephfs-top-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T00:06:05.317 INFO:teuthology.orchestra.run.smithi042.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T00:06:05.318 INFO:teuthology.orchestra.run.smithi042.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T00:06:05.319 INFO:teuthology.orchestra.run.smithi042.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T00:06:05.320 INFO:teuthology.orchestra.run.smithi042.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T00:06:05.321 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T00:06:05.322 INFO:teuthology.orchestra.run.smithi042.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T00:06:05.323 INFO:teuthology.orchestra.run.smithi042.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T00:06:05.323 INFO:teuthology.orchestra.run.smithi042.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T00:06:05.323 INFO:teuthology.orchestra.run.smithi042.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T00:06:05.323 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T00:06:05.323 INFO:teuthology.orchestra.run.smithi042.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T00:06:05.323 INFO:teuthology.orchestra.run.smithi042.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T00:06:05.323 INFO:teuthology.orchestra.run.smithi042.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T00:06:05.323 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T00:06:05.323 INFO:teuthology.orchestra.run.smithi042.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T00:06:05.323 INFO:teuthology.orchestra.run.smithi042.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T00:06:05.324 INFO:teuthology.orchestra.run.smithi042.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T00:06:05.325 INFO:teuthology.orchestra.run.smithi042.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T00:06:05.326 INFO:teuthology.orchestra.run.smithi042.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T00:06:05.327 INFO:teuthology.orchestra.run.smithi042.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T00:06:05.327 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:06:05.327 INFO:teuthology.orchestra.run.smithi042.stdout:Complete! 2024-04-25T00:06:05.938 DEBUG:teuthology.parallel:result is None 2024-04-25T00:06:05.938 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:06:06.095 DEBUG:teuthology.orchestra.run.smithi042:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:06:06.152 INFO:teuthology.orchestra.run.smithi042.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:06:06.153 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:06:06.153 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:06:06.155 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:06:06.296 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:06:06.322 INFO:teuthology.orchestra.run.smithi106.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:06:06.322 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:06:06.322 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:06:06.324 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:06:06.542 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T00:06:06.568 INFO:teuthology.orchestra.run.smithi136.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-25T00:06:06.568 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-25T00:06:06.568 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-25T00:06:06.570 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T00:06:06.570 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:06:06.570 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:06:06.653 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T00:06:06.654 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:06:06.686 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T00:06:06.686 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T00:06:06.717 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T00:06:06.717 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:06:06.717 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:06:06.780 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:06:06.884 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T00:06:06.884 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:06:06.912 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:06:06.981 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T00:06:06.981 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T00:06:07.005 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T00:06:07.069 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T00:06:07.069 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:06:07.069 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:06:07.138 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:06:07.259 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T00:06:07.260 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:06:07.283 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:06:07.347 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T00:06:07.348 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T00:06:07.372 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T00:06:07.435 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T00:06:07.436 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:06:07.436 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:06:07.502 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:06:07.609 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T00:06:07.609 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:06:07.633 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:06:07.697 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T00:06:07.697 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T00:06:07.720 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T00:06:07.783 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T00:06:07.889 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T00:06:07.889 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:06:07.890 INFO:tasks.cephadm:Cluster fsid is 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:06:07.890 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T00:06:07.890 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.42', 'mon.b': '172.21.15.106', 'mon.c': '172.21.15.136'} 2024-04-25T00:06:07.890 INFO:tasks.cephadm:First mon is mon.a on smithi042 2024-04-25T00:06:07.890 INFO:tasks.cephadm:First mgr is x 2024-04-25T00:06:07.890 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T00:06:07.891 DEBUG:teuthology.orchestra.run.smithi042:> sudo hostname $(hostname -s) 2024-04-25T00:06:07.953 DEBUG:teuthology.orchestra.run.smithi106:> sudo hostname $(hostname -s) 2024-04-25T00:06:07.977 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-04-25T00:06:08.001 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T00:06:08.001 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:06:08.226 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54987/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 22:37:46.422067', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54981/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2445-gd2c45d97', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2445.gd2c45d97'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-24 17:00:00.872117', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T00:06:08.374 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 d2c45d977142e024a59b68a8f1dce8f0b9840551 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:06:08.375 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm 2024-04-25T00:06:08.376 DEBUG:teuthology.orchestra.run.smithi042:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:06:08.945 INFO:teuthology.orchestra.run.smithi042.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:06 /home/ubuntu/cephtest/cephadm 2024-04-25T00:06:08.946 DEBUG:teuthology.orchestra.run.smithi106:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:06:09.469 INFO:teuthology.orchestra.run.smithi106.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:06 /home/ubuntu/cephtest/cephadm 2024-04-25T00:06:09.470 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T00:06:10.002 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 781409 Apr 25 00:06 /home/ubuntu/cephtest/cephadm 2024-04-25T00:06:10.002 DEBUG:teuthology.orchestra.run.smithi042:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:06:10.052 DEBUG:teuthology.orchestra.run.smithi106:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:06:10.071 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T00:06:10.105 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 on all hosts... 2024-04-25T00:06:10.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:06:10.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:06:10.113 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 pull 2024-04-25T00:06:11.578 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:06:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:06:12.310 INFO:teuthology.orchestra.run.smithi042.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:06:38.925 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T00:06:38.926 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:06:38.926 INFO:teuthology.orchestra.run.smithi106.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:06:38.926 INFO:teuthology.orchestra.run.smithi106.stdout: "repo_digests": [ 2024-04-25T00:06:38.926 INFO:teuthology.orchestra.run.smithi106.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:06:38.926 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2024-04-25T00:06:38.926 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T00:06:48.128 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-25T00:06:48.129 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:06:48.129 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:06:48.129 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-04-25T00:06:48.129 INFO:teuthology.orchestra.run.smithi136.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:06:48.129 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-25T00:06:48.129 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-25T00:06:53.168 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:06:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_version": "ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev)", 2024-04-25T00:06:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "image_id": "f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e", 2024-04-25T00:06:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "repo_digests": [ 2024-04-25T00:06:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901" 2024-04-25T00:06:53.169 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-25T00:06:53.169 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:06:53.226 DEBUG:teuthology.orchestra.run.smithi042:> sudo mkdir -p /etc/ceph 2024-04-25T00:06:53.318 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph 2024-04-25T00:06:53.350 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-04-25T00:06:53.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 777 /etc/ceph 2024-04-25T00:06:53.474 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /etc/ceph 2024-04-25T00:06:53.502 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-04-25T00:06:53.530 INFO:tasks.cephadm:Writing seed config... 2024-04-25T00:06:53.531 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T00:06:53.531 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T00:06:53.531 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T00:06:53.532 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T00:06:53.533 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T00:06:53.534 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T00:06:53.535 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:06:53.535 DEBUG:teuthology.orchestra.run.smithi042:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T00:06:53.597 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9d95543e-0297-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T00:06:53.597 DEBUG:teuthology.orchestra.run.smithi042:mon.a> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T00:06:53.644 DEBUG:teuthology.orchestra.run.smithi042:mgr.x> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T00:06:53.690 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T00:06:53.690 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 -v bootstrap --fsid 9d95543e-0297-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.42 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:06:53.964 INFO:teuthology.orchestra.run.smithi042.stdout:-------------------------------------------------------------------------------- 2024-04-25T00:06:53.964 INFO:teuthology.orchestra.run.smithi042.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551', '-v', 'bootstrap', '--fsid', '9d95543e-0297-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.42', '--skip-admin-label'] 2024-04-25T00:06:53.964 INFO:teuthology.orchestra.run.smithi042.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T00:06:53.965 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying podman|docker is present... 2024-04-25T00:06:54.021 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T00:06:54.021 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying lvm2 is present... 2024-04-25T00:06:54.022 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying time synchronization is in place... 2024-04-25T00:06:54.043 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T00:06:54.043 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T00:06:54.060 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T00:06:54.060 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-25T00:06:54.081 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-04-25T00:06:54.101 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-04-25T00:06:54.101 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-04-25T00:06:54.101 INFO:teuthology.orchestra.run.smithi042.stdout:Repeating the final host check... 2024-04-25T00:06:54.171 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T00:06:54.172 INFO:teuthology.orchestra.run.smithi042.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T00:06:54.172 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl is present 2024-04-25T00:06:54.172 INFO:teuthology.orchestra.run.smithi042.stdout:lvcreate is present 2024-04-25T00:06:54.191 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T00:06:54.192 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T00:06:54.214 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T00:06:54.214 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-25T00:06:54.240 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout enabled 2024-04-25T00:06:54.264 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout active 2024-04-25T00:06:54.264 INFO:teuthology.orchestra.run.smithi042.stdout:Unit chronyd.service is enabled and running 2024-04-25T00:06:54.264 INFO:teuthology.orchestra.run.smithi042.stdout:Host looks OK 2024-04-25T00:06:54.264 INFO:teuthology.orchestra.run.smithi042.stdout:Cluster fsid: 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:06:54.265 INFO:teuthology.orchestra.run.smithi042.stdout:Acquiring lock 140031185392544 on /run/cephadm/9d95543e-0297-11ef-bc93-c7b262605968.lock 2024-04-25T00:06:54.265 INFO:teuthology.orchestra.run.smithi042.stdout:Lock 140031185392544 acquired on /run/cephadm/9d95543e-0297-11ef-bc93-c7b262605968.lock 2024-04-25T00:06:54.267 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 3300 ... 2024-04-25T00:06:54.268 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying IP 172.21.15.42 port 6789 ... 2024-04-25T00:06:54.268 INFO:teuthology.orchestra.run.smithi042.stdout:Base mon IP(s) is [172.21.15.42:3300, 172.21.15.42:6789], mon addrv is [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-04-25T00:06:54.283 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.42 metric 100 2024-04-25T00:06:54.284 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.42 metric 100 2024-04-25T00:06:54.297 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T00:06:54.297 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-25T00:06:54.297 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-25T00:06:54.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T00:06:54.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T00:06:54.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T00:06:54.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-25T00:06:54.310 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-04-25T00:06:54.311 INFO:teuthology.orchestra.run.smithi042.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T00:06:54.313 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-04-25T00:06:54.313 INFO:teuthology.orchestra.run.smithi042.stdout:Mon IP `172.21.15.42` is in CIDR network `172.21.0.0/20` 2024-04-25T00:06:54.313 INFO:teuthology.orchestra.run.smithi042.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T00:06:54.316 INFO:teuthology.orchestra.run.smithi042.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T00:06:54.319 INFO:teuthology.orchestra.run.smithi042.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:06:55.037 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stdout f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T00:06:55.037 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551... 2024-04-25T00:06:55.037 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T00:06:55.037 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:58a99251f41e8a467bafa0ec849428709568906ca732f10972304f37056be6c5 2024-04-25T00:06:55.037 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-25T00:06:55.038 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Copying config sha256:f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e 2024-04-25T00:06:55.038 INFO:teuthology.orchestra.run.smithi042.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T00:06:56.261 INFO:teuthology.orchestra.run.smithi042.stdout:ceph: stdout ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T00:06:56.262 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph version: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev) 2024-04-25T00:06:56.262 INFO:teuthology.orchestra.run.smithi042.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T00:06:57.556 INFO:teuthology.orchestra.run.smithi042.stdout:stat: stdout 167 167 2024-04-25T00:06:57.556 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial keys... 2024-04-25T00:07:08.779 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCinilmgP+0AhAA+zFIZmLvioCK81GiRp7Vxg== 2024-04-25T00:07:10.140 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCtnilmGiUhGBAAaqlGT/Xwwpxg6LRtdruqKw== 2024-04-25T00:07:11.355 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph-authtool: stdout AQCunilmtWq4JBAA7A1ecqG0uFrzGeoWn486PQ== 2024-04-25T00:07:11.357 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial monmap... 2024-04-25T00:07:12.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T00:07:12.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T00:07:12.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:12.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T00:07:12.585 INFO:teuthology.orchestra.run.smithi042.stdout:monmaptool for a [v2:172.21.15.42:3300,v1:172.21.15.42:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T00:07:12.585 INFO:teuthology.orchestra.run.smithi042.stdout:setting min_mon_release = quincy 2024-04-25T00:07:12.585 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: set fsid to 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:12.586 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T00:07:12.586 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:07:12.586 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mon... 2024-04-25T00:07:13.948 INFO:teuthology.orchestra.run.smithi042.stdout:create mon.a on 2024-04-25T00:07:14.335 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T00:07:14.577 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T00:07:14.861 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9d95543e-0297-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-9d95543e-0297-11ef-bc93-c7b262605968.target. 2024-04-25T00:07:14.862 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9d95543e-0297-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-9d95543e-0297-11ef-bc93-c7b262605968.target. 2024-04-25T00:07:15.242 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.a 2024-04-25T00:07:15.243 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T00:07:15.542 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9d95543e-0297-11ef-bc93-c7b262605968.target.wants/ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9d95543e-0297-11ef-bc93-c7b262605968@.service. 2024-04-25T00:07:16.268 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:15 smithi042 podman[27746]: 2024-04-25 00:07:15.995741181 +0000 UTC m=+0.156915927 container create 91843f380f0252256a72f109ba6eca3022059599ab9e2ba2f2ffd4ef7ea7878d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T00:07:16.268 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:16 smithi042 podman[27746]: 2024-04-25 00:07:16.26691309 +0000 UTC m=+0.428087843 container init 91843f380f0252256a72f109ba6eca3022059599ab9e2ba2f2ffd4ef7ea7878d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T00:07:16.432 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:07:16.432 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-04-25T00:07:16.452 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:07:16.452 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-25T00:07:16.452 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-04-25T00:07:16.453 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T00:07:16.454 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon to start... 2024-04-25T00:07:16.454 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mon... 2024-04-25T00:07:16.617 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:16 smithi042 podman[27746]: 2024-04-25 00:07:16.278748007 +0000 UTC m=+0.439922763 container start 91843f380f0252256a72f109ba6eca3022059599ab9e2ba2f2ffd4ef7ea7878d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:07:16.617 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:16 smithi042 ceph-mon[27764]: mkfs 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:16.617 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:16 smithi042 ceph-mon[27764]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:07:16.617 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:16 smithi042 bash[27746]: 91843f380f0252256a72f109ba6eca3022059599ab9e2ba2f2ffd4ef7ea7878d 2024-04-25T00:07:16.617 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:16 smithi042 systemd[1]: Started Ceph mon.a for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:07:17.499 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:07:17.499 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: monmap epoch 1 2024-04-25T00:07:17.499 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:17.499 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: last_changed 2024-04-25T00:07:11.904046+0000 2024-04-25T00:07:17.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: created 2024-04-25T00:07:11.904046+0000 2024-04-25T00:07:17.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: min_mon_release 19 (squid) 2024-04-25T00:07:17.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: election_strategy: 1 2024-04-25T00:07:17.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-25T00:07:17.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: fsmap 2024-04-25T00:07:17.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T00:07:17.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: mgrmap e1: no daemons active 2024-04-25T00:07:17.500 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:17 smithi042 ceph-mon[27764]: from='client.? 172.21.15.42:0/1896204092' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T00:07:17.798 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T00:07:17.798 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout id: 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:17.798 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T00:07:17.798 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:17.798 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout services: 2024-04-25T00:07:17.798 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.821173s) 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout data: 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.084+0000 7f02b7600700 1 Processor -- start 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.085+0000 7f02b7600700 1 -- start start 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.085+0000 7f02b7600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.799 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.085+0000 7f02b7600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f02b01115e0 con 0x7f02b010ecb0 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.086+0000 7f02b4c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.086+0000 7f02b4c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01110a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50572/0 (socket says 172.21.15.42:50572) 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.086+0000 7f02b4c00700 1 -- 172.21.15.42:0/1420028355 learned_addr learned my addr 172.21.15.42:0/1420028355 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.087+0000 7f02b4c00700 1 -- 172.21.15.42:0/1420028355 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b0111760 con 0x7f02b010ecb0 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.087+0000 7f02b4c00700 1 --2- 172.21.15.42:0/1420028355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01110a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f02980098a0 tx=0x7f0298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f499abe0d7639f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.088+0000 7f02afe00700 1 -- 172.21.15.42:0/1420028355 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0298004160 con 0x7f02b010ecb0 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.088+0000 7f02afe00700 1 -- 172.21.15.42:0/1420028355 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0298004300 con 0x7f02b010ecb0 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.088+0000 7f02afe00700 1 -- 172.21.15.42:0/1420028355 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02980036a0 con 0x7f02b010ecb0 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.088+0000 7f02b7600700 1 -- 172.21.15.42:0/1420028355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 msgr2=0x7f02b01110a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.088+0000 7f02b7600700 1 --2- 172.21.15.42:0/1420028355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01110a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f02980098a0 tx=0x7f0298009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.089+0000 7f02b7600700 1 -- 172.21.15.42:0/1420028355 shutdown_connections 2024-04-25T00:07:17.800 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.089+0000 7f02b7600700 1 --2- 172.21.15.42:0/1420028355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01110a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.089+0000 7f02b7600700 1 -- 172.21.15.42:0/1420028355 >> 172.21.15.42:0/1420028355 conn(0x7f02b007d320 msgr2=0x7f02b007f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:17.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.089+0000 7f02b7600700 1 -- 172.21.15.42:0/1420028355 shutdown_connections 2024-04-25T00:07:17.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.089+0000 7f02b7600700 1 -- 172.21.15.42:0/1420028355 wait complete. 2024-04-25T00:07:17.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.090+0000 7f02b7600700 1 Processor -- start 2024-04-25T00:07:17.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.090+0000 7f02b7600700 1 -- start start 2024-04-25T00:07:17.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.091+0000 7f02b7600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01a1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:17.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.091+0000 7f02b7600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f02b0111bd0 con 0x7f02b010ecb0 2024-04-25T00:07:17.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.091+0000 7f02b4c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01a1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:17.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.091+0000 7f02b4c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01a1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50576/0 (socket says 172.21.15.42:50576) 2024-04-25T00:07:17.801 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.091+0000 7f02b4c00700 1 -- 172.21.15.42:0/1896204092 learned_addr learned my addr 172.21.15.42:0/1896204092 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:17.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.091+0000 7f02b4c00700 1 -- 172.21.15.42:0/1896204092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b01a1f60 con 0x7f02b010ecb0 2024-04-25T00:07:17.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.092+0000 7f02b4c00700 1 --2- 172.21.15.42:0/1896204092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01a1a20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0298000c00 tx=0x7f0298004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:17.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.092+0000 7f02ae000700 1 -- 172.21.15.42:0/1896204092 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0298003db0 con 0x7f02b010ecb0 2024-04-25T00:07:17.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.092+0000 7f02ae000700 1 -- 172.21.15.42:0/1896204092 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0298011630 con 0x7f02b010ecb0 2024-04-25T00:07:17.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.092+0000 7f02ae000700 1 -- 172.21.15.42:0/1896204092 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0298011950 con 0x7f02b010ecb0 2024-04-25T00:07:17.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.092+0000 7f02b7600700 1 -- 172.21.15.42:0/1896204092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b01a21f0 con 0x7f02b010ecb0 2024-04-25T00:07:17.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.092+0000 7f02b7600700 1 -- 172.21.15.42:0/1896204092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b01a4ee0 con 0x7f02b010ecb0 2024-04-25T00:07:17.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.092+0000 7f02ae000700 1 -- 172.21.15.42:0/1896204092 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0298011af0 con 0x7f02b010ecb0 2024-04-25T00:07:17.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.093+0000 7f02ae000700 1 -- 172.21.15.42:0/1896204092 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f029801c450 con 0x7f02b010ecb0 2024-04-25T00:07:17.802 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.093+0000 7f02b7600700 1 -- 172.21.15.42:0/1896204092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b0063890 con 0x7f02b010ecb0 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.097+0000 7f02ae000700 1 -- 172.21.15.42:0/1896204092 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0298032020 con 0x7f02b010ecb0 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.178+0000 7f02b7600700 1 -- 172.21.15.42:0/1896204092 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f02b002dda0 con 0x7f02b010ecb0 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.179+0000 7f02ae000700 1 -- 172.21.15.42:0/1896204092 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f0298004260 con 0x7f02b010ecb0 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.180+0000 7f02b7600700 1 -- 172.21.15.42:0/1896204092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 msgr2=0x7f02b01a1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.180+0000 7f02b7600700 1 --2- 172.21.15.42:0/1896204092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01a1a20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f0298000c00 tx=0x7f0298004750 comp rx=0 tx=0).stop 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.180+0000 7f02b7600700 1 -- 172.21.15.42:0/1896204092 shutdown_connections 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.180+0000 7f02b7600700 1 --2- 172.21.15.42:0/1896204092 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f02b010ecb0 0x7f02b01a1a20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.180+0000 7f02b7600700 1 -- 172.21.15.42:0/1896204092 >> 172.21.15.42:0/1896204092 conn(0x7f02b007d320 msgr2=0x7f02b007f620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.181+0000 7f02b7600700 1 -- 172.21.15.42:0/1896204092 shutdown_connections 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:17.181+0000 7f02b7600700 1 -- 172.21.15.42:0/1896204092 wait complete. 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:mon is available 2024-04-25T00:07:17.803 INFO:teuthology.orchestra.run.smithi042.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T00:07:18.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:18 smithi042 ceph-mon[27764]: from='client.? 172.21.15.42:0/2490321010' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T00:07:18.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:18 smithi042 ceph-mon[27764]: from='client.? 172.21.15.42:0/2490321010' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T00:07:19.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:19.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T00:07:19.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:19.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.42:3300,v1:172.21.15.42:6789] 2024-04-25T00:07:19.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T00:07:19.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T00:07:19.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T00:07:19.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T00:07:19.271 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:19.271 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T00:07:19.271 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T00:07:19.271 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:19.271 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.463+0000 7fa31ca00700 1 Processor -- start 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.464+0000 7fa31ca00700 1 -- start start 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.464+0000 7fa31ca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa318108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.464+0000 7fa31ca00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa318108740 con 0x7fa318107de0 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.464+0000 7fa315e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa318108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.464+0000 7fa315e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa318108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50578/0 (socket says 172.21.15.42:50578) 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.464+0000 7fa315e00700 1 -- 172.21.15.42:0/2544000594 learned_addr learned my addr 172.21.15.42:0/2544000594 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.465+0000 7fa315e00700 1 -- 172.21.15.42:0/2544000594 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3181088c0 con 0x7fa318107de0 2024-04-25T00:07:19.272 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.465+0000 7fa315e00700 1 --2- 172.21.15.42:0/2544000594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa318108200 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa308009e80 tx=0x7fa30800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafe82364a39267 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:19.273 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.466+0000 7fa314a00700 1 -- 172.21.15.42:0/2544000594 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa308017070 con 0x7fa318107de0 2024-04-25T00:07:19.273 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.466+0000 7fa314a00700 1 -- 172.21.15.42:0/2544000594 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa308003a20 con 0x7fa318107de0 2024-04-25T00:07:19.273 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.466+0000 7fa314a00700 1 -- 172.21.15.42:0/2544000594 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa308003d20 con 0x7fa318107de0 2024-04-25T00:07:19.273 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.466+0000 7fa31ca00700 1 -- 172.21.15.42:0/2544000594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 msgr2=0x7fa318108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:19.273 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.466+0000 7fa31ca00700 1 --2- 172.21.15.42:0/2544000594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa318108200 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa308009e80 tx=0x7fa30800b330 comp rx=0 tx=0).stop 2024-04-25T00:07:19.273 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.466+0000 7fa31ca00700 1 -- 172.21.15.42:0/2544000594 shutdown_connections 2024-04-25T00:07:19.273 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.466+0000 7fa31ca00700 1 --2- 172.21.15.42:0/2544000594 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa318108200 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:19.273 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.466+0000 7fa31ca00700 1 -- 172.21.15.42:0/2544000594 >> 172.21.15.42:0/2544000594 conn(0x7fa318103460 msgr2=0x7fa318105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:19.273 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.467+0000 7fa31ca00700 1 -- 172.21.15.42:0/2544000594 shutdown_connections 2024-04-25T00:07:19.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.467+0000 7fa31ca00700 1 -- 172.21.15.42:0/2544000594 wait complete. 2024-04-25T00:07:19.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.468+0000 7fa31ca00700 1 Processor -- start 2024-04-25T00:07:19.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.468+0000 7fa31ca00700 1 -- start start 2024-04-25T00:07:19.276 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.468+0000 7fa31ca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa31819d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.468+0000 7fa31ca00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa318109290 con 0x7fa318107de0 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.468+0000 7fa315e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa31819d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.468+0000 7fa315e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa31819d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50594/0 (socket says 172.21.15.42:50594) 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.468+0000 7fa315e00700 1 -- 172.21.15.42:0/2490321010 learned_addr learned my addr 172.21.15.42:0/2490321010 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.469+0000 7fa315e00700 1 -- 172.21.15.42:0/2490321010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31819dc60 con 0x7fa318107de0 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.469+0000 7fa315e00700 1 --2- 172.21.15.42:0/2490321010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa31819d720 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa318108b10 tx=0x7fa308004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.469+0000 7fa30ea00700 1 -- 172.21.15.42:0/2490321010 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa308022020 con 0x7fa318107de0 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.469+0000 7fa30ea00700 1 -- 172.21.15.42:0/2490321010 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa308018040 con 0x7fa318107de0 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.469+0000 7fa31ca00700 1 -- 172.21.15.42:0/2490321010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31819def0 con 0x7fa318107de0 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.469+0000 7fa30ea00700 1 -- 172.21.15.42:0/2490321010 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa308017070 con 0x7fa318107de0 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.469+0000 7fa31ca00700 1 -- 172.21.15.42:0/2490321010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3181a0b90 con 0x7fa318107de0 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.470+0000 7fa30ea00700 1 -- 172.21.15.42:0/2490321010 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa308025020 con 0x7fa318107de0 2024-04-25T00:07:19.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.470+0000 7fa30ea00700 1 -- 172.21.15.42:0/2490321010 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa30800f050 con 0x7fa318107de0 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.471+0000 7fa31ca00700 1 -- 172.21.15.42:0/2490321010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa318063890 con 0x7fa318107de0 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.473+0000 7fa30ea00700 1 -- 172.21.15.42:0/2490321010 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fa30802c030 con 0x7fa318107de0 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.552+0000 7fa31ca00700 1 -- 172.21.15.42:0/2490321010 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa3181a0d90 con 0x7fa318107de0 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.562+0000 7fa30ea00700 1 -- 172.21.15.42:0/2490321010 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7fa30801e480 con 0x7fa318107de0 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.563+0000 7fa31ca00700 1 -- 172.21.15.42:0/2490321010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 msgr2=0x7fa31819d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.563+0000 7fa30ea00700 1 -- 172.21.15.42:0/2490321010 <== mon.0 v2:172.21.15.42:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fa308035070 con 0x7fa318107de0 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.563+0000 7fa31ca00700 1 --2- 172.21.15.42:0/2490321010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa31819d720 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa318108b10 tx=0x7fa308004ad0 comp rx=0 tx=0).stop 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.564+0000 7fa31ca00700 1 -- 172.21.15.42:0/2490321010 shutdown_connections 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.564+0000 7fa31ca00700 1 --2- 172.21.15.42:0/2490321010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa318107de0 0x7fa31819d720 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.564+0000 7fa31ca00700 1 -- 172.21.15.42:0/2490321010 >> 172.21.15.42:0/2490321010 conn(0x7fa318103460 msgr2=0x7fa318105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.564+0000 7fa31ca00700 1 -- 172.21.15.42:0/2490321010 shutdown_connections 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:18.564+0000 7fa31ca00700 1 -- 172.21.15.42:0/2490321010 wait complete. 2024-04-25T00:07:19.278 INFO:teuthology.orchestra.run.smithi042.stdout:Generating new minimal ceph.conf... 2024-04-25T00:07:20.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:20 smithi042 ceph-mon[27764]: from='client.? 172.21.15.42:0/1973895309' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:07:20.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.900+0000 7fbaec800700 1 Processor -- start 2024-04-25T00:07:20.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.900+0000 7fbaec800700 1 -- start start 2024-04-25T00:07:20.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.900+0000 7fbaec800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:20.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.900+0000 7fbaec800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbae410a9b0 con 0x7fbae4109fc0 2024-04-25T00:07:20.624 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.901+0000 7fbae9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:20.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.901+0000 7fbae9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae410a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50600/0 (socket says 172.21.15.42:50600) 2024-04-25T00:07:20.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.901+0000 7fbae9e00700 1 -- 172.21.15.42:0/1697122774 learned_addr learned my addr 172.21.15.42:0/1697122774 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:20.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.901+0000 7fbae9e00700 1 -- 172.21.15.42:0/1697122774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae410b170 con 0x7fbae4109fc0 2024-04-25T00:07:20.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.901+0000 7fbae9e00700 1 --2- 172.21.15.42:0/1697122774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae410a3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbad80098a0 tx=0x7fbad8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27de20cc30fe8b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:20.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.902+0000 7fbae8a00700 1 -- 172.21.15.42:0/1697122774 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad8004160 con 0x7fbae4109fc0 2024-04-25T00:07:20.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.902+0000 7fbae8a00700 1 -- 172.21.15.42:0/1697122774 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbad8004300 con 0x7fbae4109fc0 2024-04-25T00:07:20.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.902+0000 7fbae8a00700 1 -- 172.21.15.42:0/1697122774 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad8011b50 con 0x7fbae4109fc0 2024-04-25T00:07:20.625 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.902+0000 7fbaec800700 1 -- 172.21.15.42:0/1697122774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 msgr2=0x7fbae410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:20.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.903+0000 7fbaec800700 1 --2- 172.21.15.42:0/1697122774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae410a3e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fbad80098a0 tx=0x7fbad8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.903+0000 7fbaec800700 1 -- 172.21.15.42:0/1697122774 shutdown_connections 2024-04-25T00:07:20.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.903+0000 7fbaec800700 1 --2- 172.21.15.42:0/1697122774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae410a3e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.903+0000 7fbaec800700 1 -- 172.21.15.42:0/1697122774 >> 172.21.15.42:0/1697122774 conn(0x7fbae407d320 msgr2=0x7fbae407d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:20.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.903+0000 7fbaec800700 1 -- 172.21.15.42:0/1697122774 shutdown_connections 2024-04-25T00:07:20.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.903+0000 7fbaec800700 1 -- 172.21.15.42:0/1697122774 wait complete. 2024-04-25T00:07:20.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.904+0000 7fbaec800700 1 Processor -- start 2024-04-25T00:07:20.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.904+0000 7fbaec800700 1 -- start start 2024-04-25T00:07:20.626 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.905+0000 7fbaec800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae419df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:20.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.905+0000 7fbaec800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbae410e520 con 0x7fbae4109fc0 2024-04-25T00:07:20.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.905+0000 7fbae9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae419df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:20.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.905+0000 7fbae9e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae419df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50610/0 (socket says 172.21.15.42:50610) 2024-04-25T00:07:20.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.905+0000 7fbae9e00700 1 -- 172.21.15.42:0/1973895309 learned_addr learned my addr 172.21.15.42:0/1973895309 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:20.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.905+0000 7fbae9e00700 1 -- 172.21.15.42:0/1973895309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae419e4a0 con 0x7fbae4109fc0 2024-04-25T00:07:20.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.906+0000 7fbae9e00700 1 --2- 172.21.15.42:0/1973895309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae419df60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbad8000c00 tx=0x7fbad8011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:20.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.906+0000 7fbadea00700 1 -- 172.21.15.42:0/1973895309 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad8003b20 con 0x7fbae4109fc0 2024-04-25T00:07:20.627 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.906+0000 7fbadea00700 1 -- 172.21.15.42:0/1973895309 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fbad8003cc0 con 0x7fbae4109fc0 2024-04-25T00:07:20.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.906+0000 7fbadea00700 1 -- 172.21.15.42:0/1973895309 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbad8024910 con 0x7fbae4109fc0 2024-04-25T00:07:20.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.906+0000 7fbaec800700 1 -- 172.21.15.42:0/1973895309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae419e730 con 0x7fbae4109fc0 2024-04-25T00:07:20.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.906+0000 7fbaec800700 1 -- 172.21.15.42:0/1973895309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae41a1420 con 0x7fbae4109fc0 2024-04-25T00:07:20.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.907+0000 7fbadea00700 1 -- 172.21.15.42:0/1973895309 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbad8022020 con 0x7fbae4109fc0 2024-04-25T00:07:20.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.907+0000 7fbadea00700 1 -- 172.21.15.42:0/1973895309 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbad8011e00 con 0x7fbae4109fc0 2024-04-25T00:07:20.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.908+0000 7fbaec800700 1 -- 172.21.15.42:0/1973895309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae4063890 con 0x7fbae4109fc0 2024-04-25T00:07:20.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.911+0000 7fbadea00700 1 -- 172.21.15.42:0/1973895309 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbad801acd0 con 0x7fbae4109fc0 2024-04-25T00:07:20.628 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.988+0000 7fbaec800700 1 -- 172.21.15.42:0/1973895309 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fbae41a15f0 con 0x7fbae4109fc0 2024-04-25T00:07:20.629 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.989+0000 7fbadea00700 1 -- 172.21.15.42:0/1973895309 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fbad8032030 con 0x7fbae4109fc0 2024-04-25T00:07:20.629 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.991+0000 7fbaec800700 1 -- 172.21.15.42:0/1973895309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 msgr2=0x7fbae419df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:20.629 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.991+0000 7fbaec800700 1 --2- 172.21.15.42:0/1973895309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae419df60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fbad8000c00 tx=0x7fbad8011c10 comp rx=0 tx=0).stop 2024-04-25T00:07:20.629 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.991+0000 7fbaec800700 1 -- 172.21.15.42:0/1973895309 shutdown_connections 2024-04-25T00:07:20.629 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.991+0000 7fbaec800700 1 --2- 172.21.15.42:0/1973895309 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbae4109fc0 0x7fbae419df60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:20.629 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.991+0000 7fbaec800700 1 -- 172.21.15.42:0/1973895309 >> 172.21.15.42:0/1973895309 conn(0x7fbae407d320 msgr2=0x7fbae4107500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:20.630 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.991+0000 7fbaec800700 1 -- 172.21.15.42:0/1973895309 shutdown_connections 2024-04-25T00:07:20.630 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:19.991+0000 7fbaec800700 1 -- 172.21.15.42:0/1973895309 wait complete. 2024-04-25T00:07:20.630 INFO:teuthology.orchestra.run.smithi042.stdout:Restarting the monitor... 2024-04-25T00:07:20.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:20 smithi042 systemd[1]: Stopping Ceph mon.a for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:07:21.494 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:21 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a[27760]: 2024-04-25T00:07:21.129+0000 7f61fac00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:07:21.495 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:21 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a[27760]: 2024-04-25T00:07:21.129+0000 7f61fac00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T00:07:21.495 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:21 smithi042 podman[28008]: 2024-04-25 00:07:21.181333508 +0000 UTC m=+0.186517334 container died 91843f380f0252256a72f109ba6eca3022059599ab9e2ba2f2ffd4ef7ea7878d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:07:21.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:21 smithi042 podman[28008]: 2024-04-25 00:07:21.638874216 +0000 UTC m=+0.644058070 container cleanup 91843f380f0252256a72f109ba6eca3022059599ab9e2ba2f2ffd4ef7ea7878d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T00:07:21.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:21 smithi042 bash[28008]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a 2024-04-25T00:07:22.161 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:21 smithi042 podman[28022]: 2024-04-25 00:07:21.881105961 +0000 UTC m=+0.674281115 container remove 91843f380f0252256a72f109ba6eca3022059599ab9e2ba2f2ffd4ef7ea7878d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-25T00:07:22.412 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:22 smithi042 systemd[1]: ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T00:07:22.412 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:22 smithi042 systemd[1]: Stopped Ceph mon.a for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:07:22.820 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:22 smithi042 systemd[1]: ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.a.service: Consumed 2.047s CPU time. 2024-04-25T00:07:22.820 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:22 smithi042 systemd[1]: Starting Ceph mon.a for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:07:23.072 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:22 smithi042 podman[28111]: 2024-04-25 00:07:22.724814383 +0000 UTC m=+0.035417309 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:07:23.073 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:22 smithi042 podman[28111]: 2024-04-25 00:07:22.840521523 +0000 UTC m=+0.151124489 container create 7bf7a651ead32d2297e340c4997b25c820961bb8e304cfa2febca2431bbe4dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:07:23.073 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 podman[28111]: 2024-04-25 00:07:23.061394099 +0000 UTC m=+0.371997072 container init 7bf7a651ead32d2297e340c4997b25c820961bb8e304cfa2febca2431bbe4dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9) 2024-04-25T00:07:23.073 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 podman[28111]: 2024-04-25 00:07:23.072483125 +0000 UTC m=+0.383086094 container start 7bf7a651ead32d2297e340c4997b25c820961bb8e304cfa2febca2431bbe4dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:07:23.153 INFO:teuthology.orchestra.run.smithi042.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T00:07:23.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: pidfile_write: ignore empty --pid-file 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: load: jerasure load: lrc 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: RocksDB version: 7.9.2 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Git sha 0 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: DB SUMMARY 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: DB Session ID: 39EPQ3V6RIPYHEZQ5GD5 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: CURRENT file: CURRENT 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T00:07:23.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105672 ; 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.error_if_exists: 0 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.create_if_missing: 0 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.paranoid_checks: 1 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.env: 0x558911094c60 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.info_log: 0x558919a1bae0 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.statistics: (nil) 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.use_fsync: 0 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_log_file_size: 0 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T00:07:23.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.allow_fallocate: 1 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.use_direct_reads: 0 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.db_log_dir: 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.wal_dir: 2024-04-25T00:07:23.381 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T00:07:23.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T00:07:23.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T00:07:23.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T00:07:23.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T00:07:23.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T00:07:23.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T00:07:23.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T00:07:23.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.write_buffer_manager: 0x5589199ee1e0 2024-04-25T00:07:23.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T00:07:23.382 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.unordered_write: 0 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.row_cache: None 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.wal_filter: None 2024-04-25T00:07:23.383 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.two_write_queues: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.wal_compression: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.atomic_flush: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.log_readahead_size: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T00:07:23.384 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_background_jobs: 2 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_background_compactions: -1 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_subcompactions: 1 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_open_files: -1 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T00:07:23.385 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_background_flushes: -1 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Compression algorithms supported: 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: kZSTD supported: 0 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: kXpressCompression supported: 0 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: kZlibCompression supported: 1 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: kSnappyCompression supported: 1 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: kLZ4Compression supported: 1 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: kBZip2Compression supported: 0 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T00:07:23.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.merge_operator: 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_filter: None 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558919a1bbe0) 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks: 1 2024-04-25T00:07:23.387 INFO:journalctl@ceph.mon.a.smithi042.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T00:07:23.388 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T00:07:23.388 INFO:journalctl@ceph.mon.a.smithi042.stdout: pin_top_level_index_and_filter: 1 2024-04-25T00:07:23.388 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_type: 0 2024-04-25T00:07:23.388 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_index_type: 0 2024-04-25T00:07:23.388 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_shortening: 1 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: checksum: 4 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: no_block_cache: 0 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache: 0x558919a1f090 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_name: BinnedLRUCache 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_options: 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: capacity : 536870912 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_shard_bits : 4 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: strict_capacity_limit : 0 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: high_pri_pool_ratio: 0.000 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_cache_compressed: (nil) 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: persistent_cache: (nil) 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size: 4096 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_size_deviation: 10 2024-04-25T00:07:23.389 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_restart_interval: 16 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: index_block_restart_interval: 1 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: metadata_block_size: 4096 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: partition_filters: 0 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: use_delta_encoding: 1 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: filter_policy: bloomfilter 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: whole_key_filtering: 1 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: verify_compression: 0 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: read_amp_bytes_per_bit: 0 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: format_version: 5 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_index_compression: 1 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: block_align: 0 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: max_auto_readahead_size: 262144 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: prepopulate_block_cache: 0 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: initial_auto_readahead_size: 8192 2024-04-25T00:07:23.390 INFO:journalctl@ceph.mon.a.smithi042.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compression: NoCompression 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.num_levels: 7 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T00:07:23.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 bash[28111]: 7bf7a651ead32d2297e340c4997b25c820961bb8e304cfa2febca2431bbe4dad 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T00:07:23.392 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T00:07:23.393 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T00:07:23.394 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.table_properties_collectors: 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.inplace_update_support: 0 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.bloom_locality: 0 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.max_successive_merges: 0 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.ttl: 2592000 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T00:07:23.395 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.enable_blob_files: false 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.min_blob_size: 0 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4e9335be-b82d-4000-8786-d1f8fe73f527 2024-04-25T00:07:23.396 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003643133400, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003643135564, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100628, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98548, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 90792, "raw_average_value_size": 314, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714003643, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4e9335be-b82d-4000-8786-d1f8fe73f527", "db_session_id": "39EPQ3V6RIPYHEZQ5GD5", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003643135737, "job": 1, "event": "recovery_finished"} 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558919b10000 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: DB pointer 0x558919afa000 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** DB Stats ** 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:07:23.397 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: L0 2/0 100.10 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: Sum 2/0 100.10 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 73.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** Compaction Stats [default] ** 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T00:07:23.398 INFO:journalctl@ceph.mon.a.smithi042.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 73.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: Cumulative compaction: 0.00 GB write, 12.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: Interval compaction: 0.00 GB write, 12.32 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache BinnedLRUCache@0x558919a1f090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T00:07:23.399 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: starting mon.a rank 0 at public addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] at bind addrs [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: mon.a@-1(???) e1 preinit fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: mon.a@-1(???).mds e1 new map 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: mon.a@-1(???).mds e1 print_map 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout: e1 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout: legacy client fscid: -1 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout: 2024-04-25T00:07:23.400 INFO:journalctl@ceph.mon.a.smithi042.stdout: No filesystems configured 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 systemd[1]: Started Ceph mon.a for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: monmap epoch 1 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: last_changed 2024-04-25T00:07:11.904046+0000 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: created 2024-04-25T00:07:11.904046+0000 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: min_mon_release 19 (squid) 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: election_strategy: 1 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-25T00:07:23.401 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: fsmap 2024-04-25T00:07:23.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T00:07:23.402 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:23 smithi042 ceph-mon[28130]: mgrmap e1: no daemons active 2024-04-25T00:07:24.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.191+0000 7fc3a5a00700 1 Processor -- start 2024-04-25T00:07:24.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.191+0000 7fc3a5a00700 1 -- start start 2024-04-25T00:07:24.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.191+0000 7fc3a5a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a0108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.191+0000 7fc3a5a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a0108770 con 0x7fc3a0107e10 2024-04-25T00:07:24.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.192+0000 7fc39ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a0108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.192+0000 7fc39ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a0108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50626/0 (socket says 172.21.15.42:50626) 2024-04-25T00:07:24.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.192+0000 7fc39ea00700 1 -- 172.21.15.42:0/2729506423 learned_addr learned my addr 172.21.15.42:0/2729506423 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:24.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.193+0000 7fc39ea00700 1 -- 172.21.15.42:0/2729506423 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a01088f0 con 0x7fc3a0107e10 2024-04-25T00:07:24.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.193+0000 7fc39ea00700 1 --2- 172.21.15.42:0/2729506423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a0108230 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3900098a0 tx=0x7fc390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2baf855f78413 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.194+0000 7fc39d600700 1 -- 172.21.15.42:0/2729506423 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc390004160 con 0x7fc3a0107e10 2024-04-25T00:07:24.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.194+0000 7fc39d600700 1 -- 172.21.15.42:0/2729506423 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc390004300 con 0x7fc3a0107e10 2024-04-25T00:07:24.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.194+0000 7fc39d600700 1 -- 172.21.15.42:0/2729506423 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc390011b50 con 0x7fc3a0107e10 2024-04-25T00:07:24.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.194+0000 7fc3a5a00700 1 -- 172.21.15.42:0/2729506423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 msgr2=0x7fc3a0108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.194+0000 7fc3a5a00700 1 --2- 172.21.15.42:0/2729506423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a0108230 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3900098a0 tx=0x7fc390009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.195+0000 7fc3a5a00700 1 -- 172.21.15.42:0/2729506423 shutdown_connections 2024-04-25T00:07:24.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.195+0000 7fc3a5a00700 1 --2- 172.21.15.42:0/2729506423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a0108230 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.195+0000 7fc3a5a00700 1 -- 172.21.15.42:0/2729506423 >> 172.21.15.42:0/2729506423 conn(0x7fc3a0103470 msgr2=0x7fc3a0105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:24.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.195+0000 7fc3a5a00700 1 -- 172.21.15.42:0/2729506423 shutdown_connections 2024-04-25T00:07:24.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.195+0000 7fc3a5a00700 1 -- 172.21.15.42:0/2729506423 wait complete. 2024-04-25T00:07:24.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.196+0000 7fc3a5a00700 1 Processor -- start 2024-04-25T00:07:24.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.196+0000 7fc3a5a00700 1 -- start start 2024-04-25T00:07:24.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.196+0000 7fc3a5a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a007db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:24.954 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.196+0000 7fc3a5a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a010c230 con 0x7fc3a0107e10 2024-04-25T00:07:24.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.197+0000 7fc39ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:24.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.197+0000 7fc39ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50640/0 (socket says 172.21.15.42:50640) 2024-04-25T00:07:24.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.197+0000 7fc39ea00700 1 -- 172.21.15.42:0/4021526294 learned_addr learned my addr 172.21.15.42:0/4021526294 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:24.955 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.197+0000 7fc39ea00700 1 -- 172.21.15.42:0/4021526294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a007e090 con 0x7fc3a0107e10 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.198+0000 7fc39ea00700 1 --2- 172.21.15.42:0/4021526294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a007db50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc390011b60 tx=0x7fc390011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.198+0000 7fc397400700 1 -- 172.21.15.42:0/4021526294 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc390003b20 con 0x7fc3a0107e10 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.198+0000 7fc397400700 1 -- 172.21.15.42:0/4021526294 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc390003cc0 con 0x7fc3a0107e10 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.198+0000 7fc3a5a00700 1 -- 172.21.15.42:0/4021526294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a007c280 con 0x7fc3a0107e10 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.198+0000 7fc397400700 1 -- 172.21.15.42:0/4021526294 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc390024840 con 0x7fc3a0107e10 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.198+0000 7fc3a5a00700 1 -- 172.21.15.42:0/4021526294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a007c7c0 con 0x7fc3a0107e10 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.199+0000 7fc397400700 1 -- 172.21.15.42:0/4021526294 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc390022020 con 0x7fc3a0107e10 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.199+0000 7fc397400700 1 -- 172.21.15.42:0/4021526294 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc390011e00 con 0x7fc3a0107e10 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.199+0000 7fc3a5a00700 1 -- 172.21.15.42:0/4021526294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a0063890 con 0x7fc3a0107e10 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.203+0000 7fc397400700 1 -- 172.21.15.42:0/4021526294 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fc390043dd0 con 0x7fc3a0107e10 2024-04-25T00:07:24.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.281+0000 7fc3a5a00700 1 -- 172.21.15.42:0/4021526294 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fc3a007cb00 con 0x7fc3a0107e10 2024-04-25T00:07:24.958 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.283+0000 7fc397400700 1 -- 172.21.15.42:0/4021526294 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fc390032030 con 0x7fc3a0107e10 2024-04-25T00:07:24.958 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.285+0000 7fc3a5a00700 1 -- 172.21.15.42:0/4021526294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 msgr2=0x7fc3a007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:24.958 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.285+0000 7fc3a5a00700 1 --2- 172.21.15.42:0/4021526294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a007db50 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc390011b60 tx=0x7fc390011c40 comp rx=0 tx=0).stop 2024-04-25T00:07:24.958 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.285+0000 7fc3a5a00700 1 -- 172.21.15.42:0/4021526294 shutdown_connections 2024-04-25T00:07:24.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.285+0000 7fc3a5a00700 1 --2- 172.21.15.42:0/4021526294 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc3a0107e10 0x7fc3a007db50 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:24.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.285+0000 7fc3a5a00700 1 -- 172.21.15.42:0/4021526294 >> 172.21.15.42:0/4021526294 conn(0x7fc3a0103470 msgr2=0x7fc3a0103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:24.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.286+0000 7fc3a5a00700 1 -- 172.21.15.42:0/4021526294 shutdown_connections 2024-04-25T00:07:24.959 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:24.286+0000 7fc3a5a00700 1 -- 172.21.15.42:0/4021526294 wait complete. 2024-04-25T00:07:24.959 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T00:07:24.990 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:07:24.990 INFO:teuthology.orchestra.run.smithi042.stdout:Creating mgr... 2024-04-25T00:07:24.992 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T00:07:24.993 INFO:teuthology.orchestra.run.smithi042.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T00:07:25.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:25 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/4021526294' entity='client.admin' 2024-04-25T00:07:25.421 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.x 2024-04-25T00:07:25.421 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T00:07:25.684 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9d95543e-0297-11ef-bc93-c7b262605968.target.wants/ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9d95543e-0297-11ef-bc93-c7b262605968@.service. 2024-04-25T00:07:26.085 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:25 smithi042 systemd[1]: Starting Ceph mgr.x for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:07:26.339 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:26 smithi042 podman[28351]: 2024-04-25 00:07:25.989090421 +0000 UTC m=+0.032885754 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:07:26.339 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:26 smithi042 podman[28351]: 2024-04-25 00:07:26.126024784 +0000 UTC m=+0.169820128 container create c05d3001fc487063fa5bc2e14f9a0099d22349fcd71664cecbbfd7c8f9a6b037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=) 2024-04-25T00:07:26.339 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:26 smithi042 podman[28351]: 2024-04-25 00:07:26.337995363 +0000 UTC m=+0.381790741 container init c05d3001fc487063fa5bc2e14f9a0099d22349fcd71664cecbbfd7c8f9a6b037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T00:07:26.469 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:07:26.469 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-04-25T00:07:26.487 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:07:26.488 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-25T00:07:26.488 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-04-25T00:07:26.488 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T00:07:26.509 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:07:26.510 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-04-25T00:07:26.531 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:07:26.531 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-25T00:07:26.531 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-04-25T00:07:26.531 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T00:07:26.531 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr to start... 2024-04-25T00:07:26.532 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr... 2024-04-25T00:07:26.614 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:26 smithi042 podman[28351]: 2024-04-25 00:07:26.348865061 +0000 UTC m=+0.392660405 container start c05d3001fc487063fa5bc2e14f9a0099d22349fcd71664cecbbfd7c8f9a6b037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-25T00:07:26.614 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:26 smithi042 bash[28351]: c05d3001fc487063fa5bc2e14f9a0099d22349fcd71664cecbbfd7c8f9a6b037 2024-04-25T00:07:26.614 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:26 smithi042 systemd[1]: Started Ceph mgr.x for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:07:26.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:26 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:26.613+0000 7f0285571200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:07:27.166 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:26 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:26.892+0000 7f0285571200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:07:27.167 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:27 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:27.029+0000 7f0285571200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:07:27.454 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:27 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/681721131' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:07:27.878 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:27 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:27.452+0000 7f0285571200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:07:27.878 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:27 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:27.571+0000 7f0285571200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:07:27.948 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:27.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:07:27.949 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "9d95543e-0297-11ef-bc93-c7b262605968", 2024-04-25T00:07:27.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:07:27.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:07:27.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:07:27.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:07:27.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:27.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:07:27.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:07:27.950 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:07:27.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:27.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:07:27.951 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:07:27.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:27.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-04-25T00:07:27.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:07:27.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:27.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:07:27.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:07:27.952 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:27.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:07:27.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:27.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:07:27.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:07:27.953 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:07:27.956 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:07:27.957 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:07:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:07:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:07:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:07:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:07:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:07:16.360238+0000", 2024-04-25T00:07:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:07:27.960 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:27.961 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:07:27.961 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:07:27.961 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.165+0000 7fded7e00700 1 Processor -- start 2024-04-25T00:07:27.961 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.165+0000 7fded7e00700 1 -- start start 2024-04-25T00:07:27.961 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.165+0000 7fded7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded807c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.961 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.165+0000 7fded7e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fded807e140 con 0x7fded807dd20 2024-04-25T00:07:27.961 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.166+0000 7fded6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded807c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.961 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.166+0000 7fded6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded807c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58020/0 (socket says 172.21.15.42:58020) 2024-04-25T00:07:27.961 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.166+0000 7fded6a00700 1 -- 172.21.15.42:0/93510343 learned_addr learned my addr 172.21.15.42:0/93510343 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.166+0000 7fded6a00700 1 -- 172.21.15.42:0/93510343 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded807c820 con 0x7fded807dd20 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.167+0000 7fded6a00700 1 --2- 172.21.15.42:0/93510343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded807c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdec80098a0 tx=0x7fdec8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed1255aee92be91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.167+0000 7fded6000700 1 -- 172.21.15.42:0/93510343 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec8004160 con 0x7fded807dd20 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.167+0000 7fded6000700 1 -- 172.21.15.42:0/93510343 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdec8004300 con 0x7fded807dd20 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.167+0000 7fded6000700 1 -- 172.21.15.42:0/93510343 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec8011b50 con 0x7fded807dd20 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.168+0000 7fded7e00700 1 -- 172.21.15.42:0/93510343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 msgr2=0x7fded807c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.168+0000 7fded7e00700 1 --2- 172.21.15.42:0/93510343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded807c2e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fdec80098a0 tx=0x7fdec8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.168+0000 7fded7e00700 1 -- 172.21.15.42:0/93510343 shutdown_connections 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.168+0000 7fded7e00700 1 --2- 172.21.15.42:0/93510343 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded807c2e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.168+0000 7fded7e00700 1 -- 172.21.15.42:0/93510343 >> 172.21.15.42:0/93510343 conn(0x7fded8103250 msgr2=0x7fded8105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.964 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.168+0000 7fded7e00700 1 -- 172.21.15.42:0/93510343 shutdown_connections 2024-04-25T00:07:27.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.168+0000 7fded7e00700 1 -- 172.21.15.42:0/93510343 wait complete. 2024-04-25T00:07:27.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.169+0000 7fded7e00700 1 Processor -- start 2024-04-25T00:07:27.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.170+0000 7fded7e00700 1 -- start start 2024-04-25T00:07:27.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.170+0000 7fded7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded819dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:27.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.170+0000 7fded7e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fded810e060 con 0x7fded807dd20 2024-04-25T00:07:27.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.170+0000 7fded6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded819dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:27.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.170+0000 7fded6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded819dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58028/0 (socket says 172.21.15.42:58028) 2024-04-25T00:07:27.965 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.170+0000 7fded6a00700 1 -- 172.21.15.42:0/681721131 learned_addr learned my addr 172.21.15.42:0/681721131 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:27.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.171+0000 7fded6a00700 1 -- 172.21.15.42:0/681721131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded819e290 con 0x7fded807dd20 2024-04-25T00:07:27.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.171+0000 7fded6a00700 1 --2- 172.21.15.42:0/681721131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded819dd50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdec8011670 tx=0x7fdec8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:27.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.171+0000 7fdecfe00700 1 -- 172.21.15.42:0/681721131 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec80039c0 con 0x7fded807dd20 2024-04-25T00:07:27.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.171+0000 7fdecfe00700 1 -- 172.21.15.42:0/681721131 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fdec8011e50 con 0x7fded807dd20 2024-04-25T00:07:27.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.171+0000 7fdecfe00700 1 -- 172.21.15.42:0/681721131 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdec801a5c0 con 0x7fded807dd20 2024-04-25T00:07:27.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.171+0000 7fded7e00700 1 -- 172.21.15.42:0/681721131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fded819e520 con 0x7fded807dd20 2024-04-25T00:07:27.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.172+0000 7fded7e00700 1 -- 172.21.15.42:0/681721131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fded81a1210 con 0x7fded807dd20 2024-04-25T00:07:27.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.172+0000 7fdecfe00700 1 -- 172.21.15.42:0/681721131 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdec801ac50 con 0x7fded807dd20 2024-04-25T00:07:27.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.172+0000 7fdecfe00700 1 -- 172.21.15.42:0/681721131 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fdec8019700 con 0x7fded807dd20 2024-04-25T00:07:27.968 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.173+0000 7fded7e00700 1 -- 172.21.15.42:0/681721131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fded8063890 con 0x7fded807dd20 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.175+0000 7fdecfe00700 1 -- 172.21.15.42:0/681721131 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fdec8003b60 con 0x7fded807dd20 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.256+0000 7fded7e00700 1 -- 172.21.15.42:0/681721131 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fded81a17e0 con 0x7fded807dd20 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.256+0000 7fdecfe00700 1 -- 172.21.15.42:0/681721131 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fdec80199a0 con 0x7fded807dd20 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.257+0000 7fded7e00700 1 -- 172.21.15.42:0/681721131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 msgr2=0x7fded819dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.257+0000 7fded7e00700 1 --2- 172.21.15.42:0/681721131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded819dd50 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fdec8011670 tx=0x7fdec8011b60 comp rx=0 tx=0).stop 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.258+0000 7fded7e00700 1 -- 172.21.15.42:0/681721131 shutdown_connections 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.258+0000 7fded7e00700 1 --2- 172.21.15.42:0/681721131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fded807dd20 0x7fded819dd50 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.258+0000 7fded7e00700 1 -- 172.21.15.42:0/681721131 >> 172.21.15.42:0/681721131 conn(0x7fded8103250 msgr2=0x7fded8103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.258+0000 7fded7e00700 1 -- 172.21.15.42:0/681721131 shutdown_connections 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:27.258+0000 7fded7e00700 1 -- 172.21.15.42:0/681721131 wait complete. 2024-04-25T00:07:27.969 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (1/15)... 2024-04-25T00:07:29.626 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:29 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:29.200+0000 7f0285571200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:07:29.626 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:29 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:29.433+0000 7f0285571200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:07:30.049 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:29 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:29.649+0000 7f0285571200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:07:30.049 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:30 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:30.048+0000 7f0285571200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:07:30.309 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:30 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:30.174+0000 7f0285571200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:07:30.562 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:30 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:30.437+0000 7f0285571200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:07:30.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:30 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/173946333' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:07:30.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:30 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:30.698+0000 7f0285571200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:07:31.626 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:31 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:31.532+0000 7f0285571200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:07:31.746 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:31.746 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:07:31.746 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "9d95543e-0297-11ef-bc93-c7b262605968", 2024-04-25T00:07:31.746 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:07:31.746 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:07:31.746 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:07:31.746 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:07:31.746 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:07:31.747 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:07:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:07:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:07:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:07:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:07:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:07:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:07:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:31.748 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:07:31.749 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:07:31.749 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:07:31.749 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:07:31.749 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:07:31.749 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:07:31.749 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:07:31.749 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:07:31.749 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:07:31.749 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:07:31.750 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:07:16.360238+0000", 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.638+0000 7f72a5400700 1 Processor -- start 2024-04-25T00:07:31.751 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.638+0000 7f72a5400700 1 -- start start 2024-04-25T00:07:31.752 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.638+0000 7f72a5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a007c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:31.752 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.638+0000 7f72a5400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72a007e100 con 0x7f72a007dce0 2024-04-25T00:07:31.752 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.639+0000 7f729fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a007c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:31.752 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.639+0000 7f729fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a007c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58036/0 (socket says 172.21.15.42:58036) 2024-04-25T00:07:31.752 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.639+0000 7f729fe00700 1 -- 172.21.15.42:0/2611985655 learned_addr learned my addr 172.21.15.42:0/2611985655 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:31.752 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.639+0000 7f729fe00700 1 -- 172.21.15.42:0/2611985655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a007c7e0 con 0x7f72a007dce0 2024-04-25T00:07:31.753 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.640+0000 7f729fe00700 1 --2- 172.21.15.42:0/2611985655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a007c2a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f72900098a0 tx=0x7f7290009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f2672e18e0c19f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:31.753 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.640+0000 7f729ea00700 1 -- 172.21.15.42:0/2611985655 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7290004160 con 0x7f72a007dce0 2024-04-25T00:07:31.753 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.640+0000 7f729ea00700 1 -- 172.21.15.42:0/2611985655 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7290004300 con 0x7f72a007dce0 2024-04-25T00:07:31.753 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.640+0000 7f729ea00700 1 -- 172.21.15.42:0/2611985655 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7290011b50 con 0x7f72a007dce0 2024-04-25T00:07:31.753 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.641+0000 7f72a5400700 1 -- 172.21.15.42:0/2611985655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 msgr2=0x7f72a007c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:31.753 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.641+0000 7f72a5400700 1 --2- 172.21.15.42:0/2611985655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a007c2a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f72900098a0 tx=0x7f7290009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:31.753 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.641+0000 7f72a5400700 1 -- 172.21.15.42:0/2611985655 shutdown_connections 2024-04-25T00:07:31.753 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.641+0000 7f72a5400700 1 --2- 172.21.15.42:0/2611985655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a007c2a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:31.754 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.641+0000 7f72a5400700 1 -- 172.21.15.42:0/2611985655 >> 172.21.15.42:0/2611985655 conn(0x7f72a0103250 msgr2=0x7f72a0105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:31.754 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.641+0000 7f72a5400700 1 -- 172.21.15.42:0/2611985655 shutdown_connections 2024-04-25T00:07:31.754 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.641+0000 7f72a5400700 1 -- 172.21.15.42:0/2611985655 wait complete. 2024-04-25T00:07:31.754 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.642+0000 7f72a5400700 1 Processor -- start 2024-04-25T00:07:31.754 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.642+0000 7f72a5400700 1 -- start start 2024-04-25T00:07:31.754 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.642+0000 7f72a5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a019dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:31.754 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.642+0000 7f72a5400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72a010e0a0 con 0x7f72a007dce0 2024-04-25T00:07:31.754 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.643+0000 7f729fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a019dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:31.754 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.643+0000 7f729fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a019dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58048/0 (socket says 172.21.15.42:58048) 2024-04-25T00:07:31.754 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.643+0000 7f729fe00700 1 -- 172.21.15.42:0/173946333 learned_addr learned my addr 172.21.15.42:0/173946333 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.643+0000 7f729fe00700 1 -- 172.21.15.42:0/173946333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a019e290 con 0x7f72a007dce0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.643+0000 7f729fe00700 1 --2- 172.21.15.42:0/173946333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a019dd50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7290011b60 tx=0x7f7290011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.644+0000 7f729cc00700 1 -- 172.21.15.42:0/173946333 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7290003b20 con 0x7f72a007dce0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.644+0000 7f729cc00700 1 -- 172.21.15.42:0/173946333 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7290003cc0 con 0x7f72a007dce0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.644+0000 7f729cc00700 1 -- 172.21.15.42:0/173946333 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7290024840 con 0x7f72a007dce0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.644+0000 7f72a5400700 1 -- 172.21.15.42:0/173946333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a019e520 con 0x7f72a007dce0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.644+0000 7f72a5400700 1 -- 172.21.15.42:0/173946333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a01a1210 con 0x7f72a007dce0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.644+0000 7f729cc00700 1 -- 172.21.15.42:0/173946333 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7290022020 con 0x7f72a007dce0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.644+0000 7f729cc00700 1 -- 172.21.15.42:0/173946333 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7290011e00 con 0x7f72a007dce0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.645+0000 7f72a5400700 1 -- 172.21.15.42:0/173946333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a0063890 con 0x7f72a007dce0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.648+0000 7f729cc00700 1 -- 172.21.15.42:0/173946333 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7290043dd0 con 0x7f72a007dce0 2024-04-25T00:07:31.755 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.727+0000 7f72a5400700 1 -- 172.21.15.42:0/173946333 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f72a01a17e0 con 0x7f72a007dce0 2024-04-25T00:07:31.756 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.728+0000 7f729cc00700 1 -- 172.21.15.42:0/173946333 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7290032030 con 0x7f72a007dce0 2024-04-25T00:07:31.756 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.729+0000 7f72a5400700 1 -- 172.21.15.42:0/173946333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 msgr2=0x7f72a019dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:31.756 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.729+0000 7f72a5400700 1 --2- 172.21.15.42:0/173946333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a019dd50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7290011b60 tx=0x7f7290011c40 comp rx=0 tx=0).stop 2024-04-25T00:07:31.756 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.729+0000 7f72a5400700 1 -- 172.21.15.42:0/173946333 shutdown_connections 2024-04-25T00:07:31.756 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.729+0000 7f72a5400700 1 --2- 172.21.15.42:0/173946333 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f72a007dce0 0x7f72a019dd50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:31.756 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.729+0000 7f72a5400700 1 -- 172.21.15.42:0/173946333 >> 172.21.15.42:0/173946333 conn(0x7f72a0103250 msgr2=0x7f72a0103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:31.756 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.729+0000 7f72a5400700 1 -- 172.21.15.42:0/173946333 shutdown_connections 2024-04-25T00:07:31.756 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:30.730+0000 7f72a5400700 1 -- 172.21.15.42:0/173946333 wait complete. 2024-04-25T00:07:31.756 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (2/15)... 2024-04-25T00:07:32.378 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:32 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:32.106+0000 7f0285571200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:07:32.378 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:32 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:32.237+0000 7f0285571200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:07:32.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:32 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:32.506+0000 7f0285571200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:07:33.619 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:33.168+0000 7f0285571200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:07:33.619 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:33.293+0000 7f0285571200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:07:33.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:33.618+0000 7f0285571200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:07:33.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:33.807+0000 7f0285571200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:07:34.155 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:33.943+0000 7f0285571200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:07:34.155 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:34 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:34.101+0000 7f0285571200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:07:34.476 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:34 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:34.228+0000 7f0285571200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:07:34.837 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:34 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/560874603' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:07:35.144 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:35.144 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:07:35.144 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "9d95543e-0297-11ef-bc93-c7b262605968", 2024-04-25T00:07:35.144 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:35.145 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:07:35.146 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:07:35.146 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:35.146 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:07:35.146 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:35.146 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:07:35.146 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:07:35.146 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:07:35.154 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:07:35.155 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:07:35.156 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:07:16.360238+0000", 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.385+0000 7fca24c00700 1 Processor -- start 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.385+0000 7fca24c00700 1 -- start start 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.385+0000 7fca24c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.385+0000 7fca24c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fca2007c820 con 0x7fca2007dd70 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.386+0000 7fca1de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.386+0000 7fca1de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58054/0 (socket says 172.21.15.42:58054) 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.386+0000 7fca1de00700 1 -- 172.21.15.42:0/4139821765 learned_addr learned my addr 172.21.15.42:0/4139821765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:35.157 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.386+0000 7fca1de00700 1 -- 172.21.15.42:0/4139821765 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2007c9a0 con 0x7fca2007dd70 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.387+0000 7fca1de00700 1 --2- 172.21.15.42:0/4139821765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2007c2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca10009e80 tx=0x7fca1000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0adda8ba919e4f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.387+0000 7fca1ca00700 1 -- 172.21.15.42:0/4139821765 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca10017070 con 0x7fca2007dd70 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.387+0000 7fca1ca00700 1 -- 172.21.15.42:0/4139821765 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fca1000b8c0 con 0x7fca2007dd70 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.387+0000 7fca1ca00700 1 -- 172.21.15.42:0/4139821765 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca100045c0 con 0x7fca2007dd70 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.388+0000 7fca24c00700 1 -- 172.21.15.42:0/4139821765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 msgr2=0x7fca2007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.388+0000 7fca24c00700 1 --2- 172.21.15.42:0/4139821765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2007c2e0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca10009e80 tx=0x7fca1000b330 comp rx=0 tx=0).stop 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.388+0000 7fca24c00700 1 -- 172.21.15.42:0/4139821765 shutdown_connections 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.388+0000 7fca24c00700 1 --2- 172.21.15.42:0/4139821765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2007c2e0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.388+0000 7fca24c00700 1 -- 172.21.15.42:0/4139821765 >> 172.21.15.42:0/4139821765 conn(0x7fca20103460 msgr2=0x7fca20105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.388+0000 7fca24c00700 1 -- 172.21.15.42:0/4139821765 shutdown_connections 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.388+0000 7fca24c00700 1 -- 172.21.15.42:0/4139821765 wait complete. 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.389+0000 7fca24c00700 1 Processor -- start 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.389+0000 7fca24c00700 1 -- start start 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.390+0000 7fca24c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:35.158 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.390+0000 7fca24c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fca2010e2b0 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.390+0000 7fca1de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.390+0000 7fca1de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2019dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58070/0 (socket says 172.21.15.42:58070) 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.390+0000 7fca1de00700 1 -- 172.21.15.42:0/560874603 learned_addr learned my addr 172.21.15.42:0/560874603 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.390+0000 7fca1de00700 1 -- 172.21.15.42:0/560874603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2019e400 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.391+0000 7fca1de00700 1 --2- 172.21.15.42:0/560874603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2019dec0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fca10004350 tx=0x7fca1000bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.391+0000 7fca16a00700 1 -- 172.21.15.42:0/560874603 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca10020020 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.391+0000 7fca16a00700 1 -- 172.21.15.42:0/560874603 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fca10003c10 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.391+0000 7fca24c00700 1 -- 172.21.15.42:0/560874603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2019e690 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.391+0000 7fca16a00700 1 -- 172.21.15.42:0/560874603 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca10017040 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.391+0000 7fca24c00700 1 -- 172.21.15.42:0/560874603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca201a1380 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.391+0000 7fca16a00700 1 -- 172.21.15.42:0/560874603 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fca10023020 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.391+0000 7fca16a00700 1 -- 172.21.15.42:0/560874603 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fca1000f050 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.392+0000 7fca24c00700 1 -- 172.21.15.42:0/560874603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca20063890 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.395+0000 7fca16a00700 1 -- 172.21.15.42:0/560874603 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fca1002a020 con 0x7fca2007dd70 2024-04-25T00:07:35.159 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.475+0000 7fca24c00700 1 -- 172.21.15.42:0/560874603 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fca201a1550 con 0x7fca2007dd70 2024-04-25T00:07:35.160 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.476+0000 7fca16a00700 1 -- 172.21.15.42:0/560874603 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7fca1001ccd0 con 0x7fca2007dd70 2024-04-25T00:07:35.160 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.478+0000 7fca24c00700 1 -- 172.21.15.42:0/560874603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 msgr2=0x7fca2019dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:35.160 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.478+0000 7fca24c00700 1 --2- 172.21.15.42:0/560874603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2019dec0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fca10004350 tx=0x7fca1000bd50 comp rx=0 tx=0).stop 2024-04-25T00:07:35.160 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.478+0000 7fca24c00700 1 -- 172.21.15.42:0/560874603 shutdown_connections 2024-04-25T00:07:35.160 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.478+0000 7fca24c00700 1 --2- 172.21.15.42:0/560874603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fca2007dd70 0x7fca2019dec0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:35.161 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.478+0000 7fca24c00700 1 -- 172.21.15.42:0/560874603 >> 172.21.15.42:0/560874603 conn(0x7fca20103460 msgr2=0x7fca20103f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:35.161 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.478+0000 7fca24c00700 1 -- 172.21.15.42:0/560874603 shutdown_connections 2024-04-25T00:07:35.161 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:34.478+0000 7fca24c00700 1 -- 172.21.15.42:0/560874603 wait complete. 2024-04-25T00:07:35.161 INFO:teuthology.orchestra.run.smithi042.stdout:mgr not available, waiting (3/15)... 2024-04-25T00:07:35.571 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:35.285+0000 7f0285571200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:07:35.572 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:35.571+0000 7f0285571200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:07:35.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: Activating manager daemon x 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: mgrmap e2: x(active, starting, since 0.0108896s) 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: from='mgr.14100 172.21.15.42:0/2656804720' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: from='mgr.14100 172.21.15.42:0/2656804720' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: from='mgr.14100 172.21.15.42:0/2656804720' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: from='mgr.14100 172.21.15.42:0/2656804720' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: from='mgr.14100 172.21.15.42:0/2656804720' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: Manager daemon x is now available 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: from='mgr.14100 172.21.15.42:0/2656804720' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: from='mgr.14100 172.21.15.42:0/2656804720' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: from='mgr.14100 172.21.15.42:0/2656804720' entity='mgr.x' 2024-04-25T00:07:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: from='mgr.14100 172.21.15.42:0/2656804720' entity='mgr.x' 2024-04-25T00:07:35.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:35 smithi042 ceph-mon[28130]: from='mgr.14100 172.21.15.42:0/2656804720' entity='mgr.x' 2024-04-25T00:07:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:37 smithi042 ceph-mon[28130]: mgrmap e3: x(active, since 1.02105s) 2024-04-25T00:07:38.745 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:38 smithi042 ceph-mon[28130]: mgrmap e4: x(active, since 2s) 2024-04-25T00:07:38.745 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:38 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2267331597' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T00:07:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:07:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsid": "9d95543e-0297-11ef-bc93-c7b262605968", 2024-04-25T00:07:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T00:07:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T00:07:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T00:07:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T00:07:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T00:07:38.758 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 0 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:38.759 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T00:07:38.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:38.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T00:07:38.760 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T00:07:38.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T00:07:38.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T00:07:38.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T00:07:38.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T00:07:38.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:38.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T00:07:38.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T00:07:38.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T00:07:38.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T00:07:38.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T00:07:38.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T00:07:38.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T00:07:38.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T00:07:38.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T00:07:38.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T00:07:38.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ], 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T00:07:16.360238+0000", 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout }, 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.795+0000 7fd771e00700 1 Processor -- start 2024-04-25T00:07:38.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.795+0000 7fd771e00700 1 -- start start 2024-04-25T00:07:38.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.795+0000 7fd771e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c10a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:38.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.795+0000 7fd771e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c10a810 con 0x7fd76c109e20 2024-04-25T00:07:38.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.795+0000 7fd770a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c10a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:38.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.796+0000 7fd770a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c10a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51030/0 (socket says 172.21.15.42:51030) 2024-04-25T00:07:38.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.796+0000 7fd770a00700 1 -- 172.21.15.42:0/1628686550 learned_addr learned my addr 172.21.15.42:0/1628686550 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:38.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.796+0000 7fd770a00700 1 -- 172.21.15.42:0/1628686550 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c10afd0 con 0x7fd76c109e20 2024-04-25T00:07:38.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.796+0000 7fd770a00700 1 --2- 172.21.15.42:0/1628686550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c10a240 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7600098a0 tx=0x7fd760009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5af6dfa0a0023f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:38.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.796+0000 7fd76b400700 1 -- 172.21.15.42:0/1628686550 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd760004160 con 0x7fd76c109e20 2024-04-25T00:07:38.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.797+0000 7fd76b400700 1 -- 172.21.15.42:0/1628686550 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd760004300 con 0x7fd76c109e20 2024-04-25T00:07:38.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.797+0000 7fd76b400700 1 -- 172.21.15.42:0/1628686550 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd760011b50 con 0x7fd76c109e20 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.797+0000 7fd771e00700 1 -- 172.21.15.42:0/1628686550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 msgr2=0x7fd76c10a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.797+0000 7fd771e00700 1 --2- 172.21.15.42:0/1628686550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c10a240 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7600098a0 tx=0x7fd760009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.797+0000 7fd771e00700 1 -- 172.21.15.42:0/1628686550 shutdown_connections 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.797+0000 7fd771e00700 1 --2- 172.21.15.42:0/1628686550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c10a240 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.797+0000 7fd771e00700 1 -- 172.21.15.42:0/1628686550 >> 172.21.15.42:0/1628686550 conn(0x7fd76c07d240 msgr2=0x7fd76c07d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.797+0000 7fd771e00700 1 -- 172.21.15.42:0/1628686550 shutdown_connections 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.797+0000 7fd771e00700 1 -- 172.21.15.42:0/1628686550 wait complete. 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.798+0000 7fd771e00700 1 Processor -- start 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.799+0000 7fd771e00700 1 -- start start 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.799+0000 7fd771e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:38.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.799+0000 7fd771e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd76c10e380 con 0x7fd76c109e20 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.799+0000 7fd770a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.799+0000 7fd770a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c19de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51038/0 (socket says 172.21.15.42:51038) 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.799+0000 7fd770a00700 1 -- 172.21.15.42:0/2267331597 learned_addr learned my addr 172.21.15.42:0/2267331597 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.800+0000 7fd770a00700 1 -- 172.21.15.42:0/2267331597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd76c19e350 con 0x7fd76c109e20 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.800+0000 7fd770a00700 1 --2- 172.21.15.42:0/2267331597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c19de10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd760009fb0 tx=0x7fd760011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.800+0000 7fd769600700 1 -- 172.21.15.42:0/2267331597 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7600038b0 con 0x7fd76c109e20 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.800+0000 7fd769600700 1 -- 172.21.15.42:0/2267331597 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd760011e50 con 0x7fd76c109e20 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.800+0000 7fd771e00700 1 -- 172.21.15.42:0/2267331597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd76c19e5e0 con 0x7fd76c109e20 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.800+0000 7fd769600700 1 -- 172.21.15.42:0/2267331597 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd760019550 con 0x7fd76c109e20 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.800+0000 7fd771e00700 1 -- 172.21.15.42:0/2267331597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd76c1a12d0 con 0x7fd76c109e20 2024-04-25T00:07:38.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.801+0000 7fd769600700 1 -- 172.21.15.42:0/2267331597 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fd76001a430 con 0x7fd76c109e20 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.802+0000 7fd771e00700 1 -- 172.21.15.42:0/2267331597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd76c063890 con 0x7fd76c109e20 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.802+0000 7fd769600700 1 --2- 172.21.15.42:0/2267331597 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7fd754039000 0x7fd75403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.802+0000 7fd769600700 1 -- 172.21.15.42:0/2267331597 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd76004d1e0 con 0x7fd76c109e20 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.802+0000 7fd76be00700 1 --2- 172.21.15.42:0/2267331597 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7fd754039000 0x7fd75403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.803+0000 7fd76be00700 1 --2- 172.21.15.42:0/2267331597 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7fd754039000 0x7fd75403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd758006fd0 tx=0x7fd758006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:37.809+0000 7fd769600700 1 -- 172.21.15.42:0/2267331597 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd76004cc40 con 0x7fd76c109e20 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.124+0000 7fd771e00700 1 -- 172.21.15.42:0/2267331597 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd76c1a14a0 con 0x7fd76c109e20 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.125+0000 7fd769600700 1 -- 172.21.15.42:0/2267331597 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fd76c1a14a0 con 0x7fd76c109e20 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.128+0000 7fd771e00700 1 -- 172.21.15.42:0/2267331597 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7fd754039000 msgr2=0x7fd75403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.128+0000 7fd771e00700 1 --2- 172.21.15.42:0/2267331597 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7fd754039000 0x7fd75403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd758006fd0 tx=0x7fd758006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.128+0000 7fd771e00700 1 -- 172.21.15.42:0/2267331597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 msgr2=0x7fd76c19de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:38.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.128+0000 7fd771e00700 1 --2- 172.21.15.42:0/2267331597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c19de10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd760009fb0 tx=0x7fd760011bf0 comp rx=0 tx=0).stop 2024-04-25T00:07:38.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.128+0000 7fd771e00700 1 -- 172.21.15.42:0/2267331597 shutdown_connections 2024-04-25T00:07:38.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.128+0000 7fd771e00700 1 --2- 172.21.15.42:0/2267331597 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7fd754039000 0x7fd75403b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:38.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.128+0000 7fd771e00700 1 --2- 172.21.15.42:0/2267331597 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd76c109e20 0x7fd76c19de10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:38.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.128+0000 7fd771e00700 1 -- 172.21.15.42:0/2267331597 >> 172.21.15.42:0/2267331597 conn(0x7fd76c07d240 msgr2=0x7fd76c1075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:38.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.128+0000 7fd771e00700 1 -- 172.21.15.42:0/2267331597 shutdown_connections 2024-04-25T00:07:38.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:38.128+0000 7fd771e00700 1 -- 172.21.15.42:0/2267331597 wait complete. 2024-04-25T00:07:38.778 INFO:teuthology.orchestra.run.smithi042.stdout:mgr is available 2024-04-25T00:07:40.005 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:39 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2486481105' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T00:07:40.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:40.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T00:07:40.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout fsid = 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:40.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T00:07:40.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T00:07:40.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T00:07:40.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T00:07:40.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:40.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.365+0000 7f38eba00700 1 Processor -- start 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.365+0000 7f38eba00700 1 -- start start 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.366+0000 7f38eba00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e410acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.366+0000 7f38eba00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f38e407bdd0 con 0x7f38e4108900 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.366+0000 7f38e9000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e410acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:40.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.366+0000 7f38e9000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e410acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51050/0 (socket says 172.21.15.42:51050) 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.366+0000 7f38e9000700 1 -- 172.21.15.42:0/1237959647 learned_addr learned my addr 172.21.15.42:0/1237959647 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.367+0000 7f38e9000700 1 -- 172.21.15.42:0/1237959647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e410b230 con 0x7f38e4108900 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.367+0000 7f38e9000700 1 --2- 172.21.15.42:0/1237959647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e410acf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f38d80098a0 tx=0x7f38d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e5351e2870b1ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.367+0000 7f38e3400700 1 -- 172.21.15.42:0/1237959647 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d8004160 con 0x7f38e4108900 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.367+0000 7f38e3400700 1 -- 172.21.15.42:0/1237959647 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38d8004300 con 0x7f38e4108900 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.368+0000 7f38e3400700 1 -- 172.21.15.42:0/1237959647 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d8011b50 con 0x7f38e4108900 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.368+0000 7f38eba00700 1 -- 172.21.15.42:0/1237959647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 msgr2=0x7f38e410acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.368+0000 7f38eba00700 1 --2- 172.21.15.42:0/1237959647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e410acf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f38d80098a0 tx=0x7f38d8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.368+0000 7f38eba00700 1 -- 172.21.15.42:0/1237959647 shutdown_connections 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.368+0000 7f38eba00700 1 --2- 172.21.15.42:0/1237959647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e410acf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.368+0000 7f38eba00700 1 -- 172.21.15.42:0/1237959647 >> 172.21.15.42:0/1237959647 conn(0x7f38e41022b0 msgr2=0x7f38e41046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.368+0000 7f38eba00700 1 -- 172.21.15.42:0/1237959647 shutdown_connections 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.368+0000 7f38eba00700 1 -- 172.21.15.42:0/1237959647 wait complete. 2024-04-25T00:07:40.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.369+0000 7f38eba00700 1 Processor -- start 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.370+0000 7f38eba00700 1 -- start start 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.370+0000 7f38eba00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e419dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.370+0000 7f38eba00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f38e410c1a0 con 0x7f38e4108900 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.370+0000 7f38e9000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e419dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.370+0000 7f38e9000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e419dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51058/0 (socket says 172.21.15.42:51058) 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.370+0000 7f38e9000700 1 -- 172.21.15.42:0/2486481105 learned_addr learned my addr 172.21.15.42:0/2486481105 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.371+0000 7f38e9000700 1 -- 172.21.15.42:0/2486481105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e419e4f0 con 0x7f38e4108900 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.371+0000 7f38e9000700 1 --2- 172.21.15.42:0/2486481105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e419dfb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f38d8011b60 tx=0x7f38d8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.371+0000 7f38e1600700 1 -- 172.21.15.42:0/2486481105 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d8003b20 con 0x7f38e4108900 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.371+0000 7f38e1600700 1 -- 172.21.15.42:0/2486481105 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38d8003cc0 con 0x7f38e4108900 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.371+0000 7f38e1600700 1 -- 172.21.15.42:0/2486481105 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38d8024840 con 0x7f38e4108900 2024-04-25T00:07:40.284 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.371+0000 7f38eba00700 1 -- 172.21.15.42:0/2486481105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e419e780 con 0x7f38e4108900 2024-04-25T00:07:40.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.371+0000 7f38eba00700 1 -- 172.21.15.42:0/2486481105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e41a1420 con 0x7f38e4108900 2024-04-25T00:07:40.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.372+0000 7f38e1600700 1 -- 172.21.15.42:0/2486481105 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f38d8022020 con 0x7f38e4108900 2024-04-25T00:07:40.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.373+0000 7f38eba00700 1 -- 172.21.15.42:0/2486481105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e4063890 con 0x7f38e4108900 2024-04-25T00:07:40.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.373+0000 7f38e1600700 1 --2- 172.21.15.42:0/2486481105 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f38cc039050 0x7f38cc03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:40.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.373+0000 7f38e1600700 1 -- 172.21.15.42:0/2486481105 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f38d8011e00 con 0x7f38e4108900 2024-04-25T00:07:40.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.373+0000 7f38e3e00700 1 --2- 172.21.15.42:0/2486481105 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f38cc039050 0x7f38cc03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:40.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.374+0000 7f38e3e00700 1 --2- 172.21.15.42:0/2486481105 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f38cc039050 0x7f38cc03b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38d0006fd0 tx=0x7f38d0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:40.285 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.378+0000 7f38e1600700 1 -- 172.21.15.42:0/2486481105 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f38d800b3e0 con 0x7f38e4108900 2024-04-25T00:07:40.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.607+0000 7f38eba00700 1 -- 172.21.15.42:0/2486481105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f38e41a15f0 con 0x7f38e4108900 2024-04-25T00:07:40.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.609+0000 7f38e1600700 1 -- 172.21.15.42:0/2486481105 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f38e41a15f0 con 0x7f38e4108900 2024-04-25T00:07:40.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.611+0000 7f38eba00700 1 -- 172.21.15.42:0/2486481105 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f38cc039050 msgr2=0x7f38cc03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:40.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.611+0000 7f38eba00700 1 --2- 172.21.15.42:0/2486481105 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f38cc039050 0x7f38cc03b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f38d0006fd0 tx=0x7f38d0006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:40.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.611+0000 7f38eba00700 1 -- 172.21.15.42:0/2486481105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 msgr2=0x7f38e419dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:40.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.611+0000 7f38eba00700 1 --2- 172.21.15.42:0/2486481105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e419dfb0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f38d8011b60 tx=0x7f38d8011c40 comp rx=0 tx=0).stop 2024-04-25T00:07:40.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.611+0000 7f38eba00700 1 -- 172.21.15.42:0/2486481105 shutdown_connections 2024-04-25T00:07:40.286 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.612+0000 7f38eba00700 1 --2- 172.21.15.42:0/2486481105 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f38cc039050 0x7f38cc03b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:40.287 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.612+0000 7f38eba00700 1 --2- 172.21.15.42:0/2486481105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f38e4108900 0x7f38e419dfb0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:40.287 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.612+0000 7f38eba00700 1 -- 172.21.15.42:0/2486481105 >> 172.21.15.42:0/2486481105 conn(0x7f38e41022b0 msgr2=0x7f38e4102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:40.287 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.612+0000 7f38eba00700 1 -- 172.21.15.42:0/2486481105 shutdown_connections 2024-04-25T00:07:40.287 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:39.612+0000 7f38eba00700 1 -- 172.21.15.42:0/2486481105 wait complete. 2024-04-25T00:07:40.287 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling cephadm module... 2024-04-25T00:07:41.619 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:41 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/631661577' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T00:07:41.619 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:41 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: ignoring --setuser ceph since I am not root 2024-04-25T00:07:41.619 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:41 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: ignoring --setgroup ceph since I am not root 2024-04-25T00:07:41.619 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:41 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:41.618+0000 7fb2af20d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:07:42.114 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.959+0000 7f2d23800700 1 Processor -- start 2024-04-25T00:07:42.114 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.959+0000 7f2d23800700 1 -- start start 2024-04-25T00:07:42.114 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.959+0000 7f2d23800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c10a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:42.114 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.959+0000 7f2d23800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c10a9e0 con 0x7f2d1c109ff0 2024-04-25T00:07:42.114 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.960+0000 7f2d20e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c10a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:42.114 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.960+0000 7f2d20e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c10a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51062/0 (socket says 172.21.15.42:51062) 2024-04-25T00:07:42.114 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.960+0000 7f2d20e00700 1 -- 172.21.15.42:0/4154927585 learned_addr learned my addr 172.21.15.42:0/4154927585 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:42.115 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.960+0000 7f2d20e00700 1 -- 172.21.15.42:0/4154927585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c10b1a0 con 0x7f2d1c109ff0 2024-04-25T00:07:42.115 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.961+0000 7f2d20e00700 1 --2- 172.21.15.42:0/4154927585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c10a410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0098a0 tx=0x7f2d0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54360351fb85990 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:42.115 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.961+0000 7f2d1be00700 1 -- 172.21.15.42:0/4154927585 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d0c004160 con 0x7f2d1c109ff0 2024-04-25T00:07:42.115 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.961+0000 7f2d1be00700 1 -- 172.21.15.42:0/4154927585 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d0c004300 con 0x7f2d1c109ff0 2024-04-25T00:07:42.115 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.961+0000 7f2d1be00700 1 -- 172.21.15.42:0/4154927585 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d0c011b50 con 0x7f2d1c109ff0 2024-04-25T00:07:42.115 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.962+0000 7f2d23800700 1 -- 172.21.15.42:0/4154927585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 msgr2=0x7f2d1c10a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:42.115 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.962+0000 7f2d23800700 1 --2- 172.21.15.42:0/4154927585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c10a410 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c0098a0 tx=0x7f2d0c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:42.115 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.962+0000 7f2d23800700 1 -- 172.21.15.42:0/4154927585 shutdown_connections 2024-04-25T00:07:42.115 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.962+0000 7f2d23800700 1 --2- 172.21.15.42:0/4154927585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c10a410 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:42.116 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.962+0000 7f2d23800700 1 -- 172.21.15.42:0/4154927585 >> 172.21.15.42:0/4154927585 conn(0x7f2d1c07d3b0 msgr2=0x7f2d1c07d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:42.116 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.962+0000 7f2d23800700 1 -- 172.21.15.42:0/4154927585 shutdown_connections 2024-04-25T00:07:42.116 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.962+0000 7f2d23800700 1 -- 172.21.15.42:0/4154927585 wait complete. 2024-04-25T00:07:42.116 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.963+0000 7f2d23800700 1 Processor -- start 2024-04-25T00:07:42.116 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.963+0000 7f2d23800700 1 -- start start 2024-04-25T00:07:42.116 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.963+0000 7f2d23800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c19dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:42.116 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.963+0000 7f2d23800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2d1c10e550 con 0x7f2d1c109ff0 2024-04-25T00:07:42.117 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.964+0000 7f2d20e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c19dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:42.117 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.964+0000 7f2d20e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c19dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51066/0 (socket says 172.21.15.42:51066) 2024-04-25T00:07:42.117 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.964+0000 7f2d20e00700 1 -- 172.21.15.42:0/631661577 learned_addr learned my addr 172.21.15.42:0/631661577 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:42.117 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.964+0000 7f2d20e00700 1 -- 172.21.15.42:0/631661577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d1c19e500 con 0x7f2d1c109ff0 2024-04-25T00:07:42.117 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.964+0000 7f2d20e00700 1 --2- 172.21.15.42:0/631661577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c19dfc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c011850 tx=0x7f2d0c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:42.117 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.965+0000 7f2d1a000700 1 -- 172.21.15.42:0/631661577 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d0c003af0 con 0x7f2d1c109ff0 2024-04-25T00:07:42.117 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.965+0000 7f2d1a000700 1 -- 172.21.15.42:0/631661577 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2d0c020070 con 0x7f2d1c109ff0 2024-04-25T00:07:42.117 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.965+0000 7f2d1a000700 1 -- 172.21.15.42:0/631661577 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d0c0195a0 con 0x7f2d1c109ff0 2024-04-25T00:07:42.118 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.965+0000 7f2d23800700 1 -- 172.21.15.42:0/631661577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d1c19e790 con 0x7f2d1c109ff0 2024-04-25T00:07:42.118 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.965+0000 7f2d23800700 1 -- 172.21.15.42:0/631661577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d1c1a1480 con 0x7f2d1c109ff0 2024-04-25T00:07:42.118 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.966+0000 7f2d23800700 1 -- 172.21.15.42:0/631661577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d1c063890 con 0x7f2d1c109ff0 2024-04-25T00:07:42.118 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.966+0000 7f2d1a000700 1 -- 172.21.15.42:0/631661577 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f2d0c019850 con 0x7f2d1c109ff0 2024-04-25T00:07:42.118 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.967+0000 7f2d1a000700 1 --2- 172.21.15.42:0/631661577 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f2d00039050 0x7f2d0003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:42.118 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.967+0000 7f2d1a000700 1 -- 172.21.15.42:0/631661577 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2d0c011e00 con 0x7f2d1c109ff0 2024-04-25T00:07:42.118 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.967+0000 7f2d13e00700 1 --2- 172.21.15.42:0/631661577 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f2d00039050 0x7f2d0003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:42.118 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.968+0000 7f2d13e00700 1 --2- 172.21.15.42:0/631661577 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f2d00039050 0x7f2d0003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d04006fd0 tx=0x7f2d04006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:42.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:40.972+0000 7f2d1a000700 1 -- 172.21.15.42:0/631661577 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2d0c00e260 con 0x7f2d1c109ff0 2024-04-25T00:07:42.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.265+0000 7f2d23800700 1 -- 172.21.15.42:0/631661577 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2d1c1a1650 con 0x7f2d1c109ff0 2024-04-25T00:07:42.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.319+0000 7f2d1a000700 1 -- 172.21.15.42:0/631661577 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2d1c1a1650 con 0x7f2d1c109ff0 2024-04-25T00:07:42.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.323+0000 7f2d23800700 1 -- 172.21.15.42:0/631661577 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f2d00039050 msgr2=0x7f2d0003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:42.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.323+0000 7f2d23800700 1 --2- 172.21.15.42:0/631661577 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f2d00039050 0x7f2d0003b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2d04006fd0 tx=0x7f2d04006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:42.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.323+0000 7f2d23800700 1 -- 172.21.15.42:0/631661577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 msgr2=0x7f2d1c19dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:42.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.324+0000 7f2d1a000700 1 -- 172.21.15.42:0/631661577 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f2d0c01a430 con 0x7f2d1c109ff0 2024-04-25T00:07:42.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.324+0000 7f2d23800700 1 --2- 172.21.15.42:0/631661577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c19dfc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c011850 tx=0x7f2d0c011c10 comp rx=0 tx=0).stop 2024-04-25T00:07:42.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.324+0000 7f2d23800700 1 -- 172.21.15.42:0/631661577 shutdown_connections 2024-04-25T00:07:42.119 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.324+0000 7f2d23800700 1 --2- 172.21.15.42:0/631661577 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f2d00039050 0x7f2d0003b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:42.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.324+0000 7f2d23800700 1 --2- 172.21.15.42:0/631661577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2d1c109ff0 0x7f2d1c19dfc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:42.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.324+0000 7f2d23800700 1 -- 172.21.15.42:0/631661577 >> 172.21.15.42:0/631661577 conn(0x7f2d1c07d3b0 msgr2=0x7f2d1c1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:42.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.324+0000 7f2d23800700 1 -- 172.21.15.42:0/631661577 shutdown_connections 2024-04-25T00:07:42.120 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:41.324+0000 7f2d23800700 1 -- 172.21.15.42:0/631661577 wait complete. 2024-04-25T00:07:42.266 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:41 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:41.880+0000 7fb2af20d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:07:42.266 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:42 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:42.018+0000 7fb2af20d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:07:42.558 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:42 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/631661577' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T00:07:42.558 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:42 smithi042 ceph-mon[28130]: mgrmap e5: x(active, since 5s) 2024-04-25T00:07:42.558 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:42 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:42.439+0000 7fb2af20d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:07:42.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:42 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:42.557+0000 7fb2af20d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:07:43.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:43 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3367035777' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T00:07:43.735 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:07:43.735 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T00:07:43.735 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:07:43.735 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T00:07:43.735 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T00:07:43.735 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:07:43.735 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.717+0000 7f701fe00700 1 Processor -- start 2024-04-25T00:07:43.735 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.717+0000 7f701fe00700 1 -- start start 2024-04-25T00:07:43.735 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.718+0000 7f701fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f7020108060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.718+0000 7f701fe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f70201085a0 con 0x7f7020107c40 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.718+0000 7f701ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f7020108060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.718+0000 7f701ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f7020108060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51090/0 (socket says 172.21.15.42:51090) 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.718+0000 7f701ea00700 1 -- 172.21.15.42:0/957366588 learned_addr learned my addr 172.21.15.42:0/957366588 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.719+0000 7f701ea00700 1 -- 172.21.15.42:0/957366588 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7020108720 con 0x7f7020107c40 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.719+0000 7f701ea00700 1 --2- 172.21.15.42:0/957366588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f7020108060 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f70100098a0 tx=0x7f7010009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44382c08771101 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.719+0000 7f701d600700 1 -- 172.21.15.42:0/957366588 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7010004160 con 0x7f7020107c40 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.719+0000 7f701d600700 1 -- 172.21.15.42:0/957366588 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7010004300 con 0x7f7020107c40 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.719+0000 7f701d600700 1 -- 172.21.15.42:0/957366588 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7010011b50 con 0x7f7020107c40 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.720+0000 7f701fe00700 1 -- 172.21.15.42:0/957366588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 msgr2=0x7f7020108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.720+0000 7f701fe00700 1 --2- 172.21.15.42:0/957366588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f7020108060 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f70100098a0 tx=0x7f7010009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.720+0000 7f701fe00700 1 -- 172.21.15.42:0/957366588 shutdown_connections 2024-04-25T00:07:43.736 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.720+0000 7f701fe00700 1 --2- 172.21.15.42:0/957366588 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f7020108060 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.720+0000 7f701fe00700 1 -- 172.21.15.42:0/957366588 >> 172.21.15.42:0/957366588 conn(0x7f70201032a0 msgr2=0x7f70201056c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.720+0000 7f701fe00700 1 -- 172.21.15.42:0/957366588 shutdown_connections 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.720+0000 7f701fe00700 1 -- 172.21.15.42:0/957366588 wait complete. 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.721+0000 7f701fe00700 1 Processor -- start 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.722+0000 7f701fe00700 1 -- start start 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.722+0000 7f701fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f702019dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.722+0000 7f701fe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f702010c060 con 0x7f7020107c40 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.722+0000 7f701ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f702019dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.722+0000 7f701ea00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f702019dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51094/0 (socket says 172.21.15.42:51094) 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.722+0000 7f701ea00700 1 -- 172.21.15.42:0/3367035777 learned_addr learned my addr 172.21.15.42:0/3367035777 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.723+0000 7f701ea00700 1 -- 172.21.15.42:0/3367035777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702019e280 con 0x7f7020107c40 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.723+0000 7f701ea00700 1 --2- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f702019dd40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7010011b60 tx=0x7f7010011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.723+0000 7f7017400700 1 -- 172.21.15.42:0/3367035777 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7010003b20 con 0x7f7020107c40 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.723+0000 7f7017400700 1 -- 172.21.15.42:0/3367035777 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f7010003cc0 con 0x7f7020107c40 2024-04-25T00:07:43.737 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.723+0000 7f701fe00700 1 -- 172.21.15.42:0/3367035777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702019e510 con 0x7f7020107c40 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.724+0000 7f7017400700 1 -- 172.21.15.42:0/3367035777 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7010024840 con 0x7f7020107c40 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.724+0000 7f701fe00700 1 -- 172.21.15.42:0/3367035777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70201a1200 con 0x7f7020107c40 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.725+0000 7f7017400700 1 -- 172.21.15.42:0/3367035777 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f7010022020 con 0x7f7020107c40 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.725+0000 7f701fe00700 1 -- 172.21.15.42:0/3367035777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7020063890 con 0x7f7020107c40 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.725+0000 7f7017400700 1 --2- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f7004039100 0x7f700403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.725+0000 7f7017400700 1 -- 172.21.15.42:0/3367035777 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7010011e00 con 0x7f7020107c40 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.725+0000 7f701e000700 1 -- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f7004039100 msgr2=0x7f700403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/174637222 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.726+0000 7f701e000700 1 --2- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f7004039100 0x7f700403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.730+0000 7f7017400700 1 -- 172.21.15.42:0/3367035777 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7010024bb0 con 0x7f7020107c40 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.927+0000 7f701e000700 1 -- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f7004039100 msgr2=0x7f700403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/174637222 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:42.927+0000 7f701e000700 1 --2- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f7004039100 0x7f700403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.020+0000 7f701fe00700 1 -- 172.21.15.42:0/3367035777 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f70201a13d0 con 0x7f7020107c40 2024-04-25T00:07:43.738 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.021+0000 7f7017400700 1 -- 172.21.15.42:0/3367035777 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f70201a13d0 con 0x7f7020107c40 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.024+0000 7f701fe00700 1 -- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f7004039100 msgr2=0x7f700403b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.024+0000 7f701fe00700 1 --2- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f7004039100 0x7f700403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.024+0000 7f701fe00700 1 -- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 msgr2=0x7f702019dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.024+0000 7f701fe00700 1 --2- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f702019dd40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f7010011b60 tx=0x7f7010011c40 comp rx=0 tx=0).stop 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.024+0000 7f701fe00700 1 -- 172.21.15.42:0/3367035777 shutdown_connections 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.024+0000 7f701fe00700 1 --2- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f7004039100 0x7f700403b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.024+0000 7f701fe00700 1 --2- 172.21.15.42:0/3367035777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7020107c40 0x7f702019dd40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.024+0000 7f701fe00700 1 -- 172.21.15.42:0/3367035777 >> 172.21.15.42:0/3367035777 conn(0x7f70201032a0 msgr2=0x7f7020103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.024+0000 7f701fe00700 1 -- 172.21.15.42:0/3367035777 shutdown_connections 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:43.024+0000 7f701fe00700 1 -- 172.21.15.42:0/3367035777 wait complete. 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-04-25T00:07:43.739 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 5... 2024-04-25T00:07:44.333 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:44 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:44.162+0000 7fb2af20d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:07:44.626 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:44 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:44.404+0000 7fb2af20d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:07:44.980 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:44 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:44.643+0000 7fb2af20d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:07:45.376 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:45 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:45.044+0000 7fb2af20d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:07:45.376 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:45 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:45.168+0000 7fb2af20d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:07:45.687 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:45 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:45.427+0000 7fb2af20d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:07:46.126 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:45 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:45.686+0000 7fb2af20d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:07:46.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:46 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:46.500+0000 7fb2af20d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:07:47.376 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:47 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:47.068+0000 7fb2af20d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:07:47.376 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:47 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:47.197+0000 7fb2af20d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:07:47.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:47 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:47.469+0000 7fb2af20d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:07:48.591 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:48 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:48.135+0000 7fb2af20d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:07:48.591 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:48 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:48.261+0000 7fb2af20d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:07:48.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:48 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:48.590+0000 7fb2af20d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:07:48.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:48 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:48.774+0000 7fb2af20d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:07:49.189 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:48 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:48.905+0000 7fb2af20d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:07:49.189 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:49 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:49.061+0000 7fb2af20d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:07:49.626 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:49 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:49.187+0000 7fb2af20d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:07:50.512 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:50.220+0000 7fb2af20d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:07:50.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:07:50.511+0000 7fb2af20d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: Active manager daemon x restarted 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: Activating manager daemon x 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: mgrmap e6: x(active, starting, since 0.00535737s) 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: Manager daemon x is now available 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:07:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:50 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:07:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:51 smithi042 ceph-mon[28130]: Found migration_current of "None". Setting to last migration. 2024-04-25T00:07:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:51 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:51 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:51 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:07:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:51 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:07:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:51 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:07:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:51 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:07:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:51 smithi042 ceph-mon[28130]: mgrmap e7: x(active, since 1.0158s) 2024-04-25T00:07:52.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:07:52.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T00:07:52.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T00:07:52.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:07:52.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.368+0000 7f858a000700 1 Processor -- start 2024-04-25T00:07:52.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.369+0000 7f858a000700 1 -- start start 2024-04-25T00:07:52.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.369+0000 7f858a000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f858410acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:52.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.369+0000 7f858a000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f858407be10 con 0x7f85841088c0 2024-04-25T00:07:52.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.369+0000 7f8583400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f858410acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:52.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.370+0000 7f8583400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f858410acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51102/0 (socket says 172.21.15.42:51102) 2024-04-25T00:07:52.263 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.370+0000 7f8583400700 1 -- 172.21.15.42:0/1882447192 learned_addr learned my addr 172.21.15.42:0/1882447192 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:52.263 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.370+0000 7f8583400700 1 -- 172.21.15.42:0/1882447192 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f858410b1f0 con 0x7f85841088c0 2024-04-25T00:07:52.263 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.370+0000 7f8583400700 1 --2- 172.21.15.42:0/1882447192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f858410acb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f85740098a0 tx=0x7f8574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28aeee1bc669937a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:52.263 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.371+0000 7f8582000700 1 -- 172.21.15.42:0/1882447192 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8574004160 con 0x7f85841088c0 2024-04-25T00:07:52.263 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.371+0000 7f8582000700 1 -- 172.21.15.42:0/1882447192 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8574004300 con 0x7f85841088c0 2024-04-25T00:07:52.263 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.371+0000 7f8582000700 1 -- 172.21.15.42:0/1882447192 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8574011b50 con 0x7f85841088c0 2024-04-25T00:07:52.263 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.371+0000 7f858a000700 1 -- 172.21.15.42:0/1882447192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 msgr2=0x7f858410acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:52.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.371+0000 7f858a000700 1 --2- 172.21.15.42:0/1882447192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f858410acb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f85740098a0 tx=0x7f8574009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:52.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.371+0000 7f858a000700 1 -- 172.21.15.42:0/1882447192 shutdown_connections 2024-04-25T00:07:52.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.371+0000 7f858a000700 1 --2- 172.21.15.42:0/1882447192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f858410acb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:52.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.372+0000 7f858a000700 1 -- 172.21.15.42:0/1882447192 >> 172.21.15.42:0/1882447192 conn(0x7f8584102270 msgr2=0x7f85841046b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:52.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.372+0000 7f858a000700 1 -- 172.21.15.42:0/1882447192 shutdown_connections 2024-04-25T00:07:52.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.372+0000 7f858a000700 1 -- 172.21.15.42:0/1882447192 wait complete. 2024-04-25T00:07:52.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.374+0000 7f858a000700 1 Processor -- start 2024-04-25T00:07:52.264 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.374+0000 7f858a000700 1 -- start start 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.374+0000 7f858a000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f8584108140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.374+0000 7f858a000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f858410c1b0 con 0x7f85841088c0 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.375+0000 7f8583400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f8584108140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.375+0000 7f8583400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f8584108140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51110/0 (socket says 172.21.15.42:51110) 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.375+0000 7f8583400700 1 -- 172.21.15.42:0/961688421 learned_addr learned my addr 172.21.15.42:0/961688421 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.375+0000 7f8583400700 1 -- 172.21.15.42:0/961688421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8584108680 con 0x7f85841088c0 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.375+0000 7f8583400700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f8584108140 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8574011b60 tx=0x7f8574011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.376+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8574003b20 con 0x7f85841088c0 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.376+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8574003cc0 con 0x7f85841088c0 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.376+0000 7f858a000700 1 -- 172.21.15.42:0/961688421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8584106870 con 0x7f85841088c0 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.376+0000 7f858a000700 1 -- 172.21.15.42:0/961688421 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8584106db0 con 0x7f85841088c0 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.376+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8574004300 con 0x7f85841088c0 2024-04-25T00:07:52.265 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.377+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f8574022020 con 0x7f85841088c0 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.378+0000 7f857be00700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 0x7f856803b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.378+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 --> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8574004300 con 0x7f85680391f0 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.378+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8574011e00 con 0x7f85841088c0 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.378+0000 7f8582a00700 1 -- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 msgr2=0x7f856803b6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/174637222 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.378+0000 7f8582a00700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 0x7f856803b6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.579+0000 7f8582a00700 1 -- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 msgr2=0x7f856803b6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/174637222 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.579+0000 7f8582a00700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 0x7f856803b6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.980+0000 7f8582a00700 1 -- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 msgr2=0x7f856803b6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/174637222 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:44.980+0000 7f8582a00700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 0x7f856803b6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:45.781+0000 7f8582a00700 1 -- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 msgr2=0x7f856803b6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/174637222 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:45.781+0000 7f8582a00700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 0x7f856803b6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:47.383+0000 7f8582a00700 1 -- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 msgr2=0x7f856803b6b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/174637222 2024-04-25T00:07:52.266 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:47.383+0000 7f8582a00700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 0x7f856803b6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:50.518+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f857401a610 con 0x7f85841088c0 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:50.518+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 msgr2=0x7f856803b6b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:50.518+0000 7f857be00700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 0x7f856803b6b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.525+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f857404e160 con 0x7f85841088c0 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.525+0000 7f857be00700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f856803c970 0x7f856803ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.525+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 --> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8574012ea0 con 0x7f856803c970 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.527+0000 7f8582a00700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f856803c970 0x7f856803ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.531+0000 7f8582a00700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f856803c970 0x7f856803ed60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8570003a10 tx=0x7f8570009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.532+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 <== mgr.14120 v2:172.21.15.42:6800/3681869279 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f8574012ea0 con 0x7f856803c970 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.541+0000 7f858a000700 1 -- 172.21.15.42:0/961688421 --> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8584063890 con 0x7f856803c970 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.542+0000 7f857be00700 1 -- 172.21.15.42:0/961688421 <== mgr.14120 v2:172.21.15.42:6800/3681869279 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8584063890 con 0x7f856803c970 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.542+0000 7f858a000700 1 -- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f856803c970 msgr2=0x7f856803ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:52.267 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.542+0000 7f858a000700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f856803c970 0x7f856803ed60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8570003a10 tx=0x7f8570009290 comp rx=0 tx=0).stop 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.542+0000 7f858a000700 1 -- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 msgr2=0x7f8584108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.542+0000 7f858a000700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f8584108140 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8574011b60 tx=0x7f8574011c40 comp rx=0 tx=0).stop 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.543+0000 7f858a000700 1 -- 172.21.15.42:0/961688421 shutdown_connections 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.543+0000 7f858a000700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f856803c970 0x7f856803ed60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.543+0000 7f858a000700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85841088c0 0x7f8584108140 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.543+0000 7f858a000700 1 --2- 172.21.15.42:0/961688421 >> [v2:172.21.15.42:6800/174637222,v1:172.21.15.42:6801/174637222] conn(0x7f85680391f0 0x7f856803b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.543+0000 7f858a000700 1 -- 172.21.15.42:0/961688421 >> 172.21.15.42:0/961688421 conn(0x7f8584102270 msgr2=0x7f8584102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.543+0000 7f858a000700 1 -- 172.21.15.42:0/961688421 shutdown_connections 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:51.543+0000 7f858a000700 1 -- 172.21.15.42:0/961688421 wait complete. 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 5 is available 2024-04-25T00:07:52.268 INFO:teuthology.orchestra.run.smithi042.stdout:Setting orchestrator backend to cephadm... 2024-04-25T00:07:52.822 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:52 smithi042 ceph-mon[28130]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T00:07:52.823 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:52 smithi042 ceph-mon[28130]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T00:07:52.823 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:52 smithi042 ceph-mon[28130]: [25/Apr/2024:00:07:51] ENGINE Bus STARTING 2024-04-25T00:07:52.823 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:52 smithi042 ceph-mon[28130]: [25/Apr/2024:00:07:52] ENGINE Serving on https://172.21.15.42:7150 2024-04-25T00:07:52.823 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:52 smithi042 ceph-mon[28130]: [25/Apr/2024:00:07:52] ENGINE Serving on http://172.21.15.42:8765 2024-04-25T00:07:52.823 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:52 smithi042 ceph-mon[28130]: [25/Apr/2024:00:07:52] ENGINE Bus STARTED 2024-04-25T00:07:52.823 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:52 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:53.867 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:53 smithi042 ceph-mon[28130]: mgrmap e8: x(active, since 2s) 2024-04-25T00:07:53.867 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:53 smithi042 ceph-mon[28130]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:53.867 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:53 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:07:53.867 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:53 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:07:53.874 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.890+0000 7f666b000700 1 Processor -- start 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.890+0000 7f666b000700 1 -- start start 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.891+0000 7f666b000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.891+0000 7f666b000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f666407c840 con 0x7f666407dd40 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.891+0000 7f6669c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.891+0000 7f6669c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45140/0 (socket says 172.21.15.42:45140) 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.891+0000 7f6669c00700 1 -- 172.21.15.42:0/478997310 learned_addr learned my addr 172.21.15.42:0/478997310 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.892+0000 7f6669c00700 1 -- 172.21.15.42:0/478997310 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666407c9c0 con 0x7f666407dd40 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.892+0000 7f6669c00700 1 --2- 172.21.15.42:0/478997310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666407c300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f66580098a0 tx=0x7f6658009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adfe6c037ce46e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.893+0000 7f6663e00700 1 -- 172.21.15.42:0/478997310 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6658004160 con 0x7f666407dd40 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.893+0000 7f6663e00700 1 -- 172.21.15.42:0/478997310 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6658004300 con 0x7f666407dd40 2024-04-25T00:07:53.880 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.893+0000 7f6663e00700 1 -- 172.21.15.42:0/478997310 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6658011b50 con 0x7f666407dd40 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.893+0000 7f666b000700 1 -- 172.21.15.42:0/478997310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 msgr2=0x7f666407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.893+0000 7f666b000700 1 --2- 172.21.15.42:0/478997310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666407c300 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f66580098a0 tx=0x7f6658009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.893+0000 7f666b000700 1 -- 172.21.15.42:0/478997310 shutdown_connections 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.893+0000 7f666b000700 1 --2- 172.21.15.42:0/478997310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666407c300 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.893+0000 7f666b000700 1 -- 172.21.15.42:0/478997310 >> 172.21.15.42:0/478997310 conn(0x7f6664103250 msgr2=0x7f6664105670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.893+0000 7f666b000700 1 -- 172.21.15.42:0/478997310 shutdown_connections 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.893+0000 7f666b000700 1 -- 172.21.15.42:0/478997310 wait complete. 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.894+0000 7f666b000700 1 Processor -- start 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.895+0000 7f666b000700 1 -- start start 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.895+0000 7f666b000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666419dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.895+0000 7f666b000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f666410e0a0 con 0x7f666407dd40 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.895+0000 7f6669c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666419dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.895+0000 7f6669c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666419dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45156/0 (socket says 172.21.15.42:45156) 2024-04-25T00:07:53.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.895+0000 7f6669c00700 1 -- 172.21.15.42:0/2522498083 learned_addr learned my addr 172.21.15.42:0/2522498083 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.896+0000 7f6669c00700 1 -- 172.21.15.42:0/2522498083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666419e280 con 0x7f666407dd40 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.896+0000 7f6669c00700 1 --2- 172.21.15.42:0/2522498083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666419dd40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6658011670 tx=0x7f6658011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.896+0000 7f6662000700 1 -- 172.21.15.42:0/2522498083 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66580039c0 con 0x7f666407dd40 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.896+0000 7f6662000700 1 -- 172.21.15.42:0/2522498083 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6658011e50 con 0x7f666407dd40 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.896+0000 7f666b000700 1 -- 172.21.15.42:0/2522498083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666419e510 con 0x7f666407dd40 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.896+0000 7f6662000700 1 -- 172.21.15.42:0/2522498083 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f665801a5c0 con 0x7f666407dd40 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.896+0000 7f666b000700 1 -- 172.21.15.42:0/2522498083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66641a1200 con 0x7f666407dd40 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.898+0000 7f6662000700 1 -- 172.21.15.42:0/2522498083 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6658024430 con 0x7f666407dd40 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.898+0000 7f666b000700 1 -- 172.21.15.42:0/2522498083 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6664063890 con 0x7f666407dd40 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.898+0000 7f6662000700 1 --2- 172.21.15.42:0/2522498083 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f664c039050 0x7f664c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.898+0000 7f6662000700 1 -- 172.21.15.42:0/2522498083 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f665804d4e0 con 0x7f666407dd40 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.898+0000 7f6669200700 1 --2- 172.21.15.42:0/2522498083 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f664c039050 0x7f664c03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.899+0000 7f6669200700 1 --2- 172.21.15.42:0/2522498083 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f664c039050 0x7f664c03b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6650006fd0 tx=0x7f6650006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:53.882 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:52.903+0000 7f6662000700 1 -- 172.21.15.42:0/2522498083 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f665804ce30 con 0x7f666407dd40 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.150+0000 7f666b000700 1 -- 172.21.15.42:0/2522498083 --> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f6664104540 con 0x7f664c039050 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.159+0000 7f6662000700 1 -- 172.21.15.42:0/2522498083 <== mgr.14120 v2:172.21.15.42:6800/3681869279 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6664104540 con 0x7f664c039050 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.162+0000 7f666b000700 1 -- 172.21.15.42:0/2522498083 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f664c039050 msgr2=0x7f664c03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.162+0000 7f666b000700 1 --2- 172.21.15.42:0/2522498083 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f664c039050 0x7f664c03b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6650006fd0 tx=0x7f6650006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.162+0000 7f666b000700 1 -- 172.21.15.42:0/2522498083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 msgr2=0x7f666419dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.162+0000 7f666b000700 1 --2- 172.21.15.42:0/2522498083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666419dd40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6658011670 tx=0x7f6658011b60 comp rx=0 tx=0).stop 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.163+0000 7f666b000700 1 -- 172.21.15.42:0/2522498083 shutdown_connections 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.163+0000 7f666b000700 1 --2- 172.21.15.42:0/2522498083 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f664c039050 0x7f664c03b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.163+0000 7f666b000700 1 --2- 172.21.15.42:0/2522498083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f666407dd40 0x7f666419dd40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.163+0000 7f666b000700 1 -- 172.21.15.42:0/2522498083 >> 172.21.15.42:0/2522498083 conn(0x7f6664103250 msgr2=0x7f6664103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.163+0000 7f666b000700 1 -- 172.21.15.42:0/2522498083 shutdown_connections 2024-04-25T00:07:53.883 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:53.163+0000 7f666b000700 1 -- 172.21.15.42:0/2522498083 wait complete. 2024-04-25T00:07:55.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T00:07:55.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.483+0000 7f8b6d000700 1 Processor -- start 2024-04-25T00:07:55.407 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.483+0000 7f8b6d000700 1 -- start start 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.483+0000 7f8b6d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6807c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.483+0000 7f8b6d000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6807c840 con 0x7f8b6807de00 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.484+0000 7f8b65e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6807c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.484+0000 7f8b65e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6807c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45172/0 (socket says 172.21.15.42:45172) 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.484+0000 7f8b65e00700 1 -- 172.21.15.42:0/469412841 learned_addr learned my addr 172.21.15.42:0/469412841 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.484+0000 7f8b65e00700 1 -- 172.21.15.42:0/469412841 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6807c9c0 con 0x7f8b6807de00 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.485+0000 7f8b65e00700 1 --2- 172.21.15.42:0/469412841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6807c300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8b580098a0 tx=0x7f8b58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3a730f0fc9b6dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.485+0000 7f8b65400700 1 -- 172.21.15.42:0/469412841 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b58004160 con 0x7f8b6807de00 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.485+0000 7f8b65400700 1 -- 172.21.15.42:0/469412841 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b58004300 con 0x7f8b6807de00 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.485+0000 7f8b65400700 1 -- 172.21.15.42:0/469412841 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b58011b50 con 0x7f8b6807de00 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.485+0000 7f8b6d000700 1 -- 172.21.15.42:0/469412841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 msgr2=0x7f8b6807c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.486+0000 7f8b6d000700 1 --2- 172.21.15.42:0/469412841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6807c300 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8b580098a0 tx=0x7f8b58009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:55.408 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.486+0000 7f8b6d000700 1 -- 172.21.15.42:0/469412841 shutdown_connections 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.486+0000 7f8b6d000700 1 --2- 172.21.15.42:0/469412841 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6807c300 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.486+0000 7f8b6d000700 1 -- 172.21.15.42:0/469412841 >> 172.21.15.42:0/469412841 conn(0x7f8b68103400 msgr2=0x7f8b68105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.486+0000 7f8b6d000700 1 -- 172.21.15.42:0/469412841 shutdown_connections 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.486+0000 7f8b6d000700 1 -- 172.21.15.42:0/469412841 wait complete. 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.487+0000 7f8b6d000700 1 Processor -- start 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.487+0000 7f8b6d000700 1 -- start start 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.487+0000 7f8b6d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6819df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.487+0000 7f8b6d000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8b6810e3d0 con 0x7f8b6807de00 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.488+0000 7f8b65e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6819df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.488+0000 7f8b65e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6819df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45176/0 (socket says 172.21.15.42:45176) 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.488+0000 7f8b65e00700 1 -- 172.21.15.42:0/880978717 learned_addr learned my addr 172.21.15.42:0/880978717 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.488+0000 7f8b65e00700 1 -- 172.21.15.42:0/880978717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6819e480 con 0x7f8b6807de00 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.488+0000 7f8b65e00700 1 --2- 172.21.15.42:0/880978717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6819df40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8b58011b60 tx=0x7f8b58011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.489+0000 7f8b5ea00700 1 -- 172.21.15.42:0/880978717 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b58003b20 con 0x7f8b6807de00 2024-04-25T00:07:55.409 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.489+0000 7f8b5ea00700 1 -- 172.21.15.42:0/880978717 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8b58003cc0 con 0x7f8b6807de00 2024-04-25T00:07:55.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.489+0000 7f8b5ea00700 1 -- 172.21.15.42:0/880978717 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8b58024840 con 0x7f8b6807de00 2024-04-25T00:07:55.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.489+0000 7f8b6d000700 1 -- 172.21.15.42:0/880978717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6819e710 con 0x7f8b6807de00 2024-04-25T00:07:55.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.489+0000 7f8b6d000700 1 -- 172.21.15.42:0/880978717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b681a1400 con 0x7f8b6807de00 2024-04-25T00:07:55.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.490+0000 7f8b6d000700 1 -- 172.21.15.42:0/880978717 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b68063890 con 0x7f8b6807de00 2024-04-25T00:07:55.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.490+0000 7f8b5ea00700 1 -- 172.21.15.42:0/880978717 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8b58022020 con 0x7f8b6807de00 2024-04-25T00:07:55.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.491+0000 7f8b5ea00700 1 --2- 172.21.15.42:0/880978717 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8b4c0390a0 0x7f8b4c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:55.410 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.491+0000 7f8b5ea00700 1 -- 172.21.15.42:0/880978717 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8b58011e00 con 0x7f8b6807de00 2024-04-25T00:07:55.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.491+0000 7f8b5fe00700 1 --2- 172.21.15.42:0/880978717 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8b4c0390a0 0x7f8b4c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:55.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.492+0000 7f8b5fe00700 1 --2- 172.21.15.42:0/880978717 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8b4c0390a0 0x7f8b4c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8b50006fd0 tx=0x7f8b50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:55.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.497+0000 7f8b5ea00700 1 -- 172.21.15.42:0/880978717 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8b5800b3e0 con 0x7f8b6807de00 2024-04-25T00:07:55.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.721+0000 7f8b6d000700 1 -- 172.21.15.42:0/880978717 --> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f8b68104670 con 0x7f8b4c0390a0 2024-04-25T00:07:55.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.723+0000 7f8b5ea00700 1 -- 172.21.15.42:0/880978717 <== mgr.14120 v2:172.21.15.42:6800/3681869279 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f8b68104670 con 0x7f8b4c0390a0 2024-04-25T00:07:55.411 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.726+0000 7f8b6d000700 1 -- 172.21.15.42:0/880978717 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8b4c0390a0 msgr2=0x7f8b4c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:55.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.726+0000 7f8b6d000700 1 --2- 172.21.15.42:0/880978717 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8b4c0390a0 0x7f8b4c03b560 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8b50006fd0 tx=0x7f8b50006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:55.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.726+0000 7f8b6d000700 1 -- 172.21.15.42:0/880978717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 msgr2=0x7f8b6819df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:55.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.726+0000 7f8b6d000700 1 --2- 172.21.15.42:0/880978717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6819df40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8b58011b60 tx=0x7f8b58011c40 comp rx=0 tx=0).stop 2024-04-25T00:07:55.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.726+0000 7f8b6d000700 1 -- 172.21.15.42:0/880978717 shutdown_connections 2024-04-25T00:07:55.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.726+0000 7f8b6d000700 1 --2- 172.21.15.42:0/880978717 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8b4c0390a0 0x7f8b4c03b560 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:55.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.726+0000 7f8b6d000700 1 --2- 172.21.15.42:0/880978717 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8b6807de00 0x7f8b6819df40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:55.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.726+0000 7f8b6d000700 1 -- 172.21.15.42:0/880978717 >> 172.21.15.42:0/880978717 conn(0x7f8b68103400 msgr2=0x7f8b68103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:55.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.727+0000 7f8b6d000700 1 -- 172.21.15.42:0/880978717 shutdown_connections 2024-04-25T00:07:55.412 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:54.727+0000 7f8b6d000700 1 -- 172.21.15.42:0/880978717 wait complete. 2024-04-25T00:07:55.412 INFO:teuthology.orchestra.run.smithi042.stdout:Generating ssh key... 2024-04-25T00:07:55.713 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:55 smithi042 ceph-mon[28130]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:56.597 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-mon[28130]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:56.597 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-mon[28130]: Generating ssh key... 2024-04-25T00:07:56.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: Generating public/private rsa key pair. 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: Your identification has been saved in /tmp/tmpnfrway1h/key. 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: Your public key has been saved in /tmp/tmpnfrway1h/key.pub. 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: The key fingerprint is: 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: SHA256:emeN0k8AjrOWx+IzOfBxRnfH4p83WrAFgjn2vXHX68I ceph-9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: The key's randomart image is: 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: +---[RSA 3072]----+ 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: | | 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: | o | 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: | .= . o | 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: | oo.+ = + .| 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: | o.S..+ * oo| 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: | . .*o. +. B o| 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: | o*== = ++ + | 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: | o*+ + o E=..| 2024-04-25T00:07:56.877 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: | .+ ..oo.| 2024-04-25T00:07:56.878 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:07:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: +----[SHA256]-----+ 2024-04-25T00:07:57.246 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.057+0000 7f8bdc000700 1 Processor -- start 2024-04-25T00:07:57.247 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.057+0000 7f8bdc000700 1 -- start start 2024-04-25T00:07:57.247 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.057+0000 7f8bdc000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd4108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.247 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.057+0000 7f8bdc000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4108740 con 0x7f8bd4107de0 2024-04-25T00:07:57.248 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.058+0000 7f8bd9600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd4108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.250 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.058+0000 7f8bd9600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd4108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45182/0 (socket says 172.21.15.42:45182) 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.058+0000 7f8bd9600700 1 -- 172.21.15.42:0/1450065119 learned_addr learned my addr 172.21.15.42:0/1450065119 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.058+0000 7f8bd9600700 1 -- 172.21.15.42:0/1450065119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd41088c0 con 0x7f8bd4107de0 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.059+0000 7f8bd9600700 1 --2- 172.21.15.42:0/1450065119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd4108200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80098a0 tx=0x7f8bc8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c147074b375d648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.059+0000 7f8bd3e00700 1 -- 172.21.15.42:0/1450065119 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc8004160 con 0x7f8bd4107de0 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.059+0000 7f8bd3e00700 1 -- 172.21.15.42:0/1450065119 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8bc8004300 con 0x7f8bd4107de0 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.059+0000 7f8bd3e00700 1 -- 172.21.15.42:0/1450065119 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc8011b50 con 0x7f8bd4107de0 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.060+0000 7f8bdc000700 1 -- 172.21.15.42:0/1450065119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 msgr2=0x7f8bd4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.060+0000 7f8bdc000700 1 --2- 172.21.15.42:0/1450065119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd4108200 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80098a0 tx=0x7f8bc8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.060+0000 7f8bdc000700 1 -- 172.21.15.42:0/1450065119 shutdown_connections 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.060+0000 7f8bdc000700 1 --2- 172.21.15.42:0/1450065119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd4108200 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.060+0000 7f8bdc000700 1 -- 172.21.15.42:0/1450065119 >> 172.21.15.42:0/1450065119 conn(0x7f8bd4103460 msgr2=0x7f8bd4105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.251 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.060+0000 7f8bdc000700 1 -- 172.21.15.42:0/1450065119 shutdown_connections 2024-04-25T00:07:57.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.060+0000 7f8bdc000700 1 -- 172.21.15.42:0/1450065119 wait complete. 2024-04-25T00:07:57.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.061+0000 7f8bdc000700 1 Processor -- start 2024-04-25T00:07:57.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.062+0000 7f8bdc000700 1 -- start start 2024-04-25T00:07:57.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.062+0000 7f8bdc000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd419ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.252 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.062+0000 7f8bdc000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd410c250 con 0x7f8bd4107de0 2024-04-25T00:07:57.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.062+0000 7f8bd9600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd419ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.062+0000 7f8bd9600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd419ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45186/0 (socket says 172.21.15.42:45186) 2024-04-25T00:07:57.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.062+0000 7f8bd9600700 1 -- 172.21.15.42:0/2483644174 learned_addr learned my addr 172.21.15.42:0/2483644174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:57.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.063+0000 7f8bd9600700 1 -- 172.21.15.42:0/2483644174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd419e410 con 0x7f8bd4107de0 2024-04-25T00:07:57.253 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.063+0000 7f8bd9600700 1 --2- 172.21.15.42:0/2483644174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd419ded0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8011670 tx=0x7f8bc8011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.063+0000 7f8bd2000700 1 -- 172.21.15.42:0/2483644174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc80039c0 con 0x7f8bd4107de0 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.063+0000 7f8bd2000700 1 -- 172.21.15.42:0/2483644174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8bc8011e50 con 0x7f8bd4107de0 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.063+0000 7f8bd2000700 1 -- 172.21.15.42:0/2483644174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8bc801a5c0 con 0x7f8bd4107de0 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.063+0000 7f8bdc000700 1 -- 172.21.15.42:0/2483644174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd419e6a0 con 0x7f8bd4107de0 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.063+0000 7f8bdc000700 1 -- 172.21.15.42:0/2483644174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd41a1390 con 0x7f8bd4107de0 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.064+0000 7f8bdc000700 1 -- 172.21.15.42:0/2483644174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd4063890 con 0x7f8bd4107de0 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.065+0000 7f8bd2000700 1 -- 172.21.15.42:0/2483644174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8bc8024430 con 0x7f8bd4107de0 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.065+0000 7f8bd2000700 1 --2- 172.21.15.42:0/2483644174 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8bbc039050 0x7f8bbc03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.065+0000 7f8bd2000700 1 -- 172.21.15.42:0/2483644174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8bc804d4d0 con 0x7f8bd4107de0 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.066+0000 7f8bd8c00700 1 --2- 172.21.15.42:0/2483644174 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8bbc039050 0x7f8bbc03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:57.256 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.066+0000 7f8bd8c00700 1 --2- 172.21.15.42:0/2483644174 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8bbc039050 0x7f8bbc03b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0006fd0 tx=0x7f8bc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.070+0000 7f8bd2000700 1 -- 172.21.15.42:0/2483644174 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8bc804ce20 con 0x7f8bd4107de0 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.298+0000 7f8bdc000700 1 -- 172.21.15.42:0/2483644174 --> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f8bd4104590 con 0x7f8bbc039050 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.615+0000 7f8bd2000700 1 -- 172.21.15.42:0/2483644174 <== mgr.14120 v2:172.21.15.42:6800/3681869279 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8bd4104590 con 0x7f8bbc039050 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.618+0000 7f8bdc000700 1 -- 172.21.15.42:0/2483644174 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8bbc039050 msgr2=0x7f8bbc03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.618+0000 7f8bdc000700 1 --2- 172.21.15.42:0/2483644174 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8bbc039050 0x7f8bbc03b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8bc0006fd0 tx=0x7f8bc0006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.618+0000 7f8bdc000700 1 -- 172.21.15.42:0/2483644174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 msgr2=0x7f8bd419ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.618+0000 7f8bdc000700 1 --2- 172.21.15.42:0/2483644174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd419ded0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8011670 tx=0x7f8bc8011b60 comp rx=0 tx=0).stop 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.618+0000 7f8bdc000700 1 -- 172.21.15.42:0/2483644174 shutdown_connections 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.618+0000 7f8bdc000700 1 --2- 172.21.15.42:0/2483644174 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8bbc039050 0x7f8bbc03b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.618+0000 7f8bdc000700 1 --2- 172.21.15.42:0/2483644174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8bd4107de0 0x7f8bd419ded0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.618+0000 7f8bdc000700 1 -- 172.21.15.42:0/2483644174 >> 172.21.15.42:0/2483644174 conn(0x7f8bd4103460 msgr2=0x7f8bd4103e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.618+0000 7f8bdc000700 1 -- 172.21.15.42:0/2483644174 shutdown_connections 2024-04-25T00:07:57.257 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:56.619+0000 7f8bdc000700 1 -- 172.21.15.42:0/2483644174 wait complete. 2024-04-25T00:07:57.866 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:57 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:07:57.866 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:57 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:07:58.870 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:07:58 smithi042 ceph-mon[28130]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:07:58.881 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+yawZQKu9a1Cm8B8jvCDmyu2MGQW6dQo/CDvsUxQj22oemCgo1KT7tAhV2mwkg/vjbKju7SObWxZD/ltKmALJx1zTa7sRknntm3UCyPclw/YW3deEH8gsyqmpUoLgyW2GLOjoI30tUJdfJUVTzsbZ+CubiLwLBh1cpy7dgdkuQMYPtK5Xv83bpdZAZ/YNuulNbeQtkPdiUvKbKLYrVHIMM5pYIUUXUwaYmDqIONVB4u/eedw3WJjLtjeLaoWSH5SagJRrZAGTU6W1D0EmTAPF/PJUhf2TXx8e5dTA0tgF/R3u67ypftYOZ5pIVEBmoaQ5Q72AhM7p6J+/GsbSKkSlKG48RwdlrperUO9UBkwLp1k5953oOIjplSipDcR53c6T2AXMfMzXdyVMyKTDmFRGnFF/Ys2QPxvfFrQjGEYEvrsxiiztb2/MTyko9HJNel8XZgnNXLe5lMAZOJv7FEqsJ26k6S+vlNd3RTrtluJVJw9bdOOeFhuRAOUq8CqaH3E= ceph-9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:07:58.888 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.911+0000 7f1540600700 1 Processor -- start 2024-04-25T00:07:58.889 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.912+0000 7f1540600700 1 -- start start 2024-04-25T00:07:58.889 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.912+0000 7f1540600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f15381081d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.889 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.912+0000 7f1540600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f15381087a0 con 0x7f1538107db0 2024-04-25T00:07:58.889 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.913+0000 7f153dc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f15381081d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.889 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.913+0000 7f153dc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f15381081d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34594/0 (socket says 172.21.15.42:34594) 2024-04-25T00:07:58.889 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.913+0000 7f153dc00700 1 -- 172.21.15.42:0/3787429934 learned_addr learned my addr 172.21.15.42:0/3787429934 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:58.889 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.913+0000 7f153dc00700 1 -- 172.21.15.42:0/3787429934 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1538108f60 con 0x7f1538107db0 2024-04-25T00:07:58.890 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.914+0000 7f153dc00700 1 --2- 172.21.15.42:0/3787429934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f15381081d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f152c009e80 tx=0x7f152c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e76db8e20edeca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:58.890 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.914+0000 7f1537e00700 1 -- 172.21.15.42:0/3787429934 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152c017070 con 0x7f1538107db0 2024-04-25T00:07:58.890 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.914+0000 7f1537e00700 1 -- 172.21.15.42:0/3787429934 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f152c00b8c0 con 0x7f1538107db0 2024-04-25T00:07:58.890 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.914+0000 7f1537e00700 1 -- 172.21.15.42:0/3787429934 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152c0045c0 con 0x7f1538107db0 2024-04-25T00:07:58.890 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.914+0000 7f1540600700 1 -- 172.21.15.42:0/3787429934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 msgr2=0x7f15381081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.890 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.915+0000 7f1540600700 1 --2- 172.21.15.42:0/3787429934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f15381081d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f152c009e80 tx=0x7f152c00b330 comp rx=0 tx=0).stop 2024-04-25T00:07:58.890 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.915+0000 7f1540600700 1 -- 172.21.15.42:0/3787429934 shutdown_connections 2024-04-25T00:07:58.890 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.915+0000 7f1540600700 1 --2- 172.21.15.42:0/3787429934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f15381081d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.915+0000 7f1540600700 1 -- 172.21.15.42:0/3787429934 >> 172.21.15.42:0/3787429934 conn(0x7f1538103370 msgr2=0x7f1538105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:58.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.915+0000 7f1540600700 1 -- 172.21.15.42:0/3787429934 shutdown_connections 2024-04-25T00:07:58.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.915+0000 7f1540600700 1 -- 172.21.15.42:0/3787429934 wait complete. 2024-04-25T00:07:58.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.916+0000 7f1540600700 1 Processor -- start 2024-04-25T00:07:58.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.917+0000 7f1540600700 1 -- start start 2024-04-25T00:07:58.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.917+0000 7f1540600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f153819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.917+0000 7f1540600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f153810c310 con 0x7f1538107db0 2024-04-25T00:07:58.891 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.917+0000 7f153dc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f153819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.917+0000 7f153dc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f153819bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34606/0 (socket says 172.21.15.42:34606) 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.917+0000 7f153dc00700 1 -- 172.21.15.42:0/2801107809 learned_addr learned my addr 172.21.15.42:0/2801107809 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.918+0000 7f153dc00700 1 -- 172.21.15.42:0/2801107809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153819c2f0 con 0x7f1538107db0 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.918+0000 7f153dc00700 1 --2- 172.21.15.42:0/2801107809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f153819bdb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f152c009e50 tx=0x7f152c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.918+0000 7f1536000700 1 -- 172.21.15.42:0/2801107809 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152c020020 con 0x7f1538107db0 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.918+0000 7f1536000700 1 -- 172.21.15.42:0/2801107809 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f152c003b50 con 0x7f1538107db0 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.918+0000 7f1536000700 1 -- 172.21.15.42:0/2801107809 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f152c017040 con 0x7f1538107db0 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.918+0000 7f1540600700 1 -- 172.21.15.42:0/2801107809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153819c580 con 0x7f1538107db0 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.919+0000 7f1540600700 1 -- 172.21.15.42:0/2801107809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153819f270 con 0x7f1538107db0 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.920+0000 7f1536000700 1 -- 172.21.15.42:0/2801107809 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f152c023020 con 0x7f1538107db0 2024-04-25T00:07:58.892 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.920+0000 7f1540600700 1 -- 172.21.15.42:0/2801107809 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1538063890 con 0x7f1538107db0 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.920+0000 7f1536000700 1 --2- 172.21.15.42:0/2801107809 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f1520039050 0x7f152003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.921+0000 7f1536000700 1 -- 172.21.15.42:0/2801107809 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f152c00f050 con 0x7f1538107db0 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.921+0000 7f153d200700 1 --2- 172.21.15.42:0/2801107809 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f1520039050 0x7f152003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.922+0000 7f153d200700 1 --2- 172.21.15.42:0/2801107809 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f1520039050 0x7f152003b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1524006fd0 tx=0x7f1524006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:57.926+0000 7f1536000700 1 -- 172.21.15.42:0/2801107809 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f152c02a020 con 0x7f1538107db0 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.156+0000 7f1540600700 1 -- 172.21.15.42:0/2801107809 --> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f15381045b0 con 0x7f1520039050 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.158+0000 7f1536000700 1 -- 172.21.15.42:0/2801107809 <== mgr.14120 v2:172.21.15.42:6800/3681869279 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f15381045b0 con 0x7f1520039050 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.161+0000 7f1540600700 1 -- 172.21.15.42:0/2801107809 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f1520039050 msgr2=0x7f152003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.161+0000 7f1540600700 1 --2- 172.21.15.42:0/2801107809 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f1520039050 0x7f152003b510 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1524006fd0 tx=0x7f1524006e40 comp rx=0 tx=0).stop 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.161+0000 7f1540600700 1 -- 172.21.15.42:0/2801107809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 msgr2=0x7f153819bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:07:58.893 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.161+0000 7f1540600700 1 --2- 172.21.15.42:0/2801107809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f153819bdb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f152c009e50 tx=0x7f152c00bd30 comp rx=0 tx=0).stop 2024-04-25T00:07:58.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.161+0000 7f1540600700 1 -- 172.21.15.42:0/2801107809 shutdown_connections 2024-04-25T00:07:58.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.161+0000 7f1540600700 1 --2- 172.21.15.42:0/2801107809 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f1520039050 0x7f152003b510 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.161+0000 7f1540600700 1 --2- 172.21.15.42:0/2801107809 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1538107db0 0x7f153819bdb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:07:58.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.161+0000 7f1540600700 1 -- 172.21.15.42:0/2801107809 >> 172.21.15.42:0/2801107809 conn(0x7f1538103370 msgr2=0x7f1538103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:07:58.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.161+0000 7f1540600700 1 -- 172.21.15.42:0/2801107809 shutdown_connections 2024-04-25T00:07:58.894 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:58.161+0000 7f1540600700 1 -- 172.21.15.42:0/2801107809 wait complete. 2024-04-25T00:07:58.895 INFO:teuthology.orchestra.run.smithi042.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:07:58.896 INFO:teuthology.orchestra.run.smithi042.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T00:07:58.896 INFO:teuthology.orchestra.run.smithi042.stdout:Adding host smithi042... 2024-04-25T00:08:00.730 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:00 smithi042 ceph-mon[28130]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:01.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:01 smithi042 ceph-mon[28130]: Deploying cephadm binary to smithi042 2024-04-25T00:08:06.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Added host 'smithi042' with addr '172.21.15.42' 2024-04-25T00:08:06.769 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.504+0000 7f8ffca00700 1 Processor -- start 2024-04-25T00:08:06.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.504+0000 7f8ffca00700 1 -- start start 2024-04-25T00:08:06.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.505+0000 7f8ffca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff8108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.505+0000 7f8ffca00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff8108740 con 0x7f8ff8107de0 2024-04-25T00:08:06.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.505+0000 7f8ff5e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff8108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.770 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.505+0000 7f8ff5e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff8108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34622/0 (socket says 172.21.15.42:34622) 2024-04-25T00:08:06.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.505+0000 7f8ff5e00700 1 -- 172.21.15.42:0/1247518481 learned_addr learned my addr 172.21.15.42:0/1247518481 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:06.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.506+0000 7f8ff5e00700 1 -- 172.21.15.42:0/1247518481 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff81088c0 con 0x7f8ff8107de0 2024-04-25T00:08:06.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.506+0000 7f8ff5e00700 1 --2- 172.21.15.42:0/1247518481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff8108200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80098a0 tx=0x7f8fe8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95af14ebdc53770 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.506+0000 7f8ff5400700 1 -- 172.21.15.42:0/1247518481 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fe8004160 con 0x7f8ff8107de0 2024-04-25T00:08:06.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.506+0000 7f8ff5400700 1 -- 172.21.15.42:0/1247518481 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8fe8004300 con 0x7f8ff8107de0 2024-04-25T00:08:06.771 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.506+0000 7f8ff5400700 1 -- 172.21.15.42:0/1247518481 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fe8011b50 con 0x7f8ff8107de0 2024-04-25T00:08:06.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.507+0000 7f8ffca00700 1 -- 172.21.15.42:0/1247518481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 msgr2=0x7f8ff8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.507+0000 7f8ffca00700 1 --2- 172.21.15.42:0/1247518481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff8108200 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8fe80098a0 tx=0x7f8fe8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.507+0000 7f8ffca00700 1 -- 172.21.15.42:0/1247518481 shutdown_connections 2024-04-25T00:08:06.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.507+0000 7f8ffca00700 1 --2- 172.21.15.42:0/1247518481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff8108200 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.772 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.507+0000 7f8ffca00700 1 -- 172.21.15.42:0/1247518481 >> 172.21.15.42:0/1247518481 conn(0x7f8ff8103460 msgr2=0x7f8ff8105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:06.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.508+0000 7f8ffca00700 1 -- 172.21.15.42:0/1247518481 shutdown_connections 2024-04-25T00:08:06.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.508+0000 7f8ffca00700 1 -- 172.21.15.42:0/1247518481 wait complete. 2024-04-25T00:08:06.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.509+0000 7f8ffca00700 1 Processor -- start 2024-04-25T00:08:06.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.509+0000 7f8ffca00700 1 -- start start 2024-04-25T00:08:06.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.509+0000 7f8ffca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff819df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.773 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.509+0000 7f8ffca00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff810c250 con 0x7f8ff8107de0 2024-04-25T00:08:06.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.509+0000 7f8ff5e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff819df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.509+0000 7f8ff5e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff819df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34634/0 (socket says 172.21.15.42:34634) 2024-04-25T00:08:06.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.509+0000 7f8ff5e00700 1 -- 172.21.15.42:0/1054470710 learned_addr learned my addr 172.21.15.42:0/1054470710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:06.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.510+0000 7f8ff5e00700 1 -- 172.21.15.42:0/1054470710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff819e4a0 con 0x7f8ff8107de0 2024-04-25T00:08:06.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.510+0000 7f8ff5e00700 1 --2- 172.21.15.42:0/1054470710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff819df60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8011b60 tx=0x7f8fe8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.774 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.510+0000 7f8feea00700 1 -- 172.21.15.42:0/1054470710 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fe8003b20 con 0x7f8ff8107de0 2024-04-25T00:08:06.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.511+0000 7f8feea00700 1 -- 172.21.15.42:0/1054470710 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8fe8003cc0 con 0x7f8ff8107de0 2024-04-25T00:08:06.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.511+0000 7f8feea00700 1 -- 172.21.15.42:0/1054470710 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fe8024840 con 0x7f8ff8107de0 2024-04-25T00:08:06.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.511+0000 7f8ffca00700 1 -- 172.21.15.42:0/1054470710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff819e730 con 0x7f8ff8107de0 2024-04-25T00:08:06.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.511+0000 7f8ffca00700 1 -- 172.21.15.42:0/1054470710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff81a13d0 con 0x7f8ff8107de0 2024-04-25T00:08:06.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.512+0000 7f8feea00700 1 -- 172.21.15.42:0/1054470710 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8fe8022020 con 0x7f8ff8107de0 2024-04-25T00:08:06.775 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.512+0000 7f8ffca00700 1 -- 172.21.15.42:0/1054470710 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ff8063890 con 0x7f8ff8107de0 2024-04-25T00:08:06.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.512+0000 7f8feea00700 1 --2- 172.21.15.42:0/1054470710 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8fdc0390a0 0x7f8fdc03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:06.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.512+0000 7f8feea00700 1 -- 172.21.15.42:0/1054470710 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8fe8011e00 con 0x7f8ff8107de0 2024-04-25T00:08:06.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.513+0000 7f8fefe00700 1 --2- 172.21.15.42:0/1054470710 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8fdc0390a0 0x7f8fdc03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:06.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.513+0000 7f8fefe00700 1 --2- 172.21.15.42:0/1054470710 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8fdc0390a0 0x7f8fdc03b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0006fd0 tx=0x7f8fe0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:06.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.518+0000 7f8feea00700 1 -- 172.21.15.42:0/1054470710 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8fe804cb30 con 0x7f8ff8107de0 2024-04-25T00:08:06.776 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:07:59.750+0000 7f8ffca00700 1 -- 172.21.15.42:0/1054470710 --> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi042", "addr": "172.21.15.42", "target": ["mon-mgr", ""]}) -- 0x7f8ff8104530 con 0x7f8fdc0390a0 2024-04-25T00:08:06.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.011+0000 7f8feea00700 1 -- 172.21.15.42:0/1054470710 <== mgr.14120 v2:172.21.15.42:6800/3681869279 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8ff8104530 con 0x7f8fdc0390a0 2024-04-25T00:08:06.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.015+0000 7f8ffca00700 1 -- 172.21.15.42:0/1054470710 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8fdc0390a0 msgr2=0x7f8fdc03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.015+0000 7f8ffca00700 1 --2- 172.21.15.42:0/1054470710 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8fdc0390a0 0x7f8fdc03b560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0006fd0 tx=0x7f8fe0006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:06.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.015+0000 7f8ffca00700 1 -- 172.21.15.42:0/1054470710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 msgr2=0x7f8ff819df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:06.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.015+0000 7f8ffca00700 1 --2- 172.21.15.42:0/1054470710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff819df60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8fe8011b60 tx=0x7f8fe8011c40 comp rx=0 tx=0).stop 2024-04-25T00:08:06.777 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.015+0000 7f8ffca00700 1 -- 172.21.15.42:0/1054470710 shutdown_connections 2024-04-25T00:08:06.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.015+0000 7f8ffca00700 1 --2- 172.21.15.42:0/1054470710 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f8fdc0390a0 0x7f8fdc03b560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.015+0000 7f8ffca00700 1 --2- 172.21.15.42:0/1054470710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8ff8107de0 0x7f8ff819df60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:06.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.015+0000 7f8ffca00700 1 -- 172.21.15.42:0/1054470710 >> 172.21.15.42:0/1054470710 conn(0x7f8ff8103460 msgr2=0x7f8ff8103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:06.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.016+0000 7f8ffca00700 1 -- 172.21.15.42:0/1054470710 shutdown_connections 2024-04-25T00:08:06.778 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:06.016+0000 7f8ffca00700 1 -- 172.21.15.42:0/1054470710 wait complete. 2024-04-25T00:08:06.778 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mon service... 2024-04-25T00:08:07.258 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:07 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:08:07.258 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:07 smithi042 ceph-mon[28130]: Added host smithi042 2024-04-25T00:08:07.258 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:07 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:08.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T00:08:08.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.434+0000 7efe35a00700 1 Processor -- start 2024-04-25T00:08:08.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.434+0000 7efe35a00700 1 -- start start 2024-04-25T00:08:08.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.435+0000 7efe35a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3010a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.854 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.435+0000 7efe35a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe3010a810 con 0x7efe30109e20 2024-04-25T00:08:08.857 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.435+0000 7efe2fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3010a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.857 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.435+0000 7efe2fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3010a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59922/0 (socket says 172.21.15.42:59922) 2024-04-25T00:08:08.857 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.435+0000 7efe2fe00700 1 -- 172.21.15.42:0/1595583738 learned_addr learned my addr 172.21.15.42:0/1595583738 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:08.857 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.435+0000 7efe2fe00700 1 -- 172.21.15.42:0/1595583738 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3010afd0 con 0x7efe30109e20 2024-04-25T00:08:08.857 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.436+0000 7efe2fe00700 1 --2- 172.21.15.42:0/1595583738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3010a240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efe200098a0 tx=0x7efe20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ba2c5a0e8531a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:08.857 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.436+0000 7efe2ea00700 1 -- 172.21.15.42:0/1595583738 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe20004160 con 0x7efe30109e20 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.436+0000 7efe2ea00700 1 -- 172.21.15.42:0/1595583738 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe20004300 con 0x7efe30109e20 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.436+0000 7efe2ea00700 1 -- 172.21.15.42:0/1595583738 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe20011b50 con 0x7efe30109e20 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.437+0000 7efe35a00700 1 -- 172.21.15.42:0/1595583738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 msgr2=0x7efe3010a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.437+0000 7efe35a00700 1 --2- 172.21.15.42:0/1595583738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3010a240 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efe200098a0 tx=0x7efe20009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.437+0000 7efe35a00700 1 -- 172.21.15.42:0/1595583738 shutdown_connections 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.437+0000 7efe35a00700 1 --2- 172.21.15.42:0/1595583738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3010a240 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.437+0000 7efe35a00700 1 -- 172.21.15.42:0/1595583738 >> 172.21.15.42:0/1595583738 conn(0x7efe3007d240 msgr2=0x7efe3007d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.437+0000 7efe35a00700 1 -- 172.21.15.42:0/1595583738 shutdown_connections 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.437+0000 7efe35a00700 1 -- 172.21.15.42:0/1595583738 wait complete. 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.438+0000 7efe35a00700 1 Processor -- start 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.438+0000 7efe35a00700 1 -- start start 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.439+0000 7efe35a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3019de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.439+0000 7efe35a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe3010e380 con 0x7efe30109e20 2024-04-25T00:08:08.858 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.439+0000 7efe2fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3019de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.859 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.439+0000 7efe2fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3019de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59928/0 (socket says 172.21.15.42:59928) 2024-04-25T00:08:08.859 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.439+0000 7efe2fe00700 1 -- 172.21.15.42:0/1374813844 learned_addr learned my addr 172.21.15.42:0/1374813844 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.439+0000 7efe2fe00700 1 -- 172.21.15.42:0/1374813844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3019e350 con 0x7efe30109e20 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.440+0000 7efe2fe00700 1 --2- 172.21.15.42:0/1374813844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3019de10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efe20011850 tx=0x7efe20011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.440+0000 7efe2cc00700 1 -- 172.21.15.42:0/1374813844 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe20003af0 con 0x7efe30109e20 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.440+0000 7efe2cc00700 1 -- 172.21.15.42:0/1374813844 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7efe20020070 con 0x7efe30109e20 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.440+0000 7efe2cc00700 1 -- 172.21.15.42:0/1374813844 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe200195a0 con 0x7efe30109e20 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.440+0000 7efe35a00700 1 -- 172.21.15.42:0/1374813844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3019e5e0 con 0x7efe30109e20 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.440+0000 7efe35a00700 1 -- 172.21.15.42:0/1374813844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe301a12d0 con 0x7efe30109e20 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.441+0000 7efe35a00700 1 -- 172.21.15.42:0/1374813844 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe30063890 con 0x7efe30109e20 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.441+0000 7efe2cc00700 1 -- 172.21.15.42:0/1374813844 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7efe20019850 con 0x7efe30109e20 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.442+0000 7efe2cc00700 1 --2- 172.21.15.42:0/1374813844 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7efe140390a0 0x7efe1403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.442+0000 7efe2cc00700 1 -- 172.21.15.42:0/1374813844 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efe20011e00 con 0x7efe30109e20 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.442+0000 7efe2f400700 1 --2- 172.21.15.42:0/1374813844 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7efe140390a0 0x7efe1403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:08.862 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.443+0000 7efe2f400700 1 --2- 172.21.15.42:0/1374813844 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7efe140390a0 0x7efe1403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efe1c006fd0 tx=0x7efe1c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.448+0000 7efe2cc00700 1 -- 172.21.15.42:0/1374813844 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe2000e260 con 0x7efe30109e20 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.722+0000 7efe35a00700 1 -- 172.21.15.42:0/1374813844 --> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7efe30107cd0 con 0x7efe140390a0 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.729+0000 7efe2cc00700 1 -- 172.21.15.42:0/1374813844 <== mgr.14120 v2:172.21.15.42:6800/3681869279 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7efe30107cd0 con 0x7efe140390a0 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.733+0000 7efe35a00700 1 -- 172.21.15.42:0/1374813844 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7efe140390a0 msgr2=0x7efe1403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.733+0000 7efe35a00700 1 --2- 172.21.15.42:0/1374813844 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7efe140390a0 0x7efe1403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efe1c006fd0 tx=0x7efe1c006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.733+0000 7efe35a00700 1 -- 172.21.15.42:0/1374813844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 msgr2=0x7efe3019de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.733+0000 7efe35a00700 1 --2- 172.21.15.42:0/1374813844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3019de10 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7efe20011850 tx=0x7efe20011c10 comp rx=0 tx=0).stop 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.733+0000 7efe35a00700 1 -- 172.21.15.42:0/1374813844 shutdown_connections 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.733+0000 7efe35a00700 1 --2- 172.21.15.42:0/1374813844 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7efe140390a0 0x7efe1403b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.733+0000 7efe35a00700 1 --2- 172.21.15.42:0/1374813844 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe30109e20 0x7efe3019de10 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.733+0000 7efe35a00700 1 -- 172.21.15.42:0/1374813844 >> 172.21.15.42:0/1374813844 conn(0x7efe3007d240 msgr2=0x7efe301075c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.733+0000 7efe35a00700 1 -- 172.21.15.42:0/1374813844 shutdown_connections 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:07.734+0000 7efe35a00700 1 -- 172.21.15.42:0/1374813844 wait complete. 2024-04-25T00:08:08.863 INFO:teuthology.orchestra.run.smithi042.stdout:Deploying unmanaged mgr service... 2024-04-25T00:08:09.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:08 smithi042 ceph-mon[28130]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:09.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:08 smithi042 ceph-mon[28130]: Saving service mon spec with placement count:5 2024-04-25T00:08:09.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:08 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:08:10.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:10 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:08:10.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:10 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:08:10.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:10 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:08:10.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T00:08:10.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.974+0000 7f060d600700 1 Processor -- start 2024-04-25T00:08:10.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.975+0000 7f060d600700 1 -- start start 2024-04-25T00:08:10.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.975+0000 7f060d600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:10.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.975+0000 7f060d600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f060807c800 con 0x7f060807dd50 2024-04-25T00:08:10.997 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.976+0000 7f0606a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.976+0000 7f0606a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59942/0 (socket says 172.21.15.42:59942) 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.976+0000 7f0606a00700 1 -- 172.21.15.42:0/697162458 learned_addr learned my addr 172.21.15.42:0/697162458 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.976+0000 7f0606a00700 1 -- 172.21.15.42:0/697162458 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060807c980 con 0x7f060807dd50 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.977+0000 7f0606a00700 1 --2- 172.21.15.42:0/697162458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060807c2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05f80098a0 tx=0x7f05f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839d509c0bbb73a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.977+0000 7f0605600700 1 -- 172.21.15.42:0/697162458 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f8004160 con 0x7f060807dd50 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.977+0000 7f0605600700 1 -- 172.21.15.42:0/697162458 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05f8004300 con 0x7f060807dd50 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.977+0000 7f0605600700 1 -- 172.21.15.42:0/697162458 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f8011b20 con 0x7f060807dd50 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.978+0000 7f060d600700 1 -- 172.21.15.42:0/697162458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 msgr2=0x7f060807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.978+0000 7f060d600700 1 --2- 172.21.15.42:0/697162458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060807c2c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f05f80098a0 tx=0x7f05f8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.978+0000 7f060d600700 1 -- 172.21.15.42:0/697162458 shutdown_connections 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.978+0000 7f060d600700 1 --2- 172.21.15.42:0/697162458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060807c2c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.978+0000 7f060d600700 1 -- 172.21.15.42:0/697162458 >> 172.21.15.42:0/697162458 conn(0x7f0608103460 msgr2=0x7f0608105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:10.998 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.979+0000 7f060d600700 1 -- 172.21.15.42:0/697162458 shutdown_connections 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.979+0000 7f060d600700 1 -- 172.21.15.42:0/697162458 wait complete. 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.980+0000 7f060d600700 1 Processor -- start 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.980+0000 7f060d600700 1 -- start start 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.980+0000 7f060d600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.980+0000 7f060d600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f060810e310 con 0x7f060807dd50 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.980+0000 7f0606a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.981+0000 7f0606a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59950/0 (socket says 172.21.15.42:59950) 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.981+0000 7f0606a00700 1 -- 172.21.15.42:0/1930423358 learned_addr learned my addr 172.21.15.42:0/1930423358 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.981+0000 7f0606a00700 1 -- 172.21.15.42:0/1930423358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060819e3c0 con 0x7f060807dd50 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.981+0000 7f0606a00700 1 --2- 172.21.15.42:0/1930423358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060819de80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05f80117e0 tx=0x7f05f8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.982+0000 7f05ff400700 1 -- 172.21.15.42:0/1930423358 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f8003ba0 con 0x7f060807dd50 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.982+0000 7f060d600700 1 -- 172.21.15.42:0/1930423358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f060819e650 con 0x7f060807dd50 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.982+0000 7f05ff400700 1 -- 172.21.15.42:0/1930423358 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05f801a910 con 0x7f060807dd50 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.982+0000 7f05ff400700 1 -- 172.21.15.42:0/1930423358 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05f80248f0 con 0x7f060807dd50 2024-04-25T00:08:10.999 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.982+0000 7f060d600700 1 -- 172.21.15.42:0/1930423358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f060819eab0 con 0x7f060807dd50 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.983+0000 7f060d600700 1 -- 172.21.15.42:0/1930423358 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0608063890 con 0x7f060807dd50 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.983+0000 7f05ff400700 1 -- 172.21.15.42:0/1930423358 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f05f8022020 con 0x7f060807dd50 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.984+0000 7f05ff400700 1 --2- 172.21.15.42:0/1930423358 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f05ec0390a0 0x7f05ec03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.984+0000 7f05ff400700 1 -- 172.21.15.42:0/1930423358 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f05f804d520 con 0x7f060807dd50 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.984+0000 7f0606000700 1 --2- 172.21.15.42:0/1930423358 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f05ec0390a0 0x7f05ec03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.985+0000 7f0606000700 1 --2- 172.21.15.42:0/1930423358 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f05ec0390a0 0x7f05ec03b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f05f4006fd0 tx=0x7f05f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:09.994+0000 7f05ff400700 1 -- 172.21.15.42:0/1930423358 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f05f8020c10 con 0x7f060807dd50 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.237+0000 7f060d600700 1 -- 172.21.15.42:0/1930423358 --> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f06081045c0 con 0x7f05ec0390a0 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.244+0000 7f05ff400700 1 -- 172.21.15.42:0/1930423358 <== mgr.14120 v2:172.21.15.42:6800/3681869279 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f06081045c0 con 0x7f05ec0390a0 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.247+0000 7f060d600700 1 -- 172.21.15.42:0/1930423358 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f05ec0390a0 msgr2=0x7f05ec03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.247+0000 7f060d600700 1 --2- 172.21.15.42:0/1930423358 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f05ec0390a0 0x7f05ec03b560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f05f4006fd0 tx=0x7f05f4006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.247+0000 7f060d600700 1 -- 172.21.15.42:0/1930423358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 msgr2=0x7f060819de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:11.000 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.247+0000 7f060d600700 1 --2- 172.21.15.42:0/1930423358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060819de80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f05f80117e0 tx=0x7f05f8011ba0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.001 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.247+0000 7f060d600700 1 -- 172.21.15.42:0/1930423358 shutdown_connections 2024-04-25T00:08:11.001 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.247+0000 7f060d600700 1 --2- 172.21.15.42:0/1930423358 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f05ec0390a0 0x7f05ec03b560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.001 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.247+0000 7f060d600700 1 --2- 172.21.15.42:0/1930423358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f060807dd50 0x7f060819de80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:11.001 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.247+0000 7f060d600700 1 -- 172.21.15.42:0/1930423358 >> 172.21.15.42:0/1930423358 conn(0x7f0608103460 msgr2=0x7f0608103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:11.001 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.247+0000 7f060d600700 1 -- 172.21.15.42:0/1930423358 shutdown_connections 2024-04-25T00:08:11.001 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:10.247+0000 7f060d600700 1 -- 172.21.15.42:0/1930423358 wait complete. 2024-04-25T00:08:11.619 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:11 smithi042 ceph-mon[28130]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:11.620 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:11 smithi042 ceph-mon[28130]: Saving service mgr spec with placement count:2 2024-04-25T00:08:12.677 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:12 smithi042 ceph-mon[28130]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:12.677 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:12 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/92473454' entity='client.admin' 2024-04-25T00:08:12.977 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.794+0000 7fe9a2c00700 1 Processor -- start 2024-04-25T00:08:12.978 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.794+0000 7fe9a2c00700 1 -- start start 2024-04-25T00:08:12.978 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.795+0000 7fe9a2c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c108230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.978 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.795+0000 7fe9a2c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c108800 con 0x7fe99c107e10 2024-04-25T00:08:12.979 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.795+0000 7fe99be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c108230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.979 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.795+0000 7fe99be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c108230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59956/0 (socket says 172.21.15.42:59956) 2024-04-25T00:08:12.979 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.795+0000 7fe99be00700 1 -- 172.21.15.42:0/3930502474 learned_addr learned my addr 172.21.15.42:0/3930502474 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:12.979 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.796+0000 7fe99be00700 1 -- 172.21.15.42:0/3930502474 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c108f70 con 0x7fe99c107e10 2024-04-25T00:08:12.979 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.796+0000 7fe99be00700 1 --2- 172.21.15.42:0/3930502474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c108230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0098a0 tx=0x7fe98c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1afb87f603488c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:12.980 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.796+0000 7fe99aa00700 1 -- 172.21.15.42:0/3930502474 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe98c004160 con 0x7fe99c107e10 2024-04-25T00:08:12.980 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.796+0000 7fe99aa00700 1 -- 172.21.15.42:0/3930502474 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c004300 con 0x7fe99c107e10 2024-04-25T00:08:12.981 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.796+0000 7fe99aa00700 1 -- 172.21.15.42:0/3930502474 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe98c011b20 con 0x7fe99c107e10 2024-04-25T00:08:12.981 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.797+0000 7fe9a2c00700 1 -- 172.21.15.42:0/3930502474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 msgr2=0x7fe99c108230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.981 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.797+0000 7fe9a2c00700 1 --2- 172.21.15.42:0/3930502474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c108230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0098a0 tx=0x7fe98c009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.981 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.797+0000 7fe9a2c00700 1 -- 172.21.15.42:0/3930502474 shutdown_connections 2024-04-25T00:08:12.981 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.797+0000 7fe9a2c00700 1 --2- 172.21.15.42:0/3930502474 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c108230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.982 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.797+0000 7fe9a2c00700 1 -- 172.21.15.42:0/3930502474 >> 172.21.15.42:0/3930502474 conn(0x7fe99c1033f0 msgr2=0x7fe99c105810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:12.982 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.797+0000 7fe9a2c00700 1 -- 172.21.15.42:0/3930502474 shutdown_connections 2024-04-25T00:08:12.982 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.797+0000 7fe9a2c00700 1 -- 172.21.15.42:0/3930502474 wait complete. 2024-04-25T00:08:12.983 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.798+0000 7fe9a2c00700 1 Processor -- start 2024-04-25T00:08:12.983 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.799+0000 7fe9a2c00700 1 -- start start 2024-04-25T00:08:12.983 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.799+0000 7fe9a2c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c19df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.983 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.799+0000 7fe9a2c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe99c10c320 con 0x7fe99c107e10 2024-04-25T00:08:12.983 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.799+0000 7fe99be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c19df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.984 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.799+0000 7fe99be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c19df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59970/0 (socket says 172.21.15.42:59970) 2024-04-25T00:08:12.984 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.799+0000 7fe99be00700 1 -- 172.21.15.42:0/92473454 learned_addr learned my addr 172.21.15.42:0/92473454 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:12.984 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.800+0000 7fe99be00700 1 -- 172.21.15.42:0/92473454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe99c19e440 con 0x7fe99c107e10 2024-04-25T00:08:12.985 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.800+0000 7fe99be00700 1 --2- 172.21.15.42:0/92473454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c19df00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0117e0 tx=0x7fe98c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:12.985 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.800+0000 7fe998c00700 1 -- 172.21.15.42:0/92473454 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe98c003bf0 con 0x7fe99c107e10 2024-04-25T00:08:12.985 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.800+0000 7fe9a2c00700 1 -- 172.21.15.42:0/92473454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe99c19e6d0 con 0x7fe99c107e10 2024-04-25T00:08:12.986 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.800+0000 7fe998c00700 1 -- 172.21.15.42:0/92473454 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98c01a910 con 0x7fe99c107e10 2024-04-25T00:08:12.986 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.801+0000 7fe9a2c00700 1 -- 172.21.15.42:0/92473454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe99c19eb30 con 0x7fe99c107e10 2024-04-25T00:08:12.987 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.801+0000 7fe998c00700 1 -- 172.21.15.42:0/92473454 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe98c004300 con 0x7fe99c107e10 2024-04-25T00:08:12.987 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.802+0000 7fe9a2c00700 1 -- 172.21.15.42:0/92473454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe99c063890 con 0x7fe99c107e10 2024-04-25T00:08:12.987 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.802+0000 7fe998c00700 1 -- 172.21.15.42:0/92473454 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe98c022020 con 0x7fe99c107e10 2024-04-25T00:08:12.987 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.803+0000 7fe998c00700 1 --2- 172.21.15.42:0/92473454 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fe9800391a0 0x7fe98003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:12.987 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.803+0000 7fe998c00700 1 -- 172.21.15.42:0/92473454 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe98c04d440 con 0x7fe99c107e10 2024-04-25T00:08:12.988 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.803+0000 7fe99b400700 1 --2- 172.21.15.42:0/92473454 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fe9800391a0 0x7fe98003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:12.988 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.804+0000 7fe99b400700 1 --2- 172.21.15.42:0/92473454 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fe9800391a0 0x7fe98003b660 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe988006fd0 tx=0x7fe988006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:12.988 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:11.808+0000 7fe998c00700 1 -- 172.21.15.42:0/92473454 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe98c011e20 con 0x7fe99c107e10 2024-04-25T00:08:12.988 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.047+0000 7fe9a2c00700 1 -- 172.21.15.42:0/92473454 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fe99c1a1480 con 0x7fe99c107e10 2024-04-25T00:08:12.988 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.051+0000 7fe998c00700 1 -- 172.21.15.42:0/92473454 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fe98c017cf0 con 0x7fe99c107e10 2024-04-25T00:08:12.989 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.055+0000 7fe9a2c00700 1 -- 172.21.15.42:0/92473454 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fe9800391a0 msgr2=0x7fe98003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.989 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.055+0000 7fe9a2c00700 1 --2- 172.21.15.42:0/92473454 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fe9800391a0 0x7fe98003b660 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe988006fd0 tx=0x7fe988006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:12.989 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.055+0000 7fe9a2c00700 1 -- 172.21.15.42:0/92473454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 msgr2=0x7fe99c19df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:12.989 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.055+0000 7fe9a2c00700 1 --2- 172.21.15.42:0/92473454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c19df00 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0117e0 tx=0x7fe98c011c10 comp rx=0 tx=0).stop 2024-04-25T00:08:12.990 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.055+0000 7fe9a2c00700 1 -- 172.21.15.42:0/92473454 shutdown_connections 2024-04-25T00:08:12.990 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.055+0000 7fe9a2c00700 1 --2- 172.21.15.42:0/92473454 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fe9800391a0 0x7fe98003b660 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.990 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.055+0000 7fe9a2c00700 1 --2- 172.21.15.42:0/92473454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe99c107e10 0x7fe99c19df00 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:12.990 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.055+0000 7fe9a2c00700 1 -- 172.21.15.42:0/92473454 >> 172.21.15.42:0/92473454 conn(0x7fe99c1033f0 msgr2=0x7fe99c103f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:12.990 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.055+0000 7fe9a2c00700 1 -- 172.21.15.42:0/92473454 shutdown_connections 2024-04-25T00:08:12.990 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:12.055+0000 7fe9a2c00700 1 -- 172.21.15.42:0/92473454 wait complete. 2024-04-25T00:08:14.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:14 smithi042 ceph-mon[28130]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:14.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:14 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:08:14.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:14 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2332522334' entity='client.admin' 2024-04-25T00:08:15.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.034+0000 7fa0cd800700 1 Processor -- start 2024-04-25T00:08:15.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.034+0000 7fa0cd800700 1 -- start start 2024-04-25T00:08:15.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.035+0000 7fa0cd800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c810ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.035+0000 7fa0cd800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c807be90 con 0x7fa0c8108840 2024-04-25T00:08:15.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.035+0000 7fa0c6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c810ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.035+0000 7fa0c6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c810ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59972/0 (socket says 172.21.15.42:59972) 2024-04-25T00:08:15.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.035+0000 7fa0c6a00700 1 -- 172.21.15.42:0/2973405011 learned_addr learned my addr 172.21.15.42:0/2973405011 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:15.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.036+0000 7fa0c6a00700 1 -- 172.21.15.42:0/2973405011 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c810b170 con 0x7fa0c8108840 2024-04-25T00:08:15.014 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.036+0000 7fa0c6a00700 1 --2- 172.21.15.42:0/2973405011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c810ac30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80098a0 tx=0x7fa0b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1427950e14b7965f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.036+0000 7fa0c5600700 1 -- 172.21.15.42:0/2973405011 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b8004160 con 0x7fa0c8108840 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.036+0000 7fa0c5600700 1 -- 172.21.15.42:0/2973405011 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b8004300 con 0x7fa0c8108840 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.036+0000 7fa0c5600700 1 -- 172.21.15.42:0/2973405011 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b8011b20 con 0x7fa0c8108840 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.037+0000 7fa0cd800700 1 -- 172.21.15.42:0/2973405011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 msgr2=0x7fa0c810ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.037+0000 7fa0cd800700 1 --2- 172.21.15.42:0/2973405011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c810ac30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80098a0 tx=0x7fa0b8009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.037+0000 7fa0cd800700 1 -- 172.21.15.42:0/2973405011 shutdown_connections 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.037+0000 7fa0cd800700 1 --2- 172.21.15.42:0/2973405011 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c810ac30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.037+0000 7fa0cd800700 1 -- 172.21.15.42:0/2973405011 >> 172.21.15.42:0/2973405011 conn(0x7fa0c81022b0 msgr2=0x7fa0c81046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.037+0000 7fa0cd800700 1 -- 172.21.15.42:0/2973405011 shutdown_connections 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.038+0000 7fa0cd800700 1 -- 172.21.15.42:0/2973405011 wait complete. 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.038+0000 7fa0cd800700 1 Processor -- start 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.039+0000 7fa0cd800700 1 -- start start 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.039+0000 7fa0cd800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c819df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.015 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.039+0000 7fa0cd800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c810c1e0 con 0x7fa0c8108840 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.039+0000 7fa0c6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c819df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.039+0000 7fa0c6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c819df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59982/0 (socket says 172.21.15.42:59982) 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.039+0000 7fa0c6a00700 1 -- 172.21.15.42:0/2332522334 learned_addr learned my addr 172.21.15.42:0/2332522334 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.040+0000 7fa0c6a00700 1 -- 172.21.15.42:0/2332522334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c819e450 con 0x7fa0c8108840 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.040+0000 7fa0c6a00700 1 --2- 172.21.15.42:0/2332522334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c819df10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80117e0 tx=0x7fa0b8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.040+0000 7fa0bf400700 1 -- 172.21.15.42:0/2332522334 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b8003ba0 con 0x7fa0c8108840 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.040+0000 7fa0bf400700 1 -- 172.21.15.42:0/2332522334 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0b801a910 con 0x7fa0c8108840 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.040+0000 7fa0cd800700 1 -- 172.21.15.42:0/2332522334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c819e6e0 con 0x7fa0c8108840 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.040+0000 7fa0bf400700 1 -- 172.21.15.42:0/2332522334 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0b80248f0 con 0x7fa0c8108840 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.041+0000 7fa0cd800700 1 -- 172.21.15.42:0/2332522334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c819eb40 con 0x7fa0c8108840 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.042+0000 7fa0bf400700 1 -- 172.21.15.42:0/2332522334 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa0b8022020 con 0x7fa0c8108840 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.042+0000 7fa0cd800700 1 -- 172.21.15.42:0/2332522334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c8063890 con 0x7fa0c8108840 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.042+0000 7fa0bf400700 1 --2- 172.21.15.42:0/2332522334 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa0ac039100 0x7fa0ac03b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.042+0000 7fa0bf400700 1 -- 172.21.15.42:0/2332522334 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa0b804d560 con 0x7fa0c8108840 2024-04-25T00:08:15.016 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.042+0000 7fa0c6000700 1 --2- 172.21.15.42:0/2332522334 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa0ac039100 0x7fa0ac03b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.043+0000 7fa0c6000700 1 --2- 172.21.15.42:0/2332522334 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa0ac039100 0x7fa0ac03b5c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006fd0 tx=0x7fa0b4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.048+0000 7fa0bf400700 1 -- 172.21.15.42:0/2332522334 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa0b800f2e0 con 0x7fa0c8108840 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.291+0000 7fa0cd800700 1 -- 172.21.15.42:0/2332522334 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fa0c81a15d0 con 0x7fa0c8108840 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.295+0000 7fa0bf400700 1 -- 172.21.15.42:0/2332522334 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fa0b802d3f0 con 0x7fa0c8108840 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.299+0000 7fa0cd800700 1 -- 172.21.15.42:0/2332522334 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa0ac039100 msgr2=0x7fa0ac03b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.299+0000 7fa0cd800700 1 --2- 172.21.15.42:0/2332522334 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa0ac039100 0x7fa0ac03b5c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006fd0 tx=0x7fa0b4006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.299+0000 7fa0cd800700 1 -- 172.21.15.42:0/2332522334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 msgr2=0x7fa0c819df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.299+0000 7fa0cd800700 1 --2- 172.21.15.42:0/2332522334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c819df10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa0b80117e0 tx=0x7fa0b8011ba0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.300+0000 7fa0cd800700 1 -- 172.21.15.42:0/2332522334 shutdown_connections 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.300+0000 7fa0cd800700 1 --2- 172.21.15.42:0/2332522334 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa0ac039100 0x7fa0ac03b5c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.300+0000 7fa0cd800700 1 --2- 172.21.15.42:0/2332522334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa0c8108840 0x7fa0c819df10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.300+0000 7fa0cd800700 1 -- 172.21.15.42:0/2332522334 >> 172.21.15.42:0/2332522334 conn(0x7fa0c81022b0 msgr2=0x7fa0c8102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:15.017 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.300+0000 7fa0cd800700 1 -- 172.21.15.42:0/2332522334 shutdown_connections 2024-04-25T00:08:15.018 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:14.300+0000 7fa0cd800700 1 -- 172.21.15.42:0/2332522334 wait complete. 2024-04-25T00:08:15.018 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling the dashboard module... 2024-04-25T00:08:16.753 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:16 smithi042 ceph-mon[28130]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:16.753 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:16 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/992510449' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T00:08:16.753 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:16 smithi042 ceph-mon[28130]: from='mgr.14120 172.21.15.42:0/1611477143' entity='mgr.x' 2024-04-25T00:08:16.753 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: ignoring --setuser ceph since I am not root 2024-04-25T00:08:16.753 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: ignoring --setgroup ceph since I am not root 2024-04-25T00:08:17.129 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:16.750+0000 7f826afae200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:08:17.129 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:17.000+0000 7f826afae200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:08:17.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.614+0000 7f50a7e00700 1 Processor -- start 2024-04-25T00:08:17.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.614+0000 7f50a7e00700 1 -- start start 2024-04-25T00:08:17.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.614+0000 7f50a7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a810aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.614+0000 7f50a7e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f50a807be90 con 0x7f50a8108610 2024-04-25T00:08:17.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.615+0000 7f50a6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a810aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.615+0000 7f50a6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a810aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59984/0 (socket says 172.21.15.42:59984) 2024-04-25T00:08:17.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.615+0000 7f50a6a00700 1 -- 172.21.15.42:0/3292347119 learned_addr learned my addr 172.21.15.42:0/3292347119 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:17.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.616+0000 7f50a6a00700 1 -- 172.21.15.42:0/3292347119 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a810af40 con 0x7f50a8108610 2024-04-25T00:08:17.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.616+0000 7f50a6a00700 1 --2- 172.21.15.42:0/3292347119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a810aa00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f50980098a0 tx=0x7f5098009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f40a3aadc93b68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.616+0000 7f50a5600700 1 -- 172.21.15.42:0/3292347119 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5098004160 con 0x7f50a8108610 2024-04-25T00:08:17.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.616+0000 7f50a5600700 1 -- 172.21.15.42:0/3292347119 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5098004300 con 0x7f50a8108610 2024-04-25T00:08:17.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.616+0000 7f50a5600700 1 -- 172.21.15.42:0/3292347119 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5098011b20 con 0x7f50a8108610 2024-04-25T00:08:17.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.617+0000 7f50a7e00700 1 -- 172.21.15.42:0/3292347119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 msgr2=0x7f50a810aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.617+0000 7f50a7e00700 1 --2- 172.21.15.42:0/3292347119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a810aa00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f50980098a0 tx=0x7f5098009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.618+0000 7f50a7e00700 1 -- 172.21.15.42:0/3292347119 shutdown_connections 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.618+0000 7f50a7e00700 1 --2- 172.21.15.42:0/3292347119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a810aa00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.618+0000 7f50a7e00700 1 -- 172.21.15.42:0/3292347119 >> 172.21.15.42:0/3292347119 conn(0x7f50a81020a0 msgr2=0x7f50a81044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.618+0000 7f50a7e00700 1 -- 172.21.15.42:0/3292347119 shutdown_connections 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.618+0000 7f50a7e00700 1 -- 172.21.15.42:0/3292347119 wait complete. 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.619+0000 7f50a7e00700 1 Processor -- start 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.619+0000 7f50a7e00700 1 -- start start 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.619+0000 7f50a7e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a8107e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.619+0000 7f50a7e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f50a810bfb0 con 0x7f50a8108610 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.620+0000 7f50a6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a8107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.620+0000 7f50a6a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a8107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59986/0 (socket says 172.21.15.42:59986) 2024-04-25T00:08:17.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.620+0000 7f50a6a00700 1 -- 172.21.15.42:0/992510449 learned_addr learned my addr 172.21.15.42:0/992510449 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.621+0000 7f50a6a00700 1 -- 172.21.15.42:0/992510449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a81083d0 con 0x7f50a8108610 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.621+0000 7f50a6a00700 1 --2- 172.21.15.42:0/992510449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a8107e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50980117e0 tx=0x7f5098011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.621+0000 7f509f400700 1 -- 172.21.15.42:0/992510449 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5098003ba0 con 0x7f50a8108610 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.621+0000 7f50a7e00700 1 -- 172.21.15.42:0/992510449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a81065c0 con 0x7f50a8108610 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.621+0000 7f509f400700 1 -- 172.21.15.42:0/992510449 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f509801a910 con 0x7f50a8108610 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.621+0000 7f509f400700 1 -- 172.21.15.42:0/992510449 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50980248f0 con 0x7f50a8108610 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.622+0000 7f50a7e00700 1 -- 172.21.15.42:0/992510449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a8106a80 con 0x7f50a8108610 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.623+0000 7f50a7e00700 1 -- 172.21.15.42:0/992510449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a8063890 con 0x7f50a8108610 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.623+0000 7f509f400700 1 -- 172.21.15.42:0/992510449 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f5098022020 con 0x7f50a8108610 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.623+0000 7f509f400700 1 --2- 172.21.15.42:0/992510449 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f508c0390a0 0x7f508c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.623+0000 7f509f400700 1 -- 172.21.15.42:0/992510449 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f509804d520 con 0x7f50a8108610 2024-04-25T00:08:17.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.624+0000 7f50a6000700 1 --2- 172.21.15.42:0/992510449 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f508c0390a0 0x7f508c03b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.625+0000 7f50a6000700 1 --2- 172.21.15.42:0/992510449 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f508c0390a0 0x7f508c03b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5094006fd0 tx=0x7f5094006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.629+0000 7f509f400700 1 -- 172.21.15.42:0/992510449 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f509800f2e0 con 0x7f50a8108610 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:15.919+0000 7f50a7e00700 1 -- 172.21.15.42:0/992510449 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f50a8106dc0 con 0x7f50a8108610 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.429+0000 7f509f400700 1 -- 172.21.15.42:0/992510449 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f50a8106dc0 con 0x7f50a8108610 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.433+0000 7f509f400700 1 -- 172.21.15.42:0/992510449 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f509802d3f0 con 0x7f50a8108610 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.434+0000 7f50a7e00700 1 -- 172.21.15.42:0/992510449 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f508c0390a0 msgr2=0x7f508c03b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.434+0000 7f50a7e00700 1 --2- 172.21.15.42:0/992510449 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f508c0390a0 0x7f508c03b560 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5094006fd0 tx=0x7f5094006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.434+0000 7f50a7e00700 1 -- 172.21.15.42:0/992510449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 msgr2=0x7f50a8107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.434+0000 7f50a7e00700 1 --2- 172.21.15.42:0/992510449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a8107e90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f50980117e0 tx=0x7f5098011ba0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.434+0000 7f50a7e00700 1 -- 172.21.15.42:0/992510449 shutdown_connections 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.434+0000 7f50a7e00700 1 --2- 172.21.15.42:0/992510449 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f508c0390a0 0x7f508c03b560 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.434+0000 7f50a7e00700 1 --2- 172.21.15.42:0/992510449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f50a8108610 0x7f50a8107e90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:17.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.434+0000 7f50a7e00700 1 -- 172.21.15.42:0/992510449 >> 172.21.15.42:0/992510449 conn(0x7f50a81020a0 msgr2=0x7f50a8102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:17.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.435+0000 7f50a7e00700 1 -- 172.21.15.42:0/992510449 shutdown_connections 2024-04-25T00:08:17.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:16.435+0000 7f50a7e00700 1 -- 172.21.15.42:0/992510449 wait complete. 2024-04-25T00:08:17.423 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:17.135+0000 7f826afae200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:08:17.692 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:17 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/992510449' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T00:08:17.692 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:17 smithi042 ceph-mon[28130]: mgrmap e9: x(active, since 25s) 2024-04-25T00:08:17.692 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:17.571+0000 7f826afae200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:08:17.692 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:17.690+0000 7f826afae200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:08:18.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:18 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/162705198' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T00:08:19.191 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:08:19.192 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T00:08:19.193 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T00:08:19.193 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T00:08:19.193 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T00:08:19.193 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:08:19.193 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:17.998+0000 7f9ecf600700 1 Processor -- start 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:17.999+0000 7f9ecf600700 1 -- start start 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:17.999+0000 7f9ecf600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec8108200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:17.999+0000 7f9ecf600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec8108740 con 0x7f9ec8107de0 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:17.999+0000 7f9eccc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec8108200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.000+0000 7f9eccc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec8108200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51068/0 (socket says 172.21.15.42:51068) 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.000+0000 7f9eccc00700 1 -- 172.21.15.42:0/4092967828 learned_addr learned my addr 172.21.15.42:0/4092967828 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.000+0000 7f9eccc00700 1 -- 172.21.15.42:0/4092967828 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec81088c0 con 0x7f9ec8107de0 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.000+0000 7f9eccc00700 1 --2- 172.21.15.42:0/4092967828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec8108200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0098a0 tx=0x7f9ebc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83aed6050810bf10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.001+0000 7f9ec7400700 1 -- 172.21.15.42:0/4092967828 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ebc004160 con 0x7f9ec8107de0 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.001+0000 7f9ec7400700 1 -- 172.21.15.42:0/4092967828 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ebc004300 con 0x7f9ec8107de0 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.001+0000 7f9ec7400700 1 -- 172.21.15.42:0/4092967828 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ebc011b20 con 0x7f9ec8107de0 2024-04-25T00:08:19.196 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.001+0000 7f9ecf600700 1 -- 172.21.15.42:0/4092967828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 msgr2=0x7f9ec8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.001+0000 7f9ecf600700 1 --2- 172.21.15.42:0/4092967828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec8108200 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0098a0 tx=0x7f9ebc009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.002+0000 7f9ecf600700 1 -- 172.21.15.42:0/4092967828 shutdown_connections 2024-04-25T00:08:19.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.002+0000 7f9ecf600700 1 --2- 172.21.15.42:0/4092967828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec8108200 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.002+0000 7f9ecf600700 1 -- 172.21.15.42:0/4092967828 >> 172.21.15.42:0/4092967828 conn(0x7f9ec8103460 msgr2=0x7f9ec8105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:19.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.002+0000 7f9ecf600700 1 -- 172.21.15.42:0/4092967828 shutdown_connections 2024-04-25T00:08:19.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.002+0000 7f9ecf600700 1 -- 172.21.15.42:0/4092967828 wait complete. 2024-04-25T00:08:19.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.003+0000 7f9ecf600700 1 Processor -- start 2024-04-25T00:08:19.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.003+0000 7f9ecf600700 1 -- start start 2024-04-25T00:08:19.197 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.003+0000 7f9ecf600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.198 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.003+0000 7f9ecf600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9ec810c250 con 0x7f9ec8107de0 2024-04-25T00:08:19.201 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.004+0000 7f9eccc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.004+0000 7f9eccc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51076/0 (socket says 172.21.15.42:51076) 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.004+0000 7f9eccc00700 1 -- 172.21.15.42:0/162705198 learned_addr learned my addr 172.21.15.42:0/162705198 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.004+0000 7f9eccc00700 1 -- 172.21.15.42:0/162705198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ec819e3c0 con 0x7f9ec8107de0 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.004+0000 7f9eccc00700 1 --2- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec819de80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0117e0 tx=0x7f9ebc011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.005+0000 7f9ec5600700 1 -- 172.21.15.42:0/162705198 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ebc003ba0 con 0x7f9ec8107de0 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.005+0000 7f9ec5600700 1 -- 172.21.15.42:0/162705198 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ebc01a910 con 0x7f9ec8107de0 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.005+0000 7f9ec5600700 1 -- 172.21.15.42:0/162705198 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9ebc0248f0 con 0x7f9ec8107de0 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.006+0000 7f9ecf600700 1 -- 172.21.15.42:0/162705198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ec819e650 con 0x7f9ec8107de0 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.006+0000 7f9ecf600700 1 -- 172.21.15.42:0/162705198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ec819eab0 con 0x7f9ec8107de0 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.007+0000 7f9ec5600700 1 -- 172.21.15.42:0/162705198 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f9ebc022020 con 0x7f9ec8107de0 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.007+0000 7f9ebaa00700 1 -- 172.21.15.42:0/162705198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ea4005330 con 0x7f9ec8107de0 2024-04-25T00:08:19.202 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.008+0000 7f9ec5600700 1 --2- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f9eb0039100 0x7f9eb003b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.008+0000 7f9ec5600700 1 -- 172.21.15.42:0/162705198 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f9ebc04d530 con 0x7f9ec8107de0 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.008+0000 7f9ec7e00700 1 -- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f9eb0039100 msgr2=0x7f9eb003b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3681869279 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.008+0000 7f9ec7e00700 1 --2- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f9eb0039100 0x7f9eb003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.017+0000 7f9ec5600700 1 -- 172.21.15.42:0/162705198 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9ebc00b5a0 con 0x7f9ec8107de0 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.209+0000 7f9ec7e00700 1 -- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f9eb0039100 msgr2=0x7f9eb003b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3681869279 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.209+0000 7f9ec7e00700 1 --2- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f9eb0039100 0x7f9eb003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.349+0000 7f9ebaa00700 1 -- 172.21.15.42:0/162705198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f9ea4005e30 con 0x7f9ec8107de0 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.349+0000 7f9ec5600700 1 -- 172.21.15.42:0/162705198 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f9ebc04ce80 con 0x7f9ec8107de0 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.352+0000 7f9ebaa00700 1 -- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f9eb0039100 msgr2=0x7f9eb003b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.352+0000 7f9ebaa00700 1 --2- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f9eb0039100 0x7f9eb003b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.352+0000 7f9ebaa00700 1 -- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 msgr2=0x7f9ec819de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.352+0000 7f9ebaa00700 1 --2- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec819de80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc0117e0 tx=0x7f9ebc011ba0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.352+0000 7f9ebaa00700 1 -- 172.21.15.42:0/162705198 shutdown_connections 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.352+0000 7f9ebaa00700 1 --2- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7f9eb0039100 0x7f9eb003b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.203 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.352+0000 7f9ebaa00700 1 --2- 172.21.15.42:0/162705198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9ec8107de0 0x7f9ec819de80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:19.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.352+0000 7f9ebaa00700 1 -- 172.21.15.42:0/162705198 >> 172.21.15.42:0/162705198 conn(0x7f9ec8103460 msgr2=0x7f9ec8103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:19.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.353+0000 7f9ebaa00700 1 -- 172.21.15.42:0/162705198 shutdown_connections 2024-04-25T00:08:19.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:18.353+0000 7f9ebaa00700 1 -- 172.21.15.42:0/162705198 wait complete. 2024-04-25T00:08:19.204 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for the mgr to restart... 2024-04-25T00:08:19.204 INFO:teuthology.orchestra.run.smithi042.stdout:Waiting for mgr epoch 9... 2024-04-25T00:08:19.626 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:19 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:19.361+0000 7f826afae200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:08:19.626 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:19 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:19.611+0000 7f826afae200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:08:20.028 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:19 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:19.835+0000 7f826afae200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:08:20.364 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:20 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:20.236+0000 7f826afae200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:08:20.623 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:20 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:20.362+0000 7f826afae200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:08:20.623 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:20 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:20.621+0000 7f826afae200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:08:21.376 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:20 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:20.885+0000 7f826afae200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:08:22.054 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:21 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:21.709+0000 7f826afae200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:08:22.376 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:22 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:22.297+0000 7f826afae200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:08:22.708 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:22 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:22.432+0000 7f826afae200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:08:22.971 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:22 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:22.707+0000 7f826afae200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:08:23.834 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:23 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:23.376+0000 7f826afae200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:08:23.834 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:23 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:23.502+0000 7f826afae200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:08:24.126 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:23 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:23.832+0000 7f826afae200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:08:24.126 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:24 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:24.019+0000 7f826afae200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:08:24.434 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:24 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:24.149+0000 7f826afae200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:08:24.434 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:24 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:24.305+0000 7f826afae200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:08:24.876 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:24 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:24.432+0000 7f826afae200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:08:25.760 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:25.467+0000 7f826afae200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:08:26.126 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:08:25.759+0000 7f826afae200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:08:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-mon[28130]: Active manager daemon x restarted 2024-04-25T00:08:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-mon[28130]: Activating manager daemon x 2024-04-25T00:08:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-mon[28130]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T00:08:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-mon[28130]: mgrmap e10: x(active, starting, since 0.00597698s) 2024-04-25T00:08:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:08:26.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T00:08:26.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T00:08:26.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T00:08:26.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T00:08:26.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:25 smithi042 ceph-mon[28130]: Manager daemon x is now available 2024-04-25T00:08:27.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:27.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T00:08:27.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T00:08:27.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:26 smithi042 ceph-mon[28130]: [25/Apr/2024:00:08:26] ENGINE Bus STARTING 2024-04-25T00:08:27.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:26 smithi042 ceph-mon[28130]: [25/Apr/2024:00:08:26] ENGINE Serving on http://172.21.15.42:8765 2024-04-25T00:08:27.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:26 smithi042 ceph-mon[28130]: [25/Apr/2024:00:08:26] ENGINE Serving on https://172.21.15.42:7150 2024-04-25T00:08:27.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:26 smithi042 ceph-mon[28130]: [25/Apr/2024:00:08:26] ENGINE Bus STARTED 2024-04-25T00:08:27.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:26 smithi042 ceph-mon[28130]: mgrmap e11: x(active, since 1.01072s) 2024-04-25T00:08:27.566 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout { 2024-04-25T00:08:27.567 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T00:08:27.567 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T00:08:27.567 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout } 2024-04-25T00:08:27.567 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.026+0000 7fa2a6200700 1 Processor -- start 2024-04-25T00:08:27.567 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.026+0000 7fa2a6200700 1 -- start start 2024-04-25T00:08:27.567 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.027+0000 7fa2a6200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a0108b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:27.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.027+0000 7fa2a6200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a007be30 con 0x7fa2a0106700 2024-04-25T00:08:27.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.027+0000 7fa29f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a0108b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:27.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.027+0000 7fa29f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a0108b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51086/0 (socket says 172.21.15.42:51086) 2024-04-25T00:08:27.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.027+0000 7fa29f400700 1 -- 172.21.15.42:0/3696203335 learned_addr learned my addr 172.21.15.42:0/3696203335 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:27.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.028+0000 7fa29f400700 1 -- 172.21.15.42:0/3696203335 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a0109070 con 0x7fa2a0106700 2024-04-25T00:08:27.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.028+0000 7fa29f400700 1 --2- 172.21.15.42:0/3696203335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a0108b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2900098a0 tx=0x7fa290009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118e8e640612a891 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:27.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.029+0000 7fa29e000700 1 -- 172.21.15.42:0/3696203335 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa290004160 con 0x7fa2a0106700 2024-04-25T00:08:27.569 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.029+0000 7fa29e000700 1 -- 172.21.15.42:0/3696203335 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa290004300 con 0x7fa2a0106700 2024-04-25T00:08:27.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.029+0000 7fa29e000700 1 -- 172.21.15.42:0/3696203335 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa290011b20 con 0x7fa2a0106700 2024-04-25T00:08:27.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.030+0000 7fa2a6200700 1 -- 172.21.15.42:0/3696203335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 msgr2=0x7fa2a0108b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:27.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.030+0000 7fa2a6200700 1 --2- 172.21.15.42:0/3696203335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a0108b30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa2900098a0 tx=0x7fa290009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:27.570 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.030+0000 7fa2a6200700 1 -- 172.21.15.42:0/3696203335 shutdown_connections 2024-04-25T00:08:27.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.030+0000 7fa2a6200700 1 --2- 172.21.15.42:0/3696203335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a0108b30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:27.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.030+0000 7fa2a6200700 1 -- 172.21.15.42:0/3696203335 >> 172.21.15.42:0/3696203335 conn(0x7fa2a0102360 msgr2=0x7fa2a0104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:27.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.030+0000 7fa2a6200700 1 -- 172.21.15.42:0/3696203335 shutdown_connections 2024-04-25T00:08:27.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.030+0000 7fa2a6200700 1 -- 172.21.15.42:0/3696203335 wait complete. 2024-04-25T00:08:27.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.032+0000 7fa2a6200700 1 Processor -- start 2024-04-25T00:08:27.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.032+0000 7fa2a6200700 1 -- start start 2024-04-25T00:08:27.574 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.033+0000 7fa2a6200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a01a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.033+0000 7fa2a6200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a010a140 con 0x7fa2a0106700 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.033+0000 7fa29f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a01a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.033+0000 7fa29f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a01a2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51104/0 (socket says 172.21.15.42:51104) 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.033+0000 7fa29f400700 1 -- 172.21.15.42:0/1184968771 learned_addr learned my addr 172.21.15.42:0/1184968771 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.034+0000 7fa29f400700 1 -- 172.21.15.42:0/1184968771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a01a2850 con 0x7fa2a0106700 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.034+0000 7fa29f400700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a01a2310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa2900117e0 tx=0x7fa290011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.034+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa290003ba0 con 0x7fa2a0106700 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.034+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa29001a910 con 0x7fa2a0106700 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.034+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa2900248f0 con 0x7fa2a0106700 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.034+0000 7fa2a6200700 1 -- 172.21.15.42:0/1184968771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a01a2ae0 con 0x7fa2a0106700 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.035+0000 7fa2a6200700 1 -- 172.21.15.42:0/1184968771 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a01a2f40 con 0x7fa2a0106700 2024-04-25T00:08:27.575 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.036+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fa290022020 con 0x7fa2a0106700 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.036+0000 7fa297e00700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 0x7fa28403b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.036+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 --> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa28403bcd0 con 0x7fa284039100 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.036+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa29004d4f0 con 0x7fa2a0106700 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.036+0000 7fa29ea00700 1 -- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 msgr2=0x7fa28403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3681869279 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.036+0000 7fa29ea00700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 0x7fa28403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.238+0000 7fa29ea00700 1 -- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 msgr2=0x7fa28403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3681869279 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.238+0000 7fa29ea00700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 0x7fa28403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.639+0000 7fa29ea00700 1 -- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 msgr2=0x7fa28403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3681869279 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:20.639+0000 7fa29ea00700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 0x7fa28403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:21.440+0000 7fa29ea00700 1 -- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 msgr2=0x7fa28403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3681869279 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:21.440+0000 7fa29ea00700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 0x7fa28403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:23.042+0000 7fa29ea00700 1 -- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 msgr2=0x7fa28403b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.42:6800/3681869279 2024-04-25T00:08:27.576 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:23.042+0000 7fa29ea00700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 0x7fa28403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T00:08:27.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:25.767+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7fa29004bd10 con 0x7fa2a0106700 2024-04-25T00:08:27.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:25.768+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 msgr2=0x7fa28403b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:08:27.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:25.768+0000 7fa297e00700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 0x7fa28403b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:27.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.770+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7fa290019cf0 con 0x7fa2a0106700 2024-04-25T00:08:27.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.771+0000 7fa297e00700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa28403cbc0 0x7fa28403efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:27.577 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.771+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa28403bcd0 con 0x7fa28403cbc0 2024-04-25T00:08:27.578 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.772+0000 7fa29ea00700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa28403cbc0 0x7fa28403efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:27.578 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.773+0000 7fa29ea00700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa28403cbc0 0x7fa28403efb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa28c003a10 tx=0x7fa28c009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.774+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fa28403bcd0 con 0x7fa28403cbc0 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.786+0000 7fa2a6200700 1 -- 172.21.15.42:0/1184968771 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fa2a0063890 con 0x7fa28403cbc0 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.786+0000 7fa297e00700 1 -- 172.21.15.42:0/1184968771 <== mgr.14152 v2:172.21.15.42:6800/384891702 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7fa2a0063890 con 0x7fa28403cbc0 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.787+0000 7fa2a6200700 1 -- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa28403cbc0 msgr2=0x7fa28403efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.787+0000 7fa2a6200700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa28403cbc0 0x7fa28403efb0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa28c003a10 tx=0x7fa28c009270 comp rx=0 tx=0).stop 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.787+0000 7fa2a6200700 1 -- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 msgr2=0x7fa2a01a2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.787+0000 7fa2a6200700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a01a2310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa2900117e0 tx=0x7fa290011ba0 comp rx=0 tx=0).stop 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.787+0000 7fa2a6200700 1 -- 172.21.15.42:0/1184968771 shutdown_connections 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.787+0000 7fa2a6200700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa28403cbc0 0x7fa28403efb0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.787+0000 7fa2a6200700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2a0106700 0x7fa2a01a2310 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.787+0000 7fa2a6200700 1 --2- 172.21.15.42:0/1184968771 >> [v2:172.21.15.42:6800/3681869279,v1:172.21.15.42:6801/3681869279] conn(0x7fa284039100 0x7fa28403b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.787+0000 7fa2a6200700 1 -- 172.21.15.42:0/1184968771 >> 172.21.15.42:0/1184968771 conn(0x7fa2a0102360 msgr2=0x7fa2a0104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:27.579 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.788+0000 7fa2a6200700 1 -- 172.21.15.42:0/1184968771 shutdown_connections 2024-04-25T00:08:27.580 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:26.788+0000 7fa2a6200700 1 -- 172.21.15.42:0/1184968771 wait complete. 2024-04-25T00:08:27.580 INFO:teuthology.orchestra.run.smithi042.stdout:mgr epoch 9 is available 2024-04-25T00:08:27.580 INFO:teuthology.orchestra.run.smithi042.stdout:Generating a dashboard self-signed certificate... 2024-04-25T00:08:28.123 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:27 smithi042 ceph-mon[28130]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T00:08:28.123 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:27 smithi042 ceph-mon[28130]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T00:08:28.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:28 smithi042 ceph-mon[28130]: mgrmap e12: x(active, since 2s) 2024-04-25T00:08:28.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:28 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:28.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:28 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:29.557 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T00:08:29.557 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.357+0000 7ff851200700 1 Processor -- start 2024-04-25T00:08:29.557 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.358+0000 7ff851200700 1 -- start start 2024-04-25T00:08:29.557 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.358+0000 7ff851200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c1070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.557 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.358+0000 7ff851200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c1076c0 con 0x7ff84c106cd0 2024-04-25T00:08:29.557 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.358+0000 7ff849e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c1070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.557 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.358+0000 7ff849e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c1070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49950/0 (socket says 172.21.15.42:49950) 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.358+0000 7ff849e00700 1 -- 172.21.15.42:0/3123232562 learned_addr learned my addr 172.21.15.42:0/3123232562 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.359+0000 7ff849e00700 1 -- 172.21.15.42:0/3123232562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c107ed0 con 0x7ff84c106cd0 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.359+0000 7ff849e00700 1 --2- 172.21.15.42:0/3123232562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c1070f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009e80 tx=0x7ff83c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1075c3a705661c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.359+0000 7ff849400700 1 -- 172.21.15.42:0/3123232562 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff83c017070 con 0x7ff84c106cd0 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.359+0000 7ff849400700 1 -- 172.21.15.42:0/3123232562 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c00b8c0 con 0x7ff84c106cd0 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.359+0000 7ff849400700 1 -- 172.21.15.42:0/3123232562 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff83c0045c0 con 0x7ff84c106cd0 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.360+0000 7ff851200700 1 -- 172.21.15.42:0/3123232562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 msgr2=0x7ff84c1070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.360+0000 7ff851200700 1 --2- 172.21.15.42:0/3123232562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c1070f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009e80 tx=0x7ff83c00b330 comp rx=0 tx=0).stop 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.360+0000 7ff851200700 1 -- 172.21.15.42:0/3123232562 shutdown_connections 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.360+0000 7ff851200700 1 --2- 172.21.15.42:0/3123232562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c1070f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.360+0000 7ff851200700 1 -- 172.21.15.42:0/3123232562 >> 172.21.15.42:0/3123232562 conn(0x7ff84c1022b0 msgr2=0x7ff84c1046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.360+0000 7ff851200700 1 -- 172.21.15.42:0/3123232562 shutdown_connections 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.360+0000 7ff851200700 1 -- 172.21.15.42:0/3123232562 wait complete. 2024-04-25T00:08:29.558 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.361+0000 7ff851200700 1 Processor -- start 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.362+0000 7ff851200700 1 -- start start 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.362+0000 7ff851200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c19dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.362+0000 7ff851200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c10b280 con 0x7ff84c106cd0 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.362+0000 7ff849e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c19dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.362+0000 7ff849e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c19dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49964/0 (socket says 172.21.15.42:49964) 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.362+0000 7ff849e00700 1 -- 172.21.15.42:0/596158218 learned_addr learned my addr 172.21.15.42:0/596158218 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.362+0000 7ff849e00700 1 -- 172.21.15.42:0/596158218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c19e530 con 0x7ff84c106cd0 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.363+0000 7ff849e00700 1 --2- 172.21.15.42:0/596158218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c19dff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009530 tx=0x7ff83c003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.363+0000 7ff843400700 1 -- 172.21.15.42:0/596158218 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff83c020020 con 0x7ff84c106cd0 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.363+0000 7ff851200700 1 -- 172.21.15.42:0/596158218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c19e7c0 con 0x7ff84c106cd0 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.363+0000 7ff843400700 1 -- 172.21.15.42:0/596158218 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c00bba0 con 0x7ff84c106cd0 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.363+0000 7ff843400700 1 -- 172.21.15.42:0/596158218 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff83c017040 con 0x7ff84c106cd0 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.363+0000 7ff851200700 1 -- 172.21.15.42:0/596158218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c1a14b0 con 0x7ff84c106cd0 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.364+0000 7ff843400700 1 -- 172.21.15.42:0/596158218 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ff83c023020 con 0x7ff84c106cd0 2024-04-25T00:08:29.559 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.364+0000 7ff851200700 1 -- 172.21.15.42:0/596158218 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84c063890 con 0x7ff84c106cd0 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.365+0000 7ff843400700 1 --2- 172.21.15.42:0/596158218 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8300390a0 0x7ff83003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.365+0000 7ff843400700 1 -- 172.21.15.42:0/596158218 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff83c00f050 con 0x7ff84c106cd0 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.365+0000 7ff841400700 1 --2- 172.21.15.42:0/596158218 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8300390a0 0x7ff83003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.366+0000 7ff841400700 1 --2- 172.21.15.42:0/596158218 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8300390a0 0x7ff83003b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff834006fd0 tx=0x7ff834006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.371+0000 7ff843400700 1 -- 172.21.15.42:0/596158218 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff83c02a020 con 0x7ff84c106cd0 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.607+0000 7ff851200700 1 -- 172.21.15.42:0/596158218 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7ff84c02de70 con 0x7ff8300390a0 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.697+0000 7ff843400700 1 -- 172.21.15.42:0/596158218 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7ff84c02de70 con 0x7ff8300390a0 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.699+0000 7ff851200700 1 -- 172.21.15.42:0/596158218 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8300390a0 msgr2=0x7ff83003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.699+0000 7ff851200700 1 --2- 172.21.15.42:0/596158218 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8300390a0 0x7ff83003b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff834006fd0 tx=0x7ff834006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.699+0000 7ff851200700 1 -- 172.21.15.42:0/596158218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 msgr2=0x7ff84c19dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.699+0000 7ff851200700 1 --2- 172.21.15.42:0/596158218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c19dff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009530 tx=0x7ff83c003d00 comp rx=0 tx=0).stop 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.700+0000 7ff851200700 1 -- 172.21.15.42:0/596158218 shutdown_connections 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.700+0000 7ff851200700 1 --2- 172.21.15.42:0/596158218 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8300390a0 0x7ff83003b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.700+0000 7ff851200700 1 --2- 172.21.15.42:0/596158218 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff84c106cd0 0x7ff84c19dff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:29.560 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.700+0000 7ff851200700 1 -- 172.21.15.42:0/596158218 >> 172.21.15.42:0/596158218 conn(0x7ff84c1022b0 msgr2=0x7ff84c103110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:29.561 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.700+0000 7ff851200700 1 -- 172.21.15.42:0/596158218 shutdown_connections 2024-04-25T00:08:29.561 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:28.700+0000 7ff851200700 1 -- 172.21.15.42:0/596158218 wait complete. 2024-04-25T00:08:29.561 INFO:teuthology.orchestra.run.smithi042.stdout:Creating initial admin user... 2024-04-25T00:08:30.213 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:30 smithi042 ceph-mon[28130]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:30.213 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:30 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:31.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:31 smithi042 ceph-mon[28130]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:31.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:31 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:31.501 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$lAciukA5qPPcVdWVnjABKer4thFKtJh83vIfkpXX63/EIv7rnnkeu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714003710, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T00:08:31.502 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.211+0000 7f4702800700 1 Processor -- start 2024-04-25T00:08:31.502 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.211+0000 7f4702800700 1 -- start start 2024-04-25T00:08:31.502 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.211+0000 7f4702800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:31.502 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.211+0000 7f4702800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc07c860 con 0x7f46fc07ddb0 2024-04-25T00:08:31.502 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.212+0000 7f46fb400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:31.502 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.212+0000 7f46fb400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49972/0 (socket says 172.21.15.42:49972) 2024-04-25T00:08:31.502 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.212+0000 7f46fb400700 1 -- 172.21.15.42:0/1713669177 learned_addr learned my addr 172.21.15.42:0/1713669177 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:31.502 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.213+0000 7f46fb400700 1 -- 172.21.15.42:0/1713669177 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc07c9e0 con 0x7f46fc07ddb0 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.213+0000 7f46fb400700 1 --2- 172.21.15.42:0/1713669177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc07c320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0098a0 tx=0x7f46ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d9b4ade6690a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.214+0000 7f46fa000700 1 -- 172.21.15.42:0/1713669177 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec004160 con 0x7f46fc07ddb0 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.214+0000 7f46fa000700 1 -- 172.21.15.42:0/1713669177 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ec004300 con 0x7f46fc07ddb0 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.214+0000 7f46fa000700 1 -- 172.21.15.42:0/1713669177 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec011b20 con 0x7f46fc07ddb0 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.214+0000 7f4702800700 1 -- 172.21.15.42:0/1713669177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 msgr2=0x7f46fc07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.214+0000 7f4702800700 1 --2- 172.21.15.42:0/1713669177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc07c320 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0098a0 tx=0x7f46ec009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.215+0000 7f4702800700 1 -- 172.21.15.42:0/1713669177 shutdown_connections 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.215+0000 7f4702800700 1 --2- 172.21.15.42:0/1713669177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc07c320 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.215+0000 7f4702800700 1 -- 172.21.15.42:0/1713669177 >> 172.21.15.42:0/1713669177 conn(0x7f46fc103460 msgr2=0x7f46fc105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.215+0000 7f4702800700 1 -- 172.21.15.42:0/1713669177 shutdown_connections 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.215+0000 7f4702800700 1 -- 172.21.15.42:0/1713669177 wait complete. 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.216+0000 7f4702800700 1 Processor -- start 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.216+0000 7f4702800700 1 -- start start 2024-04-25T00:08:31.503 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.217+0000 7f4702800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc1a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.217+0000 7f4702800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc109f00 con 0x7f46fc07ddb0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.217+0000 7f46fb400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc1a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.217+0000 7f46fb400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc1a2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49984/0 (socket says 172.21.15.42:49984) 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.217+0000 7f46fb400700 1 -- 172.21.15.42:0/2242634181 learned_addr learned my addr 172.21.15.42:0/2242634181 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.217+0000 7f46fb400700 1 -- 172.21.15.42:0/2242634181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc1a27b0 con 0x7f46fc07ddb0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.218+0000 7f46fb400700 1 --2- 172.21.15.42:0/2242634181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc1a2270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0117e0 tx=0x7f46ec011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.218+0000 7f46f3e00700 1 -- 172.21.15.42:0/2242634181 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec003ba0 con 0x7f46fc07ddb0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.218+0000 7f46f3e00700 1 -- 172.21.15.42:0/2242634181 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46ec01a910 con 0x7f46fc07ddb0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.218+0000 7f46f3e00700 1 -- 172.21.15.42:0/2242634181 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46ec0248f0 con 0x7f46fc07ddb0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.218+0000 7f4702800700 1 -- 172.21.15.42:0/2242634181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc1a2a40 con 0x7f46fc07ddb0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.218+0000 7f4702800700 1 -- 172.21.15.42:0/2242634181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc1a2ea0 con 0x7f46fc07ddb0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.219+0000 7f46f3e00700 1 -- 172.21.15.42:0/2242634181 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f46ec022020 con 0x7f46fc07ddb0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.220+0000 7f46f1600700 1 -- 172.21.15.42:0/2242634181 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d4005330 con 0x7f46fc07ddb0 2024-04-25T00:08:31.504 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.220+0000 7f46f3e00700 1 --2- 172.21.15.42:0/2242634181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e0039100 0x7f46e003b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.220+0000 7f46f3e00700 1 -- 172.21.15.42:0/2242634181 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f46ec04d630 con 0x7f46fc07ddb0 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.220+0000 7f46faa00700 1 --2- 172.21.15.42:0/2242634181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e0039100 0x7f46e003b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.221+0000 7f46faa00700 1 --2- 172.21.15.42:0/2242634181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e0039100 0x7f46e003b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f46e8006fd0 tx=0x7f46e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.226+0000 7f46f3e00700 1 -- 172.21.15.42:0/2242634181 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f46ec01aac0 con 0x7f46fc07ddb0 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.483+0000 7f46f1600700 1 -- 172.21.15.42:0/2242634181 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f46d4002400 con 0x7f46e0039100 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.737+0000 7f46f3e00700 1 -- 172.21.15.42:0/2242634181 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f46d4002400 con 0x7f46e0039100 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.740+0000 7f46f1600700 1 -- 172.21.15.42:0/2242634181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e0039100 msgr2=0x7f46e003b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.740+0000 7f46f1600700 1 --2- 172.21.15.42:0/2242634181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e0039100 0x7f46e003b5c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f46e8006fd0 tx=0x7f46e8006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.740+0000 7f46f1600700 1 -- 172.21.15.42:0/2242634181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 msgr2=0x7f46fc1a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.740+0000 7f46f1600700 1 --2- 172.21.15.42:0/2242634181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc1a2270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f46ec0117e0 tx=0x7f46ec011ba0 comp rx=0 tx=0).stop 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.741+0000 7f46f1600700 1 -- 172.21.15.42:0/2242634181 shutdown_connections 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.741+0000 7f46f1600700 1 --2- 172.21.15.42:0/2242634181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e0039100 0x7f46e003b5c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.741+0000 7f46f1600700 1 --2- 172.21.15.42:0/2242634181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc07ddb0 0x7f46fc1a2270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:31.505 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.741+0000 7f46f1600700 1 -- 172.21.15.42:0/2242634181 >> 172.21.15.42:0/2242634181 conn(0x7f46fc103460 msgr2=0x7f46fc199170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:31.506 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.741+0000 7f46f1600700 1 -- 172.21.15.42:0/2242634181 shutdown_connections 2024-04-25T00:08:31.506 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:30.741+0000 7f46f1600700 1 -- 172.21.15.42:0/2242634181 wait complete. 2024-04-25T00:08:31.506 INFO:teuthology.orchestra.run.smithi042.stdout:Fetching dashboard port number... 2024-04-25T00:08:32.419 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:33.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T00:08:33.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.149+0000 7f99d8200700 1 Processor -- start 2024-04-25T00:08:33.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.150+0000 7f99d8200700 1 -- start start 2024-04-25T00:08:33.277 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.150+0000 7f99d8200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:33.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.150+0000 7f99d8200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99d007c860 con 0x7f99d007ddb0 2024-04-25T00:08:33.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.150+0000 7f99d5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:33.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.150+0000 7f99d5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49986/0 (socket says 172.21.15.42:49986) 2024-04-25T00:08:33.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.151+0000 7f99d5800700 1 -- 172.21.15.42:0/1176181961 learned_addr learned my addr 172.21.15.42:0/1176181961 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:33.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.151+0000 7f99d5800700 1 -- 172.21.15.42:0/1176181961 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d007c9e0 con 0x7f99d007ddb0 2024-04-25T00:08:33.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.151+0000 7f99d5800700 1 --2- 172.21.15.42:0/1176181961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d007c320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99c40098a0 tx=0x7f99c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27598d0f33121eff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:33.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.152+0000 7f99cfe00700 1 -- 172.21.15.42:0/1176181961 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99c4004160 con 0x7f99d007ddb0 2024-04-25T00:08:33.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.152+0000 7f99cfe00700 1 -- 172.21.15.42:0/1176181961 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c4004300 con 0x7f99d007ddb0 2024-04-25T00:08:33.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.152+0000 7f99cfe00700 1 -- 172.21.15.42:0/1176181961 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99c4011b20 con 0x7f99d007ddb0 2024-04-25T00:08:33.278 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.153+0000 7f99d8200700 1 -- 172.21.15.42:0/1176181961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 msgr2=0x7f99d007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.153+0000 7f99d8200700 1 --2- 172.21.15.42:0/1176181961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d007c320 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99c40098a0 tx=0x7f99c4009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.153+0000 7f99d8200700 1 -- 172.21.15.42:0/1176181961 shutdown_connections 2024-04-25T00:08:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.153+0000 7f99d8200700 1 --2- 172.21.15.42:0/1176181961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d007c320 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.153+0000 7f99d8200700 1 -- 172.21.15.42:0/1176181961 >> 172.21.15.42:0/1176181961 conn(0x7f99d0103460 msgr2=0x7f99d0105880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.153+0000 7f99d8200700 1 -- 172.21.15.42:0/1176181961 shutdown_connections 2024-04-25T00:08:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.153+0000 7f99d8200700 1 -- 172.21.15.42:0/1176181961 wait complete. 2024-04-25T00:08:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.154+0000 7f99d8200700 1 Processor -- start 2024-04-25T00:08:33.279 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.154+0000 7f99d8200700 1 -- start start 2024-04-25T00:08:33.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.155+0000 7f99d8200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d019dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:33.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.155+0000 7f99d8200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f99d010e310 con 0x7f99d007ddb0 2024-04-25T00:08:33.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.155+0000 7f99d5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d019dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:33.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.155+0000 7f99d5800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d019dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49992/0 (socket says 172.21.15.42:49992) 2024-04-25T00:08:33.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.155+0000 7f99d5800700 1 -- 172.21.15.42:0/2244845806 learned_addr learned my addr 172.21.15.42:0/2244845806 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:33.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.156+0000 7f99d5800700 1 -- 172.21.15.42:0/2244845806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d019e3e0 con 0x7f99d007ddb0 2024-04-25T00:08:33.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.156+0000 7f99d5800700 1 --2- 172.21.15.42:0/2244845806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d019dea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f99c40117e0 tx=0x7f99c4011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:33.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.156+0000 7f99ce000700 1 -- 172.21.15.42:0/2244845806 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99c4003ba0 con 0x7f99d007ddb0 2024-04-25T00:08:33.280 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.156+0000 7f99ce000700 1 -- 172.21.15.42:0/2244845806 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c401a910 con 0x7f99d007ddb0 2024-04-25T00:08:33.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.156+0000 7f99ce000700 1 -- 172.21.15.42:0/2244845806 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f99c40248f0 con 0x7f99d007ddb0 2024-04-25T00:08:33.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.156+0000 7f99d8200700 1 -- 172.21.15.42:0/2244845806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d019e670 con 0x7f99d007ddb0 2024-04-25T00:08:33.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.156+0000 7f99d8200700 1 -- 172.21.15.42:0/2244845806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d019ead0 con 0x7f99d007ddb0 2024-04-25T00:08:33.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.158+0000 7f99ce000700 1 -- 172.21.15.42:0/2244845806 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f99c4022020 con 0x7f99d007ddb0 2024-04-25T00:08:33.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.158+0000 7f99d8200700 1 -- 172.21.15.42:0/2244845806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d0063890 con 0x7f99d007ddb0 2024-04-25T00:08:33.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.158+0000 7f99ce000700 1 --2- 172.21.15.42:0/2244845806 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f99b8039100 0x7f99b803b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:33.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.158+0000 7f99ce000700 1 -- 172.21.15.42:0/2244845806 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f99c404d960 con 0x7f99d007ddb0 2024-04-25T00:08:33.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.158+0000 7f99d4e00700 1 --2- 172.21.15.42:0/2244845806 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f99b8039100 0x7f99b803b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:33.281 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.159+0000 7f99d4e00700 1 --2- 172.21.15.42:0/2244845806 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f99b8039100 0x7f99b803b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f99bc006fd0 tx=0x7f99bc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.164+0000 7f99ce000700 1 -- 172.21.15.42:0/2244845806 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f99c401aac0 con 0x7f99d007ddb0 2024-04-25T00:08:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.331+0000 7f99ce000700 1 -- 172.21.15.42:0/2244845806 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f99c402d910 con 0x7f99d007ddb0 2024-04-25T00:08:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.421+0000 7f99d8200700 1 -- 172.21.15.42:0/2244845806 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f99d01a1560 con 0x7f99d007ddb0 2024-04-25T00:08:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.422+0000 7f99ce000700 1 -- 172.21.15.42:0/2244845806 <== mon.0 v2:172.21.15.42:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f99c4023370 con 0x7f99d007ddb0 2024-04-25T00:08:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.426+0000 7f99d8200700 1 -- 172.21.15.42:0/2244845806 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f99b8039100 msgr2=0x7f99b803b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.426+0000 7f99d8200700 1 --2- 172.21.15.42:0/2244845806 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f99b8039100 0x7f99b803b5c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f99bc006fd0 tx=0x7f99bc006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.426+0000 7f99d8200700 1 -- 172.21.15.42:0/2244845806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 msgr2=0x7f99d019dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.426+0000 7f99d8200700 1 --2- 172.21.15.42:0/2244845806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d019dea0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f99c40117e0 tx=0x7f99c4011ba0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.426+0000 7f99d8200700 1 -- 172.21.15.42:0/2244845806 shutdown_connections 2024-04-25T00:08:33.282 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.426+0000 7f99d8200700 1 --2- 172.21.15.42:0/2244845806 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f99b8039100 0x7f99b803b5c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.426+0000 7f99d8200700 1 --2- 172.21.15.42:0/2244845806 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f99d007ddb0 0x7f99d019dea0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:33.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.426+0000 7f99d8200700 1 -- 172.21.15.42:0/2244845806 >> 172.21.15.42:0/2244845806 conn(0x7f99d0103460 msgr2=0x7f99d0103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:33.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.426+0000 7f99d8200700 1 -- 172.21.15.42:0/2244845806 shutdown_connections 2024-04-25T00:08:33.283 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:32.426+0000 7f99d8200700 1 -- 172.21.15.42:0/2244845806 wait complete. 2024-04-25T00:08:33.303 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T00:08:33.303 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout disabled 2024-04-25T00:08:33.322 INFO:teuthology.orchestra.run.smithi042.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T00:08:33.322 INFO:teuthology.orchestra.run.smithi042.stdout:systemctl: stdout inactive 2024-04-25T00:08:33.322 INFO:teuthology.orchestra.run.smithi042.stdout:firewalld.service is not enabled 2024-04-25T00:08:33.322 INFO:teuthology.orchestra.run.smithi042.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T00:08:33.325 INFO:teuthology.orchestra.run.smithi042.stdout:Ceph Dashboard is now available at: 2024-04-25T00:08:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: URL: https://smithi042.front.sepia.ceph.com:8443/ 2024-04-25T00:08:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: User: admin 2024-04-25T00:08:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: Password: v0segf3ocf 2024-04-25T00:08:33.325 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:33.326 INFO:teuthology.orchestra.run.smithi042.stdout:Saving cluster configuration to /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config directory 2024-04-25T00:08:33.326 INFO:teuthology.orchestra.run.smithi042.stdout:Enabling autotune for osd_memory_target 2024-04-25T00:08:33.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:33 smithi042 ceph-mon[28130]: mgrmap e13: x(active, since 6s) 2024-04-25T00:08:33.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:33 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2244845806' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T00:08:35.531 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.616+0000 7f8a4da00700 1 Processor -- start 2024-04-25T00:08:35.532 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.617+0000 7f8a4da00700 1 -- start start 2024-04-25T00:08:35.532 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.617+0000 7f8a4da00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4810a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.617+0000 7f8a4da00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810a990 con 0x7f8a48109fa0 2024-04-25T00:08:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.617+0000 7f8a46a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4810a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.618+0000 7f8a46a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4810a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50002/0 (socket says 172.21.15.42:50002) 2024-04-25T00:08:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.618+0000 7f8a46a00700 1 -- 172.21.15.42:0/2018377469 learned_addr learned my addr 172.21.15.42:0/2018377469 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.618+0000 7f8a46a00700 1 -- 172.21.15.42:0/2018377469 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4810b1a0 con 0x7f8a48109fa0 2024-04-25T00:08:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.618+0000 7f8a46a00700 1 --2- 172.21.15.42:0/2018377469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4810a3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a380098a0 tx=0x7f8a38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a80b2a8a4a52ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.619+0000 7f8a45600700 1 -- 172.21.15.42:0/2018377469 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a38004160 con 0x7f8a48109fa0 2024-04-25T00:08:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.619+0000 7f8a45600700 1 -- 172.21.15.42:0/2018377469 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a38004300 con 0x7f8a48109fa0 2024-04-25T00:08:35.533 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.619+0000 7f8a45600700 1 -- 172.21.15.42:0/2018377469 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a38011b20 con 0x7f8a48109fa0 2024-04-25T00:08:35.536 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.619+0000 7f8a4da00700 1 -- 172.21.15.42:0/2018377469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 msgr2=0x7f8a4810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:35.536 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.619+0000 7f8a4da00700 1 --2- 172.21.15.42:0/2018377469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4810a3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8a380098a0 tx=0x7f8a38009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:35.536 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.620+0000 7f8a4da00700 1 -- 172.21.15.42:0/2018377469 shutdown_connections 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.620+0000 7f8a4da00700 1 --2- 172.21.15.42:0/2018377469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4810a3c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.620+0000 7f8a4da00700 1 -- 172.21.15.42:0/2018377469 >> 172.21.15.42:0/2018377469 conn(0x7f8a4807d3b0 msgr2=0x7f8a4807d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.620+0000 7f8a4da00700 1 -- 172.21.15.42:0/2018377469 shutdown_connections 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.620+0000 7f8a4da00700 1 -- 172.21.15.42:0/2018377469 wait complete. 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.621+0000 7f8a4da00700 1 Processor -- start 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.621+0000 7f8a4da00700 1 -- start start 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.621+0000 7f8a4da00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.621+0000 7f8a4da00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a4810e550 con 0x7f8a48109fa0 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.622+0000 7f8a46a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.622+0000 7f8a46a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50004/0 (socket says 172.21.15.42:50004) 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.622+0000 7f8a46a00700 1 -- 172.21.15.42:0/946741486 learned_addr learned my addr 172.21.15.42:0/946741486 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.622+0000 7f8a46a00700 1 -- 172.21.15.42:0/946741486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a4819e410 con 0x7f8a48109fa0 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.622+0000 7f8a46a00700 1 --2- 172.21.15.42:0/946741486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4819ded0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a380117e0 tx=0x7f8a38011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:35.537 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.623+0000 7f8a3f400700 1 -- 172.21.15.42:0/946741486 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a38003bf0 con 0x7f8a48109fa0 2024-04-25T00:08:35.538 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.623+0000 7f8a3f400700 1 -- 172.21.15.42:0/946741486 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a3801a910 con 0x7f8a48109fa0 2024-04-25T00:08:35.538 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.623+0000 7f8a3f400700 1 -- 172.21.15.42:0/946741486 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a38024970 con 0x7f8a48109fa0 2024-04-25T00:08:35.538 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.623+0000 7f8a4da00700 1 -- 172.21.15.42:0/946741486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a4819e6a0 con 0x7f8a48109fa0 2024-04-25T00:08:35.538 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.623+0000 7f8a4da00700 1 -- 172.21.15.42:0/946741486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a4819eb00 con 0x7f8a48109fa0 2024-04-25T00:08:35.538 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.624+0000 7f8a3f400700 1 -- 172.21.15.42:0/946741486 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8a38022020 con 0x7f8a48109fa0 2024-04-25T00:08:35.538 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.624+0000 7f8a4da00700 1 -- 172.21.15.42:0/946741486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a48063890 con 0x7f8a48109fa0 2024-04-25T00:08:35.538 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.625+0000 7f8a3f400700 1 --2- 172.21.15.42:0/946741486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a2c0391a0 0x7f8a2c03b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:35.538 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.625+0000 7f8a3f400700 1 -- 172.21.15.42:0/946741486 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8a3804d680 con 0x7f8a48109fa0 2024-04-25T00:08:35.540 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.625+0000 7f8a46000700 1 --2- 172.21.15.42:0/946741486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a2c0391a0 0x7f8a2c03b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:35.540 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.626+0000 7f8a46000700 1 --2- 172.21.15.42:0/946741486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a2c0391a0 0x7f8a2c03b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8a34006fd0 tx=0x7f8a34006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:35.540 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.631+0000 7f8a3f400700 1 -- 172.21.15.42:0/946741486 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8a38010380 con 0x7f8a48109fa0 2024-04-25T00:08:35.540 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.855+0000 7f8a4da00700 1 -- 172.21.15.42:0/946741486 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f8a481a1590 con 0x7f8a48109fa0 2024-04-25T00:08:35.540 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.855+0000 7f8a3f400700 1 -- 172.21.15.42:0/946741486 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f8a380212c0 con 0x7f8a48109fa0 2024-04-25T00:08:35.540 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.858+0000 7f8a4da00700 1 -- 172.21.15.42:0/946741486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a2c0391a0 msgr2=0x7f8a2c03b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:35.540 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.858+0000 7f8a4da00700 1 --2- 172.21.15.42:0/946741486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a2c0391a0 0x7f8a2c03b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8a34006fd0 tx=0x7f8a34006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:35.540 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.858+0000 7f8a4da00700 1 -- 172.21.15.42:0/946741486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 msgr2=0x7f8a4819ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:35.541 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.858+0000 7f8a4da00700 1 --2- 172.21.15.42:0/946741486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4819ded0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8a380117e0 tx=0x7f8a38011ba0 comp rx=0 tx=0).stop 2024-04-25T00:08:35.541 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.859+0000 7f8a4da00700 1 -- 172.21.15.42:0/946741486 shutdown_connections 2024-04-25T00:08:35.541 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.859+0000 7f8a4da00700 1 --2- 172.21.15.42:0/946741486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a2c0391a0 0x7f8a2c03b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:35.541 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.859+0000 7f8a4da00700 1 --2- 172.21.15.42:0/946741486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a48109fa0 0x7f8a4819ded0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:35.541 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.859+0000 7f8a4da00700 1 -- 172.21.15.42:0/946741486 >> 172.21.15.42:0/946741486 conn(0x7f8a4807d3b0 msgr2=0x7f8a481073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:35.541 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.859+0000 7f8a4da00700 1 -- 172.21.15.42:0/946741486 shutdown_connections 2024-04-25T00:08:35.541 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:34.859+0000 7f8a4da00700 1 -- 172.21.15.42:0/946741486 wait complete. 2024-04-25T00:08:37.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.184+0000 7fe633000700 1 Processor -- start 2024-04-25T00:08:37.204 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.184+0000 7fe633000700 1 -- start start 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.185+0000 7fe633000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c07c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.185+0000 7fe633000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c07c860 con 0x7fe62c07ddb0 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.185+0000 7fe631c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c07c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.185+0000 7fe631c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c07c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50018/0 (socket says 172.21.15.42:50018) 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.185+0000 7fe631c00700 1 -- 172.21.15.42:0/1308883185 learned_addr learned my addr 172.21.15.42:0/1308883185 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.186+0000 7fe631c00700 1 -- 172.21.15.42:0/1308883185 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c07c9e0 con 0x7fe62c07ddb0 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.186+0000 7fe631c00700 1 --2- 172.21.15.42:0/1308883185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c07c320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe6200098a0 tx=0x7fe620009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc76d60bc9e5d892 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.186+0000 7fe62be00700 1 -- 172.21.15.42:0/1308883185 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe620004160 con 0x7fe62c07ddb0 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.186+0000 7fe62be00700 1 -- 172.21.15.42:0/1308883185 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe620004300 con 0x7fe62c07ddb0 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.186+0000 7fe62be00700 1 -- 172.21.15.42:0/1308883185 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe620011b20 con 0x7fe62c07ddb0 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.187+0000 7fe633000700 1 -- 172.21.15.42:0/1308883185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 msgr2=0x7fe62c07c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.187+0000 7fe633000700 1 --2- 172.21.15.42:0/1308883185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c07c320 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe6200098a0 tx=0x7fe620009bb0 comp rx=0 tx=0).stop 2024-04-25T00:08:37.205 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.187+0000 7fe633000700 1 -- 172.21.15.42:0/1308883185 shutdown_connections 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.187+0000 7fe633000700 1 --2- 172.21.15.42:0/1308883185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c07c320 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.187+0000 7fe633000700 1 -- 172.21.15.42:0/1308883185 >> 172.21.15.42:0/1308883185 conn(0x7fe62c103280 msgr2=0x7fe62c1056a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.187+0000 7fe633000700 1 -- 172.21.15.42:0/1308883185 shutdown_connections 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.187+0000 7fe633000700 1 -- 172.21.15.42:0/1308883185 wait complete. 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.188+0000 7fe633000700 1 Processor -- start 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.188+0000 7fe633000700 1 -- start start 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.189+0000 7fe633000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c19dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.189+0000 7fe633000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c10e150 con 0x7fe62c07ddb0 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.189+0000 7fe631c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c19dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.189+0000 7fe631c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c19dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50032/0 (socket says 172.21.15.42:50032) 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.189+0000 7fe631c00700 1 -- 172.21.15.42:0/2590671959 learned_addr learned my addr 172.21.15.42:0/2590671959 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.189+0000 7fe631c00700 1 -- 172.21.15.42:0/2590671959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c19e2e0 con 0x7fe62c07ddb0 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.190+0000 7fe631c00700 1 --2- 172.21.15.42:0/2590671959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c19dda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe6200116a0 tx=0x7fe620011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:37.206 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.190+0000 7fe62a000700 1 -- 172.21.15.42:0/2590671959 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe620003a40 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.190+0000 7fe62a000700 1 -- 172.21.15.42:0/2590671959 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe620003be0 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.190+0000 7fe62a000700 1 -- 172.21.15.42:0/2590671959 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe62001a750 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.190+0000 7fe633000700 1 -- 172.21.15.42:0/2590671959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c19e570 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.190+0000 7fe633000700 1 -- 172.21.15.42:0/2590671959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c1a1220 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.191+0000 7fe62a000700 1 -- 172.21.15.42:0/2590671959 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe620024aa0 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.192+0000 7fe633000700 1 -- 172.21.15.42:0/2590671959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62c063890 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.192+0000 7fe62a000700 1 --2- 172.21.15.42:0/2590671959 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe614039150 0x7fe61403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.192+0000 7fe62a000700 1 -- 172.21.15.42:0/2590671959 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe62004d8c0 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.192+0000 7fe631200700 1 --2- 172.21.15.42:0/2590671959 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe614039150 0x7fe61403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.193+0000 7fe631200700 1 --2- 172.21.15.42:0/2590671959 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe614039150 0x7fe61403b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe618006fd0 tx=0x7fe618006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.198+0000 7fe62a000700 1 -- 172.21.15.42:0/2590671959 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe620011e20 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.542+0000 7fe633000700 1 -- 172.21.15.42:0/2590671959 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe62c1a13f0 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.546+0000 7fe62a000700 1 -- 172.21.15.42:0/2590671959 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fe620023390 con 0x7fe62c07ddb0 2024-04-25T00:08:37.207 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.548+0000 7fe633000700 1 -- 172.21.15.42:0/2590671959 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe614039150 msgr2=0x7fe61403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.548+0000 7fe633000700 1 --2- 172.21.15.42:0/2590671959 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe614039150 0x7fe61403b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe618006fd0 tx=0x7fe618006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.548+0000 7fe633000700 1 -- 172.21.15.42:0/2590671959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 msgr2=0x7fe62c19dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.548+0000 7fe633000700 1 --2- 172.21.15.42:0/2590671959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c19dda0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe6200116a0 tx=0x7fe620011ba0 comp rx=0 tx=0).stop 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.548+0000 7fe633000700 1 -- 172.21.15.42:0/2590671959 shutdown_connections 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.549+0000 7fe633000700 1 --2- 172.21.15.42:0/2590671959 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe614039150 0x7fe61403b610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.549+0000 7fe633000700 1 --2- 172.21.15.42:0/2590671959 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe62c07ddb0 0x7fe62c19dda0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.549+0000 7fe633000700 1 -- 172.21.15.42:0/2590671959 >> 172.21.15.42:0/2590671959 conn(0x7fe62c103280 msgr2=0x7fe62c103de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.549+0000 7fe633000700 1 -- 172.21.15.42:0/2590671959 shutdown_connections 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr 2024-04-25T00:08:36.549+0000 7fe633000700 1 -- 172.21.15.42:0/2590671959 wait complete. 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:08:37.208 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: ceph telemetry on 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout:For more information see: 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:37.209 INFO:teuthology.orchestra.run.smithi042.stdout:Bootstrap complete. 2024-04-25T00:08:37.291 INFO:tasks.cephadm:Fetching config... 2024-04-25T00:08:37.291 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:08:37.291 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T00:08:37.338 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T00:08:37.338 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:08:37.338 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T00:08:37.423 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T00:08:37.424 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:08:37.424 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T00:08:37.545 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T00:08:37.545 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:08:37.545 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T00:08:37.597 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T00:08:37.597 DEBUG:teuthology.orchestra.run.smithi042:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+yawZQKu9a1Cm8B8jvCDmyu2MGQW6dQo/CDvsUxQj22oemCgo1KT7tAhV2mwkg/vjbKju7SObWxZD/ltKmALJx1zTa7sRknntm3UCyPclw/YW3deEH8gsyqmpUoLgyW2GLOjoI30tUJdfJUVTzsbZ+CubiLwLBh1cpy7dgdkuQMYPtK5Xv83bpdZAZ/YNuulNbeQtkPdiUvKbKLYrVHIMM5pYIUUXUwaYmDqIONVB4u/eedw3WJjLtjeLaoWSH5SagJRrZAGTU6W1D0EmTAPF/PJUhf2TXx8e5dTA0tgF/R3u67ypftYOZ5pIVEBmoaQ5Q72AhM7p6J+/GsbSKkSlKG48RwdlrperUO9UBkwLp1k5953oOIjplSipDcR53c6T2AXMfMzXdyVMyKTDmFRGnFF/Ys2QPxvfFrQjGEYEvrsxiiztb2/MTyko9HJNel8XZgnNXLe5lMAZOJv7FEqsJ26k6S+vlNd3RTrtluJVJw9bdOOeFhuRAOUq8CqaH3E= ceph-9d95543e-0297-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:08:37.702 INFO:teuthology.orchestra.run.smithi042.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+yawZQKu9a1Cm8B8jvCDmyu2MGQW6dQo/CDvsUxQj22oemCgo1KT7tAhV2mwkg/vjbKju7SObWxZD/ltKmALJx1zTa7sRknntm3UCyPclw/YW3deEH8gsyqmpUoLgyW2GLOjoI30tUJdfJUVTzsbZ+CubiLwLBh1cpy7dgdkuQMYPtK5Xv83bpdZAZ/YNuulNbeQtkPdiUvKbKLYrVHIMM5pYIUUXUwaYmDqIONVB4u/eedw3WJjLtjeLaoWSH5SagJRrZAGTU6W1D0EmTAPF/PJUhf2TXx8e5dTA0tgF/R3u67ypftYOZ5pIVEBmoaQ5Q72AhM7p6J+/GsbSKkSlKG48RwdlrperUO9UBkwLp1k5953oOIjplSipDcR53c6T2AXMfMzXdyVMyKTDmFRGnFF/Ys2QPxvfFrQjGEYEvrsxiiztb2/MTyko9HJNel8XZgnNXLe5lMAZOJv7FEqsJ26k6S+vlNd3RTrtluJVJw9bdOOeFhuRAOUq8CqaH3E= ceph-9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:08:37.730 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:37 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2590671959' entity='client.admin' 2024-04-25T00:08:37.739 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+yawZQKu9a1Cm8B8jvCDmyu2MGQW6dQo/CDvsUxQj22oemCgo1KT7tAhV2mwkg/vjbKju7SObWxZD/ltKmALJx1zTa7sRknntm3UCyPclw/YW3deEH8gsyqmpUoLgyW2GLOjoI30tUJdfJUVTzsbZ+CubiLwLBh1cpy7dgdkuQMYPtK5Xv83bpdZAZ/YNuulNbeQtkPdiUvKbKLYrVHIMM5pYIUUXUwaYmDqIONVB4u/eedw3WJjLtjeLaoWSH5SagJRrZAGTU6W1D0EmTAPF/PJUhf2TXx8e5dTA0tgF/R3u67ypftYOZ5pIVEBmoaQ5Q72AhM7p6J+/GsbSKkSlKG48RwdlrperUO9UBkwLp1k5953oOIjplSipDcR53c6T2AXMfMzXdyVMyKTDmFRGnFF/Ys2QPxvfFrQjGEYEvrsxiiztb2/MTyko9HJNel8XZgnNXLe5lMAZOJv7FEqsJ26k6S+vlNd3RTrtluJVJw9bdOOeFhuRAOUq8CqaH3E= ceph-9d95543e-0297-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:08:37.782 INFO:teuthology.orchestra.run.smithi106.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+yawZQKu9a1Cm8B8jvCDmyu2MGQW6dQo/CDvsUxQj22oemCgo1KT7tAhV2mwkg/vjbKju7SObWxZD/ltKmALJx1zTa7sRknntm3UCyPclw/YW3deEH8gsyqmpUoLgyW2GLOjoI30tUJdfJUVTzsbZ+CubiLwLBh1cpy7dgdkuQMYPtK5Xv83bpdZAZ/YNuulNbeQtkPdiUvKbKLYrVHIMM5pYIUUXUwaYmDqIONVB4u/eedw3WJjLtjeLaoWSH5SagJRrZAGTU6W1D0EmTAPF/PJUhf2TXx8e5dTA0tgF/R3u67ypftYOZ5pIVEBmoaQ5Q72AhM7p6J+/GsbSKkSlKG48RwdlrperUO9UBkwLp1k5953oOIjplSipDcR53c6T2AXMfMzXdyVMyKTDmFRGnFF/Ys2QPxvfFrQjGEYEvrsxiiztb2/MTyko9HJNel8XZgnNXLe5lMAZOJv7FEqsJ26k6S+vlNd3RTrtluJVJw9bdOOeFhuRAOUq8CqaH3E= ceph-9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:08:37.793 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+yawZQKu9a1Cm8B8jvCDmyu2MGQW6dQo/CDvsUxQj22oemCgo1KT7tAhV2mwkg/vjbKju7SObWxZD/ltKmALJx1zTa7sRknntm3UCyPclw/YW3deEH8gsyqmpUoLgyW2GLOjoI30tUJdfJUVTzsbZ+CubiLwLBh1cpy7dgdkuQMYPtK5Xv83bpdZAZ/YNuulNbeQtkPdiUvKbKLYrVHIMM5pYIUUXUwaYmDqIONVB4u/eedw3WJjLtjeLaoWSH5SagJRrZAGTU6W1D0EmTAPF/PJUhf2TXx8e5dTA0tgF/R3u67ypftYOZ5pIVEBmoaQ5Q72AhM7p6J+/GsbSKkSlKG48RwdlrperUO9UBkwLp1k5953oOIjplSipDcR53c6T2AXMfMzXdyVMyKTDmFRGnFF/Ys2QPxvfFrQjGEYEvrsxiiztb2/MTyko9HJNel8XZgnNXLe5lMAZOJv7FEqsJ26k6S+vlNd3RTrtluJVJw9bdOOeFhuRAOUq8CqaH3E= ceph-9d95543e-0297-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T00:08:37.835 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+yawZQKu9a1Cm8B8jvCDmyu2MGQW6dQo/CDvsUxQj22oemCgo1KT7tAhV2mwkg/vjbKju7SObWxZD/ltKmALJx1zTa7sRknntm3UCyPclw/YW3deEH8gsyqmpUoLgyW2GLOjoI30tUJdfJUVTzsbZ+CubiLwLBh1cpy7dgdkuQMYPtK5Xv83bpdZAZ/YNuulNbeQtkPdiUvKbKLYrVHIMM5pYIUUXUwaYmDqIONVB4u/eedw3WJjLtjeLaoWSH5SagJRrZAGTU6W1D0EmTAPF/PJUhf2TXx8e5dTA0tgF/R3u67ypftYOZ5pIVEBmoaQ5Q72AhM7p6J+/GsbSKkSlKG48RwdlrperUO9UBkwLp1k5953oOIjplSipDcR53c6T2AXMfMzXdyVMyKTDmFRGnFF/Ys2QPxvfFrQjGEYEvrsxiiztb2/MTyko9HJNel8XZgnNXLe5lMAZOJv7FEqsJ26k6S+vlNd3RTrtluJVJw9bdOOeFhuRAOUq8CqaH3E= ceph-9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:08:37.846 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T00:08:38.285 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:08:40.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.825+0000 7f76a3e00700 1 -- 172.21.15.42:0/2875917733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f769c102720 msgr2=0x7f769c104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:40.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.825+0000 7f76a3e00700 1 --2- 172.21.15.42:0/2875917733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f769c102720 0x7f769c104b10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f768c009910 tx=0x7f768c009c20 comp rx=0 tx=0).stop 2024-04-25T00:08:40.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.825+0000 7f76a3e00700 1 -- 172.21.15.42:0/2875917733 shutdown_connections 2024-04-25T00:08:40.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.825+0000 7f76a3e00700 1 --2- 172.21.15.42:0/2875917733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f769c102720 0x7f769c104b10 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:40.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.825+0000 7f76a3e00700 1 -- 172.21.15.42:0/2875917733 >> 172.21.15.42:0/2875917733 conn(0x7f769c0fc130 msgr2=0x7f769c0fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.825+0000 7f76a3e00700 1 -- 172.21.15.42:0/2875917733 shutdown_connections 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.826+0000 7f76a3e00700 1 -- 172.21.15.42:0/2875917733 wait complete. 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.827+0000 7f76a3e00700 1 Processor -- start 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.827+0000 7f76a3e00700 1 -- start start 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.827+0000 7f76a3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f769c102720 0x7f769c101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.828+0000 7f76a3e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f769c107cc0 con 0x7f769c102720 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.828+0000 7f76a1400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f769c102720 0x7f769c101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.828+0000 7f76a1400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f769c102720 0x7f769c101fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39584/0 (socket says 172.21.15.42:39584) 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.828+0000 7f76a1400700 1 -- 172.21.15.42:0/1966654166 learned_addr learned my addr 172.21.15.42:0/1966654166 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.828+0000 7f76a1400700 1 -- 172.21.15.42:0/1966654166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f769c1024e0 con 0x7f769c102720 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.828+0000 7f76a1400700 1 --2- 172.21.15.42:0/1966654166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f769c102720 0x7f769c101fa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f768c005950 tx=0x7f768c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.829+0000 7f769a000700 1 -- 172.21.15.42:0/1966654166 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f768c026070 con 0x7f769c102720 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.829+0000 7f769a000700 1 -- 172.21.15.42:0/1966654166 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f768c00bc30 con 0x7f769c102720 2024-04-25T00:08:40.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.829+0000 7f769a000700 1 -- 172.21.15.42:0/1966654166 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f768c003b30 con 0x7f769c102720 2024-04-25T00:08:40.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.829+0000 7f76a3e00700 1 -- 172.21.15.42:0/1966654166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f769c1006d0 con 0x7f769c102720 2024-04-25T00:08:40.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.829+0000 7f76a3e00700 1 -- 172.21.15.42:0/1966654166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f769c100c10 con 0x7f769c102720 2024-04-25T00:08:40.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.830+0000 7f769a000700 1 -- 172.21.15.42:0/1966654166 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f768c029070 con 0x7f769c102720 2024-04-25T00:08:40.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.830+0000 7f76a3e00700 1 -- 172.21.15.42:0/1966654166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769c108440 con 0x7f769c102720 2024-04-25T00:08:40.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.830+0000 7f769a000700 1 --2- 172.21.15.42:0/1966654166 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7680039150 0x7f768003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:40.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.830+0000 7f769a000700 1 -- 172.21.15.42:0/1966654166 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f768c01e070 con 0x7f769c102720 2024-04-25T00:08:40.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.831+0000 7f76a0a00700 1 --2- 172.21.15.42:0/1966654166 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7680039150 0x7f768003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:40.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.831+0000 7f76a0a00700 1 --2- 172.21.15.42:0/1966654166 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7680039150 0x7f768003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7688006fd0 tx=0x7f7688006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:40.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:40.837+0000 7f769a000700 1 -- 172.21.15.42:0/1966654166 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f768c013070 con 0x7f769c102720 2024-04-25T00:08:41.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.070+0000 7f76a3e00700 1 -- 172.21.15.42:0/1966654166 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f769c1907b0 con 0x7f769c102720 2024-04-25T00:08:41.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.075+0000 7f769a000700 1 -- 172.21.15.42:0/1966654166 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f768c030020 con 0x7f769c102720 2024-04-25T00:08:41.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.078+0000 7f76a3e00700 1 -- 172.21.15.42:0/1966654166 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7680039150 msgr2=0x7f768003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:41.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.078+0000 7f76a3e00700 1 --2- 172.21.15.42:0/1966654166 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7680039150 0x7f768003b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7688006fd0 tx=0x7f7688006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:41.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.079+0000 7f76a3e00700 1 -- 172.21.15.42:0/1966654166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f769c102720 msgr2=0x7f769c101fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:41.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.079+0000 7f76a3e00700 1 --2- 172.21.15.42:0/1966654166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f769c102720 0x7f769c101fa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f768c005950 tx=0x7f768c005030 comp rx=0 tx=0).stop 2024-04-25T00:08:41.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.079+0000 7f76a3e00700 1 -- 172.21.15.42:0/1966654166 shutdown_connections 2024-04-25T00:08:41.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.079+0000 7f76a3e00700 1 --2- 172.21.15.42:0/1966654166 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7680039150 0x7f768003b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:41.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.079+0000 7f76a3e00700 1 --2- 172.21.15.42:0/1966654166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f769c102720 0x7f769c101fa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:41.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.079+0000 7f76a3e00700 1 -- 172.21.15.42:0/1966654166 >> 172.21.15.42:0/1966654166 conn(0x7f769c0fc130 msgr2=0x7f769c0fcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:41.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.079+0000 7f76a3e00700 1 -- 172.21.15.42:0/1966654166 shutdown_connections 2024-04-25T00:08:41.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:41.079+0000 7f76a3e00700 1 -- 172.21.15.42:0/1966654166 wait complete. 2024-04-25T00:08:41.929 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T00:08:41.929 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T00:08:42.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:42 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1966654166' entity='client.admin' 2024-04-25T00:08:42.380 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:08:44.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.753+0000 7fb410000700 1 -- 172.21.15.42:0/1165274427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4080746c0 msgr2=0x7fb408074ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:44.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.753+0000 7fb410000700 1 --2- 172.21.15.42:0/1165274427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4080746c0 0x7fb408074ae0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc009910 tx=0x7fb3fc009c20 comp rx=0 tx=0).stop 2024-04-25T00:08:44.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.753+0000 7fb410000700 1 -- 172.21.15.42:0/1165274427 shutdown_connections 2024-04-25T00:08:44.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.753+0000 7fb410000700 1 --2- 172.21.15.42:0/1165274427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4080746c0 0x7fb408074ae0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:44.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.753+0000 7fb410000700 1 -- 172.21.15.42:0/1165274427 >> 172.21.15.42:0/1165274427 conn(0x7fb4080fd280 msgr2=0x7fb4080ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:44.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.754+0000 7fb410000700 1 -- 172.21.15.42:0/1165274427 shutdown_connections 2024-04-25T00:08:44.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.754+0000 7fb410000700 1 -- 172.21.15.42:0/1165274427 wait complete. 2024-04-25T00:08:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.755+0000 7fb410000700 1 Processor -- start 2024-04-25T00:08:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.755+0000 7fb410000700 1 -- start start 2024-04-25T00:08:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.755+0000 7fb410000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4080746c0 0x7fb4081996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:44.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.755+0000 7fb410000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb40810a050 con 0x7fb4080746c0 2024-04-25T00:08:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.755+0000 7fb40d600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4080746c0 0x7fb4081996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.755+0000 7fb40d600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4080746c0 0x7fb4081996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39612/0 (socket says 172.21.15.42:39612) 2024-04-25T00:08:44.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.756+0000 7fb40d600700 1 -- 172.21.15.42:0/3274671825 learned_addr learned my addr 172.21.15.42:0/3274671825 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:44.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.756+0000 7fb40d600700 1 -- 172.21.15.42:0/3274671825 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb408199be0 con 0x7fb4080746c0 2024-04-25T00:08:44.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.756+0000 7fb40d600700 1 --2- 172.21.15.42:0/3274671825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4080746c0 0x7fb4081996a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc005950 tx=0x7fb3fc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:44.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.756+0000 7fb406000700 1 -- 172.21.15.42:0/3274671825 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3fc026070 con 0x7fb4080746c0 2024-04-25T00:08:44.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.757+0000 7fb406000700 1 -- 172.21.15.42:0/3274671825 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3fc00bc30 con 0x7fb4080746c0 2024-04-25T00:08:44.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.757+0000 7fb410000700 1 -- 172.21.15.42:0/3274671825 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb408199e70 con 0x7fb4080746c0 2024-04-25T00:08:44.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.757+0000 7fb406000700 1 -- 172.21.15.42:0/3274671825 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3fc003b30 con 0x7fb4080746c0 2024-04-25T00:08:44.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.757+0000 7fb410000700 1 -- 172.21.15.42:0/3274671825 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40819cb60 con 0x7fb4080746c0 2024-04-25T00:08:44.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.758+0000 7fb406000700 1 -- 172.21.15.42:0/3274671825 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb3fc029070 con 0x7fb4080746c0 2024-04-25T00:08:44.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.758+0000 7fb410000700 1 -- 172.21.15.42:0/3274671825 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40810a7d0 con 0x7fb4080746c0 2024-04-25T00:08:44.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.759+0000 7fb406000700 1 --2- 172.21.15.42:0/3274671825 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3f0039150 0x7fb3f003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:44.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.759+0000 7fb406000700 1 -- 172.21.15.42:0/3274671825 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb3fc01e070 con 0x7fb4080746c0 2024-04-25T00:08:44.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.759+0000 7fb40cc00700 1 --2- 172.21.15.42:0/3274671825 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3f0039150 0x7fb3f003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:44.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.760+0000 7fb40cc00700 1 --2- 172.21.15.42:0/3274671825 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3f0039150 0x7fb3f003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4006fd0 tx=0x7fb3f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:44.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:44.764+0000 7fb406000700 1 -- 172.21.15.42:0/3274671825 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb3fc013070 con 0x7fb4080746c0 2024-04-25T00:08:45.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.002+0000 7fb410000700 1 -- 172.21.15.42:0/3274671825 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fb408062670 con 0x7fb3f0039150 2024-04-25T00:08:45.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.008+0000 7fb406000700 1 -- 172.21.15.42:0/3274671825 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb408062670 con 0x7fb3f0039150 2024-04-25T00:08:45.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.011+0000 7fb410000700 1 -- 172.21.15.42:0/3274671825 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3f0039150 msgr2=0x7fb3f003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:45.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.011+0000 7fb410000700 1 --2- 172.21.15.42:0/3274671825 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3f0039150 0x7fb3f003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4006fd0 tx=0x7fb3f4006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:45.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.011+0000 7fb410000700 1 -- 172.21.15.42:0/3274671825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4080746c0 msgr2=0x7fb4081996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:45.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.011+0000 7fb410000700 1 --2- 172.21.15.42:0/3274671825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4080746c0 0x7fb4081996a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc005950 tx=0x7fb3fc005030 comp rx=0 tx=0).stop 2024-04-25T00:08:45.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.011+0000 7fb410000700 1 -- 172.21.15.42:0/3274671825 shutdown_connections 2024-04-25T00:08:45.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.011+0000 7fb410000700 1 --2- 172.21.15.42:0/3274671825 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3f0039150 0x7fb3f003b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:45.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.011+0000 7fb410000700 1 --2- 172.21.15.42:0/3274671825 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb4080746c0 0x7fb4081996a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:45.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.011+0000 7fb410000700 1 -- 172.21.15.42:0/3274671825 >> 172.21.15.42:0/3274671825 conn(0x7fb4080fd280 msgr2=0x7fb4080fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:45.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.012+0000 7fb410000700 1 -- 172.21.15.42:0/3274671825 shutdown_connections 2024-04-25T00:08:45.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:45.012+0000 7fb410000700 1 -- 172.21.15.42:0/3274671825 wait complete. 2024-04-25T00:08:45.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:45.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:45.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi042", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:08:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:08:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:08:45.853 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi106 2024-04-25T00:08:45.853 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T00:08:45.853 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.conf 2024-04-25T00:08:45.871 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T00:08:45.871 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:08:45.928 INFO:tasks.cephadm:Adding host smithi106 to orchestrator... 2024-04-25T00:08:45.929 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch host add smithi106 2024-04-25T00:08:46.343 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:08:46.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:46 smithi042 ceph-mon[28130]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:46.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:46 smithi042 ceph-mon[28130]: Updating smithi042:/etc/ceph/ceph.conf 2024-04-25T00:08:47.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:47 smithi042 ceph-mon[28130]: Updating smithi042:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:08:47.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:47 smithi042 ceph-mon[28130]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:47.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:47 smithi042 ceph-mon[28130]: Updating smithi042:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:08:48.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.434+0000 7f4192e00700 1 -- 172.21.15.42:0/2286433789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418c074550 msgr2=0x7f418c074930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:48.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.434+0000 7f4192e00700 1 --2- 172.21.15.42:0/2286433789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418c074550 0x7f418c074930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f417c009910 tx=0x7f417c009c20 comp rx=0 tx=0).stop 2024-04-25T00:08:48.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.434+0000 7f4192e00700 1 -- 172.21.15.42:0/2286433789 shutdown_connections 2024-04-25T00:08:48.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.434+0000 7f4192e00700 1 --2- 172.21.15.42:0/2286433789 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418c074550 0x7f418c074930 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:48.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.434+0000 7f4192e00700 1 -- 172.21.15.42:0/2286433789 >> 172.21.15.42:0/2286433789 conn(0x7f418c0fd770 msgr2=0x7f418c0ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:48.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.434+0000 7f4192e00700 1 -- 172.21.15.42:0/2286433789 shutdown_connections 2024-04-25T00:08:48.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.434+0000 7f4192e00700 1 -- 172.21.15.42:0/2286433789 wait complete. 2024-04-25T00:08:48.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.435+0000 7f4192e00700 1 Processor -- start 2024-04-25T00:08:48.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.436+0000 7f4192e00700 1 -- start start 2024-04-25T00:08:48.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.436+0000 7f4192e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418c074550 0x7f418c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:48.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.436+0000 7f4192e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f418c10e4d0 con 0x7f418c074550 2024-04-25T00:08:48.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.436+0000 7f418be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418c074550 0x7f418c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:48.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.436+0000 7f418be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418c074550 0x7f418c19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55698/0 (socket says 172.21.15.42:55698) 2024-04-25T00:08:48.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.436+0000 7f418be00700 1 -- 172.21.15.42:0/652287371 learned_addr learned my addr 172.21.15.42:0/652287371 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:48.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.437+0000 7f418be00700 1 -- 172.21.15.42:0/652287371 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418c19fb80 con 0x7f418c074550 2024-04-25T00:08:48.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.437+0000 7f418be00700 1 --2- 172.21.15.42:0/652287371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418c074550 0x7f418c19f640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f417c006010 tx=0x7f417c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:48.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.437+0000 7f4189600700 1 -- 172.21.15.42:0/652287371 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f417c026070 con 0x7f418c074550 2024-04-25T00:08:48.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.437+0000 7f4189600700 1 -- 172.21.15.42:0/652287371 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f417c00bc30 con 0x7f418c074550 2024-04-25T00:08:48.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.437+0000 7f4192e00700 1 -- 172.21.15.42:0/652287371 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418c19fe10 con 0x7f418c074550 2024-04-25T00:08:48.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.437+0000 7f4189600700 1 -- 172.21.15.42:0/652287371 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f417c0039a0 con 0x7f418c074550 2024-04-25T00:08:48.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.438+0000 7f4192e00700 1 -- 172.21.15.42:0/652287371 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f418c1997c0 con 0x7f418c074550 2024-04-25T00:08:48.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.439+0000 7f4189600700 1 -- 172.21.15.42:0/652287371 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f417c029070 con 0x7f418c074550 2024-04-25T00:08:48.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.439+0000 7f4192e00700 1 -- 172.21.15.42:0/652287371 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f418c075c90 con 0x7f418c074550 2024-04-25T00:08:48.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.439+0000 7f4189600700 1 --2- 172.21.15.42:0/652287371 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f416c0390f0 0x7f416c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:48.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.439+0000 7f4189600700 1 -- 172.21.15.42:0/652287371 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f417c01e070 con 0x7f418c074550 2024-04-25T00:08:48.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.440+0000 7f4183400700 1 --2- 172.21.15.42:0/652287371 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f416c0390f0 0x7f416c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:48.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.440+0000 7f4183400700 1 --2- 172.21.15.42:0/652287371 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f416c0390f0 0x7f416c03b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4174006fd0 tx=0x7f4174006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:48.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.445+0000 7f4189600700 1 -- 172.21.15.42:0/652287371 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f417c013070 con 0x7f418c074550 2024-04-25T00:08:48.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:48 smithi042 ceph-mon[28130]: Updating smithi042:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:08:48.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:48.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:48.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:48.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:48.678+0000 7f4192e00700 1 -- 172.21.15.42:0/652287371 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}) -- 0x7f418c19acf0 con 0x7f416c0390f0 2024-04-25T00:08:49.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:49 smithi042 ceph-mon[28130]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:50.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:50 smithi042 ceph-mon[28130]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:08:50.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:50 smithi042 ceph-mon[28130]: Deploying cephadm binary to smithi106 2024-04-25T00:08:51.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:51 smithi042 ceph-mon[28130]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:53.545 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:53 smithi042 ceph-mon[28130]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:53.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.543+0000 7f4189600700 1 -- 172.21.15.42:0/652287371 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f418c19acf0 con 0x7f416c0390f0 2024-04-25T00:08:53.546 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi106' with addr '172.21.15.106' 2024-04-25T00:08:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.546+0000 7f4192e00700 1 -- 172.21.15.42:0/652287371 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f416c0390f0 msgr2=0x7f416c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.546+0000 7f4192e00700 1 --2- 172.21.15.42:0/652287371 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f416c0390f0 0x7f416c03b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4174006fd0 tx=0x7f4174006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.546+0000 7f4192e00700 1 -- 172.21.15.42:0/652287371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418c074550 msgr2=0x7f418c19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:53.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.546+0000 7f4192e00700 1 --2- 172.21.15.42:0/652287371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418c074550 0x7f418c19f640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f417c006010 tx=0x7f417c005030 comp rx=0 tx=0).stop 2024-04-25T00:08:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.547+0000 7f4192e00700 1 -- 172.21.15.42:0/652287371 shutdown_connections 2024-04-25T00:08:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.547+0000 7f4192e00700 1 --2- 172.21.15.42:0/652287371 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f416c0390f0 0x7f416c03b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.547+0000 7f4192e00700 1 --2- 172.21.15.42:0/652287371 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f418c074550 0x7f418c19f640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.547+0000 7f4192e00700 1 -- 172.21.15.42:0/652287371 >> 172.21.15.42:0/652287371 conn(0x7f418c0fd770 msgr2=0x7f418c105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.547+0000 7f4192e00700 1 -- 172.21.15.42:0/652287371 shutdown_connections 2024-04-25T00:08:53.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:53.547+0000 7f4192e00700 1 -- 172.21.15.42:0/652287371 wait complete. 2024-04-25T00:08:54.373 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T00:08:54.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:54.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:54 smithi042 ceph-mon[28130]: Added host smithi106 2024-04-25T00:08:54.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:08:54.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:54.833 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:08:55.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:55 smithi042 ceph-mon[28130]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:55.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:56.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.893+0000 7fe46f800700 1 -- 172.21.15.42:0/12042902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe468106c00 msgr2=0x7fe468106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:56.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.893+0000 7fe46f800700 1 --2- 172.21.15.42:0/12042902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe468106c00 0x7fe468106fe0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe45c009910 tx=0x7fe45c009c20 comp rx=0 tx=0).stop 2024-04-25T00:08:56.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.894+0000 7fe46f800700 1 -- 172.21.15.42:0/12042902 shutdown_connections 2024-04-25T00:08:56.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.894+0000 7fe46f800700 1 --2- 172.21.15.42:0/12042902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe468106c00 0x7fe468106fe0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:56.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.894+0000 7fe46f800700 1 -- 172.21.15.42:0/12042902 >> 172.21.15.42:0/12042902 conn(0x7fe468076690 msgr2=0x7fe468076aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:56.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.894+0000 7fe46f800700 1 -- 172.21.15.42:0/12042902 shutdown_connections 2024-04-25T00:08:56.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.894+0000 7fe46f800700 1 -- 172.21.15.42:0/12042902 wait complete. 2024-04-25T00:08:56.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.896+0000 7fe46f800700 1 Processor -- start 2024-04-25T00:08:56.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.896+0000 7fe46f800700 1 -- start start 2024-04-25T00:08:56.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.897+0000 7fe46f800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe468106c00 0x7fe4681985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.897+0000 7fe46f800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe46810b1c0 con 0x7fe468106c00 2024-04-25T00:08:56.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.897+0000 7fe46ce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe468106c00 0x7fe4681985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.897+0000 7fe46ce00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe468106c00 0x7fe4681985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57362/0 (socket says 172.21.15.42:57362) 2024-04-25T00:08:56.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.897+0000 7fe46ce00700 1 -- 172.21.15.42:0/1791130979 learned_addr learned my addr 172.21.15.42:0/1791130979 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:08:56.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.898+0000 7fe46ce00700 1 -- 172.21.15.42:0/1791130979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe468198af0 con 0x7fe468106c00 2024-04-25T00:08:56.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.898+0000 7fe46ce00700 1 --2- 172.21.15.42:0/1791130979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe468106c00 0x7fe4681985b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe45c006010 tx=0x7fe45c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.898+0000 7fe465600700 1 -- 172.21.15.42:0/1791130979 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c026070 con 0x7fe468106c00 2024-04-25T00:08:56.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.898+0000 7fe465600700 1 -- 172.21.15.42:0/1791130979 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe45c00bc30 con 0x7fe468106c00 2024-04-25T00:08:56.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.898+0000 7fe465600700 1 -- 172.21.15.42:0/1791130979 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c0039d0 con 0x7fe468106c00 2024-04-25T00:08:56.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.898+0000 7fe46f800700 1 -- 172.21.15.42:0/1791130979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe468198d80 con 0x7fe468106c00 2024-04-25T00:08:56.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.898+0000 7fe46f800700 1 -- 172.21.15.42:0/1791130979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4681991e0 con 0x7fe468106c00 2024-04-25T00:08:56.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.900+0000 7fe465600700 1 -- 172.21.15.42:0/1791130979 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe45c029070 con 0x7fe468106c00 2024-04-25T00:08:56.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.900+0000 7fe46f800700 1 -- 172.21.15.42:0/1791130979 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe468102200 con 0x7fe468106c00 2024-04-25T00:08:56.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.900+0000 7fe465600700 1 --2- 172.21.15.42:0/1791130979 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4500390f0 0x7fe45003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:08:56.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.900+0000 7fe465600700 1 -- 172.21.15.42:0/1791130979 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe45c01e070 con 0x7fe468106c00 2024-04-25T00:08:56.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.901+0000 7fe467e00700 1 --2- 172.21.15.42:0/1791130979 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4500390f0 0x7fe45003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:08:56.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.901+0000 7fe467e00700 1 --2- 172.21.15.42:0/1791130979 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4500390f0 0x7fe45003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe454006fd0 tx=0x7fe454006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:08:56.907 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:56.906+0000 7fe465600700 1 -- 172.21.15.42:0/1791130979 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe45c013070 con 0x7fe468106c00 2024-04-25T00:08:57.147 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.146+0000 7fe46f800700 1 -- 172.21.15.42:0/1791130979 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe468063890 con 0x7fe4500390f0 2024-04-25T00:08:57.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.148+0000 7fe465600700 1 -- 172.21.15.42:0/1791130979 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7fe468063890 con 0x7fe4500390f0 2024-04-25T00:08:57.150 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:08:57.150 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}] 2024-04-25T00:08:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.151+0000 7fe46f800700 1 -- 172.21.15.42:0/1791130979 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4500390f0 msgr2=0x7fe45003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.151+0000 7fe46f800700 1 --2- 172.21.15.42:0/1791130979 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4500390f0 0x7fe45003b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe454006fd0 tx=0x7fe454006e40 comp rx=0 tx=0).stop 2024-04-25T00:08:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.151+0000 7fe46f800700 1 -- 172.21.15.42:0/1791130979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe468106c00 msgr2=0x7fe4681985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:08:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.151+0000 7fe46f800700 1 --2- 172.21.15.42:0/1791130979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe468106c00 0x7fe4681985b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe45c006010 tx=0x7fe45c005030 comp rx=0 tx=0).stop 2024-04-25T00:08:57.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.152+0000 7fe46f800700 1 -- 172.21.15.42:0/1791130979 shutdown_connections 2024-04-25T00:08:57.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.152+0000 7fe46f800700 1 --2- 172.21.15.42:0/1791130979 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4500390f0 0x7fe45003b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.152+0000 7fe46f800700 1 --2- 172.21.15.42:0/1791130979 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe468106c00 0x7fe4681985b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:08:57.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.152+0000 7fe46f800700 1 -- 172.21.15.42:0/1791130979 >> 172.21.15.42:0/1791130979 conn(0x7fe468076690 msgr2=0x7fe4680fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:08:57.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.152+0000 7fe46f800700 1 -- 172.21.15.42:0/1791130979 shutdown_connections 2024-04-25T00:08:57.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:08:57.152+0000 7fe46f800700 1 -- 172.21.15.42:0/1791130979 wait complete. 2024-04-25T00:08:57.507 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:57 smithi042 ceph-mon[28130]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:08:57.507 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:08:57.919 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-04-25T00:08:57.919 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T00:08:57.919 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-04-25T00:08:57.937 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T00:08:57.937 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:08:57.995 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-04-25T00:08:57.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch host add smithi136 2024-04-25T00:08:58.448 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:08:58.597 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:58 smithi042 ceph-mon[28130]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:08:59.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:08:59 smithi042 ceph-mon[28130]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:00.496 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:00.496 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:00.496 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:00.496 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:00.496 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:00.496 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:00.496 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:00.496 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:00 smithi042 ceph-mon[28130]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-25T00:09:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.525+0000 7f335e200700 1 -- 172.21.15.42:0/3078200603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3358101eb0 msgr2=0x7f3358102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.525+0000 7f335e200700 1 --2- 172.21.15.42:0/3078200603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3358101eb0 0x7f3358102290 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3348009910 tx=0x7f3348009c20 comp rx=0 tx=0).stop 2024-04-25T00:09:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.525+0000 7f335e200700 1 -- 172.21.15.42:0/3078200603 shutdown_connections 2024-04-25T00:09:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.525+0000 7f335e200700 1 --2- 172.21.15.42:0/3078200603 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3358101eb0 0x7f3358102290 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.525+0000 7f335e200700 1 -- 172.21.15.42:0/3078200603 >> 172.21.15.42:0/3078200603 conn(0x7f33580fd760 msgr2=0x7f33580ffb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.525+0000 7f335e200700 1 -- 172.21.15.42:0/3078200603 shutdown_connections 2024-04-25T00:09:00.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.525+0000 7f335e200700 1 -- 172.21.15.42:0/3078200603 wait complete. 2024-04-25T00:09:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.526+0000 7f335e200700 1 Processor -- start 2024-04-25T00:09:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.526+0000 7f335e200700 1 -- start start 2024-04-25T00:09:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.527+0000 7f335e200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3358101eb0 0x7f335819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.527+0000 7f335e200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f335810c280 con 0x7f3358101eb0 2024-04-25T00:09:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.527+0000 7f3357400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3358101eb0 0x7f335819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.527+0000 7f3357400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3358101eb0 0x7f335819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57380/0 (socket says 172.21.15.42:57380) 2024-04-25T00:09:00.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.527+0000 7f3357400700 1 -- 172.21.15.42:0/1931768744 learned_addr learned my addr 172.21.15.42:0/1931768744 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:09:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.528+0000 7f3357400700 1 -- 172.21.15.42:0/1931768744 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f335819fb50 con 0x7f3358101eb0 2024-04-25T00:09:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.528+0000 7f3357400700 1 --2- 172.21.15.42:0/1931768744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3358101eb0 0x7f335819f610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3348006010 tx=0x7f3348005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:00.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.528+0000 7f3354c00700 1 -- 172.21.15.42:0/1931768744 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3348026070 con 0x7f3358101eb0 2024-04-25T00:09:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.528+0000 7f3354c00700 1 -- 172.21.15.42:0/1931768744 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f334800bbd0 con 0x7f3358101eb0 2024-04-25T00:09:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.528+0000 7f335e200700 1 -- 172.21.15.42:0/1931768744 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f335819fe40 con 0x7f3358101eb0 2024-04-25T00:09:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.528+0000 7f3354c00700 1 -- 172.21.15.42:0/1931768744 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3348003910 con 0x7f3358101eb0 2024-04-25T00:09:00.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.529+0000 7f335e200700 1 -- 172.21.15.42:0/1931768744 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3358199790 con 0x7f3358101eb0 2024-04-25T00:09:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.530+0000 7f3354c00700 1 -- 172.21.15.42:0/1931768744 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3348029070 con 0x7f3358101eb0 2024-04-25T00:09:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.530+0000 7f335e200700 1 -- 172.21.15.42:0/1931768744 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3358103390 con 0x7f3358101eb0 2024-04-25T00:09:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.530+0000 7f3354c00700 1 --2- 172.21.15.42:0/1931768744 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f333c039150 0x7f333c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:00.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.530+0000 7f3354c00700 1 -- 172.21.15.42:0/1931768744 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f334801e070 con 0x7f3358101eb0 2024-04-25T00:09:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.531+0000 7f334ea00700 1 --2- 172.21.15.42:0/1931768744 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f333c039150 0x7f333c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:00.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.531+0000 7f334ea00700 1 --2- 172.21.15.42:0/1931768744 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f333c039150 0x7f333c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3340006fd0 tx=0x7f3340006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:00.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.536+0000 7f3354c00700 1 -- 172.21.15.42:0/1931768744 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3348050030 con 0x7f3358101eb0 2024-04-25T00:09:00.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:00.770+0000 7f335e200700 1 -- 172.21.15.42:0/1931768744 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}) -- 0x7f335819acc0 con 0x7f333c039150 2024-04-25T00:09:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:01 smithi042 ceph-mon[28130]: Updating smithi106:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:01 smithi042 ceph-mon[28130]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:01 smithi042 ceph-mon[28130]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:09:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:01 smithi042 ceph-mon[28130]: Updating smithi106:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:09:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:02.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:02 smithi042 ceph-mon[28130]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:02.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:02 smithi042 ceph-mon[28130]: Deploying cephadm binary to smithi136 2024-04-25T00:09:03.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:03 smithi042 ceph-mon[28130]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:05.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.628+0000 7f3354c00700 1 -- 172.21.15.42:0/1931768744 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f335819acc0 con 0x7f333c039150 2024-04-25T00:09:05.630 INFO:teuthology.orchestra.run.smithi042.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-04-25T00:09:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.633+0000 7f335e200700 1 -- 172.21.15.42:0/1931768744 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f333c039150 msgr2=0x7f333c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.633+0000 7f335e200700 1 --2- 172.21.15.42:0/1931768744 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f333c039150 0x7f333c03b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3340006fd0 tx=0x7f3340006e40 comp rx=0 tx=0).stop 2024-04-25T00:09:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.633+0000 7f335e200700 1 -- 172.21.15.42:0/1931768744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3358101eb0 msgr2=0x7f335819f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.633+0000 7f335e200700 1 --2- 172.21.15.42:0/1931768744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3358101eb0 0x7f335819f610 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3348006010 tx=0x7f3348005030 comp rx=0 tx=0).stop 2024-04-25T00:09:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.633+0000 7f335e200700 1 -- 172.21.15.42:0/1931768744 shutdown_connections 2024-04-25T00:09:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.633+0000 7f335e200700 1 --2- 172.21.15.42:0/1931768744 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f333c039150 0x7f333c03b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.633+0000 7f335e200700 1 --2- 172.21.15.42:0/1931768744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3358101eb0 0x7f335819f610 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.633+0000 7f335e200700 1 -- 172.21.15.42:0/1931768744 >> 172.21.15.42:0/1931768744 conn(0x7f33580fd760 msgr2=0x7f3358074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.633+0000 7f335e200700 1 -- 172.21.15.42:0/1931768744 shutdown_connections 2024-04-25T00:09:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:05.633+0000 7f335e200700 1 -- 172.21.15.42:0/1931768744 wait complete. 2024-04-25T00:09:05.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:05 smithi042 ceph-mon[28130]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:06.398 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T00:09:06.849 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:09:07.001 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:07.001 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:06 smithi042 ceph-mon[28130]: Added host smithi136 2024-04-25T00:09:07.001 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:07.001 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:06 smithi042 ceph-mon[28130]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:07.001 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:08.018 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:08.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.985+0000 7fd72b600700 1 -- 172.21.15.42:0/2024786710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd724101ea0 msgr2=0x7fd724102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:08.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.985+0000 7fd72b600700 1 --2- 172.21.15.42:0/2024786710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd724101ea0 0x7fd724102280 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd718009910 tx=0x7fd718009c20 comp rx=0 tx=0).stop 2024-04-25T00:09:08.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.986+0000 7fd72b600700 1 -- 172.21.15.42:0/2024786710 shutdown_connections 2024-04-25T00:09:08.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.986+0000 7fd72b600700 1 --2- 172.21.15.42:0/2024786710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd724101ea0 0x7fd724102280 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:08.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.986+0000 7fd72b600700 1 -- 172.21.15.42:0/2024786710 >> 172.21.15.42:0/2024786710 conn(0x7fd7240fd770 msgr2=0x7fd7240ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:08.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.986+0000 7fd72b600700 1 -- 172.21.15.42:0/2024786710 shutdown_connections 2024-04-25T00:09:08.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.986+0000 7fd72b600700 1 -- 172.21.15.42:0/2024786710 wait complete. 2024-04-25T00:09:08.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.987+0000 7fd72b600700 1 Processor -- start 2024-04-25T00:09:08.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.987+0000 7fd72b600700 1 -- start start 2024-04-25T00:09:08.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.988+0000 7fd72b600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd724101ea0 0x7fd72419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:08.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.988+0000 7fd72b600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd72410c270 con 0x7fd724101ea0 2024-04-25T00:09:08.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.988+0000 7fd728c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd724101ea0 0x7fd72419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:08.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.988+0000 7fd728c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd724101ea0 0x7fd72419f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51646/0 (socket says 172.21.15.42:51646) 2024-04-25T00:09:08.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.988+0000 7fd728c00700 1 -- 172.21.15.42:0/2402938296 learned_addr learned my addr 172.21.15.42:0/2402938296 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:09:08.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.988+0000 7fd728c00700 1 -- 172.21.15.42:0/2402938296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72419fb50 con 0x7fd724101ea0 2024-04-25T00:09:08.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.989+0000 7fd728c00700 1 --2- 172.21.15.42:0/2402938296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd724101ea0 0x7fd72419f610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd718006010 tx=0x7fd718005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:08.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.989+0000 7fd721600700 1 -- 172.21.15.42:0/2402938296 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd718026070 con 0x7fd724101ea0 2024-04-25T00:09:08.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.989+0000 7fd721600700 1 -- 172.21.15.42:0/2402938296 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd71800bbd0 con 0x7fd724101ea0 2024-04-25T00:09:08.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.989+0000 7fd72b600700 1 -- 172.21.15.42:0/2402938296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72419fe40 con 0x7fd724101ea0 2024-04-25T00:09:08.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.989+0000 7fd721600700 1 -- 172.21.15.42:0/2402938296 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd718003910 con 0x7fd724101ea0 2024-04-25T00:09:08.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.989+0000 7fd72b600700 1 -- 172.21.15.42:0/2402938296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd724199790 con 0x7fd724101ea0 2024-04-25T00:09:08.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.990+0000 7fd72b600700 1 -- 172.21.15.42:0/2402938296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd724103380 con 0x7fd724101ea0 2024-04-25T00:09:08.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.990+0000 7fd721600700 1 -- 172.21.15.42:0/2402938296 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd718029070 con 0x7fd724101ea0 2024-04-25T00:09:08.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.991+0000 7fd721600700 1 --2- 172.21.15.42:0/2402938296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd70c039150 0x7fd70c03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:08.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.991+0000 7fd721600700 1 -- 172.21.15.42:0/2402938296 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd71801e070 con 0x7fd724101ea0 2024-04-25T00:09:08.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.991+0000 7fd723e00700 1 --2- 172.21.15.42:0/2402938296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd70c039150 0x7fd70c03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:08.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.992+0000 7fd723e00700 1 --2- 172.21.15.42:0/2402938296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd70c039150 0x7fd70c03b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd710006fd0 tx=0x7fd710006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:08.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:08.997+0000 7fd721600700 1 -- 172.21.15.42:0/2402938296 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd718013070 con 0x7fd724101ea0 2024-04-25T00:09:09.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.231+0000 7fd72b600700 1 -- 172.21.15.42:0/2402938296 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd72419acc0 con 0x7fd70c039150 2024-04-25T00:09:09.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.233+0000 7fd721600700 1 -- 172.21.15.42:0/2402938296 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fd72419acc0 con 0x7fd70c039150 2024-04-25T00:09:09.234 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:09:09.234 INFO:teuthology.orchestra.run.smithi042.stdout:[{"addr": "172.21.15.42", "hostname": "smithi042", "labels": [], "status": ""}, {"addr": "172.21.15.106", "hostname": "smithi106", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-04-25T00:09:09.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.236+0000 7fd72b600700 1 -- 172.21.15.42:0/2402938296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd70c039150 msgr2=0x7fd70c03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.236+0000 7fd72b600700 1 --2- 172.21.15.42:0/2402938296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd70c039150 0x7fd70c03b610 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd710006fd0 tx=0x7fd710006e40 comp rx=0 tx=0).stop 2024-04-25T00:09:09.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.236+0000 7fd72b600700 1 -- 172.21.15.42:0/2402938296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd724101ea0 msgr2=0x7fd72419f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:09.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.236+0000 7fd72b600700 1 --2- 172.21.15.42:0/2402938296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd724101ea0 0x7fd72419f610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd718006010 tx=0x7fd718005030 comp rx=0 tx=0).stop 2024-04-25T00:09:09.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.236+0000 7fd72b600700 1 -- 172.21.15.42:0/2402938296 shutdown_connections 2024-04-25T00:09:09.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.236+0000 7fd72b600700 1 --2- 172.21.15.42:0/2402938296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd70c039150 0x7fd70c03b610 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.236+0000 7fd72b600700 1 --2- 172.21.15.42:0/2402938296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd724101ea0 0x7fd72419f610 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:09.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.236+0000 7fd72b600700 1 -- 172.21.15.42:0/2402938296 >> 172.21.15.42:0/2402938296 conn(0x7fd7240fd770 msgr2=0x7fd724074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:09.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.237+0000 7fd72b600700 1 -- 172.21.15.42:0/2402938296 shutdown_connections 2024-04-25T00:09:09.238 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:09.237+0000 7fd72b600700 1 -- 172.21.15.42:0/2402938296 wait complete. 2024-04-25T00:09:09.582 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:09 smithi042 ceph-mon[28130]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:09.582 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:09.944 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T00:09:09.944 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T00:09:10.379 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:09:10.671 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:10 smithi042 ceph-mon[28130]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:09:11.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:11 smithi042 ceph-mon[28130]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.552+0000 7f591a800700 1 -- 172.21.15.42:0/3784125184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5914103f00 msgr2=0x7f59141042e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.552+0000 7f591a800700 1 --2- 172.21.15.42:0/3784125184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5914103f00 0x7f59141042e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5908009910 tx=0x7f5908009c20 comp rx=0 tx=0).stop 2024-04-25T00:09:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.552+0000 7f591a800700 1 -- 172.21.15.42:0/3784125184 shutdown_connections 2024-04-25T00:09:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.552+0000 7f591a800700 1 --2- 172.21.15.42:0/3784125184 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5914103f00 0x7f59141042e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.552+0000 7f591a800700 1 -- 172.21.15.42:0/3784125184 >> 172.21.15.42:0/3784125184 conn(0x7f59140ff7b0 msgr2=0x7f5914101bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:12.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.552+0000 7f591a800700 1 -- 172.21.15.42:0/3784125184 shutdown_connections 2024-04-25T00:09:12.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.552+0000 7f591a800700 1 -- 172.21.15.42:0/3784125184 wait complete. 2024-04-25T00:09:12.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.553+0000 7f591a800700 1 Processor -- start 2024-04-25T00:09:12.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.554+0000 7f591a800700 1 -- start start 2024-04-25T00:09:12.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.554+0000 7f591a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5914103f00 0x7f591419f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.554+0000 7f591a800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f591410e2d0 con 0x7f5914103f00 2024-04-25T00:09:12.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.554+0000 7f5919400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5914103f00 0x7f591419f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.554+0000 7f5919400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5914103f00 0x7f591419f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51678/0 (socket says 172.21.15.42:51678) 2024-04-25T00:09:12.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.554+0000 7f5919400700 1 -- 172.21.15.42:0/1322513705 learned_addr learned my addr 172.21.15.42:0/1322513705 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:09:12.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.555+0000 7f5919400700 1 -- 172.21.15.42:0/1322513705 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f591419f940 con 0x7f5914103f00 2024-04-25T00:09:12.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.555+0000 7f5919400700 1 --2- 172.21.15.42:0/1322513705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5914103f00 0x7f591419f400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5908006010 tx=0x7f5908005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.555+0000 7f5912000700 1 -- 172.21.15.42:0/1322513705 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5908026070 con 0x7f5914103f00 2024-04-25T00:09:12.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.555+0000 7f5912000700 1 -- 172.21.15.42:0/1322513705 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f590800bbd0 con 0x7f5914103f00 2024-04-25T00:09:12.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.555+0000 7f5912000700 1 -- 172.21.15.42:0/1322513705 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5908003910 con 0x7f5914103f00 2024-04-25T00:09:12.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.555+0000 7f591a800700 1 -- 172.21.15.42:0/1322513705 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f591419fc30 con 0x7f5914103f00 2024-04-25T00:09:12.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.555+0000 7f591a800700 1 -- 172.21.15.42:0/1322513705 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59141995c0 con 0x7f5914103f00 2024-04-25T00:09:12.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.557+0000 7f5907400700 1 -- 172.21.15.42:0/1322513705 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59141053e0 con 0x7f5914103f00 2024-04-25T00:09:12.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.557+0000 7f5912000700 1 -- 172.21.15.42:0/1322513705 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f5908029070 con 0x7f5914103f00 2024-04-25T00:09:12.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.557+0000 7f5912000700 1 --2- 172.21.15.42:0/1322513705 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f58fc039150 0x7f58fc03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:12.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.557+0000 7f5912000700 1 -- 172.21.15.42:0/1322513705 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f590801e070 con 0x7f5914103f00 2024-04-25T00:09:12.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.558+0000 7f5918a00700 1 --2- 172.21.15.42:0/1322513705 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f58fc039150 0x7f58fc03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:12.559 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.558+0000 7f5918a00700 1 --2- 172.21.15.42:0/1322513705 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f58fc039150 0x7f58fc03b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5900006fd0 tx=0x7f5900006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:12.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.563+0000 7f5912000700 1 -- 172.21.15.42:0/1322513705 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5908013070 con 0x7f5914103f00 2024-04-25T00:09:12.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:12.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:12.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:12.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:12.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:12.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:12.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:12.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:12.799+0000 7f5907400700 1 -- 172.21.15.42:0/1322513705 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f5914063890 con 0x7f5914103f00 2024-04-25T00:09:13.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.601+0000 7f5912000700 1 -- 172.21.15.42:0/1322513705 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f5908030090 con 0x7f5914103f00 2024-04-25T00:09:13.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.605+0000 7f5907400700 1 -- 172.21.15.42:0/1322513705 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f58fc039150 msgr2=0x7f58fc03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.605+0000 7f5907400700 1 --2- 172.21.15.42:0/1322513705 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f58fc039150 0x7f58fc03b610 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5900006fd0 tx=0x7f5900006e40 comp rx=0 tx=0).stop 2024-04-25T00:09:13.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.605+0000 7f5907400700 1 -- 172.21.15.42:0/1322513705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5914103f00 msgr2=0x7f591419f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:13.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.605+0000 7f5907400700 1 --2- 172.21.15.42:0/1322513705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5914103f00 0x7f591419f400 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5908006010 tx=0x7f5908005030 comp rx=0 tx=0).stop 2024-04-25T00:09:13.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.605+0000 7f5907400700 1 -- 172.21.15.42:0/1322513705 shutdown_connections 2024-04-25T00:09:13.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.605+0000 7f5907400700 1 --2- 172.21.15.42:0/1322513705 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f58fc039150 0x7f58fc03b610 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.605+0000 7f5907400700 1 --2- 172.21.15.42:0/1322513705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5914103f00 0x7f591419f400 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:13.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.605+0000 7f5907400700 1 -- 172.21.15.42:0/1322513705 >> 172.21.15.42:0/1322513705 conn(0x7f59140ff7b0 msgr2=0x7f5914106c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:13.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.605+0000 7f5907400700 1 -- 172.21.15.42:0/1322513705 shutdown_connections 2024-04-25T00:09:13.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:13.605+0000 7f5907400700 1 -- 172.21.15.42:0/1322513705 wait complete. 2024-04-25T00:09:13.607 INFO:teuthology.orchestra.run.smithi042.stderr:adjusted tunables profile to default 2024-04-25T00:09:13.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:13 smithi042 ceph-mon[28130]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T00:09:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:13 smithi042 ceph-mon[28130]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:13 smithi042 ceph-mon[28130]: Updating smithi136:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:13 smithi042 ceph-mon[28130]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:09:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:13 smithi042 ceph-mon[28130]: Updating smithi136:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:09:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:13 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1322513705' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T00:09:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:14.376 INFO:tasks.cephadm:Adding mon.a on smithi042 2024-04-25T00:09:14.376 INFO:tasks.cephadm:Adding mon.b on smithi106 2024-04-25T00:09:14.376 INFO:tasks.cephadm:Adding mon.c on smithi136 2024-04-25T00:09:14.377 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi042:172.21.15.42=a;smithi106:172.21.15.106=b;smithi136:172.21.15.136=c' 2024-04-25T00:09:14.612 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:14.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:14 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1322513705' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T00:09:14.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:14 smithi042 ceph-mon[28130]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:09:14.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:14 smithi042 ceph-mon[28130]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.225+0000 7f95b501e700 1 -- 172.21.15.136:0/3119844056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95b0106bf0 msgr2=0x7f95b0106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.225+0000 7f95b501e700 1 --2- 172.21.15.136:0/3119844056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95b0106bf0 0x7f95b0106fd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f95a000b0a0 tx=0x7f95a000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.225+0000 7f95b501e700 1 -- 172.21.15.136:0/3119844056 shutdown_connections 2024-04-25T00:09:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.225+0000 7f95b501e700 1 --2- 172.21.15.136:0/3119844056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95b0106bf0 0x7f95b0106fd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.225+0000 7f95b501e700 1 -- 172.21.15.136:0/3119844056 >> 172.21.15.136:0/3119844056 conn(0x7f95b00fc630 msgr2=0x7f95b00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.225+0000 7f95b501e700 1 -- 172.21.15.136:0/3119844056 shutdown_connections 2024-04-25T00:09:16.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.225+0000 7f95b501e700 1 -- 172.21.15.136:0/3119844056 wait complete. 2024-04-25T00:09:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.226+0000 7f95b501e700 1 Processor -- start 2024-04-25T00:09:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.226+0000 7f95b501e700 1 -- start start 2024-04-25T00:09:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.226+0000 7f95b501e700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95b0106bf0 0x7f95b01965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.226+0000 7f95b501e700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95b010b1b0 con 0x7f95b0106bf0 2024-04-25T00:09:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.227+0000 7f95aed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95b0106bf0 0x7f95b01965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.227+0000 7f95aed9d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95b0106bf0 0x7f95b01965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.136:49270/0 (socket says 172.21.15.136:49270) 2024-04-25T00:09:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.227+0000 7f95aed9d700 1 -- 172.21.15.136:0/1333434120 learned_addr learned my addr 172.21.15.136:0/1333434120 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:09:16.227 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.227+0000 7f95aed9d700 1 -- 172.21.15.136:0/1333434120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b0196af0 con 0x7f95b0106bf0 2024-04-25T00:09:16.228 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.228+0000 7f95aed9d700 1 --2- 172.21.15.136:0/1333434120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95b0106bf0 0x7f95b01965b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f95a00062a0 tx=0x7f95a0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.228 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.228+0000 7f9597fff700 1 -- 172.21.15.136:0/1333434120 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95a0017040 con 0x7f95b0106bf0 2024-04-25T00:09:16.228 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.228+0000 7f95b501e700 1 -- 172.21.15.136:0/1333434120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b0196d80 con 0x7f95b0106bf0 2024-04-25T00:09:16.228 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.228+0000 7f9597fff700 1 -- 172.21.15.136:0/1333434120 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95a0022070 con 0x7f95b0106bf0 2024-04-25T00:09:16.228 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.228+0000 7f9597fff700 1 -- 172.21.15.136:0/1333434120 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95a001d440 con 0x7f95b0106bf0 2024-04-25T00:09:16.228 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.228+0000 7f95b501e700 1 -- 172.21.15.136:0/1333434120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b019ae80 con 0x7f95b0106bf0 2024-04-25T00:09:16.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.229+0000 7f95b501e700 1 -- 172.21.15.136:0/1333434120 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b01021f0 con 0x7f95b0106bf0 2024-04-25T00:09:16.229 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.230+0000 7f9597fff700 1 -- 172.21.15.136:0/1333434120 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f95a000bb90 con 0x7f95b0106bf0 2024-04-25T00:09:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.230+0000 7f9597fff700 1 --2- 172.21.15.136:0/1333434120 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95980390f0 0x7f959803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.230+0000 7f9597fff700 1 -- 172.21.15.136:0/1333434120 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f95a000f050 con 0x7f95b0106bf0 2024-04-25T00:09:16.230 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.231+0000 7f95ae59c700 1 --2- 172.21.15.136:0/1333434120 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95980390f0 0x7f959803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:16.231 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.232+0000 7f95ae59c700 1 --2- 172.21.15.136:0/1333434120 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95980390f0 0x7f959803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95a4006fd0 tx=0x7f95a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:16.248 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.249+0000 7f9597fff700 1 -- 172.21.15.136:0/1333434120 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f95a0007e20 con 0x7f95b0106bf0 2024-04-25T00:09:16.456 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.456+0000 7f95b501e700 1 -- 172.21.15.136:0/1333434120 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi106:172.21.15.106=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}) -- 0x7f95b0197910 con 0x7f95980390f0 2024-04-25T00:09:16.471 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.471+0000 7f9597fff700 1 -- 172.21.15.136:0/1333434120 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f95b0197910 con 0x7f95980390f0 2024-04-25T00:09:16.471 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mon update... 2024-04-25T00:09:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.473+0000 7f95b501e700 1 -- 172.21.15.136:0/1333434120 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95980390f0 msgr2=0x7f959803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.473+0000 7f95b501e700 1 --2- 172.21.15.136:0/1333434120 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95980390f0 0x7f959803b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95a4006fd0 tx=0x7f95a4006e40 comp rx=0 tx=0).stop 2024-04-25T00:09:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.473+0000 7f95b501e700 1 -- 172.21.15.136:0/1333434120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95b0106bf0 msgr2=0x7f95b01965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.473+0000 7f95b501e700 1 --2- 172.21.15.136:0/1333434120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95b0106bf0 0x7f95b01965b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f95a00062a0 tx=0x7f95a0003ce0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.473+0000 7f95b501e700 1 -- 172.21.15.136:0/1333434120 shutdown_connections 2024-04-25T00:09:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.473+0000 7f95b501e700 1 --2- 172.21.15.136:0/1333434120 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95980390f0 0x7f959803b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.473+0000 7f95b501e700 1 --2- 172.21.15.136:0/1333434120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95b0106bf0 0x7f95b01965b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.473+0000 7f95b501e700 1 -- 172.21.15.136:0/1333434120 >> 172.21.15.136:0/1333434120 conn(0x7f95b00fc630 msgr2=0x7f95b00fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.473+0000 7f95b501e700 1 -- 172.21.15.136:0/1333434120 shutdown_connections 2024-04-25T00:09:16.473 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:16.473+0000 7f95b501e700 1 -- 172.21.15.136:0/1333434120 wait complete. 2024-04-25T00:09:17.039 DEBUG:teuthology.orchestra.run.smithi106:mon.b> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T00:09:17.042 DEBUG:teuthology.orchestra.run.smithi136:mon.c> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T00:09:17.044 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:09:17.044 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi106:172.21.15.106=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi106:172.21.15.106=b;smithi136:172.21.15.136=c;count:3 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:17 smithi042 ceph-mon[28130]: Deploying daemon mon.c on smithi136 2024-04-25T00:09:18.091 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:19.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.567+0000 7fb3ed747700 1 -- 172.21.15.136:0/51710530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e8100da0 msgr2=0x7fb3e8101180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.567+0000 7fb3ed747700 1 --2- 172.21.15.136:0/51710530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e8100da0 0x7fb3e8101180 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8009910 tx=0x7fb3d8009c20 comp rx=0 tx=0).stop 2024-04-25T00:09:19.568 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.567+0000 7fb3ed747700 1 -- 172.21.15.136:0/51710530 shutdown_connections 2024-04-25T00:09:19.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.567+0000 7fb3ed747700 1 --2- 172.21.15.136:0/51710530 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e8100da0 0x7fb3e8101180 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.567+0000 7fb3ed747700 1 -- 172.21.15.136:0/51710530 >> 172.21.15.136:0/51710530 conn(0x7fb3e80fc630 msgr2=0x7fb3e80fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:19.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.567+0000 7fb3ed747700 1 -- 172.21.15.136:0/51710530 shutdown_connections 2024-04-25T00:09:19.569 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.567+0000 7fb3ed747700 1 -- 172.21.15.136:0/51710530 wait complete. 2024-04-25T00:09:19.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.568+0000 7fb3ed747700 1 Processor -- start 2024-04-25T00:09:19.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.568+0000 7fb3ed747700 1 -- start start 2024-04-25T00:09:19.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.568+0000 7fb3ed747700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e8100da0 0x7fb3e8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.568+0000 7fb3ed747700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3e810b170 con 0x7fb3e8100da0 2024-04-25T00:09:19.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.569+0000 7fb3e6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e8100da0 0x7fb3e8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.569+0000 7fb3e6ffd700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e8100da0 0x7fb3e8197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.136:47354/0 (socket says 172.21.15.136:47354) 2024-04-25T00:09:19.570 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.569+0000 7fb3e6ffd700 1 -- 172.21.15.136:0/365749296 learned_addr learned my addr 172.21.15.136:0/365749296 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:09:19.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.570+0000 7fb3e6ffd700 1 -- 172.21.15.136:0/365749296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3e8197c90 con 0x7fb3e8100da0 2024-04-25T00:09:19.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.570+0000 7fb3e6ffd700 1 --2- 172.21.15.136:0/365749296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e8100da0 0x7fb3e8197750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8006010 tx=0x7fb3d8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:19.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.570+0000 7fb3cffff700 1 -- 172.21.15.136:0/365749296 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3d8026070 con 0x7fb3e8100da0 2024-04-25T00:09:19.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.570+0000 7fb3ed747700 1 -- 172.21.15.136:0/365749296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3e8197f80 con 0x7fb3e8100da0 2024-04-25T00:09:19.571 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.570+0000 7fb3ed747700 1 -- 172.21.15.136:0/365749296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3e819c020 con 0x7fb3e8100da0 2024-04-25T00:09:19.572 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.570+0000 7fb3cffff700 1 -- 172.21.15.136:0/365749296 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3d800bbd0 con 0x7fb3e8100da0 2024-04-25T00:09:19.572 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.570+0000 7fb3cffff700 1 -- 172.21.15.136:0/365749296 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3d80038e0 con 0x7fb3e8100da0 2024-04-25T00:09:19.572 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.571+0000 7fb3ed747700 1 -- 172.21.15.136:0/365749296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3e8102280 con 0x7fb3e8100da0 2024-04-25T00:09:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.572+0000 7fb3cffff700 1 -- 172.21.15.136:0/365749296 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb3d8029070 con 0x7fb3e8100da0 2024-04-25T00:09:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.572+0000 7fb3cffff700 1 --2- 172.21.15.136:0/365749296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3d00390f0 0x7fb3d003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.572+0000 7fb3cffff700 1 -- 172.21.15.136:0/365749296 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb3d801e070 con 0x7fb3e8100da0 2024-04-25T00:09:19.573 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.573+0000 7fb3e67fc700 1 --2- 172.21.15.136:0/365749296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3d00390f0 0x7fb3d003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:19.574 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.574+0000 7fb3e67fc700 1 --2- 172.21.15.136:0/365749296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3d00390f0 0x7fb3d003b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc006fd0 tx=0x7fb3dc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:19.591 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.588+0000 7fb3cffff700 1 -- 172.21.15.136:0/365749296 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb3d8013070 con 0x7fb3e8100da0 2024-04-25T00:09:19.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:19 smithi042 ceph-mon[28130]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:19.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.882+0000 7fb3ed747700 1 -- 172.21.15.136:0/365749296 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb3e8198ab0 con 0x7fb3e8100da0 2024-04-25T00:09:19.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.882+0000 7fb3cffff700 1 -- 172.21.15.136:0/365749296 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fb3d800d340 con 0x7fb3e8100da0 2024-04-25T00:09:19.883 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:09:19.883 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","modified":"2024-04-25T00:07:11.904046Z","created":"2024-04-25T00:07:11.904046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T00:09:19.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.885+0000 7fb3ed747700 1 -- 172.21.15.136:0/365749296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3d00390f0 msgr2=0x7fb3d003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.885+0000 7fb3ed747700 1 --2- 172.21.15.136:0/365749296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3d00390f0 0x7fb3d003b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc006fd0 tx=0x7fb3dc006e40 comp rx=0 tx=0).stop 2024-04-25T00:09:19.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.885+0000 7fb3ed747700 1 -- 172.21.15.136:0/365749296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e8100da0 msgr2=0x7fb3e8197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:19.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.885+0000 7fb3ed747700 1 --2- 172.21.15.136:0/365749296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e8100da0 0x7fb3e8197750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8006010 tx=0x7fb3d8005030 comp rx=0 tx=0).stop 2024-04-25T00:09:19.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.885+0000 7fb3ed747700 1 -- 172.21.15.136:0/365749296 shutdown_connections 2024-04-25T00:09:19.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.885+0000 7fb3ed747700 1 --2- 172.21.15.136:0/365749296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3d00390f0 0x7fb3d003b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.885+0000 7fb3ed747700 1 --2- 172.21.15.136:0/365749296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3e8100da0 0x7fb3e8197750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:19.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.885+0000 7fb3ed747700 1 -- 172.21.15.136:0/365749296 >> 172.21.15.136:0/365749296 conn(0x7fb3e80fc630 msgr2=0x7fb3e806a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:19.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.885+0000 7fb3ed747700 1 -- 172.21.15.136:0/365749296 shutdown_connections 2024-04-25T00:09:19.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:19.885+0000 7fb3ed747700 1 -- 172.21.15.136:0/365749296 wait complete. 2024-04-25T00:09:19.887 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-25T00:09:20.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:20 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/365749296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:09:21.529 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:09:21.529 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:09:21.778 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:21 smithi042 ceph-mon[28130]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:21.935 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:09:23.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.449+0000 7febde57f700 1 -- 172.21.15.136:0/3582945355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febd8101fb0 msgr2=0x7febd8102390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:23.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.449+0000 7febde57f700 1 --2- 172.21.15.136:0/3582945355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febd8101fb0 0x7febd8102390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7febc8009910 tx=0x7febc8009c20 comp rx=0 tx=0).stop 2024-04-25T00:09:23.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.449+0000 7febde57f700 1 -- 172.21.15.136:0/3582945355 shutdown_connections 2024-04-25T00:09:23.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.449+0000 7febde57f700 1 --2- 172.21.15.136:0/3582945355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febd8101fb0 0x7febd8102390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.449+0000 7febde57f700 1 -- 172.21.15.136:0/3582945355 >> 172.21.15.136:0/3582945355 conn(0x7febd80fd840 msgr2=0x7febd80ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:23.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.449+0000 7febde57f700 1 -- 172.21.15.136:0/3582945355 shutdown_connections 2024-04-25T00:09:23.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.449+0000 7febde57f700 1 -- 172.21.15.136:0/3582945355 wait complete. 2024-04-25T00:09:23.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.450+0000 7febde57f700 1 Processor -- start 2024-04-25T00:09:23.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.450+0000 7febde57f700 1 -- start start 2024-04-25T00:09:23.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.450+0000 7febde57f700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febd8101fb0 0x7febd819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:23.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.450+0000 7febde57f700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7febd810c380 con 0x7febd8101fb0 2024-04-25T00:09:23.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.451+0000 7febd7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febd8101fb0 0x7febd819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:23.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.451+0000 7febd7fff700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febd8101fb0 0x7febd819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.136:47372/0 (socket says 172.21.15.136:47372) 2024-04-25T00:09:23.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.451+0000 7febd7fff700 1 -- 172.21.15.136:0/606207904 learned_addr learned my addr 172.21.15.136:0/606207904 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:09:23.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.451+0000 7febd7fff700 1 -- 172.21.15.136:0/606207904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd819db30 con 0x7febd8101fb0 2024-04-25T00:09:23.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.452+0000 7febd7fff700 1 --2- 172.21.15.136:0/606207904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febd8101fb0 0x7febd819d5f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7febc8005850 tx=0x7febc8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:23.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.452+0000 7febd57fa700 1 -- 172.21.15.136:0/606207904 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febc8026070 con 0x7febd8101fb0 2024-04-25T00:09:23.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.452+0000 7febde57f700 1 -- 172.21.15.136:0/606207904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd819de20 con 0x7febd8101fb0 2024-04-25T00:09:23.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.452+0000 7febde57f700 1 -- 172.21.15.136:0/606207904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd8197890 con 0x7febd8101fb0 2024-04-25T00:09:23.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.452+0000 7febd57fa700 1 -- 172.21.15.136:0/606207904 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc800bbd0 con 0x7febd8101fb0 2024-04-25T00:09:23.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.452+0000 7febd57fa700 1 -- 172.21.15.136:0/606207904 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febc80039b0 con 0x7febd8101fb0 2024-04-25T00:09:23.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.453+0000 7febde57f700 1 -- 172.21.15.136:0/606207904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd8103490 con 0x7febd8101fb0 2024-04-25T00:09:23.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.454+0000 7febd57fa700 1 -- 172.21.15.136:0/606207904 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7febc8029070 con 0x7febd8101fb0 2024-04-25T00:09:23.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.454+0000 7febd57fa700 1 --2- 172.21.15.136:0/606207904 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7febc0039150 0x7febc003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:23.455 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.454+0000 7febd57fa700 1 -- 172.21.15.136:0/606207904 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7febc801e070 con 0x7febd8101fb0 2024-04-25T00:09:23.455 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.454+0000 7febd77fe700 1 --2- 172.21.15.136:0/606207904 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7febc0039150 0x7febc003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:23.457 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.456+0000 7febd77fe700 1 --2- 172.21.15.136:0/606207904 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7febc0039150 0x7febc003b610 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7febcc006fd0 tx=0x7febcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:23.460 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.460+0000 7febd57fa700 1 -- 172.21.15.136:0/606207904 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7febc8013070 con 0x7febd8101fb0 2024-04-25T00:09:23.751 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.750+0000 7febde57f700 1 -- 172.21.15.136:0/606207904 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7febd80008d0 con 0x7febd8101fb0 2024-04-25T00:09:23.752 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.751+0000 7febd57fa700 1 -- 172.21.15.136:0/606207904 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7febc801fdd0 con 0x7febd8101fb0 2024-04-25T00:09:23.752 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:09:23.752 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":1,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","modified":"2024-04-25T00:07:11.904046Z","created":"2024-04-25T00:07:11.904046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T00:09:23.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.753+0000 7febde57f700 1 -- 172.21.15.136:0/606207904 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7febc0039150 msgr2=0x7febc003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:23.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.753+0000 7febde57f700 1 --2- 172.21.15.136:0/606207904 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7febc0039150 0x7febc003b610 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7febcc006fd0 tx=0x7febcc006e40 comp rx=0 tx=0).stop 2024-04-25T00:09:23.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.753+0000 7febde57f700 1 -- 172.21.15.136:0/606207904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febd8101fb0 msgr2=0x7febd819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:23.754 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.753+0000 7febde57f700 1 --2- 172.21.15.136:0/606207904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febd8101fb0 0x7febd819d5f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7febc8005850 tx=0x7febc8005030 comp rx=0 tx=0).stop 2024-04-25T00:09:23.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.753+0000 7febde57f700 1 -- 172.21.15.136:0/606207904 shutdown_connections 2024-04-25T00:09:23.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.753+0000 7febde57f700 1 --2- 172.21.15.136:0/606207904 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7febc0039150 0x7febc003b610 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.753+0000 7febde57f700 1 --2- 172.21.15.136:0/606207904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7febd8101fb0 0x7febd819d5f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:23.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.753+0000 7febde57f700 1 -- 172.21.15.136:0/606207904 >> 172.21.15.136:0/606207904 conn(0x7febd80fd840 msgr2=0x7febd8103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:23.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.753+0000 7febde57f700 1 -- 172.21.15.136:0/606207904 shutdown_connections 2024-04-25T00:09:23.755 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:23.753+0000 7febde57f700 1 -- 172.21.15.136:0/606207904 wait complete. 2024-04-25T00:09:23.755 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 1 2024-04-25T00:09:23.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:23 smithi042 ceph-mon[28130]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:23.889 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 podman[32890]: 2024-04-25 00:09:23.865454313 +0000 UTC m=+0.365820131 container start 02072d7cff8cdca76d1d990160dbbe3585c1c00f1b3ce7430aeed2712fab773b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-c, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T00:09:23.889 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T00:09:23.889 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: ceph version 19.0.0-2445-gd2c45d97 (d2c45d977142e024a59b68a8f1dce8f0b9840551) squid (dev), process ceph-mon, pid 2 2024-04-25T00:09:23.889 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pidfile_write: ignore empty --pid-file 2024-04-25T00:09:24.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: load: jerasure load: lrc 2024-04-25T00:09:24.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: RocksDB version: 7.9.2 2024-04-25T00:09:24.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Git sha 0 2024-04-25T00:09:24.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Compile date 2024-04-24 16:41:23 2024-04-25T00:09:24.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: DB SUMMARY 2024-04-25T00:09:24.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: DB Session ID: 53RZ31S6TISN66HNO4ND 2024-04-25T00:09:24.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: CURRENT file: CURRENT 2024-04-25T00:09:24.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T00:09:24.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T00:09:24.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T00:09:24.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T00:09:24.266 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.error_if_exists: 0 2024-04-25T00:09:24.266 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.create_if_missing: 0 2024-04-25T00:09:24.266 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.paranoid_checks: 1 2024-04-25T00:09:24.266 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T00:09:24.266 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T00:09:24.266 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T00:09:24.266 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.env: 0x55c89a80bc60 2024-04-25T00:09:24.266 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T00:09:24.266 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.info_log: 0x55c89b391aa0 2024-04-25T00:09:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T00:09:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.statistics: (nil) 2024-04-25T00:09:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.use_fsync: 0 2024-04-25T00:09:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_log_file_size: 0 2024-04-25T00:09:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T00:09:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T00:09:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T00:09:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T00:09:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.allow_fallocate: 1 2024-04-25T00:09:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T00:09:24.268 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T00:09:24.268 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.use_direct_reads: 0 2024-04-25T00:09:24.268 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T00:09:24.268 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T00:09:24.268 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.db_log_dir: 2024-04-25T00:09:24.268 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.wal_dir: 2024-04-25T00:09:24.268 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T00:09:24.268 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T00:09:24.268 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T00:09:24.268 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T00:09:24.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T00:09:24.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T00:09:24.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T00:09:24.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T00:09:24.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.write_buffer_manager: 0x55c89b3641e0 2024-04-25T00:09:24.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T00:09:24.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T00:09:24.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T00:09:24.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T00:09:24.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T00:09:24.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T00:09:24.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T00:09:24.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T00:09:24.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.unordered_write: 0 2024-04-25T00:09:24.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T00:09:24.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T00:09:24.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T00:09:24.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T00:09:24.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.row_cache: None 2024-04-25T00:09:24.270 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.wal_filter: None 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.two_write_queues: 0 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.wal_compression: 0 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.atomic_flush: 0 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.log_readahead_size: 0 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T00:09:24.271 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T00:09:24.272 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T00:09:24.272 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T00:09:24.272 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T00:09:24.272 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T00:09:24.272 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T00:09:24.272 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_background_jobs: 2 2024-04-25T00:09:24.272 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_background_compactions: -1 2024-04-25T00:09:24.272 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_subcompactions: 1 2024-04-25T00:09:24.274 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T00:09:24.274 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T00:09:24.274 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T00:09:24.274 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T00:09:24.274 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_open_files: -1 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_background_flushes: -1 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Compression algorithms supported: 2024-04-25T00:09:24.275 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: kZSTD supported: 0 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: kXpressCompression supported: 0 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: kZlibCompression supported: 1 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: kSnappyCompression supported: 1 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: kLZ4Compression supported: 1 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: kBZip2Compression supported: 0 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T00:09:24.276 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.merge_operator: 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_filter: None 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c89b391ba0) 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout: cache_index_and_filter_blocks: 1 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T00:09:24.277 INFO:journalctl@ceph.mon.c.smithi136.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: pin_top_level_index_and_filter: 1 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: index_type: 0 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: data_block_index_type: 0 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: index_shortening: 1 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: checksum: 4 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: no_block_cache: 0 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_cache: 0x55c89b395090 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_cache_name: BinnedLRUCache 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_cache_options: 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: capacity : 536870912 2024-04-25T00:09:24.278 INFO:journalctl@ceph.mon.c.smithi136.stdout: num_shard_bits : 4 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: strict_capacity_limit : 0 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: high_pri_pool_ratio: 0.000 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_cache_compressed: (nil) 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: persistent_cache: (nil) 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_size: 4096 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_size_deviation: 10 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_restart_interval: 16 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: index_block_restart_interval: 1 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: metadata_block_size: 4096 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: partition_filters: 0 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: use_delta_encoding: 1 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: filter_policy: bloomfilter 2024-04-25T00:09:24.279 INFO:journalctl@ceph.mon.c.smithi136.stdout: whole_key_filtering: 1 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout: verify_compression: 0 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout: read_amp_bytes_per_bit: 0 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout: format_version: 5 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout: enable_index_compression: 1 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout: block_align: 0 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout: max_auto_readahead_size: 262144 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout: prepopulate_block_cache: 0 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout: initial_auto_readahead_size: 8192 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T00:09:24.280 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compression: NoCompression 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.num_levels: 7 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:09:24.281 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T00:09:24.282 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T00:09:24.283 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T00:09:24.284 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.table_properties_collectors: 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.inplace_update_support: 0 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T00:09:24.285 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.bloom_locality: 0 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.max_successive_merges: 0 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.ttl: 2592000 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T00:09:24.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.enable_blob_files: false 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.min_blob_size: 0 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d7b0cde8-a51c-4ed0-ad90-aead6acae2ff 2024-04-25T00:09:24.287 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003763906419, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003763906883, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714003763, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d7b0cde8-a51c-4ed0-ad90-aead6acae2ff", "db_session_id": "53RZ31S6TISN66HNO4ND", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714003763906979, "job": 1, "event": "recovery_finished"} 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c89b486000 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: DB pointer 0x55c89b470000 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout: ** DB Stats ** 2024-04-25T00:09:24.288 INFO:journalctl@ceph.mon.c.smithi136.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: ** Compaction Stats [default] ** 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T00:09:24.289 INFO:journalctl@ceph.mon.c.smithi136.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: ** Compaction Stats [default] ** 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.6 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T00:09:24.290 INFO:journalctl@ceph.mon.c.smithi136.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout: Cumulative compaction: 0.00 GB write, 0.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout: Interval compaction: 0.00 GB write, 0.11 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout: Block cache BinnedLRUCache@0x55c89b395090#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: using public_addr v2:172.21.15.136:0/0 -> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-25T00:09:24.291 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: starting mon.c rank -1 at public addrs [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] at bind addrs [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(???) e0 preinit fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout: e1 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout: legacy client fscid: -1 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout: 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout: No filesystems configured 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T00:09:24.292 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi106", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Deploying cephadm binary to smithi106 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.293 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Added host smithi106 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.294 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi106", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Updating smithi106:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Updating smithi106:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Updating smithi106:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.295 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Deploying cephadm binary to smithi136 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Added host smithi136 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.296 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Updating smithi136:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:24.297 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Updating smithi136:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1322513705' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1322513705' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi042:172.21.15.42=a;smithi106:172.21.15.106=b;smithi136:172.21.15.136=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Saving service mon spec with placement smithi042:172.21.15.42=a;smithi106:172.21.15.106=b;smithi136:172.21.15.136=c;count:3 2024-04-25T00:09:24.298 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: Deploying daemon mon.c on smithi136 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/365749296' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 ceph-mon[32923]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T00:09:24.299 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 bash[32890]: 02072d7cff8cdca76d1d990160dbbe3585c1c00f1b3ce7430aeed2712fab773b 2024-04-25T00:09:24.300 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:23 smithi136 systemd[1]: Started Ceph mon.c for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:09:25.535 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:09:25.535 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:09:25.781 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:09:29.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: mon.a calling monitor election 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: mon.c calling monitor election 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:09:29.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: monmap epoch 2 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: last_changed 2024-04-25T00:09:23.939096+0000 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: created 2024-04-25T00:07:11.904046+0000 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: min_mon_release 19 (squid) 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: election_strategy: 1 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: fsmap 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: mgrmap e13: x(active, since 63s) 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: overall HEALTH_OK 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:29.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:29.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:29.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:29.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:29.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:29.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: mon.a calling monitor election 2024-04-25T00:09:29.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: mon.c calling monitor election 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: monmap epoch 2 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: last_changed 2024-04-25T00:09:23.939096+0000 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: created 2024-04-25T00:07:11.904046+0000 2024-04-25T00:09:29.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: min_mon_release 19 (squid) 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: election_strategy: 1 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: fsmap 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: mgrmap e13: x(active, since 63s) 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: overall HEALTH_OK 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:29.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:30.160 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.159+0000 7fd21d634700 1 -- 172.21.15.136:0/131040556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2000035c0 msgr2=0x7fd200005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.159+0000 7fd21d634700 1 --2- 172.21.15.136:0/131040556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2000035c0 0x7fd200005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd218078360 tx=0x7fd20800feb0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.159+0000 7fd21d634700 1 -- 172.21.15.136:0/131040556 shutdown_connections 2024-04-25T00:09:30.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.159+0000 7fd21d634700 1 --2- 172.21.15.136:0/131040556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2000035c0 0x7fd200005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.159+0000 7fd21d634700 1 -- 172.21.15.136:0/131040556 >> 172.21.15.136:0/131040556 conn(0x7fd218077070 msgr2=0x7fd218077480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:30.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.159+0000 7fd21d634700 1 -- 172.21.15.136:0/131040556 shutdown_connections 2024-04-25T00:09:30.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.159+0000 7fd21d634700 1 -- 172.21.15.136:0/131040556 wait complete. 2024-04-25T00:09:30.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.160+0000 7fd21d634700 1 Processor -- start 2024-04-25T00:09:30.161 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.160+0000 7fd21d634700 1 -- start start 2024-04-25T00:09:30.162 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.160+0000 7fd21d634700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2000035c0 0x7fd21819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.160+0000 7fd21d634700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21819fd00 0x7fd218199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.160+0000 7fd21d634700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd218109dc0 con 0x7fd21819fd00 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.160+0000 7fd21d634700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd218109c40 con 0x7fd2000035c0 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.160+0000 7fd216ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2000035c0 0x7fd21819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.161+0000 7fd216ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2000035c0 0x7fd21819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:37220/0 (socket says 172.21.15.136:37220) 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.161+0000 7fd216ffd700 1 -- 172.21.15.136:0/7224921 learned_addr learned my addr 172.21.15.136:0/7224921 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.161+0000 7fd216ffd700 1 -- 172.21.15.136:0/7224921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2000035c0 msgr2=0x7fd21819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.161+0000 7fd216ffd700 1 -- 172.21.15.136:0/7224921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2000035c0 msgr2=0x7fd21819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.161+0000 7fd216ffd700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2000035c0 0x7fd21819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.161+0000 7fd216ffd700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2000035c0 0x7fd21819f7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.161+0000 7fd2167fc700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21819fd00 0x7fd218199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.162+0000 7fd2167fc700 1 -- 172.21.15.136:0/7224921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2000035c0 msgr2=0x7fd21819f7c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.162+0000 7fd2167fc700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2000035c0 0x7fd21819f7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.163 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.162+0000 7fd2167fc700 1 -- 172.21.15.136:0/7224921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2181a0340 con 0x7fd21819fd00 2024-04-25T00:09:30.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.162+0000 7fd2167fc700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21819fd00 0x7fd218199980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00b4d0 tx=0x7fd20c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.162+0000 7fd1fffff700 1 -- 172.21.15.136:0/7224921 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd20c011630 con 0x7fd21819fd00 2024-04-25T00:09:30.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.163+0000 7fd21d634700 1 -- 172.21.15.136:0/7224921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd218199ec0 con 0x7fd21819fd00 2024-04-25T00:09:30.164 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.163+0000 7fd21d634700 1 -- 172.21.15.136:0/7224921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21819a450 con 0x7fd21819fd00 2024-04-25T00:09:30.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.163+0000 7fd1fffff700 1 -- 172.21.15.136:0/7224921 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd20c011cb0 con 0x7fd21819fd00 2024-04-25T00:09:30.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.163+0000 7fd1fffff700 1 -- 172.21.15.136:0/7224921 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fd20c0193b0 con 0x7fd21819fd00 2024-04-25T00:09:30.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.164+0000 7fd21d634700 1 -- 172.21.15.136:0/7224921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd200002160 con 0x7fd21819fd00 2024-04-25T00:09:30.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.164+0000 7fd1fffff700 1 -- 172.21.15.136:0/7224921 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fd20c018020 con 0x7fd21819fd00 2024-04-25T00:09:30.165 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.164+0000 7fd1fffff700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd204038e90 0x7fd20403b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:30.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.164+0000 7fd1fffff700 1 -- 172.21.15.136:0/7224921 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd20c04ba80 con 0x7fd21819fd00 2024-04-25T00:09:30.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.165+0000 7fd216ffd700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd204038e90 0x7fd20403b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:30.166 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.166+0000 7fd216ffd700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd204038e90 0x7fd20403b350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd20800fa60 tx=0x7fd208005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:30.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.180+0000 7fd1fffff700 1 -- 172.21.15.136:0/7224921 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd20c01f020 con 0x7fd21819fd00 2024-04-25T00:09:30.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:30 smithi042 ceph-mon[28130]: Deploying daemon mon.b on smithi106 2024-04-25T00:09:30.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:30 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:30.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.463+0000 7fd21d634700 1 -- 172.21.15.136:0/7224921 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd200002c50 con 0x7fd21819fd00 2024-04-25T00:09:30.465 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:30 smithi136 ceph-mon[32923]: Deploying daemon mon.b on smithi106 2024-04-25T00:09:30.465 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:30 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:30.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.465+0000 7fd1fffff700 1 -- 172.21.15.136:0/7224921 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fd20c018570 con 0x7fd21819fd00 2024-04-25T00:09:30.465 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:09:30.466 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":2,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","modified":"2024-04-25T00:09:23.939096Z","created":"2024-04-25T00:07:11.904046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T00:09:30.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 -- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd204038e90 msgr2=0x7fd20403b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd204038e90 0x7fd20403b350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd20800fa60 tx=0x7fd208005960 comp rx=0 tx=0).stop 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 -- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21819fd00 msgr2=0x7fd218199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21819fd00 0x7fd218199980 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd20c00b4d0 tx=0x7fd20c00b890 comp rx=0 tx=0).stop 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 -- 172.21.15.136:0/7224921 shutdown_connections 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd204038e90 0x7fd20403b350 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2000035c0 0x7fd21819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 --2- 172.21.15.136:0/7224921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd21819fd00 0x7fd218199980 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 -- 172.21.15.136:0/7224921 >> 172.21.15.136:0/7224921 conn(0x7fd218077070 msgr2=0x7fd2180fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 -- 172.21.15.136:0/7224921 shutdown_connections 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:30.467+0000 7fd21d634700 1 -- 172.21.15.136:0/7224921 wait complete. 2024-04-25T00:09:30.468 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 2 2024-04-25T00:09:31.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:31 smithi042 ceph-mon[28130]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:31.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:31 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/7224921' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:09:31.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:31 smithi136 ceph-mon[32923]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:31.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:31 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/7224921' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:09:32.071 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T00:09:32.071 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T00:09:32.314 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:09:33.332 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:33 smithi136 ceph-mon[32923]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:33.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:33 smithi106 systemd[1]: Started Ceph mon.b for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:09:33.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:33 smithi042 ceph-mon[28130]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:38.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.178+0000 7f7bb2296700 1 -- 172.21.15.136:0/1785606837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b940057d0 msgr2=0x7f7b94007c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:38.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.178+0000 7f7bb2296700 1 --2- 172.21.15.136:0/1785606837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b940057d0 0x7f7b94007c60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7bac100b70 tx=0x7f7b9c00fe60 comp rx=0 tx=0).stop 2024-04-25T00:09:38.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.178+0000 7f7bb2296700 1 -- 172.21.15.136:0/1785606837 shutdown_connections 2024-04-25T00:09:38.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.178+0000 7f7bb2296700 1 --2- 172.21.15.136:0/1785606837 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7b940057d0 0x7f7b94007c60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.178+0000 7f7bb2296700 1 -- 172.21.15.136:0/1785606837 >> 172.21.15.136:0/1785606837 conn(0x7f7bac0fc770 msgr2=0x7f7bac0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:38.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.178+0000 7f7bb2296700 1 -- 172.21.15.136:0/1785606837 shutdown_connections 2024-04-25T00:09:38.180 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.179+0000 7f7bb2296700 1 -- 172.21.15.136:0/1785606837 wait complete. 2024-04-25T00:09:38.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.180+0000 7f7bb2296700 1 Processor -- start 2024-04-25T00:09:38.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.180+0000 7f7bb2296700 1 -- start start 2024-04-25T00:09:38.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.180+0000 7f7bb2296700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b940057d0 0x7f7bac19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.180+0000 7f7bb2296700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac19b970 0x7f7bac1955f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7bb2296700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bac195b30 0x7f7bac195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7bb2296700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac107ca0 con 0x7f7bac195b30 2024-04-25T00:09:38.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7bb2296700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac107b20 con 0x7f7bac19b970 2024-04-25T00:09:38.181 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7bb2296700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac107e20 con 0x7f7b940057d0 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7babfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b940057d0 0x7f7bac19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7babfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b940057d0 0x7f7bac19b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:40568/0 (socket says 172.21.15.136:40568) 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7babfff700 1 -- 172.21.15.136:0/4024016140 learned_addr learned my addr 172.21.15.136:0/4024016140 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7babfff700 1 -- 172.21.15.136:0/4024016140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b940057d0 msgr2=0x7f7bac19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7babfff700 1 -- 172.21.15.136:0/4024016140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b940057d0 msgr2=0x7f7bac19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7babfff700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b940057d0 0x7f7bac19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7bab7fe700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac19b970 0x7f7bac1955f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7babfff700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b940057d0 0x7f7bac19b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7bab7fe700 1 -- 172.21.15.136:0/4024016140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac19b970 msgr2=0x7f7bac1955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7bab7fe700 1 -- 172.21.15.136:0/4024016140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac19b970 msgr2=0x7f7bac1955f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T00:09:38.183 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7bab7fe700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac19b970 0x7f7bac1955f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T00:09:38.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7bab7fe700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac19b970 0x7f7bac1955f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T00:09:38.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.181+0000 7f7bb0833700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bac195b30 0x7f7bac195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.182+0000 7f7bb0833700 1 -- 172.21.15.136:0/4024016140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b940057d0 msgr2=0x7f7bac19b430 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:09:38.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.182+0000 7f7bb0833700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b940057d0 0x7f7bac19b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.182+0000 7f7bb0833700 1 -- 172.21.15.136:0/4024016140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac19b970 msgr2=0x7f7bac1955f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:09:38.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.182+0000 7f7bb0833700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac19b970 0x7f7bac1955f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.182+0000 7f7bb0833700 1 -- 172.21.15.136:0/4024016140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac1a2300 con 0x7f7bac195b30 2024-04-25T00:09:38.184 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.182+0000 7f7bb0833700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bac195b30 0x7f7bac195fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400bd50 tx=0x7f7ba400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.183+0000 7f7ba97fa700 1 -- 172.21.15.136:0/4024016140 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4019070 con 0x7f7bac195b30 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.183+0000 7f7bb2296700 1 -- 172.21.15.136:0/4024016140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bac1a2530 con 0x7f7bac195b30 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.183+0000 7f7ba97fa700 1 -- 172.21.15.136:0/4024016140 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ba400cd10 con 0x7f7bac195b30 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.183+0000 7f7ba97fa700 1 -- 172.21.15.136:0/4024016140 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40127e0 con 0x7f7bac195b30 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.183+0000 7f7bb2296700 1 -- 172.21.15.136:0/4024016140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bac1a2ac0 con 0x7f7bac195b30 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.184+0000 7f7bb2296700 1 -- 172.21.15.136:0/4024016140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bac063890 con 0x7f7bac195b30 2024-04-25T00:09:38.185 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.184+0000 7f7ba97fa700 1 -- 172.21.15.136:0/4024016140 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7ba401c070 con 0x7f7bac195b30 2024-04-25T00:09:38.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.185+0000 7f7ba97fa700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7b98039580 0x7f7b9803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:38.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.185+0000 7f7ba97fa700 1 -- 172.21.15.136:0/4024016140 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7ba404bb40 con 0x7f7bac195b30 2024-04-25T00:09:38.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.185+0000 7f7babfff700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7b98039580 0x7f7b9803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:38.186 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.186+0000 7f7babfff700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7b98039580 0x7f7b9803ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c001eb0 tx=0x7f7b9c005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:38.202 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.201+0000 7f7ba97fa700 1 -- 172.21.15.136:0/4024016140 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7ba4014ab0 con 0x7f7bac195b30 2024-04-25T00:09:38.485 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.484+0000 7f7bb2296700 1 -- 172.21.15.136:0/4024016140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f7bac02db30 con 0x7f7bac195b30 2024-04-25T00:09:38.485 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: mon.c calling monitor election 2024-04-25T00:09:38.485 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: mon.a calling monitor election 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: monmap epoch 3 2024-04-25T00:09:38.486 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: last_changed 2024-04-25T00:09:33.075473+0000 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: created 2024-04-25T00:07:11.904046+0000 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: min_mon_release 19 (squid) 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: election_strategy: 1 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: fsmap 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: mgrmap e13: x(active, since 72s) 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: overall HEALTH_OK 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:38.487 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:38.488 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.485+0000 7f7ba97fa700 1 -- 172.21.15.136:0/4024016140 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f7ba404a450 con 0x7f7bac195b30 2024-04-25T00:09:38.488 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-25T00:09:38.488 INFO:teuthology.orchestra.run.smithi136.stdout:{"epoch":3,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","modified":"2024-04-25T00:09:33.075473Z","created":"2024-04-25T00:07:11.904046Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:3300","nonce":0},{"type":"v1","addr":"172.21.15.42:6789","nonce":0}]},"addr":"172.21.15.42:6789/0","public_addr":"172.21.15.42:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:3300","nonce":0},{"type":"v1","addr":"172.21.15.106:6789","nonce":0}]},"addr":"172.21.15.106:6789/0","public_addr":"172.21.15.106:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T00:09:38.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 -- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7b98039580 msgr2=0x7f7b9803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:38.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7b98039580 0x7f7b9803ba40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c001eb0 tx=0x7f7b9c005de0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 -- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bac195b30 msgr2=0x7f7bac195fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:38.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bac195b30 0x7f7bac195fe0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400bd50 tx=0x7f7ba400be30 comp rx=0 tx=0).stop 2024-04-25T00:09:38.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 -- 172.21.15.136:0/4024016140 shutdown_connections 2024-04-25T00:09:38.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7b98039580 0x7f7b9803ba40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7bac195b30 0x7f7bac195fe0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7b940057d0 0x7f7bac19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 --2- 172.21.15.136:0/4024016140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bac19b970 0x7f7bac1955f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:38.489 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 -- 172.21.15.136:0/4024016140 >> 172.21.15.136:0/4024016140 conn(0x7f7bac0fc770 msgr2=0x7f7bac0fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:38.490 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 -- 172.21.15.136:0/4024016140 shutdown_connections 2024-04-25T00:09:38.490 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:38.488+0000 7f7bb2296700 1 -- 172.21.15.136:0/4024016140 wait complete. 2024-04-25T00:09:38.490 INFO:teuthology.orchestra.run.smithi136.stderr:dumped monmap epoch 3 2024-04-25T00:09:38.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: mon.c calling monitor election 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: mon.a calling monitor election 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:38.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: monmap epoch 3 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: last_changed 2024-04-25T00:09:33.075473+0000 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: created 2024-04-25T00:07:11.904046+0000 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: min_mon_release 19 (squid) 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: election_strategy: 1 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: fsmap 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: mgrmap e13: x(active, since 72s) 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: overall HEALTH_OK 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:38.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:39.104 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T00:09:39.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T00:09:39.400 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:39 smithi136 ceph-mon[32923]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:39.400 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:39 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:39.400 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:39 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:39.400 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:39 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:39.400 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:39 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/4024016140' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:09:39.400 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:39 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:39.529 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:09:39.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:39 smithi042 ceph-mon[28130]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:39.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:39 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:39.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:39 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:39.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:39 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:39.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:39 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/4024016140' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:09:39.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:39 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:40.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: mon.c calling monitor election 2024-04-25T00:09:40.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:09:40.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: mon.a calling monitor election 2024-04-25T00:09:40.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: monmap epoch 3 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: last_changed 2024-04-25T00:09:33.075473+0000 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: created 2024-04-25T00:07:11.904046+0000 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: min_mon_release 19 (squid) 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: election_strategy: 1 2024-04-25T00:09:40.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: fsmap 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: mgrmap e13: x(active, since 72s) 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: overall HEALTH_OK 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='client.? 172.21.15.136:0/4024016140' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T00:09:40.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:41.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: mon.b calling monitor election 2024-04-25T00:09:41.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:41.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: Updating smithi042:/etc/ceph/ceph.conf 2024-04-25T00:09:41.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-25T00:09:41.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T00:09:41.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: mon.b calling monitor election 2024-04-25T00:09:41.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: mon.c calling monitor election 2024-04-25T00:09:41.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: mon.a calling monitor election 2024-04-25T00:09:41.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:09:41.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: monmap epoch 3 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: last_changed 2024-04-25T00:09:33.075473+0000 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: created 2024-04-25T00:07:11.904046+0000 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: min_mon_release 19 (squid) 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: election_strategy: 1 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: fsmap 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: mgrmap e13: x(active, since 74s) 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: overall HEALTH_OK 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: Updating smithi106:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: Updating smithi136:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:41.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: Updating smithi042:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:41.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: mon.b calling monitor election 2024-04-25T00:09:41.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: Updating smithi042:/etc/ceph/ceph.conf 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: mon.b calling monitor election 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: mon.c calling monitor election 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: mon.a calling monitor election 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: monmap epoch 3 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: last_changed 2024-04-25T00:09:33.075473+0000 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: created 2024-04-25T00:07:11.904046+0000 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: min_mon_release 19 (squid) 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: election_strategy: 1 2024-04-25T00:09:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: fsmap 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: mgrmap e13: x(active, since 74s) 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: overall HEALTH_OK 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: Updating smithi106:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: Updating smithi136:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: Updating smithi042:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:41.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: mon.b calling monitor election 2024-04-25T00:09:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: Updating smithi042:/etc/ceph/ceph.conf 2024-04-25T00:09:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: Updating smithi106:/etc/ceph/ceph.conf 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: Updating smithi136:/etc/ceph/ceph.conf 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: mon.b calling monitor election 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: mon.c calling monitor election 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: mon.a calling monitor election 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: monmap epoch 3 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: last_changed 2024-04-25T00:09:33.075473+0000 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: created 2024-04-25T00:07:11.904046+0000 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: min_mon_release 19 (squid) 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: election_strategy: 1 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: 0: [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] mon.a 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: 1: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-25T00:09:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: 2: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.b 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: fsmap 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: mgrmap e13: x(active, since 74s) 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: overall HEALTH_OK 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: Updating smithi106:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: Updating smithi136:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: Updating smithi042:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T00:09:41.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:41.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.759+0000 7efe25200700 1 -- 172.21.15.42:0/2135782795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe20074660 msgr2=0x7efe20074a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.759+0000 7efe25200700 1 --2- 172.21.15.42:0/2135782795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe20074660 0x7efe20074a40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efe10009910 tx=0x7efe10009c20 comp rx=0 tx=0).stop 2024-04-25T00:09:41.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.759+0000 7efe25200700 1 -- 172.21.15.42:0/2135782795 shutdown_connections 2024-04-25T00:09:41.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.759+0000 7efe25200700 1 --2- 172.21.15.42:0/2135782795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe20074660 0x7efe20074a40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.759+0000 7efe25200700 1 -- 172.21.15.42:0/2135782795 >> 172.21.15.42:0/2135782795 conn(0x7efe200fd770 msgr2=0x7efe200ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:41.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.759+0000 7efe25200700 1 -- 172.21.15.42:0/2135782795 shutdown_connections 2024-04-25T00:09:41.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.759+0000 7efe25200700 1 -- 172.21.15.42:0/2135782795 wait complete. 2024-04-25T00:09:41.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.760+0000 7efe25200700 1 Processor -- start 2024-04-25T00:09:41.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.760+0000 7efe25200700 1 -- start start 2024-04-25T00:09:41.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.761+0000 7efe25200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe20074660 0x7efe2019f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.761+0000 7efe25200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe2019fe10 0x7efe20199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.761+0000 7efe25200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe20199fd0 0x7efe2019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.761+0000 7efe25200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010e4f0 con 0x7efe20074660 2024-04-25T00:09:41.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.761+0000 7efe25200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010e370 con 0x7efe20199fd0 2024-04-25T00:09:41.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.761+0000 7efe25200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efe2010e670 con 0x7efe2019fe10 2024-04-25T00:09:41.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.761+0000 7efe1de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe20074660 0x7efe2019f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.761+0000 7efe1de00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe20074660 0x7efe2019f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56220/0 (socket says 172.21.15.42:56220) 2024-04-25T00:09:41.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.761+0000 7efe1de00700 1 -- 172.21.15.42:0/1476969763 learned_addr learned my addr 172.21.15.42:0/1476969763 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:09:41.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.762+0000 7efe1d400700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe2019fe10 0x7efe20199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.762+0000 7efe1e800700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe20199fd0 0x7efe2019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.762 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.762+0000 7efe1de00700 1 -- 172.21.15.42:0/1476969763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe2019fe10 msgr2=0x7efe20199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.762+0000 7efe1de00700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe2019fe10 0x7efe20199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.762+0000 7efe1de00700 1 -- 172.21.15.42:0/1476969763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe20199fd0 msgr2=0x7efe2019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:41.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.762+0000 7efe1de00700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe20199fd0 0x7efe2019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:41.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.762+0000 7efe1de00700 1 -- 172.21.15.42:0/1476969763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe201a67a0 con 0x7efe20074660 2024-04-25T00:09:41.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.762+0000 7efe1de00700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe20074660 0x7efe2019f8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efe100092e0 tx=0x7efe10004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:41.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.763+0000 7efe16a00700 1 -- 172.21.15.42:0/1476969763 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10026070 con 0x7efe20074660 2024-04-25T00:09:41.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.763+0000 7efe16a00700 1 -- 172.21.15.42:0/1476969763 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe1000b750 con 0x7efe20074660 2024-04-25T00:09:41.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.763+0000 7efe16a00700 1 -- 172.21.15.42:0/1476969763 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10003a00 con 0x7efe20074660 2024-04-25T00:09:41.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.763+0000 7efe25200700 1 -- 172.21.15.42:0/1476969763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe201a6970 con 0x7efe20074660 2024-04-25T00:09:41.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.763+0000 7efe25200700 1 -- 172.21.15.42:0/1476969763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe201a6ea0 con 0x7efe20074660 2024-04-25T00:09:41.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.765+0000 7efe25200700 1 -- 172.21.15.42:0/1476969763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe20063890 con 0x7efe20074660 2024-04-25T00:09:41.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.765+0000 7efe16a00700 1 -- 172.21.15.42:0/1476969763 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7efe10029070 con 0x7efe20074660 2024-04-25T00:09:41.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.765+0000 7efe16a00700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efe04039580 0x7efe0403ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:41.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.765+0000 7efe16a00700 1 -- 172.21.15.42:0/1476969763 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7efe1004cde0 con 0x7efe20074660 2024-04-25T00:09:41.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.766+0000 7efe1d400700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efe04039580 0x7efe0403ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:41.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.767+0000 7efe1d400700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efe04039580 0x7efe0403ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efe08009ad0 tx=0x7efe08009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:41.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:41.771+0000 7efe16a00700 1 -- 172.21.15.42:0/1476969763 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efe10052020 con 0x7efe20074660 2024-04-25T00:09:42.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.003+0000 7efe25200700 1 -- 172.21.15.42:0/1476969763 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efe201a7130 con 0x7efe20074660 2024-04-25T00:09:42.004 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.004+0000 7efe16a00700 1 -- 172.21.15.42:0/1476969763 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7efe10030020 con 0x7efe20074660 2024-04-25T00:09:42.005 INFO:teuthology.orchestra.run.smithi042.stdout:# minimal ceph.conf for 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:42.005 INFO:teuthology.orchestra.run.smithi042.stdout:[global] 2024-04-25T00:09:42.005 INFO:teuthology.orchestra.run.smithi042.stdout: fsid = 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:09:42.005 INFO:teuthology.orchestra.run.smithi042.stdout: mon_host = [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] 2024-04-25T00:09:42.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.007+0000 7efe25200700 1 -- 172.21.15.42:0/1476969763 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efe04039580 msgr2=0x7efe0403ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.007+0000 7efe25200700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efe04039580 0x7efe0403ba40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efe08009ad0 tx=0x7efe08009210 comp rx=0 tx=0).stop 2024-04-25T00:09:42.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.008+0000 7efe25200700 1 -- 172.21.15.42:0/1476969763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe20074660 msgr2=0x7efe2019f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:42.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.008+0000 7efe25200700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe20074660 0x7efe2019f8d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efe100092e0 tx=0x7efe10004fa0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.008 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.008+0000 7efe25200700 1 -- 172.21.15.42:0/1476969763 shutdown_connections 2024-04-25T00:09:42.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.008+0000 7efe25200700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efe04039580 0x7efe0403ba40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.008+0000 7efe25200700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe20199fd0 0x7efe2019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.008+0000 7efe25200700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efe20074660 0x7efe2019f8d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.008+0000 7efe25200700 1 --2- 172.21.15.42:0/1476969763 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efe2019fe10 0x7efe20199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:42.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.008+0000 7efe25200700 1 -- 172.21.15.42:0/1476969763 >> 172.21.15.42:0/1476969763 conn(0x7efe200fd770 msgr2=0x7efe20106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:42.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.008+0000 7efe25200700 1 -- 172.21.15.42:0/1476969763 shutdown_connections 2024-04-25T00:09:42.009 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:42.008+0000 7efe25200700 1 -- 172.21.15.42:0/1476969763 wait complete. 2024-04-25T00:09:42.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:42.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.028 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.029 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:09:42.029 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:42.029 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:09:42.029 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:42.029 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:41 smithi042 ceph-mon[28130]: Reconfiguring daemon mon.a on smithi042 2024-04-25T00:09:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:09:42.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:42.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:09:42.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:42.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:41 smithi136 ceph-mon[32923]: Reconfiguring daemon mon.a on smithi042 2024-04-25T00:09:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:09:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:42.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T00:09:42.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:42.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:09:42.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:42.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:41 smithi106 ceph-mon[32510]: Reconfiguring daemon mon.a on smithi042 2024-04-25T00:09:42.842 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T00:09:42.843 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:09:42.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:09:42.918 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:09:42.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:09:43.022 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:42 smithi042 ceph-mon[28130]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:43.023 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:42 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1476969763' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:43.034 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T00:09:43.034 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:09:43.067 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T00:09:43.067 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:09:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:42 smithi106 ceph-mon[32510]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:43.128 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:42 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1476969763' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:43.131 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T00:09:43.131 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T00:09:43.158 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:42 smithi136 ceph-mon[32923]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:43.158 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:42 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1476969763' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:43.164 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T00:09:43.164 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T00:09:43.231 INFO:tasks.cephadm:Adding mgr.x on smithi042 2024-04-25T00:09:43.231 INFO:tasks.cephadm:Adding mgr.y on smithi106 2024-04-25T00:09:43.231 INFO:tasks.cephadm:Adding mgr.z on smithi136 2024-04-25T00:09:43.232 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi042=x;smithi106=y;smithi136=z' 2024-04-25T00:09:43.500 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:09:44.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.995+0000 7fb0268a1700 1 -- 172.21.15.136:0/3904128525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb020101dc0 msgr2=0x7fb0201021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:44.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.995+0000 7fb0268a1700 1 --2- 172.21.15.136:0/3904128525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb020101dc0 0x7fb0201021e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb010009910 tx=0x7fb010009c20 comp rx=0 tx=0).stop 2024-04-25T00:09:44.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.995+0000 7fb0268a1700 1 -- 172.21.15.136:0/3904128525 shutdown_connections 2024-04-25T00:09:44.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.995+0000 7fb0268a1700 1 --2- 172.21.15.136:0/3904128525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb020101dc0 0x7fb0201021e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:44.998 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.995+0000 7fb0268a1700 1 -- 172.21.15.136:0/3904128525 >> 172.21.15.136:0/3904128525 conn(0x7fb0200fd360 msgr2=0x7fb0200ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.995+0000 7fb0268a1700 1 -- 172.21.15.136:0/3904128525 shutdown_connections 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.995+0000 7fb0268a1700 1 -- 172.21.15.136:0/3904128525 wait complete. 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb0268a1700 1 Processor -- start 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb0268a1700 1 -- start start 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb0268a1700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb020101dc0 0x7fb0201975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb0268a1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb020197b10 0x7fb02019cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb0268a1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb020197f90 0x7fb02019f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb0268a1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb020107e50 con 0x7fb020197b10 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb0268a1700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb020107cd0 con 0x7fb020101dc0 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb0268a1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb020107fd0 con 0x7fb020197f90 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb024e3e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb020197f90 0x7fb02019f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb024e3e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb020197f90 0x7fb02019f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:40572/0 (socket says 172.21.15.136:40572) 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.996+0000 7fb024e3e700 1 -- 172.21.15.136:0/1624848125 learned_addr learned my addr 172.21.15.136:0/1624848125 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:09:44.999 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb024e3e700 1 -- 172.21.15.136:0/1624848125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb020101dc0 msgr2=0x7fb0201975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb024e3e700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb020101dc0 0x7fb0201975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb024e3e700 1 -- 172.21.15.136:0/1624848125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb020197b10 msgr2=0x7fb02019cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb024e3e700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb020197b10 0x7fb02019cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb024e3e700 1 -- 172.21.15.136:0/1624848125 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02019f9e0 con 0x7fb020197f90 2024-04-25T00:09:45.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb024e3e700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb020197f90 0x7fb02019f110 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb01800ba50 tx=0x7fb01800be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb01d7fa700 1 -- 172.21.15.136:0/1624848125 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018019070 con 0x7fb020197f90 2024-04-25T00:09:45.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb01d7fa700 1 -- 172.21.15.136:0/1624848125 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb018014440 con 0x7fb020197f90 2024-04-25T00:09:45.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb01d7fa700 1 -- 172.21.15.136:0/1624848125 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018007590 con 0x7fb020197f90 2024-04-25T00:09:45.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb0268a1700 1 -- 172.21.15.136:0/1624848125 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02019fcd0 con 0x7fb020197f90 2024-04-25T00:09:45.000 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.997+0000 7fb0268a1700 1 -- 172.21.15.136:0/1624848125 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0201a0240 con 0x7fb020197f90 2024-04-25T00:09:45.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.998+0000 7fb01d7fa700 1 -- 172.21.15.136:0/1624848125 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb01801c070 con 0x7fb020197f90 2024-04-25T00:09:45.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.998+0000 7fb0268a1700 1 -- 172.21.15.136:0/1624848125 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb020063890 con 0x7fb020197f90 2024-04-25T00:09:45.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.998+0000 7fb01d7fa700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb008039580 0x7fb00803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:45.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.998+0000 7fb01d7fa700 1 -- 172.21.15.136:0/1624848125 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb01804baa0 con 0x7fb020197f90 2024-04-25T00:09:45.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.999+0000 7fb01ffff700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb008039580 0x7fb00803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:45.001 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:44.999+0000 7fb01ffff700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb008039580 0x7fb00803ba40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb010005e00 tx=0x7fb0100052d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:45.004 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.003+0000 7fb01d7fa700 1 -- 172.21.15.136:0/1624848125 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb018014910 con 0x7fb020197f90 2024-04-25T00:09:45.209 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.209+0000 7fb0268a1700 1 -- 172.21.15.136:0/1624848125 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi106=y;smithi136=z", "target": ["mon-mgr", ""]}) -- 0x7fb020062670 con 0x7fb008039580 2024-04-25T00:09:45.222 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.222+0000 7fb01d7fa700 1 -- 172.21.15.136:0/1624848125 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb020062670 con 0x7fb008039580 2024-04-25T00:09:45.223 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mgr update... 2024-04-25T00:09:45.224 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 -- 172.21.15.136:0/1624848125 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb008039580 msgr2=0x7fb00803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb008039580 0x7fb00803ba40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb010005e00 tx=0x7fb0100052d0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 -- 172.21.15.136:0/1624848125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb020197f90 msgr2=0x7fb02019f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb020197f90 0x7fb02019f110 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb01800ba50 tx=0x7fb01800be10 comp rx=0 tx=0).stop 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 -- 172.21.15.136:0/1624848125 shutdown_connections 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb008039580 0x7fb00803ba40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb020197f90 0x7fb02019f110 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb020101dc0 0x7fb0201975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 --2- 172.21.15.136:0/1624848125 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb020197b10 0x7fb02019cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 -- 172.21.15.136:0/1624848125 >> 172.21.15.136:0/1624848125 conn(0x7fb0200fd360 msgr2=0x7fb0200fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 -- 172.21.15.136:0/1624848125 shutdown_connections 2024-04-25T00:09:45.225 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:09:45.224+0000 7fb0268a1700 1 -- 172.21.15.136:0/1624848125 wait complete. 2024-04-25T00:09:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:45 smithi136 ceph-mon[32923]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:45 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:45 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:45 smithi136 ceph-mon[32923]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:09:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:45 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:45 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:09:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:45 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:45 smithi136 ceph-mon[32923]: Reconfiguring daemon mon.b on smithi106 2024-04-25T00:09:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:45 smithi106 ceph-mon[32510]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:45 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:45 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:45 smithi106 ceph-mon[32510]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:09:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:45 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:45 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:09:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:45 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:45 smithi106 ceph-mon[32510]: Reconfiguring daemon mon.b on smithi106 2024-04-25T00:09:45.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:45 smithi042 ceph-mon[28130]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:45.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:45 smithi042 ceph-mon[28130]: Reconfiguring mon.b (monmap changed)... 2024-04-25T00:09:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:09:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:45.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:45 smithi042 ceph-mon[28130]: Reconfiguring daemon mon.b on smithi106 2024-04-25T00:09:45.835 DEBUG:teuthology.orchestra.run.smithi106:mgr.y> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T00:09:45.838 DEBUG:teuthology.orchestra.run.smithi136:mgr.z> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T00:09:45.840 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T00:09:45.840 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:09:45.840 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:09:45.901 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:09:45.901 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_1 2024-04-25T00:09:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:09:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:45.954 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-04-25T00:09:45.954 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:45.954 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:45.954 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-25 00:08:42.116788228 +0000 2024-04-25T00:09:45.954 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-25 00:03:32.660856166 +0000 2024-04-25T00:09:45.954 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-25 00:03:32.660856166 +0000 2024-04-25T00:09:45.954 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-25 00:03:32.660856166 +0000 2024-04-25T00:09:45.955 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:09:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-25T00:09:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-25T00:09:46.038 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000294416 s, 1.7 MB/s 2024-04-25T00:09:46.042 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:09:46.102 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_2 2024-04-25T00:09:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:09:46.153 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:46.153 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-25T00:09:46.153 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:46.153 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:46.153 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-25 00:08:42.194787613 +0000 2024-04-25T00:09:46.154 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-25 00:03:33.263851410 +0000 2024-04-25T00:09:46.154 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-25 00:03:33.263851410 +0000 2024-04-25T00:09:46.154 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-25 00:03:33.263851410 +0000 2024-04-25T00:09:46.154 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:09:46.236 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-25T00:09:46.236 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-25T00:09:46.236 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000259642 s, 2.0 MB/s 2024-04-25T00:09:46.240 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:09:46.301 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_3 2024-04-25T00:09:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:09:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-25T00:09:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-25 00:08:42.257787117 +0000 2024-04-25T00:09:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-25 00:03:33.865846661 +0000 2024-04-25T00:09:46.355 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-25 00:03:33.865846661 +0000 2024-04-25T00:09:46.355 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-25 00:03:33.865846661 +0000 2024-04-25T00:09:46.355 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:09:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:46 smithi042 ceph-mon[28130]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi106=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:46 smithi042 ceph-mon[28130]: Saving service mgr spec with placement smithi042=x;smithi106=y;smithi136=z;count:3 2024-04-25T00:09:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:46 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:46 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:46 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:46 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:46 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:09:46.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:46 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:46.437 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-25T00:09:46.437 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-25T00:09:46.437 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000298126 s, 1.7 MB/s 2024-04-25T00:09:46.441 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:09:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:46 smithi136 ceph-mon[32923]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi106=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:46 smithi136 ceph-mon[32923]: Saving service mgr spec with placement smithi042=x;smithi106=y;smithi136=z;count:3 2024-04-25T00:09:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:46 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:46 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:46 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:46 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:46 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:09:46.488 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:46 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:46.545 DEBUG:teuthology.orchestra.run.smithi042:> stat /dev/vg_nvme/lv_4 2024-04-25T00:09:46.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:46 smithi106 ceph-mon[32510]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi042=x;smithi106=y;smithi136=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:46.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:46 smithi106 ceph-mon[32510]: Saving service mgr spec with placement smithi042=x;smithi106=y;smithi136=z;count:3 2024-04-25T00:09:46.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:46 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:46.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:46 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:46.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:46 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:46.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:46 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T00:09:46.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:46 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T00:09:46.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:46 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:46.598 INFO:teuthology.orchestra.run.smithi042.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:09:46.598 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:46.599 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-25T00:09:46.599 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:46.599 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:46.599 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-25 00:08:42.317786644 +0000 2024-04-25T00:09:46.599 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-25 00:03:34.523841469 +0000 2024-04-25T00:09:46.599 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-25 00:03:34.523841469 +0000 2024-04-25T00:09:46.599 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-25 00:03:34.523841469 +0000 2024-04-25T00:09:46.599 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:09:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records in 2024-04-25T00:09:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:1+0 records out 2024-04-25T00:09:46.684 INFO:teuthology.orchestra.run.smithi042.stderr:512 bytes copied, 0.000294459 s, 1.7 MB/s 2024-04-25T00:09:46.689 DEBUG:teuthology.orchestra.run.smithi042:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:09:46.751 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2024-04-25T00:09:46.751 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:09:46.770 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:09:46.770 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2024-04-25T00:09:46.828 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:09:46.828 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:46.828 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T00:09:46.828 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:46.828 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:46.828 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 00:09:32.499977096 +0000 2024-04-25T00:09:46.828 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 00:02:25.713916592 +0000 2024-04-25T00:09:46.829 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 00:02:25.713916592 +0000 2024-04-25T00:09:46.829 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-25 00:02:25.713916592 +0000 2024-04-25T00:09:46.829 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:09:46.894 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T00:09:46.894 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T00:09:46.895 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000231612 s, 2.2 MB/s 2024-04-25T00:09:46.896 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:09:46.955 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2024-04-25T00:09:47.012 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:09:47.012 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:47.012 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-25T00:09:47.012 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:47.012 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:47.012 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 00:09:32.499977096 +0000 2024-04-25T00:09:47.012 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 00:02:26.090907299 +0000 2024-04-25T00:09:47.012 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 00:02:26.090907299 +0000 2024-04-25T00:09:47.012 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-25 00:02:26.090907299 +0000 2024-04-25T00:09:47.013 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:09:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T00:09:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T00:09:47.077 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000247861 s, 2.1 MB/s 2024-04-25T00:09:47.078 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:09:47.136 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2024-04-25T00:09:47.194 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:09:47.194 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:47.195 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T00:09:47.195 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:47.195 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:47.195 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 00:09:32.499977096 +0000 2024-04-25T00:09:47.195 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 00:02:26.420899163 +0000 2024-04-25T00:09:47.195 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 00:02:26.420899163 +0000 2024-04-25T00:09:47.195 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-25 00:02:26.420899163 +0000 2024-04-25T00:09:47.195 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:09:47.263 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T00:09:47.263 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T00:09:47.263 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000212526 s, 2.4 MB/s 2024-04-25T00:09:47.264 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:09:47.321 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2024-04-25T00:09:47.377 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:09:47.377 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:47.377 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-25T00:09:47.377 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:47.377 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:47.377 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2024-04-25 00:09:32.500977072 +0000 2024-04-25T00:09:47.377 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2024-04-25 00:02:26.757890854 +0000 2024-04-25T00:09:47.377 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2024-04-25 00:02:26.757890854 +0000 2024-04-25T00:09:47.377 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2024-04-25 00:02:26.757890854 +0000 2024-04-25T00:09:47.378 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:09:47.439 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2024-04-25T00:09:47.439 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2024-04-25T00:09:47.439 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000199377 s, 2.6 MB/s 2024-04-25T00:09:47.440 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:09:47.497 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-25T00:09:47.497 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T00:09:47.513 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T00:09:47.513 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-04-25T00:09:47.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:47 smithi136 ceph-mon[32923]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:47.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:47 smithi136 ceph-mon[32923]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:09:47.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:47 smithi136 ceph-mon[32923]: Reconfiguring daemon mon.c on smithi136 2024-04-25T00:09:47.514 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:47 smithi106 ceph-mon[32510]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:47.514 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:47 smithi106 ceph-mon[32510]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:09:47.514 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:47 smithi106 ceph-mon[32510]: Reconfiguring daemon mon.c on smithi136 2024-04-25T00:09:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T00:09:47.567 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T00:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-25 00:09:21.832908151 +0000 2024-04-25T00:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-25 00:02:45.661911618 +0000 2024-04-25T00:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-25 00:02:45.661911618 +0000 2024-04-25T00:09:47.568 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-25 00:02:45.661911618 +0000 2024-04-25T00:09:47.568 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T00:09:47.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:47 smithi042 ceph-mon[28130]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:47.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:47 smithi042 ceph-mon[28130]: Reconfiguring mon.c (monmap changed)... 2024-04-25T00:09:47.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:47 smithi042 ceph-mon[28130]: Reconfiguring daemon mon.c on smithi136 2024-04-25T00:09:47.630 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-25T00:09:47.630 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-25T00:09:47.631 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.00022666 s, 2.3 MB/s 2024-04-25T00:09:47.632 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T00:09:47.688 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-04-25T00:09:47.743 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T00:09:47.743 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:47.743 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T00:09:47.743 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:47.744 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:47.744 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-25 00:09:21.832908151 +0000 2024-04-25T00:09:47.744 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-25 00:02:45.985904073 +0000 2024-04-25T00:09:47.744 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-25 00:02:45.985904073 +0000 2024-04-25T00:09:47.744 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-25 00:02:45.985904073 +0000 2024-04-25T00:09:47.744 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T00:09:47.806 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-25T00:09:47.806 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-25T00:09:47.806 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000296305 s, 1.7 MB/s 2024-04-25T00:09:47.807 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T00:09:47.865 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-04-25T00:09:47.928 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T00:09:47.928 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T00:09:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:47.928 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-25 00:09:21.833908128 +0000 2024-04-25T00:09:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-25 00:02:46.466892871 +0000 2024-04-25T00:09:47.929 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-25 00:02:46.466892871 +0000 2024-04-25T00:09:47.929 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-25 00:02:46.466892871 +0000 2024-04-25T00:09:47.929 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T00:09:47.992 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-25T00:09:47.992 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-25T00:09:47.992 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000238338 s, 2.1 MB/s 2024-04-25T00:09:47.993 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T00:09:48.053 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-04-25T00:09:48.109 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T00:09:48.110 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T00:09:48.110 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-25T00:09:48.110 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:09:48.110 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T00:09:48.110 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-25 00:09:21.833908128 +0000 2024-04-25T00:09:48.110 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-25 00:02:46.803885024 +0000 2024-04-25T00:09:48.111 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-25 00:02:46.803885024 +0000 2024-04-25T00:09:48.111 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-04-25 00:02:46.803885024 +0000 2024-04-25T00:09:48.111 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T00:09:48.173 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-25T00:09:48.173 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-25T00:09:48.174 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000200447 s, 2.6 MB/s 2024-04-25T00:09:48.174 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T00:09:48.230 INFO:tasks.cephadm:Deploying osd.0 on smithi042 with /dev/vg_nvme/lv_4... 2024-04-25T00:09:48.231 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:09:48.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:48.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:48.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:48.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:48.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:48.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:48.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:09:48.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:09:48.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:09:48.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:48.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:48 smithi106 ceph-mon[32510]: Deploying daemon mgr.z on smithi136 2024-04-25T00:09:48.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:48.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:48.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:48.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:48.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:48.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:48.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:09:48.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:09:48.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:09:48.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:48.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:48 smithi042 ceph-mon[28130]: Deploying daemon mgr.z on smithi136 2024-04-25T00:09:48.693 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:09:48.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:48.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:48.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:48.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:48.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:48.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:48.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:09:48.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:09:48.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:09:48.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:48.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:48 smithi136 ceph-mon[32923]: Deploying daemon mgr.z on smithi136 2024-04-25T00:09:49.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:49 smithi136 ceph-mon[32923]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:49.513 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:49 smithi136 systemd[1]: Starting Ceph mgr.z for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:09:49.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:49 smithi042 ceph-mon[28130]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:49.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:49 smithi106 ceph-mon[32510]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:49.931 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:49 smithi136 podman[34765]: 2024-04-25 00:09:49.628912448 +0000 UTC m=+0.086702472 container create d8b0fe708cd71aa4937b2273b7d05ba6b7785e64ea5c458349a4ebe956e390d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T00:09:49.932 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:49 smithi136 podman[34765]: 2024-04-25 00:09:49.560241687 +0000 UTC m=+0.018031728 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:09:49.932 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:49 smithi136 podman[34765]: 2024-04-25 00:09:49.848254686 +0000 UTC m=+0.306044710 container init d8b0fe708cd71aa4937b2273b7d05ba6b7785e64ea5c458349a4ebe956e390d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T00:09:49.932 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:49 smithi136 podman[34765]: 2024-04-25 00:09:49.850936649 +0000 UTC m=+0.308726676 container start d8b0fe708cd71aa4937b2273b7d05ba6b7785e64ea5c458349a4ebe956e390d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:09:49.932 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:49 smithi136 ceph-mgr[34784]: -- 172.21.15.136:0/222454658 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56538b2525a0 con 0x56538b227000 2024-04-25T00:09:49.932 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:49 smithi136 bash[34765]: d8b0fe708cd71aa4937b2273b7d05ba6b7785e64ea5c458349a4ebe956e390d9 2024-04-25T00:09:50.190 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:49 smithi136 systemd[1]: Started Ceph mgr.z for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:09:50.190 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:50.005+0000 7fccb35a9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:09:50.190 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:50.190+0000 7fccb35a9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:09:50.513 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:50.291+0000 7fccb35a9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:09:51.006 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:50.597+0000 7fccb35a9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:09:51.006 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:50.681+0000 7fccb35a9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:09:51.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:51 smithi042 ceph-mon[28130]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:51.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:51.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:51.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:51.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:09:51.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:09:51.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:09:51.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:51.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:51 smithi042 ceph-mon[28130]: Deploying daemon mgr.y on smithi106 2024-04-25T00:09:51.228 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:51 smithi106 ceph-mon[32510]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:51.228 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:51.228 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:51.229 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:51.229 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:09:51.229 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:09:51.229 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:09:51.229 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:51.229 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:51 smithi106 ceph-mon[32510]: Deploying daemon mgr.y on smithi106 2024-04-25T00:09:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:51 smithi136 ceph-mon[32923]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:09:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T00:09:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:09:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:51 smithi136 ceph-mon[32923]: Deploying daemon mgr.y on smithi106 2024-04-25T00:09:51.794 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:09:51.794 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:09:51.794 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-04-25T00:09:51.794 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-04-25T00:09:51.794 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.03802 s, 276 MB/s 2024-04-25T00:09:51.794 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-04-25T00:09:52.088 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:51 smithi106 systemd[1]: Starting Ceph mgr.y for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:09:52.263 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:51 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:51.923+0000 7fccb35a9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:09:52.263 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:52.097+0000 7fccb35a9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:09:52.442 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:52 smithi106 podman[33886]: 2024-04-25 00:09:52.063118353 +0000 UTC m=+0.017862723 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:09:52.442 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:52 smithi106 podman[33886]: 2024-04-25 00:09:52.182281925 +0000 UTC m=+0.137026295 container create 5077203a74ec9fa8b616b5221bd42d31147727051055918ebd78cdf0c959dbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:09:52.443 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:52 smithi106 podman[33886]: 2024-04-25 00:09:52.376118409 +0000 UTC m=+0.330862779 container init 5077203a74ec9fa8b616b5221bd42d31147727051055918ebd78cdf0c959dbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T00:09:52.443 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:52 smithi106 podman[33886]: 2024-04-25 00:09:52.378672853 +0000 UTC m=+0.333417223 container start 5077203a74ec9fa8b616b5221bd42d31147727051055918ebd78cdf0c959dbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T00:09:52.443 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:52 smithi106 ceph-mgr[33905]: -- 172.21.15.106:0/3253371734 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647283365a0 con 0x56472830b000 2024-04-25T00:09:52.443 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:52 smithi106 bash[33886]: 5077203a74ec9fa8b616b5221bd42d31147727051055918ebd78cdf0c959dbc0 2024-04-25T00:09:52.556 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:52.273+0000 7fccb35a9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:09:52.556 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:52.556+0000 7fccb35a9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:09:52.659 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_4 2024-04-25T00:09:52.712 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:52 smithi106 systemd[1]: Started Ceph mgr.y for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:09:52.712 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:52.530+0000 7fa93949e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T00:09:52.712 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:52.712+0000 7fa93949e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:09:52.855 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:52.649+0000 7fccb35a9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:09:53.079 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:09:53.088 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:52.816+0000 7fa93949e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T00:09:53.264 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:52.855+0000 7fccb35a9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:09:53.264 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:53.056+0000 7fccb35a9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:09:53.518 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:53 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:53.138+0000 7fa93949e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T00:09:53.518 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:53 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:53.225+0000 7fa93949e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T00:09:53.835 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:53.689+0000 7fccb35a9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:09:53.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:53 smithi136 ceph-mon[32923]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:53.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.836 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:53 smithi106 ceph-mon[32510]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:53.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:53 smithi042 ceph-mon[28130]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:53.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:53.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:54.431 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:54.134+0000 7fccb35a9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:09:54.431 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:54.233+0000 7fccb35a9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:09:54.431 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:54.431+0000 7fccb35a9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:09:54.818 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:54 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:54.483+0000 7fa93949e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T00:09:54.819 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:54 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:54.650+0000 7fa93949e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T00:09:54.819 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:54 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:54.818+0000 7fa93949e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T00:09:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.125+0000 7fa671200700 1 -- 172.21.15.42:0/2135601297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c076260 msgr2=0x7fa66c0746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.126+0000 7fa671200700 1 --2- 172.21.15.42:0/2135601297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c076260 0x7fa66c0746c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009990 tx=0x7fa65c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:09:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.126+0000 7fa671200700 1 -- 172.21.15.42:0/2135601297 shutdown_connections 2024-04-25T00:09:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.126+0000 7fa671200700 1 --2- 172.21.15.42:0/2135601297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c10a170 0x7fa66c10c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:55.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.126+0000 7fa671200700 1 --2- 172.21.15.42:0/2135601297 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c076260 0x7fa66c0746c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:55.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.126+0000 7fa671200700 1 --2- 172.21.15.42:0/2135601297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c074c90 0x7fa66c075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:55.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.126+0000 7fa671200700 1 -- 172.21.15.42:0/2135601297 >> 172.21.15.42:0/2135601297 conn(0x7fa66c0fd800 msgr2=0x7fa66c0ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:09:55.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.126+0000 7fa671200700 1 -- 172.21.15.42:0/2135601297 shutdown_connections 2024-04-25T00:09:55.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.126+0000 7fa671200700 1 -- 172.21.15.42:0/2135601297 wait complete. 2024-04-25T00:09:55.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.127+0000 7fa671200700 1 Processor -- start 2024-04-25T00:09:55.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.127+0000 7fa671200700 1 -- start start 2024-04-25T00:09:55.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.127+0000 7fa671200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c074c90 0x7fa66c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:55.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.127+0000 7fa671200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c076260 0x7fa66c19ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:55.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.127+0000 7fa671200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c10a170 0x7fa66c1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:55.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.127+0000 7fa671200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1100f0 con 0x7fa66c10a170 2024-04-25T00:09:55.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.127+0000 7fa671200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c10ff70 con 0x7fa66c074c90 2024-04-25T00:09:55.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.127+0000 7fa671200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c110270 con 0x7fa66c076260 2024-04-25T00:09:55.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa669e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c074c90 0x7fa66c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:55.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa669e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c074c90 0x7fa66c19a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43314/0 (socket says 172.21.15.42:43314) 2024-04-25T00:09:55.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa669400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c076260 0x7fa66c19ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:55.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa669e00700 1 -- 172.21.15.42:0/3363354350 learned_addr learned my addr 172.21.15.42:0/3363354350 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:09:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa66a800700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c10a170 0x7fa66c1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa669e00700 1 -- 172.21.15.42:0/3363354350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c076260 msgr2=0x7fa66c19ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa669e00700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c076260 0x7fa66c19ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:55.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa669e00700 1 -- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c10a170 msgr2=0x7fa66c1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:09:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa669e00700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c10a170 0x7fa66c1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:09:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa669e00700 1 -- 172.21.15.42:0/3363354350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c1a2390 con 0x7fa66c074c90 2024-04-25T00:09:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.128+0000 7fa66a800700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c10a170 0x7fa66c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:09:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.129+0000 7fa669e00700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c074c90 0x7fa66c19a730 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b8e0 tx=0x7fa65c00bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.129+0000 7fa662a00700 1 -- 172.21.15.42:0/3363354350 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c026070 con 0x7fa66c074c90 2024-04-25T00:09:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.129+0000 7fa662a00700 1 -- 172.21.15.42:0/3363354350 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65c00f460 con 0x7fa66c074c90 2024-04-25T00:09:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.129+0000 7fa662a00700 1 -- 172.21.15.42:0/3363354350 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c021670 con 0x7fa66c074c90 2024-04-25T00:09:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.129+0000 7fa671200700 1 -- 172.21.15.42:0/3363354350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c1a2620 con 0x7fa66c074c90 2024-04-25T00:09:55.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.129+0000 7fa671200700 1 -- 172.21.15.42:0/3363354350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c1a2ad0 con 0x7fa66c074c90 2024-04-25T00:09:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.130+0000 7fa662a00700 1 -- 172.21.15.42:0/3363354350 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa65c029070 con 0x7fa66c074c90 2024-04-25T00:09:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.130+0000 7fa671200700 1 -- 172.21.15.42:0/3363354350 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa66c068f20 con 0x7fa66c074c90 2024-04-25T00:09:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.131+0000 7fa662a00700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa650039330 0x7fa65003b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:09:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.131+0000 7fa662a00700 1 -- 172.21.15.42:0/3363354350 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa65c04d1e0 con 0x7fa66c074c90 2024-04-25T00:09:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.131+0000 7fa669400700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa650039330 0x7fa65003b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:09:55.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.131+0000 7fa669400700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa650039330 0x7fa65003b7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa658009680 tx=0x7fa658009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:09:55.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.136+0000 7fa662a00700 1 -- 172.21.15.42:0/3363354350 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa65c013070 con 0x7fa66c074c90 2024-04-25T00:09:55.263 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:54.958+0000 7fccb35a9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:09:55.263 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:55.049+0000 7fccb35a9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:09:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:55.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:55.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:09:55.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:09:55.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:09:55.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:55.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-mon[32923]: Reconfiguring daemon mgr.x on smithi042 2024-04-25T00:09:55.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:55.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:55.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:55.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:09:55.345 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:09:55.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:09:55.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:55.346 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:55 smithi042 ceph-mon[28130]: Reconfiguring daemon mgr.x on smithi042 2024-04-25T00:09:55.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:55.386+0000 7fa671200700 1 -- 172.21.15.42:0/3363354350 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa66c062670 con 0x7fa650039330 2024-04-25T00:09:55.393 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:55.394 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-mon[32510]: Reconfiguring daemon mgr.x on smithi042 2024-04-25T00:09:55.395 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:55.102+0000 7fa93949e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T00:09:55.395 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:55.194+0000 7fa93949e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T00:09:55.395 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:55.393+0000 7fa93949e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T00:09:55.627 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:55.288+0000 7fccb35a9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:09:55.627 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:55.419+0000 7fccb35a9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:09:55.627 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:55.513+0000 7fccb35a9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:09:55.627 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:55.627+0000 7fccb35a9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:09:55.838 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:55 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:55.584+0000 7fa93949e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T00:09:56.013 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:55.718+0000 7fccb35a9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:09:56.475 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:56 smithi042 ceph-mon[28130]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:56.475 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:56.475 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:56.475 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:56.476 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:56.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:56 smithi136 ceph-mon[32923]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:56.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:56.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:56.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:56.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:56.584 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:56 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:56.173+0000 7fa93949e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T00:09:56.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:56 smithi106 ceph-mon[32510]: from='client.24113 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:09:56.585 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:09:56.585 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:09:56.585 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:56.585 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:56.838 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:56 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:56.584+0000 7fa93949e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T00:09:56.838 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:56 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:56.678+0000 7fa93949e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T00:09:57.013 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:56.554+0000 7fccb35a9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:09:57.013 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:09:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z[34780]: 2024-04-25T00:09:56.769+0000 7fccb35a9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:09:57.173 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:56 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:56.883+0000 7fa93949e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T00:09:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:57 smithi042 ceph-mon[28130]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:57.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:57 smithi042 ceph-mon[28130]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:09:57.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:57 smithi042 ceph-mon[28130]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:09:57.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:57 smithi042 ceph-mon[28130]: Standby manager daemon z started 2024-04-25T00:09:57.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:57 smithi042 ceph-mon[28130]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:09:57.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:57 smithi042 ceph-mon[28130]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:09:57.508 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:57.417+0000 7fa93949e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T00:09:57.508 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:57.507+0000 7fa93949e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T00:09:57.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-mon[32510]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:57.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:57.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:57.508 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:57.509 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-mon[32510]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:09:57.509 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-mon[32510]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:09:57.509 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-mon[32510]: Standby manager daemon z started 2024-04-25T00:09:57.509 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-mon[32510]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:09:57.509 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-mon[32510]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:09:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:57 smithi136 ceph-mon[32923]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:09:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:57 smithi136 ceph-mon[32923]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T00:09:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:57 smithi136 ceph-mon[32923]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:09:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:57 smithi136 ceph-mon[32923]: Standby manager daemon z started 2024-04-25T00:09:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:57 smithi136 ceph-mon[32923]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T00:09:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:57 smithi136 ceph-mon[32923]: from='mgr.? 172.21.15.136:0/1160260946' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:09:57.655 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:09:57.654+0000 7fa662a00700 1 -- 172.21.15.42:0/3363354350 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fa65c009d10 con 0x7fa66c074c90 2024-04-25T00:09:57.838 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:57.746+0000 7fa93949e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T00:09:58.175 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:57.876+0000 7fa93949e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T00:09:58.175 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:57.986+0000 7fa93949e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T00:09:58.176 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:58 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:58.098+0000 7fa93949e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T00:09:58.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:58.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:58.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:58.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:58 smithi042 ceph-mon[28130]: mgrmap e14: x(active, since 91s), standbys: z 2024-04-25T00:09:58.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:09:58.493 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:58 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:58.188+0000 7fa93949e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T00:09:58.493 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:58.493 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:58.493 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:58.493 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:58 smithi106 ceph-mon[32510]: mgrmap e14: x(active, since 91s), standbys: z 2024-04-25T00:09:58.493 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:09:58.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:09:58.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:09:58.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:09:58.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:58 smithi136 ceph-mon[32923]: mgrmap e14: x(active, since 91s), standbys: z 2024-04-25T00:09:58.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T00:09:59.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:09:59 smithi106 ceph-mon[32510]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:59.338 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:58 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:58.990+0000 7fa93949e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T00:09:59.338 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:09:59 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y[33901]: 2024-04-25T00:09:59.202+0000 7fa93949e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T00:09:59.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:09:59 smithi136 ceph-mon[32923]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:09:59.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:09:59 smithi042 ceph-mon[28130]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:00.205 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:00.202+0000 7fa662a00700 1 -- 172.21.15.42:0/3363354350 <== mon.2 v2:172.21.15.106:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa65c022cd0 con 0x7fa66c074c90 2024-04-25T00:10:00.479 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:00 smithi042 ceph-mon[28130]: Standby manager daemon y started 2024-04-25T00:10:00.479 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:00 smithi042 ceph-mon[28130]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:10:00.479 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:00 smithi042 ceph-mon[28130]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:10:00.479 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:00 smithi042 ceph-mon[28130]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:10:00.479 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:00 smithi042 ceph-mon[28130]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:10:00.480 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:00 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3496039752' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e126226-45cb-411a-ac44-64c6f80f8eaa"}]: dispatch 2024-04-25T00:10:00.480 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:00 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3496039752' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e126226-45cb-411a-ac44-64c6f80f8eaa"}]': finished 2024-04-25T00:10:00.480 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:00 smithi042 ceph-mon[28130]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:10:00.480 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:00.480 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:00 smithi042 ceph-mon[28130]: overall HEALTH_OK 2024-04-25T00:10:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:00 smithi136 ceph-mon[32923]: Standby manager daemon y started 2024-04-25T00:10:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:00 smithi136 ceph-mon[32923]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:10:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:00 smithi136 ceph-mon[32923]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:10:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:00 smithi136 ceph-mon[32923]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:10:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:00 smithi136 ceph-mon[32923]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:10:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:00 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3496039752' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e126226-45cb-411a-ac44-64c6f80f8eaa"}]: dispatch 2024-04-25T00:10:00.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:00 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3496039752' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e126226-45cb-411a-ac44-64c6f80f8eaa"}]': finished 2024-04-25T00:10:00.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:00 smithi136 ceph-mon[32923]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:10:00.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:00.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:00 smithi136 ceph-mon[32923]: overall HEALTH_OK 2024-04-25T00:10:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:00 smithi106 ceph-mon[32510]: Standby manager daemon y started 2024-04-25T00:10:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:00 smithi106 ceph-mon[32510]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T00:10:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:00 smithi106 ceph-mon[32510]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T00:10:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:00 smithi106 ceph-mon[32510]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T00:10:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:00 smithi106 ceph-mon[32510]: from='mgr.? 172.21.15.106:0/476784476' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T00:10:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:00 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3496039752' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e126226-45cb-411a-ac44-64c6f80f8eaa"}]: dispatch 2024-04-25T00:10:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:00 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3496039752' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e126226-45cb-411a-ac44-64c6f80f8eaa"}]': finished 2024-04-25T00:10:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:00 smithi106 ceph-mon[32510]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T00:10:00.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:00.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:00 smithi106 ceph-mon[32510]: overall HEALTH_OK 2024-04-25T00:10:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:01 smithi136 ceph-mon[32923]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:01 smithi136 ceph-mon[32923]: mgrmap e15: x(active, since 94s), standbys: y, z 2024-04-25T00:10:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:10:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:01 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1782343555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:01.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:01 smithi106 ceph-mon[32510]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:01.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:01 smithi106 ceph-mon[32510]: mgrmap e15: x(active, since 94s), standbys: y, z 2024-04-25T00:10:01.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:10:01.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:01 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1782343555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:01.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:01 smithi042 ceph-mon[28130]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:01 smithi042 ceph-mon[28130]: mgrmap e15: x(active, since 94s), standbys: y, z 2024-04-25T00:10:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T00:10:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:01 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1782343555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:03.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:03 smithi136 ceph-mon[32923]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:03 smithi106 ceph-mon[32510]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:03.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:03 smithi042 ceph-mon[28130]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:05 smithi136 ceph-mon[32923]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:05 smithi106 ceph-mon[32510]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:05.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:05 smithi042 ceph-mon[28130]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:07.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:07 smithi042 ceph-mon[28130]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:07.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:07 smithi136 ceph-mon[32923]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:07 smithi106 ceph-mon[32510]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:09.260 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:09 smithi042 ceph-mon[28130]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:09 smithi136 ceph-mon[32923]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:09 smithi106 ceph-mon[32510]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:10.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:10 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:10:10.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:10 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:10.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:10 smithi136 ceph-mon[32923]: Deploying daemon osd.0 on smithi042 2024-04-25T00:10:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:10 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:10:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:10 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:10.566 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:10 smithi042 ceph-mon[28130]: Deploying daemon osd.0 on smithi042 2024-04-25T00:10:10.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:10 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T00:10:10.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:10 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:10.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:10 smithi106 ceph-mon[32510]: Deploying daemon osd.0 on smithi042 2024-04-25T00:10:11.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:11 smithi042 ceph-mon[28130]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:11.500 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:11 smithi136 ceph-mon[32923]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:11 smithi106 ceph-mon[32510]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:13 smithi136 ceph-mon[32923]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:13 smithi106 ceph-mon[32510]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:13.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:13 smithi042 ceph-mon[28130]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:15.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:15 smithi042 ceph-mon[28130]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:15 smithi136 ceph-mon[32923]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:15 smithi106 ceph-mon[32510]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:17.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:17 smithi136 ceph-mon[32923]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:17 smithi106 ceph-mon[32510]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:17.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:17 smithi042 ceph-mon[28130]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:18.747 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:18 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:18.747 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:18 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:18.747 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:18 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:18.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:18 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:18.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:18 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:18.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:18 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:18.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:18 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:18.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:18 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:18.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:18 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:19 smithi136 ceph-mon[32923]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:19 smithi106 ceph-mon[32510]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:19.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:19 smithi042 ceph-mon[28130]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:21 smithi136 ceph-mon[32923]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:21.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:21 smithi136 ceph-mon[32923]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:10:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:21 smithi106 ceph-mon[32510]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:21 smithi106 ceph-mon[32510]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:10:21.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:21 smithi042 ceph-mon[28130]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:21.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:21.865 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:21.866 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:21.866 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:21.866 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:21.866 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:21 smithi042 ceph-mon[28130]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T00:10:22.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.347+0000 7fa662a00700 1 -- 172.21.15.42:0/3363354350 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa66c062670 con 0x7fa650039330 2024-04-25T00:10:22.350 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 0 on host 'smithi042' 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.351+0000 7fa671200700 1 -- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa650039330 msgr2=0x7fa65003b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.351+0000 7fa671200700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa650039330 0x7fa65003b7f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa658009680 tx=0x7fa658009260 comp rx=0 tx=0).stop 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.351+0000 7fa671200700 1 -- 172.21.15.42:0/3363354350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c074c90 msgr2=0x7fa66c19a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.351+0000 7fa671200700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c074c90 0x7fa66c19a730 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa65c00b8e0 tx=0x7fa65c00bb90 comp rx=0 tx=0).stop 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.351+0000 7fa671200700 1 -- 172.21.15.42:0/3363354350 shutdown_connections 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.351+0000 7fa671200700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa650039330 0x7fa65003b7f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.351+0000 7fa671200700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c10a170 0x7fa66c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.351+0000 7fa671200700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c074c90 0x7fa66c19a730 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.351+0000 7fa671200700 1 --2- 172.21.15.42:0/3363354350 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c076260 0x7fa66c19ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.351+0000 7fa671200700 1 -- 172.21.15.42:0/3363354350 >> 172.21.15.42:0/3363354350 conn(0x7fa66c0fd800 msgr2=0x7fa66c106070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:22.353 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.352+0000 7fa671200700 1 -- 172.21.15.42:0/3363354350 shutdown_connections 2024-04-25T00:10:22.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:10:22.352+0000 7fa671200700 1 -- 172.21.15.42:0/3363354350 wait complete. 2024-04-25T00:10:23.071 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:22 smithi042 ceph-mon[28130]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:23.071 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:22 smithi042 ceph-mon[28130]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:10:23.071 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:22 smithi042 ceph-mon[28130]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:10:23.071 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:22 smithi042 ceph-mon[28130]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:10:23.071 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:22 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:23.071 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:22 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:23.072 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:22 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:23.072 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:22 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:23.119 DEBUG:teuthology.orchestra.run.smithi042:osd.0> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T00:10:23.123 INFO:tasks.cephadm:Deploying osd.1 on smithi106 with /dev/vg_nvme/lv_4... 2024-04-25T00:10:23.123 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:10:23.150 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:22 smithi106 ceph-mon[32510]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:23.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:22 smithi106 ceph-mon[32510]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:10:23.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:22 smithi106 ceph-mon[32510]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:10:23.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:22 smithi106 ceph-mon[32510]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:10:23.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:22 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:23.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:22 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:23.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:22 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:23.151 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:22 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:22 smithi136 ceph-mon[32923]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:22 smithi136 ceph-mon[32923]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T00:10:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:22 smithi136 ceph-mon[32923]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T00:10:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:22 smithi136 ceph-mon[32923]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:10:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:22 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:22 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:22 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:23.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:22 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:23.367 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:10:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:23 smithi106 ceph-mon[32510]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:10:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:23 smithi106 ceph-mon[32510]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:10:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:23 smithi042 ceph-mon[28130]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:10:24.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:23 smithi042 ceph-mon[28130]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:10:24.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:24.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:23 smithi136 ceph-mon[32923]: from='osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:10:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:23 smithi136 ceph-mon[32923]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T00:10:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:24.267 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:25.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:24 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:10:25.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:24 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:10:25.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:24 smithi106 ceph-mon[32510]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:25.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:24 smithi106 ceph-mon[32510]: osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] boot 2024-04-25T00:10:25.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:24 smithi106 ceph-mon[32510]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:10:25.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:24 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:25.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:24 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:10:25.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:24 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:10:25.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:24 smithi042 ceph-mon[28130]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:25.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:24 smithi042 ceph-mon[28130]: osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] boot 2024-04-25T00:10:25.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:24 smithi042 ceph-mon[28130]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:10:25.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:24 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:25.210 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:10:25.210 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:10:25.210 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-25T00:10:25.210 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-25T00:10:25.210 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115086 s, 911 MB/s 2024-04-25T00:10:25.210 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-25T00:10:25.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:24 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:10:25.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:24 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:10:25.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:24 smithi136 ceph-mon[32923]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T00:10:25.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:24 smithi136 ceph-mon[32923]: osd.0 [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] boot 2024-04-25T00:10:25.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:24 smithi136 ceph-mon[32923]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T00:10:25.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:24 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T00:10:25.810 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_4 2024-04-25T00:10:26.085 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:10:26.087 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:25 smithi106 ceph-mon[32510]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:10:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:25 smithi042 ceph-mon[28130]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:10:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:25 smithi136 ceph-mon[32923]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T00:10:27.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:26 smithi106 ceph-mon[32510]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:27.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:26 smithi136 ceph-mon[32923]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:27.368 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:26 smithi042 ceph-mon[28130]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:27.402 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.402+0000 7f5b03d3b700 1 -- 172.21.15.106:0/894994715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5afc100540 msgr2=0x7f5afc100960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:27.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.402+0000 7f5b03d3b700 1 --2- 172.21.15.106:0/894994715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5afc100540 0x7f5afc100960 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00b0a0 tx=0x7f5aec00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:10:27.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.402+0000 7f5b03d3b700 1 -- 172.21.15.106:0/894994715 shutdown_connections 2024-04-25T00:10:27.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.402+0000 7f5b03d3b700 1 --2- 172.21.15.106:0/894994715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5afc103ec0 0x7f5afc10a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:27.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.402+0000 7f5b03d3b700 1 --2- 172.21.15.106:0/894994715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5afc100540 0x7f5afc100960 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:27.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.402+0000 7f5b03d3b700 1 --2- 172.21.15.106:0/894994715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5afc103520 0x7f5afc103980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:27.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.402+0000 7f5b03d3b700 1 -- 172.21.15.106:0/894994715 >> 172.21.15.106:0/894994715 conn(0x7f5afc0fbec0 msgr2=0x7f5afc0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:27.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.402+0000 7f5b03d3b700 1 -- 172.21.15.106:0/894994715 shutdown_connections 2024-04-25T00:10:27.403 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.402+0000 7f5b03d3b700 1 -- 172.21.15.106:0/894994715 wait complete. 2024-04-25T00:10:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b03d3b700 1 Processor -- start 2024-04-25T00:10:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b03d3b700 1 -- start start 2024-04-25T00:10:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b03d3b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5afc100540 0x7f5afc198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:27.405 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b03d3b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5afc103520 0x7f5afc198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b03d3b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5afc103ec0 0x7f5afc19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b03d3b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc10df30 con 0x7f5afc103520 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b03d3b700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc10ddb0 con 0x7f5afc103ec0 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b03d3b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc10e0b0 con 0x7f5afc100540 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b022d8700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5afc103ec0 0x7f5afc19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b022d8700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5afc103ec0 0x7f5afc19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:52354/0 (socket says 172.21.15.106:52354) 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b022d8700 1 -- 172.21.15.106:0/2625392743 learned_addr learned my addr 172.21.15.106:0/2625392743 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.403+0000 7f5b01ad7700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5afc100540 0x7f5afc198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5b022d8700 1 -- 172.21.15.106:0/2625392743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5afc100540 msgr2=0x7f5afc198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5b022d8700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5afc100540 0x7f5afc198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5b012d6700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5afc103520 0x7f5afc198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:27.406 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5b022d8700 1 -- 172.21.15.106:0/2625392743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5afc103520 msgr2=0x7f5afc198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:27.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5b022d8700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5afc103520 0x7f5afc198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:27.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5b022d8700 1 -- 172.21.15.106:0/2625392743 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc1a01d0 con 0x7f5afc103ec0 2024-04-25T00:10:27.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5b01ad7700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5afc100540 0x7f5afc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:10:27.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5b022d8700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5afc103ec0 0x7f5afc19fb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5af400b550 tx=0x7f5af400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:27.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5afaffd700 1 -- 172.21.15.106:0/2625392743 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af4015020 con 0x7f5afc103ec0 2024-04-25T00:10:27.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5afaffd700 1 -- 172.21.15.106:0/2625392743 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5af4011d10 con 0x7f5afc103ec0 2024-04-25T00:10:27.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5b03d3b700 1 -- 172.21.15.106:0/2625392743 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5afc1a04c0 con 0x7f5afc103ec0 2024-04-25T00:10:27.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5afaffd700 1 -- 172.21.15.106:0/2625392743 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af400f2e0 con 0x7f5afc103ec0 2024-04-25T00:10:27.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.404+0000 7f5b03d3b700 1 -- 172.21.15.106:0/2625392743 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5afc1a0a80 con 0x7f5afc103ec0 2024-04-25T00:10:27.407 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.405+0000 7f5b03d3b700 1 -- 172.21.15.106:0/2625392743 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5afc068f20 con 0x7f5afc103ec0 2024-04-25T00:10:27.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.406+0000 7f5afaffd700 1 -- 172.21.15.106:0/2625392743 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f5af4018020 con 0x7f5afc103ec0 2024-04-25T00:10:27.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.407+0000 7f5afaffd700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ae40a2c10 0x7f5ae40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:27.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.407+0000 7f5afaffd700 1 -- 172.21.15.106:0/2625392743 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f5af40cccf0 con 0x7f5afc103ec0 2024-04-25T00:10:27.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.407+0000 7f5b01ad7700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ae40a2c10 0x7f5ae40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:27.408 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.408+0000 7f5b01ad7700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ae40a2c10 0x7f5ae40a50d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009fd0 tx=0x7f5aec009da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:27.411 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.411+0000 7f5afaffd700 1 -- 172.21.15.106:0/2625392743 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5af401f020 con 0x7f5afc103ec0 2024-04-25T00:10:27.615 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:27.615+0000 7f5b03d3b700 1 -- 172.21.15.106:0/2625392743 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5afc062670 con 0x7f5ae40a2c10 2024-04-25T00:10:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.251 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:28.252 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:28.252 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:28.252 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.252 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.252 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:28.252 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.252 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:28.252 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:28.252 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:29.156 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:28 smithi106 ceph-mon[32510]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:29.156 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:28 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi042 2024-04-25T00:10:29.156 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:28 smithi106 ceph-mon[32510]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:29.156 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:28 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi042 to 17094M 2024-04-25T00:10:29.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:28 smithi136 ceph-mon[32923]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:29.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:28 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi042 2024-04-25T00:10:29.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:28 smithi136 ceph-mon[32923]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:29.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:28 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi042 to 17094M 2024-04-25T00:10:29.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:28 smithi042 ceph-mon[28130]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:29.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:28 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi042 2024-04-25T00:10:29.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:28 smithi042 ceph-mon[28130]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:29.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:28 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi042 to 17094M 2024-04-25T00:10:31.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:30 smithi042 ceph-mon[28130]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:31.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:30 smithi042 ceph-mon[28130]: from='client.? 172.21.15.106:0/904462006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df2777b5-5078-4e63-b3f9-6f20ea649a3d"}]: dispatch 2024-04-25T00:10:31.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:30 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df2777b5-5078-4e63-b3f9-6f20ea649a3d"}]: dispatch 2024-04-25T00:10:31.204 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:30 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df2777b5-5078-4e63-b3f9-6f20ea649a3d"}]': finished 2024-04-25T00:10:31.205 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:30 smithi042 ceph-mon[28130]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:10:31.205 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:30 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:31.205 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:30 smithi042 ceph-mon[28130]: from='client.? 172.21.15.106:0/485193547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:31.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:30 smithi136 ceph-mon[32923]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:31.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:30 smithi136 ceph-mon[32923]: from='client.? 172.21.15.106:0/904462006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df2777b5-5078-4e63-b3f9-6f20ea649a3d"}]: dispatch 2024-04-25T00:10:31.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:30 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df2777b5-5078-4e63-b3f9-6f20ea649a3d"}]: dispatch 2024-04-25T00:10:31.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:30 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df2777b5-5078-4e63-b3f9-6f20ea649a3d"}]': finished 2024-04-25T00:10:31.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:30 smithi136 ceph-mon[32923]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:10:31.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:30 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:31.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:30 smithi136 ceph-mon[32923]: from='client.? 172.21.15.106:0/485193547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:30 smithi106 ceph-mon[32510]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:30 smithi106 ceph-mon[32510]: from='client.? 172.21.15.106:0/904462006' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df2777b5-5078-4e63-b3f9-6f20ea649a3d"}]: dispatch 2024-04-25T00:10:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:30 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df2777b5-5078-4e63-b3f9-6f20ea649a3d"}]: dispatch 2024-04-25T00:10:31.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:30 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df2777b5-5078-4e63-b3f9-6f20ea649a3d"}]': finished 2024-04-25T00:10:31.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:30 smithi106 ceph-mon[32510]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T00:10:31.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:30 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:31.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:30 smithi106 ceph-mon[32510]: from='client.? 172.21.15.106:0/485193547' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:33.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:32 smithi136 ceph-mon[32923]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:33.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:32 smithi106 ceph-mon[32510]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:33.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:32 smithi042 ceph-mon[28130]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:35.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:34 smithi136 ceph-mon[32923]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:35.301 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:34 smithi106 ceph-mon[32510]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:35.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:34 smithi042 ceph-mon[28130]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:37.260 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:36 smithi106 ceph-mon[32510]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:37.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:36 smithi136 ceph-mon[32923]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:37.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:36 smithi042 ceph-mon[28130]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:38.202 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:10:38.202 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:38.203 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:37 smithi106 ceph-mon[32510]: Deploying daemon osd.1 on smithi106 2024-04-25T00:10:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:10:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:37 smithi136 ceph-mon[32923]: Deploying daemon osd.1 on smithi106 2024-04-25T00:10:38.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T00:10:38.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:38.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:37 smithi042 ceph-mon[28130]: Deploying daemon osd.1 on smithi106 2024-04-25T00:10:39.205 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:38 smithi106 ceph-mon[32510]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:39.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:38 smithi136 ceph-mon[32923]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:39.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:38 smithi042 ceph-mon[28130]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:41.201 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:40 smithi106 ceph-mon[32510]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:41.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:40 smithi136 ceph-mon[32923]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:41.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:40 smithi042 ceph-mon[28130]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:43.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:42 smithi136 ceph-mon[32923]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:43.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:42 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:43.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:42 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:43.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:42 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:43.333 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:42 smithi106 ceph-mon[32510]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:43.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:42 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:43.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:42 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:43.334 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:42 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:43.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:42 smithi042 ceph-mon[28130]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:43.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:42 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:43.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:42 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:43.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:42 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:45.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:44 smithi106 ceph-mon[32510]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:45.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:44 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:45.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:44 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:45.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:44 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:45.160 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:44 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:45.161 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:44 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:45.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:44 smithi136 ceph-mon[32923]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:45.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:44 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:45.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:44 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:45.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:44 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:45.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:44 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:45.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:44 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:45.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:44 smithi042 ceph-mon[28130]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:45.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:44 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:45.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:44 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:45.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:44 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:45.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:44 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:45.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:44 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:46.161 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.161+0000 7f5afaffd700 1 -- 172.21.15.106:0/2625392743 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5afc062670 con 0x7f5ae40a2c10 2024-04-25T00:10:46.162 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 1 on host 'smithi106' 2024-04-25T00:10:46.163 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 -- 172.21.15.106:0/2625392743 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ae40a2c10 msgr2=0x7f5ae40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ae40a2c10 0x7f5ae40a50d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5aec009fd0 tx=0x7f5aec009da0 comp rx=0 tx=0).stop 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 -- 172.21.15.106:0/2625392743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5afc103ec0 msgr2=0x7f5afc19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5afc103ec0 0x7f5afc19fb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5af400b550 tx=0x7f5af400b910 comp rx=0 tx=0).stop 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 -- 172.21.15.106:0/2625392743 shutdown_connections 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ae40a2c10 0x7f5ae40a50d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5afc103ec0 0x7f5afc19fb30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5afc100540 0x7f5afc198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 --2- 172.21.15.106:0/2625392743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5afc103520 0x7f5afc198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 -- 172.21.15.106:0/2625392743 >> 172.21.15.106:0/2625392743 conn(0x7f5afc0fbec0 msgr2=0x7f5afc1068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 -- 172.21.15.106:0/2625392743 shutdown_connections 2024-04-25T00:10:46.164 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:10:46.163+0000 7f5b03d3b700 1 -- 172.21.15.106:0/2625392743 wait complete. 2024-04-25T00:10:46.743 DEBUG:teuthology.orchestra.run.smithi106:osd.1> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T00:10:46.745 INFO:tasks.cephadm:Deploying osd.2 on smithi136 with /dev/vg_nvme/lv_4... 2024-04-25T00:10:46.745 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T00:10:46.988 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:10:47.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:47 smithi106 ceph-mon[32510]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:47.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:47.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:47.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:47.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:47 smithi106 ceph-mon[32510]: from='osd.1 [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:10:47.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:47 smithi106 ceph-mon[32510]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:10:47.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:47 smithi042 ceph-mon[28130]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:47.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:47.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:47.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:47.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:47 smithi042 ceph-mon[28130]: from='osd.1 [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:10:47.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:47 smithi042 ceph-mon[28130]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:10:47.421 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:47 smithi136 ceph-mon[32923]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:47.421 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:47.421 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:47.421 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:10:47.421 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:47 smithi136 ceph-mon[32923]: from='osd.1 [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:10:47.421 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:47 smithi136 ceph-mon[32923]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T00:10:48.335 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:48 smithi136 ceph-mon[32923]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:10:48.335 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:48 smithi136 ceph-mon[32923]: from='osd.1 [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:10:48.335 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:48 smithi136 ceph-mon[32923]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:10:48.335 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:48.335 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:48 smithi136 ceph-mon[32923]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:10:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:48 smithi106 ceph-mon[32510]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:10:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:48 smithi106 ceph-mon[32510]: from='osd.1 [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:10:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:48 smithi106 ceph-mon[32510]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:10:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:48 smithi106 ceph-mon[32510]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:10:48.339 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:10:48 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1[37115]: 2024-04-25T00:10:48.121+0000 7ffa90962700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T00:10:48.339 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:10:48 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1[37115]: 2024-04-25T00:10:48.125+0000 7ffa8cf56700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:10:48.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:48 smithi042 ceph-mon[28130]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T00:10:48.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:48 smithi042 ceph-mon[28130]: from='osd.1 [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:10:48.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:48 smithi042 ceph-mon[28130]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T00:10:48.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:48.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:48 smithi042 ceph-mon[28130]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:10:48.877 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T00:10:48.878 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T00:10:48.878 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-25T00:10:48.878 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-25T00:10:48.878 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00938661 s, 1.1 GB/s 2024-04-25T00:10:48.878 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-25T00:10:49.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:49 smithi106 ceph-mon[32510]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:49.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:49 smithi106 ceph-mon[32510]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:10:49.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:49 smithi106 ceph-mon[32510]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:10:49.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:49 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:49.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:49 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:49.447 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:49 smithi136 ceph-mon[32923]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:49.447 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:49 smithi136 ceph-mon[32923]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:10:49.447 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:49 smithi136 ceph-mon[32923]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:10:49.447 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:49 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:49.447 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:49 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:49.473 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_4 2024-04-25T00:10:49.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:49 smithi042 ceph-mon[28130]: pgmap v72: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:49.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:49 smithi042 ceph-mon[28130]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:10:49.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:49 smithi042 ceph-mon[28130]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T00:10:49.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:49 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:49.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:49 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:49.728 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:10:50.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:10:50.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:10:50.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: osd.1 [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] boot 2024-04-25T00:10:50.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:10:50.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:50.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi106 2024-04-25T00:10:50.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:50.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi106 to 17088M 2024-04-25T00:10:50.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:50.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:50.340 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:10:50.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:10:50.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: osd.1 [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] boot 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi106 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi106 to 17088M 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:50.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:10:50.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:10:50.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: osd.1 [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] boot 2024-04-25T00:10:50.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T00:10:50.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T00:10:50.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi106 2024-04-25T00:10:50.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:10:50.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi106 to 17088M 2024-04-25T00:10:50.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:50.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:50.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:10:50.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:10:51.293 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.292+0000 7fc188a40700 1 -- 172.21.15.136:0/1229215089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184103e30 msgr2=0x7fc18410a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:51.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.292+0000 7fc188a40700 1 --2- 172.21.15.136:0/1229215089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184103e30 0x7fc18410a6e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00b0a0 tx=0x7fc17c00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:10:51.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.292+0000 7fc188a40700 1 -- 172.21.15.136:0/1229215089 shutdown_connections 2024-04-25T00:10:51.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.292+0000 7fc188a40700 1 --2- 172.21.15.136:0/1229215089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184103e30 0x7fc18410a6e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:51.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.292+0000 7fc188a40700 1 --2- 172.21.15.136:0/1229215089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc184102290 0x7fc1841026b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:51.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.292+0000 7fc188a40700 1 --2- 172.21.15.136:0/1229215089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc184103490 0x7fc1841038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:51.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.292+0000 7fc188a40700 1 -- 172.21.15.136:0/1229215089 >> 172.21.15.136:0/1229215089 conn(0x7fc1840fd830 msgr2=0x7fc1840ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:10:51.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.292+0000 7fc188a40700 1 -- 172.21.15.136:0/1229215089 shutdown_connections 2024-04-25T00:10:51.294 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.293+0000 7fc188a40700 1 -- 172.21.15.136:0/1229215089 wait complete. 2024-04-25T00:10:51.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.293+0000 7fc188a40700 1 Processor -- start 2024-04-25T00:10:51.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.293+0000 7fc188a40700 1 -- start start 2024-04-25T00:10:51.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc188a40700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc184102290 0x7fc184198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:51.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc188a40700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc184103490 0x7fc1841989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:51.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc188a40700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184103e30 0x7fc18419fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:51.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc188a40700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc18410df40 con 0x7fc184103490 2024-04-25T00:10:51.296 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc188a40700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc18410ddc0 con 0x7fc184102290 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc188a40700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc18410e0c0 con 0x7fc184103e30 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc182d9d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184103e30 0x7fc18419fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc182d9d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184103e30 0x7fc18419fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:48602/0 (socket says 172.21.15.136:48602) 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc182d9d700 1 -- 172.21.15.136:0/2322727321 learned_addr learned my addr 172.21.15.136:0/2322727321 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc18259c700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc184102290 0x7fc184198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc182d9d700 1 -- 172.21.15.136:0/2322727321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc184102290 msgr2=0x7fc184198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc182d9d700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc184102290 0x7fc184198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc182d9d700 1 -- 172.21.15.136:0/2322727321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc184103490 msgr2=0x7fc1841989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc182d9d700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc184103490 0x7fc1841989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc182d9d700 1 -- 172.21.15.136:0/2322727321 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1841a00f0 con 0x7fc184103e30 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc182d9d700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184103e30 0x7fc18419fa50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0095e0 tx=0x7fc17c007770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc17b7fe700 1 -- 172.21.15.136:0/2322727321 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c021070 con 0x7fc184103e30 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc17b7fe700 1 -- 172.21.15.136:0/2322727321 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc17c004020 con 0x7fc184103e30 2024-04-25T00:10:51.297 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc188a40700 1 -- 172.21.15.136:0/2322727321 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1841a0380 con 0x7fc184103e30 2024-04-25T00:10:51.298 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc17b7fe700 1 -- 172.21.15.136:0/2322727321 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c005280 con 0x7fc184103e30 2024-04-25T00:10:51.298 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.294+0000 7fc188a40700 1 -- 172.21.15.136:0/2322727321 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1841a08b0 con 0x7fc184103e30 2024-04-25T00:10:51.298 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.295+0000 7fc188a40700 1 -- 172.21.15.136:0/2322727321 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc184068f20 con 0x7fc184103e30 2024-04-25T00:10:51.298 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.297+0000 7fc17b7fe700 1 -- 172.21.15.136:0/2322727321 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fc17c0041d0 con 0x7fc184103e30 2024-04-25T00:10:51.298 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.298+0000 7fc17b7fe700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc1700a2b90 0x7fc1700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:10:51.299 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.298+0000 7fc17b7fe700 1 -- 172.21.15.136:0/2322727321 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fc17c00f020 con 0x7fc184103e30 2024-04-25T00:10:51.299 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.298+0000 7fc18259c700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc1700a2b90 0x7fc1700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:10:51.300 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.299+0000 7fc18259c700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc1700a2b90 0x7fc1700a5050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc174006fd0 tx=0x7fc174008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:10:51.302 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.301+0000 7fc17b7fe700 1 -- 172.21.15.136:0/2322727321 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc17c09cdb0 con 0x7fc184103e30 2024-04-25T00:10:51.508 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:10:51.507+0000 7fc188a40700 1 -- 172.21.15.136:0/2322727321 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fc184062670 con 0x7fc1700a2b90 2024-04-25T00:10:51.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:51 smithi136 ceph-mon[32923]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:51.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:51 smithi136 ceph-mon[32923]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:10:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:51 smithi106 ceph-mon[32510]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:51 smithi106 ceph-mon[32510]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:10:51.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:51 smithi042 ceph-mon[28130]: pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-25T00:10:51.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:51 smithi042 ceph-mon[28130]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T00:10:52.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:52 smithi042 ceph-mon[28130]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:52.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:52.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:52.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:52.745 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:52 smithi136 ceph-mon[32923]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:52.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:52.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:52.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:52 smithi106 ceph-mon[32510]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:10:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:10:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:10:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:10:53.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:53 smithi136 ceph-mon[32923]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:53 smithi106 ceph-mon[32510]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:53.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:53 smithi042 ceph-mon[28130]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:54 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/747571204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f53ac73-5268-4b0f-9256-bf92ba350ce5"}]: dispatch 2024-04-25T00:10:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:54 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f53ac73-5268-4b0f-9256-bf92ba350ce5"}]: dispatch 2024-04-25T00:10:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:54 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f53ac73-5268-4b0f-9256-bf92ba350ce5"}]': finished 2024-04-25T00:10:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:54 smithi136 ceph-mon[32923]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:10:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:10:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:54 smithi106 ceph-mon[32510]: from='client.? 172.21.15.136:0/747571204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f53ac73-5268-4b0f-9256-bf92ba350ce5"}]: dispatch 2024-04-25T00:10:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:54 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f53ac73-5268-4b0f-9256-bf92ba350ce5"}]: dispatch 2024-04-25T00:10:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:54 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f53ac73-5268-4b0f-9256-bf92ba350ce5"}]': finished 2024-04-25T00:10:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:54 smithi106 ceph-mon[32510]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:10:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:10:54.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:54 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/747571204' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f53ac73-5268-4b0f-9256-bf92ba350ce5"}]: dispatch 2024-04-25T00:10:54.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:54 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f53ac73-5268-4b0f-9256-bf92ba350ce5"}]: dispatch 2024-04-25T00:10:54.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:54 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f53ac73-5268-4b0f-9256-bf92ba350ce5"}]': finished 2024-04-25T00:10:54.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:54 smithi042 ceph-mon[28130]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T00:10:54.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:10:55.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:55 smithi136 ceph-mon[32923]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:55.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:55 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/2974677639' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:55 smithi106 ceph-mon[32510]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:55 smithi106 ceph-mon[32510]: from='client.? 172.21.15.136:0/2974677639' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:55 smithi042 ceph-mon[28130]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:55 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/2974677639' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:10:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:57 smithi136 ceph-mon[32923]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:57 smithi106 ceph-mon[32510]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:57 smithi042 ceph-mon[28130]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:59.616 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:10:59 smithi136 ceph-mon[32923]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:59.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:10:59 smithi106 ceph-mon[32510]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:10:59.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:10:59 smithi042 ceph-mon[28130]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:01.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:01 smithi136 ceph-mon[32923]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:01 smithi106 ceph-mon[32510]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:01 smithi042 ceph-mon[28130]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:02.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:02 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:11:02.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:02 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:02.723 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:02 smithi136 ceph-mon[32923]: Deploying daemon osd.2 on smithi136 2024-04-25T00:11:02.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:02 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:11:02.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:02 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:02.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:02 smithi106 ceph-mon[32510]: Deploying daemon osd.2 on smithi136 2024-04-25T00:11:02.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:02 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T00:11:02.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:02 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:02.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:02 smithi042 ceph-mon[28130]: Deploying daemon osd.2 on smithi136 2024-04-25T00:11:03.620 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:03 smithi136 ceph-mon[32923]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:03.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:03 smithi042 ceph-mon[28130]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:03.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:03 smithi106 ceph-mon[32510]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:05.804 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:05 smithi136 ceph-mon[32923]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:05 smithi106 ceph-mon[32510]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:05.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:05 smithi042 ceph-mon[28130]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:07.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:07 smithi042 ceph-mon[28130]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:07.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:07.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:07.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:07.956 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:07 smithi136 ceph-mon[32923]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:07.956 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:07.956 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:07.956 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:07 smithi106 ceph-mon[32510]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:08.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:08.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:08.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:09.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:09 smithi042 ceph-mon[28130]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:09.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:09.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:09.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:09.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:09.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:09.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:09 smithi136 ceph-mon[32923]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:09.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:09.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:09.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:09.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:09.927 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:09 smithi106 ceph-mon[32510]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:10.251 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 2 on host 'smithi136' 2024-04-25T00:11:10.252 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.250+0000 7fc17b7fe700 1 -- 172.21.15.136:0/2322727321 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc184062670 con 0x7fc1700a2b90 2024-04-25T00:11:10.253 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 -- 172.21.15.136:0/2322727321 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc1700a2b90 msgr2=0x7fc1700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:10.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc1700a2b90 0x7fc1700a5050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc174006fd0 tx=0x7fc174008040 comp rx=0 tx=0).stop 2024-04-25T00:11:10.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 -- 172.21.15.136:0/2322727321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184103e30 msgr2=0x7fc18419fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:10.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184103e30 0x7fc18419fa50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc17c0095e0 tx=0x7fc17c007770 comp rx=0 tx=0).stop 2024-04-25T00:11:10.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 -- 172.21.15.136:0/2322727321 shutdown_connections 2024-04-25T00:11:10.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc1700a2b90 0x7fc1700a5050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc184103e30 0x7fc18419fa50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.254 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc184102290 0x7fc184198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 --2- 172.21.15.136:0/2322727321 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc184103490 0x7fc1841989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:10.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 -- 172.21.15.136:0/2322727321 >> 172.21.15.136:0/2322727321 conn(0x7fc1840fd830 msgr2=0x7fc184106730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:10.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 -- 172.21.15.136:0/2322727321 shutdown_connections 2024-04-25T00:11:10.255 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:11:10.252+0000 7fc188a40700 1 -- 172.21.15.136:0/2322727321 wait complete. 2024-04-25T00:11:10.848 DEBUG:teuthology.orchestra.run.smithi136:osd.2> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T00:11:10.851 INFO:tasks.cephadm:Deploying osd.3 on smithi042 with /dev/vg_nvme/lv_3... 2024-04-25T00:11:10.852 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:11:11.313 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:11:11.466 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:11 smithi042 ceph-mon[28130]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:11.467 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:11.467 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:11.467 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:11.467 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:11 smithi042 ceph-mon[28130]: from='osd.2 [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:11:11.467 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:11 smithi042 ceph-mon[28130]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:11:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:11 smithi136 ceph-mon[32923]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:11 smithi136 ceph-mon[32923]: from='osd.2 [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:11:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:11 smithi136 ceph-mon[32923]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:11:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:11 smithi106 ceph-mon[32510]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:11 smithi106 ceph-mon[32510]: from='osd.2 [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:11:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:11 smithi106 ceph-mon[32510]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T00:11:12.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:12 smithi042 ceph-mon[28130]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:11:12.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:12 smithi042 ceph-mon[28130]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:11:12.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:12.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:12 smithi042 ceph-mon[28130]: from='osd.2 [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:11:12.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:12 smithi042 ceph-mon[28130]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:11:12.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:12 smithi136 ceph-mon[32923]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:11:12.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:12 smithi136 ceph-mon[32923]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:11:12.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:12 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:12.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:12 smithi136 ceph-mon[32923]: from='osd.2 [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:11:12.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:12 smithi136 ceph-mon[32923]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:11:12.514 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:11:12 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2[38101]: 2024-04-25T00:11:12.235+0000 7f5eebf97700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T00:11:12.514 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:11:12 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2[38101]: 2024-04-25T00:11:12.240+0000 7f5ee858b700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:11:12.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:12 smithi106 ceph-mon[32510]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T00:11:12.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:12 smithi106 ceph-mon[32510]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T00:11:12.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:12 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:12.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:12 smithi106 ceph-mon[32510]: from='osd.2 [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:11:12.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:12 smithi106 ceph-mon[32510]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:11:13.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:13 smithi136 ceph-mon[32923]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:13.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:13 smithi136 ceph-mon[32923]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:11:13.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:13 smithi136 ceph-mon[32923]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:11:13.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:13.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:13.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:13 smithi042 ceph-mon[28130]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:13.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:13 smithi042 ceph-mon[28130]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:11:13.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:13 smithi042 ceph-mon[28130]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:11:13.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:13.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:13 smithi106 ceph-mon[32510]: pgmap v89: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:13 smithi106 ceph-mon[32510]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:11:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:13 smithi106 ceph-mon[32510]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T00:11:13.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:13.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:13.991 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:11:13.991 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:11:13.991 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-04-25T00:11:13.991 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-04-25T00:11:13.991 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0329436 s, 318 MB/s 2024-04-25T00:11:13.991 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-04-25T00:11:14.735 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:11:14.735 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:11:14.736 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: osd.2 [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] boot 2024-04-25T00:11:14.736 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:11:14.736 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:14.736 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.736 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.736 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:14.736 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.736 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:14.736 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:14.736 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:11:14.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:11:14.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: osd.2 [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] boot 2024-04-25T00:11:14.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:11:14.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:14.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:14.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:14.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:14.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_3 2024-04-25T00:11:14.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:11:14.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:11:14.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: osd.2 [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] boot 2024-04-25T00:11:14.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T00:11:14.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T00:11:14.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:14.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:14.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:14.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:14.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:15.238 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:11:15.756 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:15 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi136 2024-04-25T00:11:15.756 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:15 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi136 to 17088M 2024-04-25T00:11:15.756 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:15 smithi042 ceph-mon[28130]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:15.756 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:11:15.756 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:11:15.756 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:15 smithi042 ceph-mon[28130]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:11:15.756 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:11:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:15 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi136 2024-04-25T00:11:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:15 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi136 to 17088M 2024-04-25T00:11:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:15 smithi136 ceph-mon[32923]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:11:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:11:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:15 smithi136 ceph-mon[32923]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:11:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:11:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:15 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi136 2024-04-25T00:11:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:15 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi136 to 17088M 2024-04-25T00:11:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:15 smithi106 ceph-mon[32510]: pgmap v92: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-25T00:11:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:11:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T00:11:15.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:15 smithi106 ceph-mon[32510]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T00:11:15.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:11:16.509 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:11:16 smithi042 sudo[47167]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:11:16.509 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:11:16 smithi042 sudo[47167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:16.509 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:11:16 smithi042 sudo[47167]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:16.509 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:11:16 smithi042 sudo[47167]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:16.510 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:16 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:11:16.510 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:16 smithi042 ceph-mon[28130]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:11:16.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:16 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:11:16.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:16 smithi136 ceph-mon[32923]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:11:16.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:16 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:11:16.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:16 smithi106 ceph-mon[32510]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T00:11:17.123 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:11:16 smithi042 sudo[47190]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:11:17.123 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:11:16 smithi042 sudo[47190]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:17.123 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:11:16 smithi042 sudo[47190]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:17.123 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:11:16 smithi042 sudo[47190]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:17.402 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:11:17 smithi106 sudo[41364]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:11:17.402 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:11:17 smithi106 sudo[41364]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:17.402 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:11:17 smithi106 sudo[41364]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:17.402 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:11:17 smithi106 sudo[41364]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:17.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.409+0000 7fa4be000700 1 -- 172.21.15.42:0/580858762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b8100b70 msgr2=0x7fa4b8100f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:17.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.409+0000 7fa4be000700 1 --2- 172.21.15.42:0/580858762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b8100b70 0x7fa4b8100f90 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009990 tx=0x7fa4a8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:11:17.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.410+0000 7fa4be000700 1 -- 172.21.15.42:0/580858762 shutdown_connections 2024-04-25T00:11:17.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.410+0000 7fa4be000700 1 --2- 172.21.15.42:0/580858762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8101e90 0x7fa4b8102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:17.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.410+0000 7fa4be000700 1 --2- 172.21.15.42:0/580858762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b8100b70 0x7fa4b8100f90 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:17.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.410+0000 7fa4be000700 1 --2- 172.21.15.42:0/580858762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b81014d0 0x7fa4b8101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:17.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.410+0000 7fa4be000700 1 -- 172.21.15.42:0/580858762 >> 172.21.15.42:0/580858762 conn(0x7fa4b80fc730 msgr2=0x7fa4b80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:17.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.410+0000 7fa4be000700 1 -- 172.21.15.42:0/580858762 shutdown_connections 2024-04-25T00:11:17.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.410+0000 7fa4be000700 1 -- 172.21.15.42:0/580858762 wait complete. 2024-04-25T00:11:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.411+0000 7fa4be000700 1 Processor -- start 2024-04-25T00:11:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.411+0000 7fa4be000700 1 -- start start 2024-04-25T00:11:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.411+0000 7fa4be000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b8100b70 0x7fa4b819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:17.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.412+0000 7fa4be000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b81014d0 0x7fa4b819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.412+0000 7fa4be000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8101e90 0x7fa4b81a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.412+0000 7fa4be000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8110090 con 0x7fa4b81014d0 2024-04-25T00:11:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.412+0000 7fa4be000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b810ff10 con 0x7fa4b8100b70 2024-04-25T00:11:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.412+0000 7fa4be000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b8110210 con 0x7fa4b8101e90 2024-04-25T00:11:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.412+0000 7fa4b7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8101e90 0x7fa4b81a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.412+0000 7fa4b7e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8101e90 0x7fa4b81a1cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:42678/0 (socket says 172.21.15.42:42678) 2024-04-25T00:11:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.412+0000 7fa4b7e00700 1 -- 172.21.15.42:0/3220382548 learned_addr learned my addr 172.21.15.42:0/3220382548 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:11:17.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.412+0000 7fa4b6a00700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b81014d0 0x7fa4b819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.413+0000 7fa4b7400700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b8100b70 0x7fa4b819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.413+0000 7fa4b7e00700 1 -- 172.21.15.42:0/3220382548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b8100b70 msgr2=0x7fa4b819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.413+0000 7fa4b7e00700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b8100b70 0x7fa4b819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.413+0000 7fa4b7e00700 1 -- 172.21.15.42:0/3220382548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b81014d0 msgr2=0x7fa4b819ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:17.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.413+0000 7fa4b7e00700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b81014d0 0x7fa4b819ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.413+0000 7fa4b7e00700 1 -- 172.21.15.42:0/3220382548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b81a2360 con 0x7fa4b8101e90 2024-04-25T00:11:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.413+0000 7fa4b7e00700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8101e90 0x7fa4b81a1cc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000ef60 tx=0x7fa4b000ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.413+0000 7fa4afe00700 1 -- 172.21.15.42:0/3220382548 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b00092e0 con 0x7fa4b8101e90 2024-04-25T00:11:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.413+0000 7fa4be000700 1 -- 172.21.15.42:0/3220382548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b81a25f0 con 0x7fa4b8101e90 2024-04-25T00:11:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.413+0000 7fa4afe00700 1 -- 172.21.15.42:0/3220382548 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa4b0004bb0 con 0x7fa4b8101e90 2024-04-25T00:11:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.414+0000 7fa4afe00700 1 -- 172.21.15.42:0/3220382548 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0004280 con 0x7fa4b8101e90 2024-04-25T00:11:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.414+0000 7fa4be000700 1 -- 172.21.15.42:0/3220382548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b81a2b30 con 0x7fa4b8101e90 2024-04-25T00:11:17.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.415+0000 7fa4be000700 1 -- 172.21.15.42:0/3220382548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b8068f20 con 0x7fa4b8101e90 2024-04-25T00:11:17.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.417+0000 7fa4afe00700 1 -- 172.21.15.42:0/3220382548 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa4b001b030 con 0x7fa4b8101e90 2024-04-25T00:11:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.419+0000 7fa4afe00700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa49c0a2c20 0x7fa49c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.419+0000 7fa4afe00700 1 -- 172.21.15.42:0/3220382548 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 3237+0+0 (secure 0 0 0) 0x7fa4b00cd7a0 con 0x7fa4b8101e90 2024-04-25T00:11:17.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.419+0000 7fa4b7400700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa49c0a2c20 0x7fa49c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:17.420 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.420+0000 7fa4b7400700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa49c0a2c20 0x7fa49c0a50e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800b7b0 tx=0x7fa4a801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:17.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.425+0000 7fa4afe00700 1 -- 172.21.15.42:0/3220382548 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa4b0022020 con 0x7fa4b8101e90 2024-04-25T00:11:17.660 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:11:17 smithi106 sudo[41367]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:11:17.661 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:11:17 smithi106 sudo[41367]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:17.661 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:11:17 smithi106 sudo[41367]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:17.661 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:11:17 smithi106 sudo[41367]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:17.661 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:17 smithi106 ceph-mon[32510]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:17.661 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:17 smithi106 ceph-mon[32510]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T00:11:17.661 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:17 smithi106 ceph-mon[32510]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:11:17.661 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:17.661 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:17.665 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:17.664+0000 7fa4be000700 1 -- 172.21.15.42:0/3220382548 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa4b8062670 con 0x7fa49c0a2c20 2024-04-25T00:11:17.665 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:17 smithi042 ceph-mon[28130]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:17.665 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:17 smithi042 ceph-mon[28130]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T00:11:17.665 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:17 smithi042 ceph-mon[28130]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:11:17.665 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:17.665 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:17.763 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:11:17 smithi136 sudo[42166]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:11:17.763 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:11:17 smithi136 sudo[42166]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:17.763 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:11:17 smithi136 sudo[42166]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:17.763 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:11:17 smithi136 sudo[42166]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:17 smithi136 ceph-mon[32923]: pgmap v95: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:17 smithi136 ceph-mon[32923]: mgrmap e16: x(active, since 2m), standbys: y, z 2024-04-25T00:11:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:17 smithi136 ceph-mon[32923]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T00:11:17.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:17.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:18.263 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:11:17 smithi136 sudo[42169]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:11:18.263 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:11:17 smithi136 sudo[42169]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:18.263 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:11:17 smithi136 sudo[42169]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:18.263 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:11:17 smithi136 sudo[42169]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:18.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:18 smithi042 sudo[47333]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:11:18.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:18 smithi042 sudo[47333]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:18.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:18 smithi042 sudo[47333]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:18.810 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:18 smithi042 sudo[47333]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:19.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:19 smithi042 sudo[47361]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:11:19.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:19 smithi042 sudo[47361]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:19.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:19 smithi042 sudo[47361]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:19.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:19 smithi042 sudo[47361]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:19.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:19.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:19.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:19.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:19.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:19 smithi042 ceph-mon[28130]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:11:19.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:19 smithi042 ceph-mon[28130]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:11:19.400 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:19.400 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:19.401 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:19.401 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:19.401 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 ceph-mon[32923]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:11:19.401 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 ceph-mon[32923]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:11:19.401 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 sudo[42172]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:11:19.401 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 sudo[42172]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:19.401 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 sudo[42172]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:19.401 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 sudo[42172]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:19 smithi106 ceph-mon[32510]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:11:19.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:19 smithi106 ceph-mon[32510]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:11:19.746 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 sudo[42175]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:11:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 sudo[42175]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 sudo[42175]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:19.747 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:19 smithi136 sudo[42175]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:20.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:19 smithi106 sudo[41382]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T00:11:20.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:19 smithi106 sudo[41382]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:20.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:19 smithi106 sudo[41382]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:20.137 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:19 smithi106 sudo[41382]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:20.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:20.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:20.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:20.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:11:20.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:11:20.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:11:20.302 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:20.303 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:11:20.303 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:11:20.303 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:11:20.303 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:11:20.303 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:20.303 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:11:20.303 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:11:20.304 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:20 smithi042 ceph-mon[28130]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:11:20.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:20.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:20.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:20.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:11:20.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:20 smithi136 ceph-mon[32923]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:11:20.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 sudo[41385]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T00:11:20.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 sudo[41385]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T00:11:20.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 sudo[41385]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T00:11:20.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 sudo[41385]: pam_unix(sudo:session): session closed for user root 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: pgmap v97: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T00:11:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:20 smithi106 ceph-mon[32510]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T00:11:21.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:21 smithi136 ceph-mon[32923]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:21 smithi106 ceph-mon[32510]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:21 smithi042 ceph-mon[28130]: pgmap v98: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:22.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:22 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3060698315' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8044c10-17a2-4a89-926c-2fe500f36943"}]: dispatch 2024-04-25T00:11:22.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:22 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3060698315' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8044c10-17a2-4a89-926c-2fe500f36943"}]': finished 2024-04-25T00:11:22.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:22 smithi042 ceph-mon[28130]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T00:11:22.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:22 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:22.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:22 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3060698315' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8044c10-17a2-4a89-926c-2fe500f36943"}]: dispatch 2024-04-25T00:11:22.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:22 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3060698315' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8044c10-17a2-4a89-926c-2fe500f36943"}]': finished 2024-04-25T00:11:22.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:22 smithi136 ceph-mon[32923]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T00:11:22.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:22 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:22.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:22 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3060698315' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a8044c10-17a2-4a89-926c-2fe500f36943"}]: dispatch 2024-04-25T00:11:22.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:22 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3060698315' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a8044c10-17a2-4a89-926c-2fe500f36943"}]': finished 2024-04-25T00:11:22.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:22 smithi106 ceph-mon[32510]: osdmap e22: 4 total, 3 up, 4 in 2024-04-25T00:11:22.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:22 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:23.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:23 smithi136 ceph-mon[32923]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:23.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:23 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1445619544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:23.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:23 smithi106 ceph-mon[32510]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:23.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:23 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1445619544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:23.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:23 smithi042 ceph-mon[28130]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:23.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:23 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1445619544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:25.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:25 smithi136 ceph-mon[32923]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:25.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:25 smithi106 ceph-mon[32510]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:25.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:25 smithi042 ceph-mon[28130]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 83 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:27.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:27 smithi136 ceph-mon[32923]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 62 KiB/s wr, 5 op/s 2024-04-25T00:11:27.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:27 smithi106 ceph-mon[32510]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 62 KiB/s wr, 5 op/s 2024-04-25T00:11:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:27 smithi042 ceph-mon[28130]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 2.1 KiB/s rd, 62 KiB/s wr, 5 op/s 2024-04-25T00:11:29.357 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:29 smithi042 ceph-mon[28130]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 42 KiB/s wr, 3 op/s 2024-04-25T00:11:29.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:29 smithi136 ceph-mon[32923]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 42 KiB/s wr, 3 op/s 2024-04-25T00:11:29.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:29 smithi106 ceph-mon[32510]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.4 KiB/s rd, 42 KiB/s wr, 3 op/s 2024-04-25T00:11:31.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:31 smithi136 ceph-mon[32923]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2024-04-25T00:11:31.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:31 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:11:31.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:31 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:31.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:31 smithi106 ceph-mon[32510]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2024-04-25T00:11:31.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:31 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:11:31.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:31 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:31.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:31 smithi042 ceph-mon[28130]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 32 KiB/s wr, 2 op/s 2024-04-25T00:11:31.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:31 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T00:11:31.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:31 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:32.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:32 smithi042 ceph-mon[28130]: Deploying daemon osd.3 on smithi042 2024-04-25T00:11:32.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:32 smithi136 ceph-mon[32923]: Deploying daemon osd.3 on smithi042 2024-04-25T00:11:32.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:32 smithi106 ceph-mon[32510]: Deploying daemon osd.3 on smithi042 2024-04-25T00:11:33.450 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:33 smithi042 ceph-mon[28130]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 899 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T00:11:33.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:33 smithi136 ceph-mon[32923]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 899 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T00:11:33.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:33 smithi106 ceph-mon[32510]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 899 B/s rd, 26 KiB/s wr, 2 op/s 2024-04-25T00:11:35.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:35 smithi042 ceph-mon[28130]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:11:35.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:35 smithi136 ceph-mon[32923]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:11:35.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:35 smithi106 ceph-mon[32510]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:11:37.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:37 smithi136 ceph-mon[32923]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:11:37.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:37 smithi106 ceph-mon[32510]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:11:37.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:37 smithi042 ceph-mon[28130]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-04-25T00:11:39.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:39 smithi136 ceph-mon[32923]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:39.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:39 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:39.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:39 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:39.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:39 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:39.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:39 smithi106 ceph-mon[32510]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:39.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:39 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:39.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:39 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:39.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:39 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:39.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:39 smithi042 ceph-mon[28130]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:39.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:39 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:39.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:39 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:39.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:39 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:41.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:41 smithi136 ceph-mon[32923]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:41.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:41 smithi106 ceph-mon[32510]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:41.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:41 smithi042 ceph-mon[28130]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:43.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:43 smithi042 ceph-mon[28130]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:43.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:43.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:43.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:43.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:43.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:43.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:43 smithi042 ceph-mon[28130]: from='osd.3 [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:11:43.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:43 smithi042 ceph-mon[28130]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:11:43.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:43 smithi136 ceph-mon[32923]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:43.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:43.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:43.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:43.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:43.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:43.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:43 smithi136 ceph-mon[32923]: from='osd.3 [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:11:43.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:43 smithi136 ceph-mon[32923]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:11:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:43 smithi106 ceph-mon[32510]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:43.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:43.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:43.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:43 smithi106 ceph-mon[32510]: from='osd.3 [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:11:43.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:43 smithi106 ceph-mon[32510]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T00:11:44.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.261+0000 7fa4afe00700 1 -- 172.21.15.42:0/3220382548 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa4b8062670 con 0x7fa49c0a2c20 2024-04-25T00:11:44.263 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 3 on host 'smithi042' 2024-04-25T00:11:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.267+0000 7fa4be000700 1 -- 172.21.15.42:0/3220382548 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa49c0a2c20 msgr2=0x7fa49c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.267+0000 7fa4be000700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa49c0a2c20 0x7fa49c0a50e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa4a800b7b0 tx=0x7fa4a801a040 comp rx=0 tx=0).stop 2024-04-25T00:11:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.267+0000 7fa4be000700 1 -- 172.21.15.42:0/3220382548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8101e90 msgr2=0x7fa4b81a1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:44.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.267+0000 7fa4be000700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8101e90 0x7fa4b81a1cc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa4b000ef60 tx=0x7fa4b000ef90 comp rx=0 tx=0).stop 2024-04-25T00:11:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.267+0000 7fa4be000700 1 -- 172.21.15.42:0/3220382548 shutdown_connections 2024-04-25T00:11:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.267+0000 7fa4be000700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa49c0a2c20 0x7fa49c0a50e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.267+0000 7fa4be000700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4b8101e90 0x7fa4b81a1cc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.267+0000 7fa4be000700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4b8100b70 0x7fa4b819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.267+0000 7fa4be000700 1 --2- 172.21.15.42:0/3220382548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4b81014d0 0x7fa4b819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:44.269 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.267+0000 7fa4be000700 1 -- 172.21.15.42:0/3220382548 >> 172.21.15.42:0/3220382548 conn(0x7fa4b80fc730 msgr2=0x7fa4b80fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.268+0000 7fa4be000700 1 -- 172.21.15.42:0/3220382548 shutdown_connections 2024-04-25T00:11:44.270 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:11:44.268+0000 7fa4be000700 1 -- 172.21.15.42:0/3220382548 wait complete. 2024-04-25T00:11:44.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:44 smithi136 ceph-mon[32923]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:11:44.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:44 smithi136 ceph-mon[32923]: from='osd.3 [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:11:44.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:44 smithi136 ceph-mon[32923]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:11:44.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:44 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:44.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:44 smithi136 ceph-mon[32923]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:11:44.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:44 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:44.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:44 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:44.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:44 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:44.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:44 smithi106 ceph-mon[32510]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:11:44.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:44 smithi106 ceph-mon[32510]: from='osd.3 [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:11:44.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:44 smithi106 ceph-mon[32510]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:11:44.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:44 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:44.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:44 smithi106 ceph-mon[32510]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:11:44.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:44 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:44.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:44 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:44.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:44 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:44.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:44 smithi042 ceph-mon[28130]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T00:11:44.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:44 smithi042 ceph-mon[28130]: from='osd.3 [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:11:44.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:44 smithi042 ceph-mon[28130]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T00:11:44.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:44 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:44.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:44 smithi042 ceph-mon[28130]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:11:44.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:44 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:44.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:44 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:44.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:44 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:11:45.103 DEBUG:teuthology.orchestra.run.smithi042:osd.3> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T00:11:45.109 INFO:tasks.cephadm:Deploying osd.4 on smithi106 with /dev/vg_nvme/lv_3... 2024-04-25T00:11:45.109 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:11:45.352 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:11:45.755 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:45 smithi106 ceph-mon[32510]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:45.755 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:45 smithi106 ceph-mon[32510]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:11:45.755 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:45 smithi106 ceph-mon[32510]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:11:45.755 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:45 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:45.756 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:45 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:45.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:45 smithi136 ceph-mon[32923]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:45.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:45 smithi136 ceph-mon[32923]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:11:45.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:45 smithi136 ceph-mon[32923]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:11:45.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:45 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:45.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:45 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:45.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:45 smithi042 ceph-mon[28130]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:45.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:45 smithi042 ceph-mon[28130]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:11:45.844 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:45 smithi042 ceph-mon[28130]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T00:11:45.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:45.845 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:46.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:46 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:11:46.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:46 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:11:46.620 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:46 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:46.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:46 smithi106 ceph-mon[32510]: osd.3 [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] boot 2024-04-25T00:11:46.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:46 smithi106 ceph-mon[32510]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T00:11:46.621 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:46 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:46.727 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:46 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:11:46.727 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:46 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:11:46.727 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:46 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:46.727 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:46 smithi042 ceph-mon[28130]: osd.3 [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] boot 2024-04-25T00:11:46.727 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:46 smithi042 ceph-mon[28130]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T00:11:46.727 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:46 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:46.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:46 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:11:46.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:46 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:11:46.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:46 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:46.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:46 smithi136 ceph-mon[32923]: osd.3 [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] boot 2024-04-25T00:11:46.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:46 smithi136 ceph-mon[32923]: osdmap e25: 4 total, 4 up, 4 in 2024-04-25T00:11:46.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:46 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T00:11:47.225 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:11:47.225 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:11:47.225 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-25T00:11:47.225 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-25T00:11:47.225 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.013235 s, 792 MB/s 2024-04-25T00:11:47.225 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-25T00:11:47.731 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:47 smithi106 ceph-mon[32510]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:47.731 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:47 smithi106 ceph-mon[32510]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:11:47.759 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_3 2024-04-25T00:11:47.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:47 smithi042 ceph-mon[28130]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:47.761 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:47 smithi042 ceph-mon[28130]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:11:47.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:47 smithi136 ceph-mon[32923]: pgmap v115: 1 pgs: 1 peering; 577 KiB data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-25T00:11:47.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:47 smithi136 ceph-mon[32923]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T00:11:48.007 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:11:48.762 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:48 smithi106 ceph-mon[32510]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:11:48.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:48 smithi136 ceph-mon[32923]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:11:48.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:48 smithi042 ceph-mon[28130]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T00:11:49.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.475+0000 7fdf37ced700 1 -- 172.21.15.106:0/1844235466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30101180 msgr2=0x7fdf301015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.475+0000 7fdf37ced700 1 --2- 172.21.15.106:0/1844235466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30101180 0x7fdf301015a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0099c0 tx=0x7fdf2c009cd0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.475+0000 7fdf37ced700 1 -- 172.21.15.106:0/1844235466 shutdown_connections 2024-04-25T00:11:49.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.475+0000 7fdf37ced700 1 --2- 172.21.15.106:0/1844235466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30102d20 0x7fdf301095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.475+0000 7fdf37ced700 1 --2- 172.21.15.106:0/1844235466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30101180 0x7fdf301015a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.475+0000 7fdf37ced700 1 --2- 172.21.15.106:0/1844235466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf30102380 0x7fdf301027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.475+0000 7fdf37ced700 1 -- 172.21.15.106:0/1844235466 >> 172.21.15.106:0/1844235466 conn(0x7fdf300fc700 msgr2=0x7fdf300feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:11:49.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.475+0000 7fdf37ced700 1 -- 172.21.15.106:0/1844235466 shutdown_connections 2024-04-25T00:11:49.479 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.475+0000 7fdf37ced700 1 -- 172.21.15.106:0/1844235466 wait complete. 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.475+0000 7fdf37ced700 1 Processor -- start 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf37ced700 1 -- start start 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf37ced700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30101180 0x7fdf3006b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf37ced700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30102380 0x7fdf3019d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf37ced700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf30102d20 0x7fdf3019fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf37ced700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3010ce90 con 0x7fdf30102380 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf37ced700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3010cd10 con 0x7fdf30101180 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf37ced700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdf3010d010 con 0x7fdf30102d20 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf35a89700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30101180 0x7fdf3006b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf35a89700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30101180 0x7fdf3006b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47958/0 (socket says 172.21.15.106:47958) 2024-04-25T00:11:49.480 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf35a89700 1 -- 172.21.15.106:0/383924353 learned_addr learned my addr 172.21.15.106:0/383924353 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf3628a700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf30102d20 0x7fdf3019fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf35a89700 1 -- 172.21.15.106:0/383924353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf30102d20 msgr2=0x7fdf3019fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf35a89700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf30102d20 0x7fdf3019fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf35a89700 1 -- 172.21.15.106:0/383924353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30102380 msgr2=0x7fdf3019d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf35a89700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30102380 0x7fdf3019d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf35a89700 1 -- 172.21.15.106:0/383924353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf301a02d0 con 0x7fdf30101180 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf3628a700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf30102d20 0x7fdf3019fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf35a89700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30101180 0x7fdf3006b560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0098e0 tx=0x7fdf2c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf26ffd700 1 -- 172.21.15.106:0/383924353 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c027070 con 0x7fdf30101180 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf26ffd700 1 -- 172.21.15.106:0/383924353 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf2c022470 con 0x7fdf30101180 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf37ced700 1 -- 172.21.15.106:0/383924353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf301a0560 con 0x7fdf30101180 2024-04-25T00:11:49.481 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf26ffd700 1 -- 172.21.15.106:0/383924353 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c00f6c0 con 0x7fdf30101180 2024-04-25T00:11:49.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.476+0000 7fdf37ced700 1 -- 172.21.15.106:0/383924353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf301a0a30 con 0x7fdf30101180 2024-04-25T00:11:49.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.477+0000 7fdf37ced700 1 -- 172.21.15.106:0/383924353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf30068f20 con 0x7fdf30101180 2024-04-25T00:11:49.482 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.478+0000 7fdf26ffd700 1 -- 172.21.15.106:0/383924353 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdf2c02a070 con 0x7fdf30101180 2024-04-25T00:11:49.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.479+0000 7fdf26ffd700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2c20 0x7fdf1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:11:49.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.479+0000 7fdf26ffd700 1 -- 172.21.15.106:0/383924353 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3669+0+0 (secure 0 0 0) 0x7fdf2c0cf0e0 con 0x7fdf30101180 2024-04-25T00:11:49.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.480+0000 7fdf35288700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2c20 0x7fdf1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:11:49.483 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.480+0000 7fdf35288700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2c20 0x7fdf1c0a50e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf3006bae0 tx=0x7fdf20006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:11:49.484 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.484+0000 7fdf26ffd700 1 -- 172.21.15.106:0/383924353 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdf2c013070 con 0x7fdf30101180 2024-04-25T00:11:49.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:49 smithi106 ceph-mon[32510]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:11:49.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:49 smithi106 ceph-mon[32510]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:11:49.693 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:11:49.692+0000 7fdf37ced700 1 -- 172.21.15.106:0/383924353 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdf30062670 con 0x7fdf1c0a2c20 2024-04-25T00:11:49.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:49 smithi136 ceph-mon[32923]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:11:49.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:49 smithi136 ceph-mon[32923]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:11:49.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:49 smithi042 ceph-mon[28130]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:11:49.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:49 smithi042 ceph-mon[28130]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T00:11:50.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:50 smithi106 ceph-mon[32510]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:11:50.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:50.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:50.584 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:50.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:50 smithi136 ceph-mon[32923]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:11:50.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:50.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:50.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:50.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:50 smithi042 ceph-mon[28130]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T00:11:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:11:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:11:50.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:51.780 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:51 smithi042 ceph-mon[28130]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:51.780 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:51 smithi042 ceph-mon[28130]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:11:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:51.781 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:51.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:51.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:51.782 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:51 smithi106 ceph-mon[32510]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:51 smithi106 ceph-mon[32510]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:51 smithi136 ceph-mon[32923]: from='client.24218 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:11:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:51 smithi136 ceph-mon[32923]: pgmap v121: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:11:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:11:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:11:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:11:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:11:52.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:52 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi042 2024-04-25T00:11:52.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:52 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi042 to 8547M 2024-04-25T00:11:52.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:52 smithi042 ceph-mon[28130]: from='client.? 172.21.15.106:0/3441251455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61b5a912-f971-4678-98fb-5f95190f8c46"}]: dispatch 2024-04-25T00:11:52.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:52 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61b5a912-f971-4678-98fb-5f95190f8c46"}]: dispatch 2024-04-25T00:11:52.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:52 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61b5a912-f971-4678-98fb-5f95190f8c46"}]': finished 2024-04-25T00:11:52.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:52 smithi042 ceph-mon[28130]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T00:11:52.800 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:11:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:52 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi042 2024-04-25T00:11:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:52 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi042 to 8547M 2024-04-25T00:11:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:52 smithi106 ceph-mon[32510]: from='client.? 172.21.15.106:0/3441251455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61b5a912-f971-4678-98fb-5f95190f8c46"}]: dispatch 2024-04-25T00:11:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:52 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61b5a912-f971-4678-98fb-5f95190f8c46"}]: dispatch 2024-04-25T00:11:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:52 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61b5a912-f971-4678-98fb-5f95190f8c46"}]': finished 2024-04-25T00:11:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:52 smithi106 ceph-mon[32510]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T00:11:52.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:11:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:52 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi042 2024-04-25T00:11:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:52 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi042 to 8547M 2024-04-25T00:11:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:52 smithi136 ceph-mon[32923]: from='client.? 172.21.15.106:0/3441251455' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61b5a912-f971-4678-98fb-5f95190f8c46"}]: dispatch 2024-04-25T00:11:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:52 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "61b5a912-f971-4678-98fb-5f95190f8c46"}]: dispatch 2024-04-25T00:11:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:52 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "61b5a912-f971-4678-98fb-5f95190f8c46"}]': finished 2024-04-25T00:11:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:52 smithi136 ceph-mon[32923]: osdmap e30: 5 total, 4 up, 5 in 2024-04-25T00:11:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:11:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:53 smithi106 ceph-mon[32510]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-25T00:11:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:53 smithi106 ceph-mon[32510]: from='client.? 172.21.15.106:0/3347373356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:53.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:53 smithi042 ceph-mon[28130]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-25T00:11:53.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:53 smithi042 ceph-mon[28130]: from='client.? 172.21.15.106:0/3347373356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:53 smithi136 ceph-mon[32923]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 108 KiB/s, 0 objects/s recovering 2024-04-25T00:11:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:53 smithi136 ceph-mon[32923]: from='client.? 172.21.15.106:0/3347373356' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:11:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:55 smithi106 ceph-mon[32510]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:11:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:55 smithi042 ceph-mon[28130]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:11:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:55 smithi136 ceph-mon[32923]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:11:57.706 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:57 smithi106 ceph-mon[32510]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-25T00:11:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:57 smithi042 ceph-mon[28130]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-25T00:11:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:57 smithi136 ceph-mon[32923]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-04-25T00:11:59.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:11:59 smithi106 ceph-mon[32510]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T00:11:59.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:11:59 smithi042 ceph-mon[28130]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T00:12:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:11:59 smithi136 ceph-mon[32923]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-04-25T00:12:00.821 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:12:00.821 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:00.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:12:00.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:01.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T00:12:01.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:01 smithi106 ceph-mon[32510]: Deploying daemon osd.4 on smithi106 2024-04-25T00:12:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:01 smithi106 ceph-mon[32510]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:12:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:01 smithi042 ceph-mon[28130]: Deploying daemon osd.4 on smithi106 2024-04-25T00:12:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:01 smithi042 ceph-mon[28130]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:12:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:01 smithi136 ceph-mon[32923]: Deploying daemon osd.4 on smithi106 2024-04-25T00:12:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:01 smithi136 ceph-mon[32923]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:12:03.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:03 smithi106 ceph-mon[32510]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:03.880 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:03 smithi042 ceph-mon[28130]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:03 smithi136 ceph-mon[32923]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:05 smithi106 ceph-mon[32510]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:05.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:05 smithi042 ceph-mon[28130]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:05.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:05.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:05.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:05 smithi136 ceph-mon[32923]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:07.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:07 smithi106 ceph-mon[32510]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:07.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:07.830 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:07.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:07 smithi042 ceph-mon[28130]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:07.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:07.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:07 smithi136 ceph-mon[32923]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:08.428 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.427+0000 7fdf26ffd700 1 -- 172.21.15.106:0/383924353 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdf30062670 con 0x7fdf1c0a2c20 2024-04-25T00:12:08.428 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 4 on host 'smithi106' 2024-04-25T00:12:08.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 -- 172.21.15.106:0/383924353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2c20 msgr2=0x7fdf1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:08.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2c20 0x7fdf1c0a50e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fdf3006bae0 tx=0x7fdf20006d20 comp rx=0 tx=0).stop 2024-04-25T00:12:08.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 -- 172.21.15.106:0/383924353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30101180 msgr2=0x7fdf3006b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:08.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30101180 0x7fdf3006b560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c0098e0 tx=0x7fdf2c0047b0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.430 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 -- 172.21.15.106:0/383924353 shutdown_connections 2024-04-25T00:12:08.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2c20 0x7fdf1c0a50e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf30102d20 0x7fdf3019fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf30101180 0x7fdf3006b560 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 --2- 172.21.15.106:0/383924353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf30102380 0x7fdf3019d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:08.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 -- 172.21.15.106:0/383924353 >> 172.21.15.106:0/383924353 conn(0x7fdf300fc700 msgr2=0x7fdf301057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:08.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 -- 172.21.15.106:0/383924353 shutdown_connections 2024-04-25T00:12:08.431 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:12:08.430+0000 7fdf37ced700 1 -- 172.21.15.106:0/383924353 wait complete. 2024-04-25T00:12:08.771 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:08 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:08.772 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:08 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:08.772 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:08 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:08.772 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:08 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:08.772 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:08 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:08.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:08 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:08.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:08 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:08.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:08 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:08.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:08 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:08.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:08 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:09.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:08 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:09.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:08 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:09.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:08 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:09.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:08 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:09.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:08 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:09.064 DEBUG:teuthology.orchestra.run.smithi106:osd.4> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T00:12:09.066 INFO:tasks.cephadm:Deploying osd.5 on smithi136 with /dev/vg_nvme/lv_3... 2024-04-25T00:12:09.066 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T00:12:09.312 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:12:09.697 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:09 smithi136 ceph-mon[32923]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:09.697 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:09.698 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:09 smithi136 ceph-mon[32923]: from='osd.4 [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:12:09.698 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:09 smithi136 ceph-mon[32923]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:12:09.800 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:09 smithi106 ceph-mon[32510]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:09.801 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:09.801 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:09 smithi106 ceph-mon[32510]: from='osd.4 [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:12:09.801 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:09 smithi106 ceph-mon[32510]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:12:09.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:09 smithi042 ceph-mon[28130]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:09.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:09.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:09 smithi042 ceph-mon[28130]: from='osd.4 [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:12:09.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:09 smithi042 ceph-mon[28130]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T00:12:10.705 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:10 smithi136 ceph-mon[32923]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:12:10.705 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:10 smithi136 ceph-mon[32923]: from='osd.4 [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:12:10.705 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:10 smithi136 ceph-mon[32923]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:12:10.705 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:10 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:10.705 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:10 smithi136 ceph-mon[32923]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:12:10.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:10 smithi106 ceph-mon[32510]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:12:10.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:10 smithi106 ceph-mon[32510]: from='osd.4 [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:12:10.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:10 smithi106 ceph-mon[32510]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:12:10.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:10 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:10.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:10 smithi106 ceph-mon[32510]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:12:10.839 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:12:10 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-25T00:12:10.597+0000 7f597e4a2700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T00:12:10.839 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:12:10 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-25T00:12:10.603+0000 7f597aa96700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:12:10.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:10 smithi042 ceph-mon[28130]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T00:12:10.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:10 smithi042 ceph-mon[28130]: from='osd.4 [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:12:10.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:10 smithi042 ceph-mon[28130]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T00:12:10.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:10 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:10.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:10 smithi042 ceph-mon[28130]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:12:11.235 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T00:12:11.235 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T00:12:11.235 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-25T00:12:11.235 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-25T00:12:11.235 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00945076 s, 1.1 GB/s 2024-04-25T00:12:11.235 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-25T00:12:11.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:11 smithi136 ceph-mon[32923]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:11.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:11 smithi136 ceph-mon[32923]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:12:11.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:11 smithi136 ceph-mon[32923]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:12:11.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:11.795 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:11.823 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_3 2024-04-25T00:12:11.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:11 smithi042 ceph-mon[28130]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:11.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:11 smithi042 ceph-mon[28130]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:12:11.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:11 smithi042 ceph-mon[28130]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:12:11.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:11.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:12.058 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:12:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:11 smithi106 ceph-mon[32510]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:11 smithi106 ceph-mon[32510]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:12:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:11 smithi106 ceph-mon[32510]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T00:12:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:12.734 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:12:12.734 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:12:12.734 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: osd.4 [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] boot 2024-04-25T00:12:12.734 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T00:12:12.734 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:12.734 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:12.735 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:12.735 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:12.735 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:12.735 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:12.735 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:12.735 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:12.735 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:12 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:12.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:12:12.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:12:12.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: osd.4 [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] boot 2024-04-25T00:12:12.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T00:12:12.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:12.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:12.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:12.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:12.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:12.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:12.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:12.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:12.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:12 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:13.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:12:13.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:12:13.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: osd.4 [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] boot 2024-04-25T00:12:13.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: osdmap e33: 5 total, 5 up, 5 in 2024-04-25T00:12:13.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T00:12:13.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:13.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:13.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:13.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:13.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:13.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:13.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:13.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:12 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:13.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.447+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1925245117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0c4103e20 msgr2=0x7fb0c410a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.447+0000 7fb0cc6e1700 1 --2- 172.21.15.136:0/1925245117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0c4103e20 0x7fb0c410a6d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000b0a0 tx=0x7fb0c000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1925245117 shutdown_connections 2024-04-25T00:12:13.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 --2- 172.21.15.136:0/1925245117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0c4103e20 0x7fb0c410a6d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 --2- 172.21.15.136:0/1925245117 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c4102280 0x7fb0c41026a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 --2- 172.21.15.136:0/1925245117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c4103480 0x7fb0c41038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1925245117 >> 172.21.15.136:0/1925245117 conn(0x7fb0c40fd820 msgr2=0x7fb0c40ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1925245117 shutdown_connections 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1925245117 wait complete. 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 Processor -- start 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 -- start start 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0c4102280 0x7fb0c41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c4103480 0x7fb0c4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c4103e20 0x7fb0c419fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c410dee0 con 0x7fb0c4103480 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c410dd60 con 0x7fb0c4103e20 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0cc6e1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb0c410e060 con 0x7fb0c4102280 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0ca47d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0c4102280 0x7fb0c41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0ca47d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0c4102280 0x7fb0c41984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:36842/0 (socket says 172.21.15.136:36842) 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.448+0000 7fb0ca47d700 1 -- 172.21.15.136:0/1088655220 learned_addr learned my addr 172.21.15.136:0/1088655220 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:12:13.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0cac7e700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c4103e20 0x7fb0c419fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0ca47d700 1 -- 172.21.15.136:0/1088655220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c4103e20 msgr2=0x7fb0c419fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0ca47d700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c4103e20 0x7fb0c419fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0ca47d700 1 -- 172.21.15.136:0/1088655220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c4103480 msgr2=0x7fb0c4198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0ca47d700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c4103480 0x7fb0c4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0ca47d700 1 -- 172.21.15.136:0/1088655220 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0c41a0120 con 0x7fb0c4102280 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0ca47d700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0c4102280 0x7fb0c41984c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00b530 tx=0x7fb0bc00b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0bb7fe700 1 -- 172.21.15.136:0/1088655220 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc015020 con 0x7fb0c4102280 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0bb7fe700 1 -- 172.21.15.136:0/1088655220 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0bc011d10 con 0x7fb0c4102280 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0bb7fe700 1 -- 172.21.15.136:0/1088655220 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc00f2e0 con 0x7fb0c4102280 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1088655220 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0c41a0410 con 0x7fb0c4102280 2024-04-25T00:12:13.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.449+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1088655220 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0c41a0a20 con 0x7fb0c4102280 2024-04-25T00:12:13.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.450+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1088655220 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0c4068f20 con 0x7fb0c4102280 2024-04-25T00:12:13.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.451+0000 7fb0bb7fe700 1 -- 172.21.15.136:0/1088655220 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb0bc018020 con 0x7fb0c4102280 2024-04-25T00:12:13.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.452+0000 7fb0bb7fe700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0b00a2ca0 0x7fb0b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:13.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.452+0000 7fb0bb7fe700 1 -- 172.21.15.136:0/1088655220 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fb0bc0cd670 con 0x7fb0c4102280 2024-04-25T00:12:13.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.452+0000 7fb0c9c7c700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0b00a2ca0 0x7fb0b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:13.454 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.453+0000 7fb0c9c7c700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0b00a2ca0 0x7fb0b00a5160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4005950 tx=0x7fb0b40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:13.457 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.456+0000 7fb0bb7fe700 1 -- 172.21.15.136:0/1088655220 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb0bc01f020 con 0x7fb0c4102280 2024-04-25T00:12:13.667 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:13.666+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1088655220 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb0c4062670 con 0x7fb0b00a2ca0 2024-04-25T00:12:13.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:13 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi106 2024-04-25T00:12:13.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:13 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-25T00:12:13.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:13 smithi136 ceph-mon[32923]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:13.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:13 smithi136 ceph-mon[32923]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:12:13.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:13 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi106 2024-04-25T00:12:13.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:13 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-25T00:12:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:13 smithi042 ceph-mon[28130]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:13 smithi042 ceph-mon[28130]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:12:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:13 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi106 2024-04-25T00:12:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:13 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi106 to 8544M 2024-04-25T00:12:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:13 smithi106 ceph-mon[32510]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-25T00:12:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:13 smithi106 ceph-mon[32510]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T00:12:14.934 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:14.934 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:14.934 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:15.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:15.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:15.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:15.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:15.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:15.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:15.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:15 smithi136 ceph-mon[32923]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:15.767 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:15 smithi136 ceph-mon[32923]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:15 smithi106 ceph-mon[32510]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:15 smithi106 ceph-mon[32510]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:15 smithi042 ceph-mon[28130]: from='client.24238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:15 smithi042 ceph-mon[28130]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:17.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:16 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/3256839337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "978af6c3-4b15-4a19-bc5b-587c384520ef"}]: dispatch 2024-04-25T00:12:17.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:16 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "978af6c3-4b15-4a19-bc5b-587c384520ef"}]: dispatch 2024-04-25T00:12:17.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:16 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "978af6c3-4b15-4a19-bc5b-587c384520ef"}]': finished 2024-04-25T00:12:17.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:16 smithi136 ceph-mon[32923]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T00:12:17.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:16 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:17.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:16 smithi106 ceph-mon[32510]: from='client.? 172.21.15.136:0/3256839337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "978af6c3-4b15-4a19-bc5b-587c384520ef"}]: dispatch 2024-04-25T00:12:17.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:16 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "978af6c3-4b15-4a19-bc5b-587c384520ef"}]: dispatch 2024-04-25T00:12:17.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:16 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "978af6c3-4b15-4a19-bc5b-587c384520ef"}]': finished 2024-04-25T00:12:17.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:16 smithi106 ceph-mon[32510]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T00:12:17.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:16 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:17.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:16 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/3256839337' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "978af6c3-4b15-4a19-bc5b-587c384520ef"}]: dispatch 2024-04-25T00:12:17.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:16 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "978af6c3-4b15-4a19-bc5b-587c384520ef"}]: dispatch 2024-04-25T00:12:17.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:16 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "978af6c3-4b15-4a19-bc5b-587c384520ef"}]': finished 2024-04-25T00:12:17.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:16 smithi042 ceph-mon[28130]: osdmap e35: 6 total, 5 up, 6 in 2024-04-25T00:12:17.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:16 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:17.989 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:17 smithi106 ceph-mon[32510]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:17.989 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:17 smithi106 ceph-mon[32510]: from='client.? 172.21.15.136:0/3158105987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:17 smithi136 ceph-mon[32923]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:17 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/3158105987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:18.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:17 smithi042 ceph-mon[28130]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:18.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:17 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/3158105987' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:19.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:18 smithi136 ceph-mon[32923]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:19.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:18 smithi106 ceph-mon[32510]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:19.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:18 smithi042 ceph-mon[28130]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:21.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:20 smithi136 ceph-mon[32923]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:21.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:20 smithi106 ceph-mon[32510]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:21.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:20 smithi042 ceph-mon[28130]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:23.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:22 smithi136 ceph-mon[32923]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:23.179 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:22 smithi042 ceph-mon[28130]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:23.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:22 smithi106 ceph-mon[32510]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:12:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:24.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:12:24.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:24.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T00:12:24.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:25.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:24 smithi136 ceph-mon[32923]: Deploying daemon osd.5 on smithi136 2024-04-25T00:12:25.242 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:24 smithi136 ceph-mon[32923]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:25.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:24 smithi106 ceph-mon[32510]: Deploying daemon osd.5 on smithi136 2024-04-25T00:12:25.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:24 smithi106 ceph-mon[32510]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:25.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:24 smithi042 ceph-mon[28130]: Deploying daemon osd.5 on smithi136 2024-04-25T00:12:25.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:24 smithi042 ceph-mon[28130]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:27.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:26 smithi136 ceph-mon[32923]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:27.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:26 smithi106 ceph-mon[32510]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:27.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:26 smithi042 ceph-mon[28130]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:29.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:28 smithi106 ceph-mon[32510]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:29.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:28 smithi136 ceph-mon[32923]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:29.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:28 smithi042 ceph-mon[28130]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:29 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:29 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:29 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:30.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:30.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:30.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:31.237 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:30 smithi136 ceph-mon[32923]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:30 smithi106 ceph-mon[32510]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:31.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:30 smithi042 ceph-mon[28130]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:32.503 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.502+0000 7fb0bb7fe700 1 -- 172.21.15.136:0/1088655220 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb0c4062670 con 0x7fb0b00a2ca0 2024-04-25T00:12:32.510 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 5 on host 'smithi136' 2024-04-25T00:12:32.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1088655220 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0b00a2ca0 msgr2=0x7fb0b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:32.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0b00a2ca0 0x7fb0b00a5160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4005950 tx=0x7fb0b40058e0 comp rx=0 tx=0).stop 2024-04-25T00:12:32.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1088655220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0c4102280 msgr2=0x7fb0c41984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:32.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0c4102280 0x7fb0c41984c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc00b530 tx=0x7fb0bc00b840 comp rx=0 tx=0).stop 2024-04-25T00:12:32.510 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1088655220 shutdown_connections 2024-04-25T00:12:32.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0b00a2ca0 0x7fb0b00a5160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:32.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c4103e20 0x7fb0c419fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:32.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0c4102280 0x7fb0c41984c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:32.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 --2- 172.21.15.136:0/1088655220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0c4103480 0x7fb0c4198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:32.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1088655220 >> 172.21.15.136:0/1088655220 conn(0x7fb0c40fd820 msgr2=0x7fb0c4106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:32.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1088655220 shutdown_connections 2024-04-25T00:12:32.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:12:32.505+0000 7fb0cc6e1700 1 -- 172.21.15.136:0/1088655220 wait complete. 2024-04-25T00:12:32.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:32.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:32.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:32.842 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.842 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.842 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:32.842 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:32.842 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.842 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.842 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.842 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:32.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:32.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:32.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:32.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:12:33.107 DEBUG:teuthology.orchestra.run.smithi136:osd.5> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T00:12:33.120 INFO:tasks.cephadm:Deploying osd.6 on smithi042 with /dev/vg_nvme/lv_2... 2024-04-25T00:12:33.120 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:12:33.597 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:12:33.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:33 smithi136 ceph-mon[32923]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:33.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:33 smithi136 ceph-mon[32923]: from='osd.5 [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:12:33.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:33 smithi136 ceph-mon[32923]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:12:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:33 smithi106 ceph-mon[32510]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:33 smithi106 ceph-mon[32510]: from='osd.5 [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:12:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:33 smithi106 ceph-mon[32510]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:12:33.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:33 smithi042 ceph-mon[28130]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:33.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:33 smithi042 ceph-mon[28130]: from='osd.5 [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:12:33.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:33 smithi042 ceph-mon[28130]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T00:12:34.791 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:34 smithi042 ceph-mon[28130]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:12:34.792 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:34 smithi042 ceph-mon[28130]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:12:34.792 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:34 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:34.792 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:34 smithi042 ceph-mon[28130]: from='osd.5 [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:12:34.792 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:34 smithi042 ceph-mon[28130]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:12:34.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:34 smithi106 ceph-mon[32510]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:12:34.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:34 smithi106 ceph-mon[32510]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:12:34.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:34 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:34.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:34 smithi106 ceph-mon[32510]: from='osd.5 [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:12:34.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:34 smithi106 ceph-mon[32510]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:12:35.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:34 smithi136 ceph-mon[32923]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T00:12:35.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:34 smithi136 ceph-mon[32923]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T00:12:35.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:34 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:35.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:34 smithi136 ceph-mon[32923]: from='osd.5 [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:12:35.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:34 smithi136 ceph-mon[32923]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:12:35.513 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:12:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:12:35.214+0000 7f631780a700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T00:12:35.514 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:12:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:12:35.224+0000 7f630fdf6700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:12:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:35 smithi106 ceph-mon[32510]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:35 smithi106 ceph-mon[32510]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:12:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:35 smithi106 ceph-mon[32510]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:12:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:35 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:35 smithi106 ceph-mon[32510]: from='osd.5 ' entity='osd.5' 2024-04-25T00:12:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:35 smithi106 ceph-mon[32510]: osd.5 [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] boot 2024-04-25T00:12:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:35 smithi106 ceph-mon[32510]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T00:12:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:35 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:35 smithi042 ceph-mon[28130]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:35 smithi042 ceph-mon[28130]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:12:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:35 smithi042 ceph-mon[28130]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:12:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:35 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:35 smithi042 ceph-mon[28130]: from='osd.5 ' entity='osd.5' 2024-04-25T00:12:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:35 smithi042 ceph-mon[28130]: osd.5 [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] boot 2024-04-25T00:12:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:35 smithi042 ceph-mon[28130]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T00:12:35.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:35 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:36.004 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:35 smithi136 ceph-mon[32923]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:36.005 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:35 smithi136 ceph-mon[32923]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:12:36.005 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:35 smithi136 ceph-mon[32923]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T00:12:36.005 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:35 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:36.005 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:35 smithi136 ceph-mon[32923]: from='osd.5 ' entity='osd.5' 2024-04-25T00:12:36.005 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:35 smithi136 ceph-mon[32923]: osd.5 [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] boot 2024-04-25T00:12:36.005 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:35 smithi136 ceph-mon[32923]: osdmap e38: 6 total, 6 up, 6 in 2024-04-25T00:12:36.005 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:35 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T00:12:36.457 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:12:36.457 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:12:36.457 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-04-25T00:12:36.457 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-04-25T00:12:36.457 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0334357 s, 314 MB/s 2024-04-25T00:12:36.457 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-04-25T00:12:36.798 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:12:36.798 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:12:36.798 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:36.798 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:36.798 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:36.798 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:36.798 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:36.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:36.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:36.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:36.799 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:36 smithi042 ceph-mon[28130]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:12:36.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:12:36.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:12:36.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:36.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:36.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:36.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:36.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:36.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:36.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:36.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:36.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:36 smithi106 ceph-mon[32510]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:12:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:12:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:12:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:37.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:12:37.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:37.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:37.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:12:37.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:12:37.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:36 smithi136 ceph-mon[32923]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T00:12:37.136 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_2 2024-04-25T00:12:37.587 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:12:37.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:37 smithi106 ceph-mon[32510]: pgmap v153: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:37.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:37 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi136 2024-04-25T00:12:37.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:37 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi136 to 8544M 2024-04-25T00:12:37.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:37 smithi106 ceph-mon[32510]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:12:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:37 smithi042 ceph-mon[28130]: pgmap v153: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:37 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi136 2024-04-25T00:12:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:37 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi136 to 8544M 2024-04-25T00:12:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:37 smithi042 ceph-mon[28130]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:12:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:37 smithi136 ceph-mon[32923]: pgmap v153: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-25T00:12:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:37 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi136 2024-04-25T00:12:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:37 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi136 to 8544M 2024-04-25T00:12:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:37 smithi136 ceph-mon[32923]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T00:12:39.811 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:39 smithi042 ceph-mon[28130]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:39.811 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:39 smithi042 ceph-mon[28130]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:12:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.987+0000 7f948dc00700 1 -- 172.21.15.42:0/1022577272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881099c0 msgr2=0x7f9488109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.987+0000 7f948dc00700 1 --2- 172.21.15.42:0/1022577272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881099c0 0x7f9488109e20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f94740098f0 tx=0x7f9474009c00 comp rx=0 tx=0).stop 2024-04-25T00:12:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.987+0000 7f948dc00700 1 -- 172.21.15.42:0/1022577272 shutdown_connections 2024-04-25T00:12:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.987+0000 7f948dc00700 1 --2- 172.21.15.42:0/1022577272 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f948810a360 0x7f9488110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.987+0000 7f948dc00700 1 --2- 172.21.15.42:0/1022577272 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94881087c0 0x7f9488108be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.987+0000 7f948dc00700 1 --2- 172.21.15.42:0/1022577272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881099c0 0x7f9488109e20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.987+0000 7f948dc00700 1 -- 172.21.15.42:0/1022577272 >> 172.21.15.42:0/1022577272 conn(0x7f9488077790 msgr2=0x7f9488079bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:12:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.987+0000 7f948dc00700 1 -- 172.21.15.42:0/1022577272 shutdown_connections 2024-04-25T00:12:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.988+0000 7f948dc00700 1 -- 172.21.15.42:0/1022577272 wait complete. 2024-04-25T00:12:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.989+0000 7f948dc00700 1 Processor -- start 2024-04-25T00:12:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.989+0000 7f948dc00700 1 -- start start 2024-04-25T00:12:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.989+0000 7f948dc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881087c0 0x7f948819e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.989+0000 7f948dc00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94881099c0 0x7f948819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.989+0000 7f948dc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f948810a360 0x7f94881a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f948dc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f94881143d0 con 0x7f948810a360 2024-04-25T00:12:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f948dc00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9488114250 con 0x7f94881099c0 2024-04-25T00:12:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f948dc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9488114550 con 0x7f94881087c0 2024-04-25T00:12:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f9487e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881087c0 0x7f948819e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f9487e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881087c0 0x7f948819e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47336/0 (socket says 172.21.15.42:47336) 2024-04-25T00:12:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f9487e00700 1 -- 172.21.15.42:0/1743890229 learned_addr learned my addr 172.21.15.42:0/1743890229 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:12:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f948d200700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f948810a360 0x7f94881a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f9487400700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94881099c0 0x7f948819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f9487e00700 1 -- 172.21.15.42:0/1743890229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94881099c0 msgr2=0x7f948819ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f9487e00700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94881099c0 0x7f948819ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f9487e00700 1 -- 172.21.15.42:0/1743890229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f948810a360 msgr2=0x7f94881a5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:12:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f9487e00700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f948810a360 0x7f94881a5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:12:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.990+0000 7f9487e00700 1 -- 172.21.15.42:0/1743890229 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94881a6640 con 0x7f94881087c0 2024-04-25T00:12:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.991+0000 7f9487e00700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881087c0 0x7f948819e9e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f947800d9b0 tx=0x7f947800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.991+0000 7f9484c00700 1 -- 172.21.15.42:0/1743890229 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9478016070 con 0x7f94881087c0 2024-04-25T00:12:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.991+0000 7f9484c00700 1 -- 172.21.15.42:0/1743890229 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f947800be30 con 0x7f94881087c0 2024-04-25T00:12:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.991+0000 7f9484c00700 1 -- 172.21.15.42:0/1743890229 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947800f3c0 con 0x7f94881087c0 2024-04-25T00:12:39.995 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.991+0000 7f948dc00700 1 -- 172.21.15.42:0/1743890229 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94881a6930 con 0x7f94881087c0 2024-04-25T00:12:39.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.991+0000 7f948dc00700 1 -- 172.21.15.42:0/1743890229 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94881a6ec0 con 0x7f94881087c0 2024-04-25T00:12:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.992+0000 7f948dc00700 1 -- 172.21.15.42:0/1743890229 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9488068f20 con 0x7f94881087c0 2024-04-25T00:12:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.994+0000 7f9484c00700 1 -- 172.21.15.42:0/1743890229 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9478019070 con 0x7f94881087c0 2024-04-25T00:12:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.995+0000 7f9484c00700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f946c0a2ca0 0x7f946c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:12:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.996+0000 7f9484c00700 1 -- 172.21.15.42:0/1743890229 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7f94780cdce0 con 0x7f94881087c0 2024-04-25T00:12:39.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.996+0000 7f9487400700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f946c0a2ca0 0x7f946c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:12:39.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:39.997+0000 7f9487400700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f946c0a2ca0 0x7f946c0a5160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f94740098c0 tx=0x7f947401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:12:40.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:40.000+0000 7f9484c00700 1 -- 172.21.15.42:0/1743890229 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9478020030 con 0x7f94881087c0 2024-04-25T00:12:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:39 smithi136 ceph-mon[32923]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:39 smithi136 ceph-mon[32923]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:12:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:39 smithi106 ceph-mon[32510]: pgmap v156: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:39 smithi106 ceph-mon[32510]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T00:12:40.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:12:40.246+0000 7f948dc00700 1 -- 172.21.15.42:0/1743890229 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f9488062670 con 0x7f946c0a2ca0 2024-04-25T00:12:40.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:40 smithi042 ceph-mon[28130]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:12:40.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:40.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:40.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:41.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:40 smithi136 ceph-mon[32923]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:12:41.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:41.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:41.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:41.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:40 smithi106 ceph-mon[32510]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T00:12:41.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:12:41.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:12:41.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:41.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:41 smithi042 ceph-mon[28130]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:41.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:41 smithi042 ceph-mon[28130]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:41 smithi136 ceph-mon[32923]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:41 smithi136 ceph-mon[32923]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:41 smithi106 ceph-mon[32510]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:41 smithi106 ceph-mon[32510]: from='client.24265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:12:43.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:43 smithi042 ceph-mon[28130]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-25T00:12:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:43 smithi136 ceph-mon[32923]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-25T00:12:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:43 smithi106 ceph-mon[32510]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 109 KiB/s, 0 objects/s recovering 2024-04-25T00:12:45.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:44 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/922016078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ac3ab28-29e1-4316-af39-4ca7e9785cf3"}]: dispatch 2024-04-25T00:12:45.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:44 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/922016078' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ac3ab28-29e1-4316-af39-4ca7e9785cf3"}]': finished 2024-04-25T00:12:45.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:44 smithi136 ceph-mon[32923]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T00:12:45.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:44 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:12:45.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:44 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/922016078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ac3ab28-29e1-4316-af39-4ca7e9785cf3"}]: dispatch 2024-04-25T00:12:45.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:44 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/922016078' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ac3ab28-29e1-4316-af39-4ca7e9785cf3"}]': finished 2024-04-25T00:12:45.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:44 smithi106 ceph-mon[32510]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T00:12:45.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:44 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:12:45.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:44 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/922016078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ac3ab28-29e1-4316-af39-4ca7e9785cf3"}]: dispatch 2024-04-25T00:12:45.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:44 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/922016078' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ac3ab28-29e1-4316-af39-4ca7e9785cf3"}]': finished 2024-04-25T00:12:45.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:44 smithi042 ceph-mon[28130]: osdmap e43: 7 total, 6 up, 7 in 2024-04-25T00:12:45.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:44 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:12:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:45 smithi136 ceph-mon[32923]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-25T00:12:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:45 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/46197133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:45 smithi106 ceph-mon[32510]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-25T00:12:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:45 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/46197133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:46.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:45 smithi042 ceph-mon[28130]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-04-25T00:12:46.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:45 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/46197133' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:12:47.890 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:47 smithi106 ceph-mon[32510]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T00:12:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:47 smithi136 ceph-mon[32923]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T00:12:48.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:47 smithi042 ceph-mon[28130]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T00:12:49.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:48 smithi136 ceph-mon[32923]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T00:12:49.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:48 smithi106 ceph-mon[32510]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T00:12:49.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:48 smithi042 ceph-mon[28130]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T00:12:51.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:50 smithi042 ceph-mon[28130]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:12:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:50 smithi136 ceph-mon[32923]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:12:51.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:50 smithi106 ceph-mon[32510]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:12:53.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:52 smithi042 ceph-mon[28130]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:53.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:52 smithi136 ceph-mon[32923]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:53.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:52 smithi106 ceph-mon[32510]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:55.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:54 smithi042 ceph-mon[28130]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:55.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:12:55.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:55.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:54 smithi042 ceph-mon[28130]: Deploying daemon osd.6 on smithi042 2024-04-25T00:12:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:54 smithi136 ceph-mon[32923]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:12:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:54 smithi136 ceph-mon[32923]: Deploying daemon osd.6 on smithi042 2024-04-25T00:12:55.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:54 smithi106 ceph-mon[32510]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:55.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T00:12:55.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:12:55.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:54 smithi106 ceph-mon[32510]: Deploying daemon osd.6 on smithi042 2024-04-25T00:12:57.166 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:56 smithi042 ceph-mon[28130]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:57.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:56 smithi136 ceph-mon[32923]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:57.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:56 smithi106 ceph-mon[32510]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:59.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:12:58 smithi136 ceph-mon[32923]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:59.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:12:58 smithi106 ceph-mon[32510]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:12:59.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:12:58 smithi042 ceph-mon[28130]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:01.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:00 smithi136 ceph-mon[32923]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:01.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:00 smithi106 ceph-mon[32510]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:01.366 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:00 smithi042 ceph-mon[28130]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:03.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:02 smithi136 ceph-mon[32923]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:03.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:02 smithi106 ceph-mon[32510]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:03.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:02 smithi042 ceph-mon[28130]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:04.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:04 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:04.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:04 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:04.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:04 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:04.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:04 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:04.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:04 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:04.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:04 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:04.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:04 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:04.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:04 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:04.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:04 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:05.304 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:05 smithi042 ceph-mon[28130]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:05 smithi136 ceph-mon[32923]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:05 smithi106 ceph-mon[32510]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:07.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:07 smithi042 ceph-mon[28130]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:07.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:07.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:07.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:07 smithi042 ceph-mon[28130]: from='osd.6 [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:13:07.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:07 smithi042 ceph-mon[28130]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:13:07.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:07 smithi136 ceph-mon[32923]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:07.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:07.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:07.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:07 smithi136 ceph-mon[32923]: from='osd.6 [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:13:07.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:07 smithi136 ceph-mon[32923]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:13:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:07 smithi106 ceph-mon[32510]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:07 smithi106 ceph-mon[32510]: from='osd.6 [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:13:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:07 smithi106 ceph-mon[32510]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T00:13:08.208 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 6 on host 'smithi042' 2024-04-25T00:13:08.208 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.205+0000 7f9484c00700 1 -- 172.21.15.42:0/1743890229 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9488062670 con 0x7f946c0a2ca0 2024-04-25T00:13:08.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.209+0000 7f948dc00700 1 -- 172.21.15.42:0/1743890229 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f946c0a2ca0 msgr2=0x7f946c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.209+0000 7f948dc00700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f946c0a2ca0 0x7f946c0a5160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f94740098c0 tx=0x7f947401a040 comp rx=0 tx=0).stop 2024-04-25T00:13:08.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.209+0000 7f948dc00700 1 -- 172.21.15.42:0/1743890229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881087c0 msgr2=0x7f948819e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:08.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.209+0000 7f948dc00700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881087c0 0x7f948819e9e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f947800d9b0 tx=0x7f947800dcc0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.209+0000 7f948dc00700 1 -- 172.21.15.42:0/1743890229 shutdown_connections 2024-04-25T00:13:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.209+0000 7f948dc00700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f946c0a2ca0 0x7f946c0a5160 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.209+0000 7f948dc00700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f948810a360 0x7f94881a5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.209+0000 7f948dc00700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94881087c0 0x7f948819e9e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.209+0000 7f948dc00700 1 --2- 172.21.15.42:0/1743890229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94881099c0 0x7f948819ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.209+0000 7f948dc00700 1 -- 172.21.15.42:0/1743890229 >> 172.21.15.42:0/1743890229 conn(0x7f9488077790 msgr2=0x7f948810ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.210+0000 7f948dc00700 1 -- 172.21.15.42:0/1743890229 shutdown_connections 2024-04-25T00:13:08.211 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:13:08.210+0000 7f948dc00700 1 -- 172.21.15.42:0/1743890229 wait complete. 2024-04-25T00:13:09.274 DEBUG:teuthology.orchestra.run.smithi042:osd.6> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T00:13:09.280 INFO:tasks.cephadm:Deploying osd.7 on smithi106 with /dev/vg_nvme/lv_2... 2024-04-25T00:13:09.280 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:13:09.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:09 smithi106 ceph-mon[32510]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:09.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:09 smithi106 ceph-mon[32510]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:13:09.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:09 smithi106 ceph-mon[32510]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:13:09.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:09.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:09 smithi106 ceph-mon[32510]: from='osd.6 [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:13:09.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:09 smithi106 ceph-mon[32510]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:13:09.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:09.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:09.315 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:09 smithi136 ceph-mon[32923]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:09.315 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:09 smithi136 ceph-mon[32923]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:13:09.315 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:09 smithi136 ceph-mon[32923]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:13:09.315 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:09.315 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:09 smithi136 ceph-mon[32923]: from='osd.6 [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:13:09.315 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:09 smithi136 ceph-mon[32923]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:13:09.315 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:09.316 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:09.335 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:09 smithi042 ceph-mon[28130]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:09.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:09 smithi042 ceph-mon[28130]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T00:13:09.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:09 smithi042 ceph-mon[28130]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T00:13:09.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:09.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:09 smithi042 ceph-mon[28130]: from='osd.6 [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:13:09.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:09 smithi042 ceph-mon[28130]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:13:09.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:09.336 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:09.525 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:13:10.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:10 smithi106 ceph-mon[32510]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:13:10.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:10 smithi106 ceph-mon[32510]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:13:10.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:10 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:10.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:10 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:10.349 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:10 smithi042 ceph-mon[28130]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:13:10.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:10 smithi042 ceph-mon[28130]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:13:10.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:10 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:10.350 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:10 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:10.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:10 smithi136 ceph-mon[32923]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:13:10.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:10 smithi136 ceph-mon[32923]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T00:13:10.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:10 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:10.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:10 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:11.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:11 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:13:11.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:11 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:13:11.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:11 smithi106 ceph-mon[32510]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:11.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:11.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:11 smithi106 ceph-mon[32510]: osd.6 [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] boot 2024-04-25T00:13:11.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:11 smithi106 ceph-mon[32510]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T00:13:11.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:11.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:11 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:13:11.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:11 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:13:11.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:11 smithi042 ceph-mon[28130]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:11.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:11.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:11 smithi042 ceph-mon[28130]: osd.6 [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] boot 2024-04-25T00:13:11.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:11 smithi042 ceph-mon[28130]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T00:13:11.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:11.435 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:13:11.435 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:13:11.435 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-25T00:13:11.435 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-25T00:13:11.435 INFO:teuthology.orchestra.run.smithi106.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113674 s, 922 MB/s 2024-04-25T00:13:11.435 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-25T00:13:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:11 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:13:11.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:11 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:13:11.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:11 smithi136 ceph-mon[32923]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-04-25T00:13:11.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:11.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:11 smithi136 ceph-mon[32923]: osd.6 [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] boot 2024-04-25T00:13:11.516 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:11 smithi136 ceph-mon[32923]: osdmap e46: 7 total, 7 up, 7 in 2024-04-25T00:13:11.516 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T00:13:12.042 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_2 2024-04-25T00:13:12.320 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:13:12.326 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:12 smithi106 ceph-mon[32510]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:13:12.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:12 smithi042 ceph-mon[28130]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:13:12.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:12 smithi136 ceph-mon[32923]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T00:13:13.237 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:13 smithi106 ceph-mon[32510]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:13.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:13 smithi042 ceph-mon[28130]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:13 smithi136 ceph-mon[32923]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:13.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.718+0000 7f729cf44700 1 -- 172.21.15.106:0/1010046561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72981022c0 msgr2=0x7f72981026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:13.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.718+0000 7f729cf44700 1 --2- 172.21.15.106:0/1010046561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72981022c0 0x7f72981026e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f72880099c0 tx=0x7f7288009cd0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.718 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.718+0000 7f729cf44700 1 -- 172.21.15.106:0/1010046561 shutdown_connections 2024-04-25T00:13:13.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.718+0000 7f729cf44700 1 --2- 172.21.15.106:0/1010046561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7298103e60 0x7f729810a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.718+0000 7f729cf44700 1 --2- 172.21.15.106:0/1010046561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72981022c0 0x7f72981026e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.718+0000 7f729cf44700 1 --2- 172.21.15.106:0/1010046561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72981034c0 0x7f7298103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.718+0000 7f729cf44700 1 -- 172.21.15.106:0/1010046561 >> 172.21.15.106:0/1010046561 conn(0x7f72980fd880 msgr2=0x7f72980ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:13.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.718+0000 7f729cf44700 1 -- 172.21.15.106:0/1010046561 shutdown_connections 2024-04-25T00:13:13.719 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.718+0000 7f729cf44700 1 -- 172.21.15.106:0/1010046561 wait complete. 2024-04-25T00:13:13.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.718+0000 7f729cf44700 1 Processor -- start 2024-04-25T00:13:13.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f729cf44700 1 -- start start 2024-04-25T00:13:13.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f729cf44700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72981022c0 0x7f7298075fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:13.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f729cf44700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72981034c0 0x7f7298074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:13.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f729cf44700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7298103e60 0x7f7298074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:13.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f729cf44700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f729810df30 con 0x7f7298103e60 2024-04-25T00:13:13.721 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f729cf44700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f729810ddb0 con 0x7f72981034c0 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f729cf44700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f729810e0b0 con 0x7f72981022c0 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f728dd9b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72981034c0 0x7f7298074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f728dd9b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72981034c0 0x7f7298074620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50636/0 (socket says 172.21.15.106:50636) 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f728dd9b700 1 -- 172.21.15.106:0/4071838571 learned_addr learned my addr 172.21.15.106:0/4071838571 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f729659c700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72981022c0 0x7f7298075fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f728dd9b700 1 -- 172.21.15.106:0/4071838571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72981022c0 msgr2=0x7f7298075fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f728dd9b700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72981022c0 0x7f7298075fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f728dd9b700 1 -- 172.21.15.106:0/4071838571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7298103e60 msgr2=0x7f7298074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f728dd9b700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7298103e60 0x7f7298074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f728dd9b700 1 -- 172.21.15.106:0/4071838571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7298075410 con 0x7f72981034c0 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.719+0000 7f729659c700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72981022c0 0x7f7298075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.720+0000 7f728dd9b700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72981034c0 0x7f7298074620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f728000c970 tx=0x7f728000cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:13.722 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.720+0000 7f728ffff700 1 -- 172.21.15.106:0/4071838571 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280007aa0 con 0x7f72981034c0 2024-04-25T00:13:13.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.720+0000 7f728ffff700 1 -- 172.21.15.106:0/4071838571 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7280004d10 con 0x7f72981034c0 2024-04-25T00:13:13.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.720+0000 7f729cf44700 1 -- 172.21.15.106:0/4071838571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72981a26b0 con 0x7f72981034c0 2024-04-25T00:13:13.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.720+0000 7f728ffff700 1 -- 172.21.15.106:0/4071838571 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7280005690 con 0x7f72981034c0 2024-04-25T00:13:13.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.720+0000 7f729cf44700 1 -- 172.21.15.106:0/4071838571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72981a2bf0 con 0x7f72981034c0 2024-04-25T00:13:13.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.721+0000 7f729cf44700 1 -- 172.21.15.106:0/4071838571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7298068f20 con 0x7f72981034c0 2024-04-25T00:13:13.723 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.722+0000 7f728ffff700 1 -- 172.21.15.106:0/4071838571 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7280004750 con 0x7f72981034c0 2024-04-25T00:13:13.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.722+0000 7f728ffff700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72840a2c20 0x7f72840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:13.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.723+0000 7f728ffff700 1 -- 172.21.15.106:0/4071838571 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f72800ce460 con 0x7f72981034c0 2024-04-25T00:13:13.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.723+0000 7f729659c700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72840a2c20 0x7f72840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:13.724 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.724+0000 7f729659c700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72840a2c20 0x7f72840a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7288006010 tx=0x7f7288005c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:13.726 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.726+0000 7f728ffff700 1 -- 172.21.15.106:0/4071838571 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f728009c1f0 con 0x7f72981034c0 2024-04-25T00:13:13.935 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:13.934+0000 7f729cf44700 1 -- 172.21.15.106:0/4071838571 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f7298062670 con 0x7f72840a2c20 2024-04-25T00:13:14.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:13:14.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:13:14.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:14 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:14.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:13:14.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:13:14.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:14 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:14.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:13:14.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:13:14.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:14 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:15.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:15.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:15.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi042 2024-04-25T00:13:15.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:15.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:15.446 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:15.447 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi042 to 5698M 2024-04-25T00:13:15.447 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.447 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:15.447 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:15.447 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.447 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi042 2024-04-25T00:13:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:15.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:15.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:15.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi042 to 5698M 2024-04-25T00:13:15.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:15.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:15.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:15.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:15.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='client.24293 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi042 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi042 to 5698M 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:15.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:16.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:16 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:16.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:16 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:16.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:16 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:16.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:16 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:16.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:16 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:16.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:16 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:16.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:16 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:16.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:16 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:16.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:16 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:17.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:17 smithi042 ceph-mon[28130]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:17 smithi042 ceph-mon[28130]: from='client.? 172.21.15.106:0/3173398081' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235"}]: dispatch 2024-04-25T00:13:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:17 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235"}]: dispatch 2024-04-25T00:13:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:17 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235"}]': finished 2024-04-25T00:13:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:17 smithi042 ceph-mon[28130]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T00:13:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:17 smithi042 ceph-mon[28130]: from='client.? 172.21.15.106:0/1997471514' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:13:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:17 smithi136 ceph-mon[32923]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:17 smithi136 ceph-mon[32923]: from='client.? 172.21.15.106:0/3173398081' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235"}]: dispatch 2024-04-25T00:13:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:17 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235"}]: dispatch 2024-04-25T00:13:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:17 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235"}]': finished 2024-04-25T00:13:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:17 smithi136 ceph-mon[32923]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T00:13:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:17 smithi136 ceph-mon[32923]: from='client.? 172.21.15.106:0/1997471514' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:13:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:17 smithi106 ceph-mon[32510]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:17 smithi106 ceph-mon[32510]: from='client.? 172.21.15.106:0/3173398081' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235"}]: dispatch 2024-04-25T00:13:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:17 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235"}]: dispatch 2024-04-25T00:13:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:17 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235"}]': finished 2024-04-25T00:13:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:17 smithi106 ceph-mon[32510]: osdmap e48: 8 total, 7 up, 8 in 2024-04-25T00:13:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:17 smithi106 ceph-mon[32510]: from='client.? 172.21.15.106:0/1997471514' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:13:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:19 smithi042 ceph-mon[28130]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:19 smithi136 ceph-mon[32923]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:19 smithi106 ceph-mon[32510]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:21.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:21 smithi042 ceph-mon[28130]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:21.638 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:21 smithi106 ceph-mon[32510]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:21 smithi136 ceph-mon[32923]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:23.365 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:23 smithi106 ceph-mon[32510]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:23.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:23 smithi042 ceph-mon[28130]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:23 smithi136 ceph-mon[32923]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:24.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:24 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:13:24.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:24 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:24.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:24 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:13:24.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:24 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:24.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:24 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T00:13:24.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:24 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:25.477 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:25 smithi106 ceph-mon[32510]: Deploying daemon osd.7 on smithi106 2024-04-25T00:13:25.478 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:25 smithi106 ceph-mon[32510]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:25.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:25 smithi042 ceph-mon[28130]: Deploying daemon osd.7 on smithi106 2024-04-25T00:13:25.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:25 smithi042 ceph-mon[28130]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:25 smithi136 ceph-mon[32923]: Deploying daemon osd.7 on smithi106 2024-04-25T00:13:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:25 smithi136 ceph-mon[32923]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:27 smithi042 ceph-mon[28130]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:27 smithi136 ceph-mon[32923]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:27 smithi106 ceph-mon[32510]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:29 smithi136 ceph-mon[32923]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:29.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:29 smithi106 ceph-mon[32510]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:29.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:29 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:29.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:29 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:29.767 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:29 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:29.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:29 smithi042 ceph-mon[28130]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:29.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:29.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:29.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:31.649 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:31 smithi106 ceph-mon[32510]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:31.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:31 smithi136 ceph-mon[32923]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:31.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:31 smithi042 ceph-mon[28130]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:32.634 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 7 on host 'smithi106' 2024-04-25T00:13:32.635 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.634+0000 7f728ffff700 1 -- 172.21.15.106:0/4071838571 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f7298062670 con 0x7f72840a2c20 2024-04-25T00:13:32.636 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.636+0000 7f729cf44700 1 -- 172.21.15.106:0/4071838571 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72840a2c20 msgr2=0x7f72840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.636+0000 7f729cf44700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72840a2c20 0x7f72840a50e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7288006010 tx=0x7f7288005c20 comp rx=0 tx=0).stop 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.636+0000 7f729cf44700 1 -- 172.21.15.106:0/4071838571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72981034c0 msgr2=0x7f7298074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.636+0000 7f729cf44700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72981034c0 0x7f7298074620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f728000c970 tx=0x7f728000cd30 comp rx=0 tx=0).stop 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.636+0000 7f729cf44700 1 -- 172.21.15.106:0/4071838571 shutdown_connections 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.636+0000 7f729cf44700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72840a2c20 0x7f72840a50e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.636+0000 7f729cf44700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7298103e60 0x7f7298074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.636+0000 7f729cf44700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72981022c0 0x7f7298075fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.636+0000 7f729cf44700 1 --2- 172.21.15.106:0/4071838571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72981034c0 0x7f7298074620 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.636+0000 7f729cf44700 1 -- 172.21.15.106:0/4071838571 >> 172.21.15.106:0/4071838571 conn(0x7f72980fd880 msgr2=0x7f72981067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.637+0000 7f729cf44700 1 -- 172.21.15.106:0/4071838571 shutdown_connections 2024-04-25T00:13:32.637 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:13:32.637+0000 7f729cf44700 1 -- 172.21.15.106:0/4071838571 wait complete. 2024-04-25T00:13:33.273 DEBUG:teuthology.orchestra.run.smithi106:osd.7> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T00:13:33.275 INFO:tasks.cephadm:Deploying osd.8 on smithi136 with /dev/vg_nvme/lv_2... 2024-04-25T00:13:33.275 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T00:13:33.519 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:13:33.852 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:33 smithi136 ceph-mon[32923]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:33.852 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:33 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:33.852 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:33 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:33.852 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:33 smithi136 ceph-mon[32923]: from='osd.7 [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:13:33.852 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:33 smithi136 ceph-mon[32923]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:13:33.853 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:33 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:33.853 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:33 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:33.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:33 smithi042 ceph-mon[28130]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:33.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:33 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:33.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:33 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:33.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:33 smithi042 ceph-mon[28130]: from='osd.7 [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:13:33.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:33 smithi042 ceph-mon[28130]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:13:33.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:33 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:33.883 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:33 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:34.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:33 smithi106 ceph-mon[32510]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:34.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:33 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:34.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:33 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:34.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:33 smithi106 ceph-mon[32510]: from='osd.7 [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:13:34.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:33 smithi106 ceph-mon[32510]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T00:13:34.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:33 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:34.014 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:33 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:34.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:34 smithi106 ceph-mon[32510]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:13:34.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:34 smithi106 ceph-mon[32510]: from='osd.7 [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:13:34.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:34 smithi106 ceph-mon[32510]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:13:34.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:34 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:34.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:34 smithi106 ceph-mon[32510]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:13:34.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:13:34 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:13:34.618+0000 7fdfd43eb700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T00:13:34.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:13:34 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:13:34.628+0000 7fdfd09df700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:13:34.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:34 smithi042 ceph-mon[28130]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:13:34.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:34 smithi042 ceph-mon[28130]: from='osd.7 [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:13:34.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:34 smithi042 ceph-mon[28130]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:13:34.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:34 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:34.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:34 smithi042 ceph-mon[28130]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:13:34.878 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:34 smithi136 ceph-mon[32923]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T00:13:34.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:34 smithi136 ceph-mon[32923]: from='osd.7 [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:13:34.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:34 smithi136 ceph-mon[32923]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T00:13:34.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:34 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:34.879 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:34 smithi136 ceph-mon[32923]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:13:35.442 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T00:13:35.442 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T00:13:35.442 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-25T00:13:35.442 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-25T00:13:35.443 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00960681 s, 1.1 GB/s 2024-04-25T00:13:35.443 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-25T00:13:35.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:35 smithi136 ceph-mon[32923]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:35.808 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:35 smithi136 ceph-mon[32923]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:13:35.809 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:35 smithi136 ceph-mon[32923]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:13:35.809 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:35 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:35.809 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:35 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:35 smithi106 ceph-mon[32510]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:35 smithi106 ceph-mon[32510]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:13:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:35 smithi106 ceph-mon[32510]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:13:36.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:35 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:36.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:35 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:36.118 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_2 2024-04-25T00:13:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:35 smithi042 ceph-mon[28130]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-04-25T00:13:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:35 smithi042 ceph-mon[28130]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:13:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:35 smithi042 ceph-mon[28130]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T00:13:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:35 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:36.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:35 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:36.355 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:13:36.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:36 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:13:36.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:36 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:13:36.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:36 smithi136 ceph-mon[32923]: osd.7 [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] boot 2024-04-25T00:13:36.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:36 smithi136 ceph-mon[32923]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T00:13:36.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:36.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:36 smithi136 ceph-mon[32923]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T00:13:36.783 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:36 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:13:36.783 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:36 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:13:36.783 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:36 smithi106 ceph-mon[32510]: osd.7 [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] boot 2024-04-25T00:13:36.783 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:36 smithi106 ceph-mon[32510]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T00:13:36.783 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:36.784 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:36 smithi106 ceph-mon[32510]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T00:13:37.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:36 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:13:37.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:36 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:13:37.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:36 smithi042 ceph-mon[28130]: osd.7 [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] boot 2024-04-25T00:13:37.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:36 smithi042 ceph-mon[28130]: osdmap e51: 8 total, 8 up, 8 in 2024-04-25T00:13:37.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T00:13:37.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:36 smithi042 ceph-mon[28130]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T00:13:37.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.878+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/3605169850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f941022c0 msgr2=0x7f1f941026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.880 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.878+0000 7f1f9bc8d700 1 --2- 172.21.15.136:0/3605169850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f941022c0 0x7f1f941026e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009990 tx=0x7f1f90009ca0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.878+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/3605169850 shutdown_connections 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.878+0000 7f1f9bc8d700 1 --2- 172.21.15.136:0/3605169850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f94103e60 0x7f1f9410a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.878+0000 7f1f9bc8d700 1 --2- 172.21.15.136:0/3605169850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f941022c0 0x7f1f941026e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.878+0000 7f1f9bc8d700 1 --2- 172.21.15.136:0/3605169850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f941034c0 0x7f1f94103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.878+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/3605169850 >> 172.21.15.136:0/3605169850 conn(0x7f1f940fd840 msgr2=0x7f1f940ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.878+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/3605169850 shutdown_connections 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.878+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/3605169850 wait complete. 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9bc8d700 1 Processor -- start 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9bc8d700 1 -- start start 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9bc8d700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f941022c0 0x7f1f941984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9bc8d700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f941034c0 0x7f1f941989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9bc8d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94103e60 0x7f1f9419fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9bc8d700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9410de80 con 0x7f1f941034c0 2024-04-25T00:13:37.881 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9bc8d700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9410dd00 con 0x7f1f941022c0 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9bc8d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9410e000 con 0x7f1f94103e60 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9a22a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94103e60 0x7f1f9419fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9a22a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94103e60 0x7f1f9419fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:49904/0 (socket says 172.21.15.136:49904) 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.879+0000 7f1f9a22a700 1 -- 172.21.15.136:0/2216690683 learned_addr learned my addr 172.21.15.136:0/2216690683 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f99a29700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f941022c0 0x7f1f941984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f9a22a700 1 -- 172.21.15.136:0/2216690683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f941022c0 msgr2=0x7f1f941984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f9a22a700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f941022c0 0x7f1f941984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f9a22a700 1 -- 172.21.15.136:0/2216690683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f941034c0 msgr2=0x7f1f941989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f9a22a700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f941034c0 0x7f1f941989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f9a22a700 1 -- 172.21.15.136:0/2216690683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f941a0100 con 0x7f1f94103e60 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f99a29700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f941022c0 0x7f1f941984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f9a22a700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94103e60 0x7f1f9419fa60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00add0 tx=0x7f1f8c00c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.882 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f8affd700 1 -- 172.21.15.136:0/2216690683 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c017020 con 0x7f1f94103e60 2024-04-25T00:13:37.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/2216690683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f941a03f0 con 0x7f1f94103e60 2024-04-25T00:13:37.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f8affd700 1 -- 172.21.15.136:0/2216690683 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f8c00ce20 con 0x7f1f94103e60 2024-04-25T00:13:37.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f8affd700 1 -- 172.21.15.136:0/2216690683 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c005520 con 0x7f1f94103e60 2024-04-25T00:13:37.883 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.880+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/2216690683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f941a0930 con 0x7f1f94103e60 2024-04-25T00:13:37.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.881+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/2216690683 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f94068f20 con 0x7f1f94103e60 2024-04-25T00:13:37.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.882+0000 7f1f8affd700 1 -- 172.21.15.136:0/2216690683 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1f8c004ad0 con 0x7f1f94103e60 2024-04-25T00:13:37.884 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.883+0000 7f1f8affd700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f800a2c20 0x7f1f800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:13:37.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.883+0000 7f1f8affd700 1 -- 172.21.15.136:0/2216690683 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7f1f8c0ce2b0 con 0x7f1f94103e60 2024-04-25T00:13:37.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.883+0000 7f1f99a29700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f800a2c20 0x7f1f800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:13:37.885 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.884+0000 7f1f99a29700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f800a2c20 0x7f1f800a50e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f900039c0 tx=0x7f1f90005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:13:37.886 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:37.886+0000 7f1f8affd700 1 -- 172.21.15.136:0/2216690683 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f8c021020 con 0x7f1f94103e60 2024-04-25T00:13:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:38.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:38.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:38.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:38.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:38.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:37 smithi136 ceph-mon[32923]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:13:38.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:38.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:38.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:38.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:38.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:38.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:38.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:38.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:38.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:38.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:37 smithi106 ceph-mon[32510]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:13:38.091 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:38.092+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/2216690683 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1f94062670 con 0x7f1f800a2c20 2024-04-25T00:13:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:13:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:38.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:38.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:38.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:13:38.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:38.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:37 smithi042 ceph-mon[28130]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T00:13:38.961 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:38 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi106 2024-04-25T00:13:38.962 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:38 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-25T00:13:38.962 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:13:38.962 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:13:38.962 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:38 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi106 2024-04-25T00:13:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:38 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-25T00:13:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:13:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:13:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:39.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:38 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi106 2024-04-25T00:13:39.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:38 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi106 to 5696M 2024-04-25T00:13:39.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:13:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:13:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:39.919 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:39 smithi136 ceph-mon[32923]: pgmap v199: 1 pgs: 1 remapped; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:39.919 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:39 smithi136 ceph-mon[32923]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:39.919 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:39 smithi136 ceph-mon[32923]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T00:13:39.919 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:39 smithi136 ceph-mon[32923]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:13:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:39 smithi106 ceph-mon[32510]: pgmap v199: 1 pgs: 1 remapped; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:39 smithi106 ceph-mon[32510]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:39 smithi106 ceph-mon[32510]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T00:13:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:39 smithi106 ceph-mon[32510]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:13:40.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:39 smithi042 ceph-mon[28130]: pgmap v199: 1 pgs: 1 remapped; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:40.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:39 smithi042 ceph-mon[28130]: from='client.24298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:13:40.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:39 smithi042 ceph-mon[28130]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-04-25T00:13:40.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:39 smithi042 ceph-mon[28130]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T00:13:40.935 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:40 smithi136 ceph-mon[32923]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:13:40.935 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:40 smithi136 ceph-mon[32923]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:41.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:40 smithi106 ceph-mon[32510]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:13:41.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:40 smithi106 ceph-mon[32510]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:41.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:40 smithi042 ceph-mon[28130]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T00:13:41.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:40 smithi042 ceph-mon[28130]: pgmap v202: 1 pgs: 1 remapped; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:41 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/2218143180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff7307f4-b84a-4297-941f-544e5e06b1d0"}]: dispatch 2024-04-25T00:13:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:41 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff7307f4-b84a-4297-941f-544e5e06b1d0"}]: dispatch 2024-04-25T00:13:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:41 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff7307f4-b84a-4297-941f-544e5e06b1d0"}]': finished 2024-04-25T00:13:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:41 smithi136 ceph-mon[32923]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T00:13:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:13:42.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:41 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/969204467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:13:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:41 smithi106 ceph-mon[32510]: from='client.? 172.21.15.136:0/2218143180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff7307f4-b84a-4297-941f-544e5e06b1d0"}]: dispatch 2024-04-25T00:13:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:41 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff7307f4-b84a-4297-941f-544e5e06b1d0"}]: dispatch 2024-04-25T00:13:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:41 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff7307f4-b84a-4297-941f-544e5e06b1d0"}]': finished 2024-04-25T00:13:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:41 smithi106 ceph-mon[32510]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T00:13:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:13:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:41 smithi106 ceph-mon[32510]: from='client.? 172.21.15.136:0/969204467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:13:42.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:41 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/2218143180' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff7307f4-b84a-4297-941f-544e5e06b1d0"}]: dispatch 2024-04-25T00:13:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:41 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ff7307f4-b84a-4297-941f-544e5e06b1d0"}]: dispatch 2024-04-25T00:13:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:41 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff7307f4-b84a-4297-941f-544e5e06b1d0"}]': finished 2024-04-25T00:13:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:41 smithi042 ceph-mon[28130]: osdmap e56: 9 total, 8 up, 9 in 2024-04-25T00:13:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:13:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:41 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/969204467' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:13:43.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:42 smithi136 ceph-mon[32923]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:43.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:42 smithi106 ceph-mon[32510]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:43.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:42 smithi042 ceph-mon[28130]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:43 smithi136 ceph-mon[32923]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T00:13:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:43 smithi136 ceph-mon[32923]: Cluster is now healthy 2024-04-25T00:13:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:43 smithi106 ceph-mon[32510]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T00:13:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:43 smithi106 ceph-mon[32510]: Cluster is now healthy 2024-04-25T00:13:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:43 smithi042 ceph-mon[28130]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-04-25T00:13:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:43 smithi042 ceph-mon[28130]: Cluster is now healthy 2024-04-25T00:13:45.005 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:44 smithi136 ceph-mon[32923]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:45.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:44 smithi106 ceph-mon[32510]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:45.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:44 smithi042 ceph-mon[28130]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:47.114 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:46 smithi136 ceph-mon[32923]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:47.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:46 smithi106 ceph-mon[32510]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:47.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:46 smithi042 ceph-mon[28130]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:49.070 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:48 smithi136 ceph-mon[32923]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:49.070 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:13:49.070 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:48 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:49.070 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:48 smithi136 ceph-mon[32923]: Deploying daemon osd.8 on smithi136 2024-04-25T00:13:49.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:48 smithi106 ceph-mon[32510]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:49.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:13:49.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:48 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:49.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:48 smithi106 ceph-mon[32510]: Deploying daemon osd.8 on smithi136 2024-04-25T00:13:49.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:48 smithi042 ceph-mon[28130]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:49.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T00:13:49.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:48 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:13:49.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:48 smithi042 ceph-mon[28130]: Deploying daemon osd.8 on smithi136 2024-04-25T00:13:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:50 smithi136 ceph-mon[32923]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:51.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:50 smithi106 ceph-mon[32510]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:51.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:50 smithi042 ceph-mon[28130]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:53.181 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:52 smithi042 ceph-mon[28130]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:53.237 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:52 smithi136 ceph-mon[32923]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:53.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:52 smithi106 ceph-mon[32510]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:54.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:54.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:54.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:13:55.758 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:55 smithi136 ceph-mon[32923]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:55 smithi106 ceph-mon[32510]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:55 smithi042 ceph-mon[28130]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:56.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.874+0000 7f1f8affd700 1 -- 172.21.15.136:0/2216690683 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1f94062670 con 0x7f1f800a2c20 2024-04-25T00:13:56.876 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 8 on host 'smithi136' 2024-04-25T00:13:56.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.876+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/2216690683 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f800a2c20 msgr2=0x7f1f800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.876+0000 7f1f9bc8d700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f800a2c20 0x7f1f800a50e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1f900039c0 tx=0x7f1f90005fb0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.877+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/2216690683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94103e60 msgr2=0x7f1f9419fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:13:56.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.877+0000 7f1f9bc8d700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94103e60 0x7f1f9419fa60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c00add0 tx=0x7f1f8c00c330 comp rx=0 tx=0).stop 2024-04-25T00:13:56.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.877+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/2216690683 shutdown_connections 2024-04-25T00:13:56.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.877+0000 7f1f9bc8d700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f800a2c20 0x7f1f800a50e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.877+0000 7f1f9bc8d700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f94103e60 0x7f1f9419fa60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.877+0000 7f1f9bc8d700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f941022c0 0x7f1f941984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.877+0000 7f1f9bc8d700 1 --2- 172.21.15.136:0/2216690683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f941034c0 0x7f1f941989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:13:56.878 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.877+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/2216690683 >> 172.21.15.136:0/2216690683 conn(0x7f1f940fd840 msgr2=0x7f1f941067c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:13:56.879 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.877+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/2216690683 shutdown_connections 2024-04-25T00:13:56.879 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:13:56.877+0000 7f1f9bc8d700 1 -- 172.21.15.136:0/2216690683 wait complete. 2024-04-25T00:13:57.545 DEBUG:teuthology.orchestra.run.smithi136:osd.8> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T00:13:57.547 INFO:tasks.cephadm:Deploying osd.9 on smithi042 with /dev/vg_nvme/lv_1... 2024-04-25T00:13:57.548 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:13:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:57 smithi136 ceph-mon[32923]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:57 smithi136 ceph-mon[32923]: from='osd.8 [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:13:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:57 smithi136 ceph-mon[32923]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:13:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:57 smithi106 ceph-mon[32510]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:57 smithi106 ceph-mon[32510]: from='osd.8 [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:13:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:57 smithi106 ceph-mon[32510]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:13:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:57 smithi042 ceph-mon[28130]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:57 smithi042 ceph-mon[28130]: from='osd.8 [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:13:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:57 smithi042 ceph-mon[28130]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T00:13:58.044 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:13:58.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:58.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:58.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:58 smithi042 ceph-mon[28130]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:13:58.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:58 smithi042 ceph-mon[28130]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T00:13:58.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:58 smithi042 ceph-mon[28130]: from='osd.8 [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:13:58.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:13:58.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:58 smithi042 ceph-mon[28130]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:13:58.930 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:58.930 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:58.930 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:58 smithi136 ceph-mon[32923]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:13:58.930 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:58 smithi136 ceph-mon[32923]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T00:13:58.930 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:58 smithi136 ceph-mon[32923]: from='osd.8 [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:13:58.930 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:13:58.930 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:58 smithi136 ceph-mon[32923]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:13:59.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:59.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:13:59.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:58 smithi106 ceph-mon[32510]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T00:13:59.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:58 smithi106 ceph-mon[32510]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T00:13:59.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:58 smithi106 ceph-mon[32510]: from='osd.8 [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:13:59.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:13:59.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:58 smithi106 ceph-mon[32510]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:13:59.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:59 smithi042 ceph-mon[28130]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:13:59.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:59 smithi042 ceph-mon[28130]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:13:59.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:59 smithi042 ceph-mon[28130]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:13:59.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:13:59.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:13:59.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:59 smithi042 ceph-mon[28130]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T00:13:59.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:13:59.997 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:13:59 smithi042 ceph-mon[28130]: from='osd.8 ' entity='osd.8' 2024-04-25T00:14:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:59 smithi136 ceph-mon[32923]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:14:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:59 smithi136 ceph-mon[32923]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:14:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:59 smithi136 ceph-mon[32923]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:14:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:59 smithi136 ceph-mon[32923]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T00:14:00.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:00.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:13:59 smithi136 ceph-mon[32923]: from='osd.8 ' entity='osd.8' 2024-04-25T00:14:00.014 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:13:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:13:59.515+0000 7f451db58700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T00:14:00.014 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:13:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:13:59.526+0000 7f4516144700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:14:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:59 smithi106 ceph-mon[32510]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:14:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:59 smithi106 ceph-mon[32510]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:14:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:59 smithi106 ceph-mon[32510]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T00:14:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:59 smithi106 ceph-mon[32510]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T00:14:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:13:59 smithi106 ceph-mon[32510]: from='osd.8 ' entity='osd.8' 2024-04-25T00:14:00.910 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:14:00.911 INFO:teuthology.orchestra.run.smithi042.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:14:00.911 INFO:teuthology.orchestra.run.smithi042.stderr: stderr: 10+0 records in 2024-04-25T00:14:00.911 INFO:teuthology.orchestra.run.smithi042.stderr:10+0 records out 2024-04-25T00:14:00.911 INFO:teuthology.orchestra.run.smithi042.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0341502 s, 307 MB/s 2024-04-25T00:14:00.911 INFO:teuthology.orchestra.run.smithi042.stderr:--> Zapping successful for: 2024-04-25T00:14:00.928 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:00 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:14:00.928 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:00 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:14:00.928 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:00.928 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:00 smithi136 ceph-mon[32923]: osd.8 [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] boot 2024-04-25T00:14:00.928 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:00 smithi136 ceph-mon[32923]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T00:14:00.928 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:00.957 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:00 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:14:00.957 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:00 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:14:00.958 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:00.958 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:00 smithi042 ceph-mon[28130]: osd.8 [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] boot 2024-04-25T00:14:00.958 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:00 smithi042 ceph-mon[28130]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T00:14:00.958 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:01.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:00 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:14:01.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:00 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:14:01.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:01.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:00 smithi106 ceph-mon[32510]: osd.8 [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] boot 2024-04-25T00:14:01.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:00 smithi106 ceph-mon[32510]: osdmap e60: 9 total, 9 up, 9 in 2024-04-25T00:14:01.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T00:14:01.745 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi042:vg_nvme/lv_1 2024-04-25T00:14:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:14:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi136 2024-04-25T00:14:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi136 to 5696M 2024-04-25T00:14:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:14:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:14:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi136 2024-04-25T00:14:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi136 to 5696M 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-04-25T00:14:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi136 2024-04-25T00:14:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi136 to 5696M 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:02.205 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:14:03.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:03 smithi136 ceph-mon[32923]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:03.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:03 smithi136 ceph-mon[32923]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:14:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:03 smithi106 ceph-mon[32510]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:03 smithi106 ceph-mon[32510]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:14:03.613 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:03 smithi042 ceph-mon[28130]: pgmap v219: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:03.613 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:03 smithi042 ceph-mon[28130]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T00:14:04.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.423+0000 7fe99b000700 1 -- 172.21.15.42:0/2142366862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe994101e30 msgr2=0x7fe9941022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.423+0000 7fe99b000700 1 --2- 172.21.15.42:0/2142366862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe994101e30 0x7fe9941022b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe98c00b0a0 tx=0x7fe98c00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.423+0000 7fe99b000700 1 -- 172.21.15.42:0/2142366862 shutdown_connections 2024-04-25T00:14:04.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.423+0000 7fe99b000700 1 --2- 172.21.15.42:0/2142366862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe994101e30 0x7fe9941022b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.423+0000 7fe99b000700 1 --2- 172.21.15.42:0/2142366862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe994100b10 0x7fe994100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.423+0000 7fe99b000700 1 --2- 172.21.15.42:0/2142366862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe994101470 0x7fe9941018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.423+0000 7fe99b000700 1 -- 172.21.15.42:0/2142366862 >> 172.21.15.42:0/2142366862 conn(0x7fe9940fc730 msgr2=0x7fe9940feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:04.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.423+0000 7fe99b000700 1 -- 172.21.15.42:0/2142366862 shutdown_connections 2024-04-25T00:14:04.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.423+0000 7fe99b000700 1 -- 172.21.15.42:0/2142366862 wait complete. 2024-04-25T00:14:04.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.424+0000 7fe99b000700 1 Processor -- start 2024-04-25T00:14:04.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe99b000700 1 -- start start 2024-04-25T00:14:04.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe99b000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe994100b10 0x7fe99419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe99b000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe994101470 0x7fe99419ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe99b000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe994101e30 0x7fe9941a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe99b000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941100a0 con 0x7fe994101470 2024-04-25T00:14:04.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe99b000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe99410ff20 con 0x7fe994100b10 2024-04-25T00:14:04.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe99b000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe994110220 con 0x7fe994101e30 2024-04-25T00:14:04.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe993e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe994100b10 0x7fe99419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe993400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe994101470 0x7fe99419ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe993e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe994100b10 0x7fe99419a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:48552/0 (socket says 172.21.15.42:48552) 2024-04-25T00:14:04.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.425+0000 7fe993e00700 1 -- 172.21.15.42:0/1524313414 learned_addr learned my addr 172.21.15.42:0/1524313414 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:14:04.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.426+0000 7fe999000700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe994101e30 0x7fe9941a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.426+0000 7fe993e00700 1 -- 172.21.15.42:0/1524313414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe994101e30 msgr2=0x7fe9941a1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.426+0000 7fe993e00700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe994101e30 0x7fe9941a1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.426+0000 7fe993e00700 1 -- 172.21.15.42:0/1524313414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe994101470 msgr2=0x7fe99419ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:04.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.426+0000 7fe993e00700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe994101470 0x7fe99419ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:04.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.426+0000 7fe993e00700 1 -- 172.21.15.42:0/1524313414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9941a2340 con 0x7fe994100b10 2024-04-25T00:14:04.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.426+0000 7fe993e00700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe994100b10 0x7fe99419a6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe98400e9a0 tx=0x7fe98400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:04.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.426+0000 7fe990c00700 1 -- 172.21.15.42:0/1524313414 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe98400ca10 con 0x7fe994100b10 2024-04-25T00:14:04.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.426+0000 7fe990c00700 1 -- 172.21.15.42:0/1524313414 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe98400cbb0 con 0x7fe994100b10 2024-04-25T00:14:04.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.426+0000 7fe990c00700 1 -- 172.21.15.42:0/1524313414 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984018710 con 0x7fe994100b10 2024-04-25T00:14:04.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.427+0000 7fe99b000700 1 -- 172.21.15.42:0/1524313414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9941a2630 con 0x7fe994100b10 2024-04-25T00:14:04.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.427+0000 7fe99b000700 1 -- 172.21.15.42:0/1524313414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9941a2b70 con 0x7fe994100b10 2024-04-25T00:14:04.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.428+0000 7fe99b000700 1 -- 172.21.15.42:0/1524313414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe994068f20 con 0x7fe994100b10 2024-04-25T00:14:04.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.430+0000 7fe990c00700 1 -- 172.21.15.42:0/1524313414 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe9840188b0 con 0x7fe994100b10 2024-04-25T00:14:04.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.431+0000 7fe990c00700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9780a2c20 0x7fe9780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.431+0000 7fe990c00700 1 -- 172.21.15.42:0/1524313414 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7fe9840ce8b0 con 0x7fe994100b10 2024-04-25T00:14:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.432+0000 7fe993400700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9780a2c20 0x7fe9780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:04.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.432+0000 7fe993400700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9780a2c20 0x7fe9780a50e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe980007be0 tx=0x7fe9800073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:04.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.435+0000 7fe990c00700 1 -- 172.21.15.42:0/1524313414 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe98409c120 con 0x7fe994100b10 2024-04-25T00:14:04.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:04 smithi136 ceph-mon[32923]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:14:04.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:04 smithi106 ceph-mon[32510]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:14:04.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:04 smithi042 ceph-mon[28130]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T00:14:04.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:04.677+0000 7fe99b000700 1 -- 172.21.15.42:0/1524313414 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe994062670 con 0x7fe9780a2c20 2024-04-25T00:14:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:05 smithi136 ceph-mon[32923]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:05 smithi136 ceph-mon[32923]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T00:14:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:05 smithi106 ceph-mon[32510]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:05 smithi106 ceph-mon[32510]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T00:14:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:05.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:05 smithi042 ceph-mon[28130]: pgmap v222: 1 pgs: 1 remapped+peering; 577 KiB data, 252 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:05.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:05 smithi042 ceph-mon[28130]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T00:14:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:06.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:06 smithi042 ceph-mon[28130]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:06.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:06 smithi136 ceph-mon[32923]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:06.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:06 smithi106 ceph-mon[32510]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi042:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:07.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:07 smithi136 ceph-mon[32923]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-25T00:14:07.520 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:07 smithi042 ceph-mon[28130]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-25T00:14:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:07 smithi106 ceph-mon[32510]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-25T00:14:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:09 smithi136 ceph-mon[32923]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:14:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:09 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2433210461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "179b1772-e33e-4970-9a88-6af3a1eb19dd"}]: dispatch 2024-04-25T00:14:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:09 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2433210461' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "179b1772-e33e-4970-9a88-6af3a1eb19dd"}]': finished 2024-04-25T00:14:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:09 smithi136 ceph-mon[32923]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T00:14:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:09 smithi106 ceph-mon[32510]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:14:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:09 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2433210461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "179b1772-e33e-4970-9a88-6af3a1eb19dd"}]: dispatch 2024-04-25T00:14:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:09 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2433210461' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "179b1772-e33e-4970-9a88-6af3a1eb19dd"}]': finished 2024-04-25T00:14:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:09 smithi106 ceph-mon[32510]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T00:14:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:09.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:09 smithi042 ceph-mon[28130]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 254 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:14:09.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:09 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2433210461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "179b1772-e33e-4970-9a88-6af3a1eb19dd"}]: dispatch 2024-04-25T00:14:09.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:09 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2433210461' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "179b1772-e33e-4970-9a88-6af3a1eb19dd"}]': finished 2024-04-25T00:14:09.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:09 smithi042 ceph-mon[28130]: osdmap e65: 10 total, 9 up, 10 in 2024-04-25T00:14:09.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:10.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:10 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1662822612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:10.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:10 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1662822612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:10.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:10 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1662822612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:11 smithi136 ceph-mon[32923]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T00:14:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:11 smithi106 ceph-mon[32510]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T00:14:11.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:11 smithi042 ceph-mon[28130]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-04-25T00:14:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:13 smithi136 ceph-mon[32923]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T00:14:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:13 smithi106 ceph-mon[32510]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T00:14:13.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:13 smithi042 ceph-mon[28130]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T00:14:15.496 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:15 smithi042 ceph-mon[28130]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T00:14:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:15 smithi106 ceph-mon[32510]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T00:14:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:15 smithi136 ceph-mon[32923]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-04-25T00:14:17.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:17 smithi042 ceph-mon[28130]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:17.590 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:17 smithi106 ceph-mon[32510]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:17 smithi136 ceph-mon[32923]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:19 smithi106 ceph-mon[32510]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:14:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:19 smithi106 ceph-mon[32510]: Deploying daemon osd.9 on smithi042 2024-04-25T00:14:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:19 smithi042 ceph-mon[28130]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:14:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:19 smithi042 ceph-mon[28130]: Deploying daemon osd.9 on smithi042 2024-04-25T00:14:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:19 smithi136 ceph-mon[32923]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T00:14:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:19 smithi136 ceph-mon[32923]: Deploying daemon osd.9 on smithi042 2024-04-25T00:14:21.380 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:21 smithi042 ceph-mon[28130]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:21 smithi106 ceph-mon[32510]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:21.596 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:21 smithi136 ceph-mon[32923]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:23.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:23 smithi106 ceph-mon[32510]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:23.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:23 smithi042 ceph-mon[28130]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:23 smithi136 ceph-mon[32923]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:25.587 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:25 smithi042 ceph-mon[28130]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:25.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:25 smithi106 ceph-mon[32510]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:25 smithi136 ceph-mon[32923]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:27.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:27 smithi106 ceph-mon[32510]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:27.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:27.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:27.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:27 smithi042 ceph-mon[28130]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:27 smithi136 ceph-mon[32923]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:29.453 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:29 smithi042 ceph-mon[28130]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:29.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:29 smithi106 ceph-mon[32510]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:29 smithi136 ceph-mon[32923]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:31.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:31 smithi106 ceph-mon[32510]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:31.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:31 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:31.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:31 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:31.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:31 smithi106 ceph-mon[32510]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:14:31.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:31 smithi042 ceph-mon[28130]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:31.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:31 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:31.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:31 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:31.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:31 smithi042 ceph-mon[28130]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:14:31.726 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.725+0000 7fe990c00700 1 -- 172.21.15.42:0/1524313414 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe994062670 con 0x7fe9780a2c20 2024-04-25T00:14:31.726 INFO:teuthology.orchestra.run.smithi042.stdout:Created osd(s) 9 on host 'smithi042' 2024-04-25T00:14:31.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.729+0000 7fe99b000700 1 -- 172.21.15.42:0/1524313414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9780a2c20 msgr2=0x7fe9780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.729+0000 7fe99b000700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9780a2c20 0x7fe9780a50e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe980007be0 tx=0x7fe9800073d0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.729+0000 7fe99b000700 1 -- 172.21.15.42:0/1524313414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe994100b10 msgr2=0x7fe99419a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:31.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.729+0000 7fe99b000700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe994100b10 0x7fe99419a6e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe98400e9a0 tx=0x7fe98400ecb0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.729+0000 7fe99b000700 1 -- 172.21.15.42:0/1524313414 shutdown_connections 2024-04-25T00:14:31.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.729+0000 7fe99b000700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9780a2c20 0x7fe9780a50e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.729+0000 7fe99b000700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe994101e30 0x7fe9941a1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.729+0000 7fe99b000700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe994100b10 0x7fe99419a6e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.729+0000 7fe99b000700 1 --2- 172.21.15.42:0/1524313414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe994101470 0x7fe99419ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:31.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.729+0000 7fe99b000700 1 -- 172.21.15.42:0/1524313414 >> 172.21.15.42:0/1524313414 conn(0x7fe9940fc730 msgr2=0x7fe9940fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:31.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.730+0000 7fe99b000700 1 -- 172.21.15.42:0/1524313414 shutdown_connections 2024-04-25T00:14:31.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:14:31.730+0000 7fe99b000700 1 -- 172.21.15.42:0/1524313414 wait complete. 2024-04-25T00:14:31.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:31 smithi136 ceph-mon[32923]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:31.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:31 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:31.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:31 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:31.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:31 smithi136 ceph-mon[32923]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T00:14:32.515 DEBUG:teuthology.orchestra.run.smithi042:osd.9> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T00:14:32.521 INFO:tasks.cephadm:Deploying osd.10 on smithi106 with /dev/vg_nvme/lv_1... 2024-04-25T00:14:32.521 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:14:32.767 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:14:32.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:32 smithi106 ceph-mon[32510]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:14:32.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:32 smithi106 ceph-mon[32510]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:14:32.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:32 smithi106 ceph-mon[32510]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:14:32.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:32.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:32.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:32 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:32.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:32 smithi106 ceph-mon[32510]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:33.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:32 smithi136 ceph-mon[32923]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:14:33.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:32 smithi136 ceph-mon[32923]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:14:33.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:32 smithi136 ceph-mon[32923]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:14:33.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:33.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:33.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:32 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:33.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:32 smithi136 ceph-mon[32923]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:33.078 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:32 smithi042 ceph-mon[28130]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T00:14:33.078 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:32 smithi042 ceph-mon[28130]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T00:14:33.078 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:32 smithi042 ceph-mon[28130]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]: dispatch 2024-04-25T00:14:33.079 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:33.079 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:33.079 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:32 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:33.079 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:32 smithi042 ceph-mon[28130]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:33 smithi136 ceph-mon[32923]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:14:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:33 smithi136 ceph-mon[32923]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T00:14:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:33 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:33 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:33 smithi136 ceph-mon[32923]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' 2024-04-25T00:14:34.024 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:33 smithi106 ceph-mon[32510]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:14:34.024 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:33 smithi106 ceph-mon[32510]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T00:14:34.024 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:33 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:34.024 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:33 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:34.024 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:33 smithi106 ceph-mon[32510]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' 2024-04-25T00:14:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:33 smithi042 ceph-mon[28130]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi042", "root=default"]}]': finished 2024-04-25T00:14:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:33 smithi042 ceph-mon[28130]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T00:14:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:33 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:33 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:33 smithi042 ceph-mon[28130]: from='osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842]' entity='osd.9' 2024-04-25T00:14:34.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:14:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:14:33.673+0000 7f166e200700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T00:14:34.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:14:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:14:33.708+0000 7f1662c00700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:14:34.696 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:14:34.696 INFO:teuthology.orchestra.run.smithi106.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:14:34.696 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10+0 records in 2024-04-25T00:14:34.697 INFO:teuthology.orchestra.run.smithi106.stderr:10+0 records out 2024-04-25T00:14:34.697 INFO:teuthology.orchestra.run.smithi106.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114326 s, 917 MB/s 2024-04-25T00:14:34.697 INFO:teuthology.orchestra.run.smithi106.stderr:--> Zapping successful for: 2024-04-25T00:14:34.822 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:34 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:14:34.822 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:34 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:14:34.822 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:34 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:34.822 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:34 smithi042 ceph-mon[28130]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:35.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:34 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:14:35.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:34 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:14:35.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:34 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:35.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:34 smithi136 ceph-mon[32923]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:35.035 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:34 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:14:35.035 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:34 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:14:35.035 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:34 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:35.035 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:34 smithi106 ceph-mon[32510]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 804 GiB / 805 GiB avail 2024-04-25T00:14:35.322 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi106:vg_nvme/lv_1 2024-04-25T00:14:35.566 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:14:35.914 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:35 smithi106 ceph-mon[32510]: osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] boot 2024-04-25T00:14:35.914 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:35 smithi106 ceph-mon[32510]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T00:14:35.914 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:35 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:35 smithi136 ceph-mon[32923]: osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] boot 2024-04-25T00:14:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:35 smithi136 ceph-mon[32923]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T00:14:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:35 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:35 smithi042 ceph-mon[28130]: osd.9 [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] boot 2024-04-25T00:14:36.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:35 smithi042 ceph-mon[28130]: osdmap e68: 10 total, 10 up, 10 in 2024-04-25T00:14:36.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:35 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T00:14:36.921 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:36 smithi106 ceph-mon[32510]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T00:14:36.921 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:36 smithi106 ceph-mon[32510]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:36 smithi136 ceph-mon[32923]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T00:14:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:36 smithi136 ceph-mon[32923]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:37.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:36 smithi042 ceph-mon[28130]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T00:14:37.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:36 smithi042 ceph-mon[28130]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 288 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2465189328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c1011a0 msgr2=0x7f8d1c1015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 --2- 172.21.15.106:0/2465189328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c1011a0 0x7f8d1c1015c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8d100099c0 tx=0x7f8d10009cd0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2465189328 shutdown_connections 2024-04-25T00:14:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 --2- 172.21.15.106:0/2465189328 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d1c102d40 0x7f8d1c1095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 --2- 172.21.15.106:0/2465189328 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c1011a0 0x7f8d1c1015c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 --2- 172.21.15.106:0/2465189328 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d1c1023a0 0x7f8d1c102800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2465189328 >> 172.21.15.106:0/2465189328 conn(0x7f8d1c0fc760 msgr2=0x7f8d1c0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2465189328 shutdown_connections 2024-04-25T00:14:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2465189328 wait complete. 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 Processor -- start 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 -- start start 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c1011a0 0x7f8d1c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d1c1023a0 0x7f8d1c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d1c102d40 0x7f8d1c19fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10ceb0 con 0x7f8d1c102d40 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10cd30 con 0x7f8d1c1011a0 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d22bd3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d1c10d030 con 0x7f8d1c1023a0 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.149+0000 7f8d2096f700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c1011a0 0x7f8d1c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d2096f700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c1011a0 0x7f8d1c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:50520/0 (socket says 172.21.15.106:50520) 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d2096f700 1 -- 172.21.15.106:0/2552029796 learned_addr learned my addr 172.21.15.106:0/2552029796 (peer_addr_for_me v2:172.21.15.106:0/0) 2024-04-25T00:14:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d1bfff700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d1c1023a0 0x7f8d1c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d2096f700 1 -- 172.21.15.106:0/2552029796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d1c1023a0 msgr2=0x7f8d1c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d2096f700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d1c1023a0 0x7f8d1c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d2096f700 1 -- 172.21.15.106:0/2552029796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d1c102d40 msgr2=0x7f8d1c19fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d2096f700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d1c102d40 0x7f8d1c19fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d2096f700 1 -- 172.21.15.106:0/2552029796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1c1a0310 con 0x7f8d1c1011a0 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d2096f700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c1011a0 0x7f8d1c198700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d10003890 tx=0x7f8d10005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d19ffb700 1 -- 172.21.15.106:0/2552029796 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d10027070 con 0x7f8d1c1011a0 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2552029796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1c1a05a0 con 0x7f8d1c1011a0 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d19ffb700 1 -- 172.21.15.106:0/2552029796 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d10004210 con 0x7f8d1c1011a0 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d19ffb700 1 -- 172.21.15.106:0/2552029796 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1000f460 con 0x7f8d1c1011a0 2024-04-25T00:14:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.150+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2552029796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d1c1a0ad0 con 0x7f8d1c1011a0 2024-04-25T00:14:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.151+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2552029796 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d1c068f20 con 0x7f8d1c1011a0 2024-04-25T00:14:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.152+0000 7f8d19ffb700 1 -- 172.21.15.106:0/2552029796 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8d1002a070 con 0x7f8d1c1011a0 2024-04-25T00:14:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.153+0000 7f8d19ffb700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d0c0a2c20 0x7f8d0c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:14:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.153+0000 7f8d19ffb700 1 -- 172.21.15.106:0/2552029796 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f8d100cf900 con 0x7f8d1c1011a0 2024-04-25T00:14:37.154 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.153+0000 7f8d1bfff700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d0c0a2c20 0x7f8d0c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:14:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.154+0000 7f8d1bfff700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d0c0a2c20 0x7f8d0c0a50e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009a10 tx=0x7f8d08009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:14:37.157 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.157+0000 7f8d19ffb700 1 -- 172.21.15.106:0/2552029796 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8d1005e740 con 0x7f8d1c1011a0 2024-04-25T00:14:37.364 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:37.363+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2552029796 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8d1c06b090 con 0x7f8d0c0a2c20 2024-04-25T00:14:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:37 smithi042 ceph-mon[28130]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:37.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:37.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:37.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:37.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:37.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:37.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:38.003 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:37 smithi106 ceph-mon[32510]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:38.003 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:38.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:38.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:38.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:38.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:38.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:38.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:38.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:38.004 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:37 smithi136 ceph-mon[32923]: from='client.24362 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi106:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:14:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:14:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:14:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:14:38.969 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi042 2024-04-25T00:14:38.969 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi042 to 4273M 2024-04-25T00:14:38.969 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:38.969 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:38.969 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:38.969 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:38.969 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:38.969 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:38.970 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:38.970 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:38.970 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:38 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi042 2024-04-25T00:14:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi042 to 4273M 2024-04-25T00:14:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:39.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:39.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:39.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:38 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi042 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi042 to 4273M 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:38 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:41.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:40 smithi136 ceph-mon[32923]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:41.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:40 smithi136 ceph-mon[32923]: from='client.? 172.21.15.106:0/486699523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc443ab7-2aea-427e-9bfd-d25f4f4dc239"}]: dispatch 2024-04-25T00:14:41.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:40 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc443ab7-2aea-427e-9bfd-d25f4f4dc239"}]: dispatch 2024-04-25T00:14:41.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:40 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc443ab7-2aea-427e-9bfd-d25f4f4dc239"}]': finished 2024-04-25T00:14:41.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:40 smithi136 ceph-mon[32923]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T00:14:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:14:41.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:40 smithi136 ceph-mon[32923]: from='client.? 172.21.15.106:0/2871870500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:40 smithi106 ceph-mon[32510]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:40 smithi106 ceph-mon[32510]: from='client.? 172.21.15.106:0/486699523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc443ab7-2aea-427e-9bfd-d25f4f4dc239"}]: dispatch 2024-04-25T00:14:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:40 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc443ab7-2aea-427e-9bfd-d25f4f4dc239"}]: dispatch 2024-04-25T00:14:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:40 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc443ab7-2aea-427e-9bfd-d25f4f4dc239"}]': finished 2024-04-25T00:14:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:40 smithi106 ceph-mon[32510]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T00:14:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:14:41.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:40 smithi106 ceph-mon[32510]: from='client.? 172.21.15.106:0/2871870500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:41.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:40 smithi042 ceph-mon[28130]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:41.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:40 smithi042 ceph-mon[28130]: from='client.? 172.21.15.106:0/486699523' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc443ab7-2aea-427e-9bfd-d25f4f4dc239"}]: dispatch 2024-04-25T00:14:41.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:40 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc443ab7-2aea-427e-9bfd-d25f4f4dc239"}]: dispatch 2024-04-25T00:14:41.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:40 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc443ab7-2aea-427e-9bfd-d25f4f4dc239"}]': finished 2024-04-25T00:14:41.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:40 smithi042 ceph-mon[28130]: osdmap e70: 11 total, 10 up, 11 in 2024-04-25T00:14:41.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:14:41.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:40 smithi042 ceph-mon[28130]: from='client.? 172.21.15.106:0/2871870500' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:14:43.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:42 smithi136 ceph-mon[32923]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:43.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:42 smithi106 ceph-mon[32510]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:43.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:42 smithi042 ceph-mon[28130]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:45.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:44 smithi136 ceph-mon[32923]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:45.283 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:44 smithi106 ceph-mon[32510]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:45.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:44 smithi042 ceph-mon[28130]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:47.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:46 smithi136 ceph-mon[32923]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:47.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:46 smithi106 ceph-mon[32510]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:47.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:46 smithi042 ceph-mon[28130]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:14:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:14:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:48.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T00:14:48.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:49.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:48 smithi136 ceph-mon[32923]: Deploying daemon osd.10 on smithi106 2024-04-25T00:14:49.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:48 smithi136 ceph-mon[32923]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:49.314 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:48 smithi106 ceph-mon[32510]: Deploying daemon osd.10 on smithi106 2024-04-25T00:14:49.314 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:48 smithi106 ceph-mon[32510]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:49.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:48 smithi042 ceph-mon[28130]: Deploying daemon osd.10 on smithi106 2024-04-25T00:14:49.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:48 smithi042 ceph-mon[28130]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:51.231 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:50 smithi106 ceph-mon[32510]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:51.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:50 smithi136 ceph-mon[32923]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:51.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:50 smithi042 ceph-mon[28130]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:53.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:52 smithi106 ceph-mon[32510]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:53.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:52 smithi136 ceph-mon[32923]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:53.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:52 smithi042 ceph-mon[28130]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:54.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:54.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:54.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:54.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:55.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:55 smithi136 ceph-mon[32923]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:55 smithi106 ceph-mon[32510]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:55 smithi042 ceph-mon[28130]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:57.200 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.199+0000 7f8d19ffb700 1 -- 172.21.15.106:0/2552029796 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f8d1c06b090 con 0x7f8d0c0a2c20 2024-04-25T00:14:57.200 INFO:teuthology.orchestra.run.smithi106.stdout:Created osd(s) 10 on host 'smithi106' 2024-04-25T00:14:57.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2552029796 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d0c0a2c20 msgr2=0x7f8d0c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.202 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d0c0a2c20 0x7f8d0c0a50e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8d08009a10 tx=0x7f8d08009190 comp rx=0 tx=0).stop 2024-04-25T00:14:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2552029796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c1011a0 msgr2=0x7f8d1c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:14:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c1011a0 0x7f8d1c198700 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8d10003890 tx=0x7f8d10005e80 comp rx=0 tx=0).stop 2024-04-25T00:14:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2552029796 shutdown_connections 2024-04-25T00:14:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d0c0a2c20 0x7f8d0c0a50e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d1c102d40 0x7f8d1c19fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d1c1011a0 0x7f8d1c198700 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 --2- 172.21.15.106:0/2552029796 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d1c1023a0 0x7f8d1c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:14:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2552029796 >> 172.21.15.106:0/2552029796 conn(0x7f8d1c0fc760 msgr2=0x7f8d1c06a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:14:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2552029796 shutdown_connections 2024-04-25T00:14:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2024-04-25T00:14:57.202+0000 7f8d22bd3700 1 -- 172.21.15.106:0/2552029796 wait complete. 2024-04-25T00:14:57.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:57.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:57.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:57.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.711 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:57.712 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: from='osd.10 [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:14:57.712 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:57 smithi106 ceph-mon[32510]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:14:57.737 DEBUG:teuthology.orchestra.run.smithi106:osd.10> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T00:14:57.739 INFO:tasks.cephadm:Deploying osd.11 on smithi136 with /dev/vg_nvme/lv_1... 2024-04-25T00:14:57.740 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T00:14:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:57.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: from='osd.10 [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:14:57.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:57 smithi136 ceph-mon[32923]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:14:57.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:57.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:14:57.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:14:57.879 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.879 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.879 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:14:57.879 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:14:57.879 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: from='osd.10 [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:14:57.879 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:57 smithi042 ceph-mon[28130]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T00:14:57.984 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:14:59.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:59 smithi136 ceph-mon[32923]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:59.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:59 smithi136 ceph-mon[32923]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:14:59.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:59 smithi136 ceph-mon[32923]: from='osd.10 [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:14:59.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:59 smithi136 ceph-mon[32923]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:14:59.346 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:14:59.347 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:14:59 smithi136 ceph-mon[32923]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:14:59.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:59 smithi106 ceph-mon[32510]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:59.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:59 smithi106 ceph-mon[32510]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:14:59.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:59 smithi106 ceph-mon[32510]: from='osd.10 [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:14:59.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:59 smithi106 ceph-mon[32510]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:14:59.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:14:59.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:14:59 smithi106 ceph-mon[32510]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:14:59.589 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:14:59 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:14:59.194+0000 7ff6c12f1700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T00:14:59.589 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:14:59 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:14:59.209+0000 7ff6bd8e5700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:14:59.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:59 smithi042 ceph-mon[28130]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:14:59.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:59 smithi042 ceph-mon[28130]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T00:14:59.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:59 smithi042 ceph-mon[28130]: from='osd.10 [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:14:59.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:59 smithi042 ceph-mon[28130]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T00:14:59.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:14:59.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:14:59 smithi042 ceph-mon[28130]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]: dispatch 2024-04-25T00:14:59.889 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T00:14:59.889 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T00:14:59.889 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-04-25T00:14:59.889 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-04-25T00:14:59.889 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00994592 s, 1.1 GB/s 2024-04-25T00:14:59.889 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 2024-04-25T00:14:59.889 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-04-25T00:15:00.306 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:00 smithi106 ceph-mon[32510]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:15:00.306 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:00 smithi106 ceph-mon[32510]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T00:15:00.306 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:15:00.306 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:15:00.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:00 smithi106 ceph-mon[32510]: osd.10 [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] boot 2024-04-25T00:15:00.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:00 smithi106 ceph-mon[32510]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T00:15:00.307 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:15:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:00 smithi136 ceph-mon[32923]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:15:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:00 smithi136 ceph-mon[32923]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T00:15:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:15:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:15:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:00 smithi136 ceph-mon[32923]: osd.10 [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] boot 2024-04-25T00:15:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:00 smithi136 ceph-mon[32923]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T00:15:00.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:15:00.542 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_1 2024-04-25T00:15:00.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:00 smithi042 ceph-mon[28130]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi106", "root=default"]}]': finished 2024-04-25T00:15:00.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:00 smithi042 ceph-mon[28130]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T00:15:00.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:15:00.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:15:00.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:00 smithi042 ceph-mon[28130]: osd.10 [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] boot 2024-04-25T00:15:00.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:00 smithi042 ceph-mon[28130]: osdmap e73: 11 total, 11 up, 11 in 2024-04-25T00:15:00.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T00:15:00.830 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:15:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:15:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:15:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:15:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:15:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:01.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:01.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:15:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:15:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 289 MiB used, 894 GiB / 894 GiB avail 2024-04-25T00:15:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: osdmap e74: 11 total, 11 up, 11 in 2024-04-25T00:15:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:01.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:01.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:01.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:01.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:02.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.372+0000 7f52dd24b700 1 -- 172.21.15.136:0/3021871592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8107f80 msgr2=0x7f52d810a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.372+0000 7f52dd24b700 1 --2- 172.21.15.136:0/3021871592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8107f80 0x7f52d810a3f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f52d000b0a0 tx=0x7f52d000b3b0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.372+0000 7f52dd24b700 1 -- 172.21.15.136:0/3021871592 shutdown_connections 2024-04-25T00:15:02.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.372+0000 7f52dd24b700 1 --2- 172.21.15.136:0/3021871592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8107f80 0x7f52d810a3f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.375 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.372+0000 7f52dd24b700 1 --2- 172.21.15.136:0/3021871592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d8102d20 0x7f52d8105110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.372+0000 7f52dd24b700 1 --2- 172.21.15.136:0/3021871592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d8105650 0x7f52d8107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.372+0000 7f52dd24b700 1 -- 172.21.15.136:0/3021871592 >> 172.21.15.136:0/3021871592 conn(0x7f52d80fc750 msgr2=0x7f52d80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:02.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.372+0000 7f52dd24b700 1 -- 172.21.15.136:0/3021871592 shutdown_connections 2024-04-25T00:15:02.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.372+0000 7f52dd24b700 1 -- 172.21.15.136:0/3021871592 wait complete. 2024-04-25T00:15:02.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52dd24b700 1 Processor -- start 2024-04-25T00:15:02.376 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52dd24b700 1 -- start start 2024-04-25T00:15:02.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52dd24b700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d8102d20 0x7f52d81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52dd24b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8105650 0x7f52d81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52dd24b700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d8107f80 0x7f52d819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52dd24b700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f52d810dec0 con 0x7f52d8107f80 2024-04-25T00:15:02.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52dd24b700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f52d810dd40 con 0x7f52d8102d20 2024-04-25T00:15:02.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52dd24b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f52d810e040 con 0x7f52d8105650 2024-04-25T00:15:02.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52ce59c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8105650 0x7f52d81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.377 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52ce59c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8105650 0x7f52d81989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:54348/0 (socket says 172.21.15.136:54348) 2024-04-25T00:15:02.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52ce59c700 1 -- 172.21.15.136:0/1916940976 learned_addr learned my addr 172.21.15.136:0/1916940976 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:15:02.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52d6d9d700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d8102d20 0x7f52d81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52ce59c700 1 -- 172.21.15.136:0/1916940976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d8102d20 msgr2=0x7f52d81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52ce59c700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d8102d20 0x7f52d81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52ce59c700 1 -- 172.21.15.136:0/1916940976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d8107f80 msgr2=0x7f52d819fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:02.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.373+0000 7f52ce59c700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d8107f80 0x7f52d819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:02.378 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.374+0000 7f52ce59c700 1 -- 172.21.15.136:0/1916940976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d81a0100 con 0x7f52d8105650 2024-04-25T00:15:02.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.374+0000 7f52ce59c700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8105650 0x7f52d81989e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b560 tx=0x7f52c000b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.374+0000 7f52d4d99700 1 -- 172.21.15.136:0/1916940976 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c0015020 con 0x7f52d8105650 2024-04-25T00:15:02.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.374+0000 7f52d4d99700 1 -- 172.21.15.136:0/1916940976 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52c0011d10 con 0x7f52d8105650 2024-04-25T00:15:02.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.374+0000 7f52dd24b700 1 -- 172.21.15.136:0/1916940976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d81a03f0 con 0x7f52d8105650 2024-04-25T00:15:02.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.374+0000 7f52d4d99700 1 -- 172.21.15.136:0/1916940976 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c000f2e0 con 0x7f52d8105650 2024-04-25T00:15:02.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.374+0000 7f52dd24b700 1 -- 172.21.15.136:0/1916940976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d81a0930 con 0x7f52d8105650 2024-04-25T00:15:02.379 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.375+0000 7f52dd24b700 1 -- 172.21.15.136:0/1916940976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d8068f20 con 0x7f52d8105650 2024-04-25T00:15:02.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.376+0000 7f52d4d99700 1 -- 172.21.15.136:0/1916940976 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f52c0018020 con 0x7f52d8105650 2024-04-25T00:15:02.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.376+0000 7f52d4d99700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f52c40a2c20 0x7f52c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:02.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.377+0000 7f52d4d99700 1 -- 172.21.15.136:0/1916940976 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7f52c00ce6d0 con 0x7f52d8105650 2024-04-25T00:15:02.381 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.377+0000 7f52d6d9d700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f52c40a2c20 0x7f52c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:02.382 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.378+0000 7f52d6d9d700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f52c40a2c20 0x7f52c40a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c800a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:02.382 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.380+0000 7f52d4d99700 1 -- 172.21.15.136:0/1916940976 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f52c001f020 con 0x7f52d8105650 2024-04-25T00:15:02.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:02 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi106 2024-04-25T00:15:02.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:02 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-25T00:15:02.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:02 smithi136 ceph-mon[32923]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:15:02.586 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:02.586+0000 7f52dd24b700 1 -- 172.21.15.136:0/1916940976 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f52d8062670 con 0x7f52c40a2c20 2024-04-25T00:15:02.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:02 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi106 2024-04-25T00:15:02.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:02 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-25T00:15:02.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:02 smithi106 ceph-mon[32510]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:15:02.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:02 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi106 2024-04-25T00:15:02.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:02 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi106 to 4272M 2024-04-25T00:15:02.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:02 smithi042 ceph-mon[28130]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T00:15:03.480 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:03 smithi136 ceph-mon[32923]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:03.480 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:03 smithi136 ceph-mon[32923]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:15:03.480 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:03 smithi136 ceph-mon[32923]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:03.480 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:03 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:15:03.480 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:03 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:15:03.480 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:03 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:03 smithi106 ceph-mon[32510]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:03 smithi106 ceph-mon[32510]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:15:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:03 smithi106 ceph-mon[32510]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:03 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:15:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:03 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:15:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:03 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:03.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:03 smithi042 ceph-mon[28130]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 315 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:03.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:03 smithi042 ceph-mon[28130]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T00:15:03.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:03 smithi042 ceph-mon[28130]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:03.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:03 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T00:15:03.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:03 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T00:15:03.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:03 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:04.397 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:04 smithi136 ceph-mon[32923]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T00:15:04.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:04 smithi106 ceph-mon[32510]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T00:15:04.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:04 smithi042 ceph-mon[28130]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T00:15:05.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:05 smithi136 ceph-mon[32923]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:05.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:05 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/507058436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbdadbeb-aa71-49d8-bffb-4977f2dbaa44"}]: dispatch 2024-04-25T00:15:05.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:05 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbdadbeb-aa71-49d8-bffb-4977f2dbaa44"}]: dispatch 2024-04-25T00:15:05.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:05 smithi136 ceph-mon[32923]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbdadbeb-aa71-49d8-bffb-4977f2dbaa44"}]': finished 2024-04-25T00:15:05.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:05 smithi136 ceph-mon[32923]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T00:15:05.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:05 smithi106 ceph-mon[32510]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:05 smithi106 ceph-mon[32510]: from='client.? 172.21.15.136:0/507058436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbdadbeb-aa71-49d8-bffb-4977f2dbaa44"}]: dispatch 2024-04-25T00:15:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:05 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbdadbeb-aa71-49d8-bffb-4977f2dbaa44"}]: dispatch 2024-04-25T00:15:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:05 smithi106 ceph-mon[32510]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbdadbeb-aa71-49d8-bffb-4977f2dbaa44"}]': finished 2024-04-25T00:15:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:05 smithi106 ceph-mon[32510]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T00:15:05.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:05 smithi042 ceph-mon[28130]: pgmap v266: 1 pgs: 1 remapped+peering; 577 KiB data, 316 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:05 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/507058436' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbdadbeb-aa71-49d8-bffb-4977f2dbaa44"}]: dispatch 2024-04-25T00:15:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:05 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbdadbeb-aa71-49d8-bffb-4977f2dbaa44"}]: dispatch 2024-04-25T00:15:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:05 smithi042 ceph-mon[28130]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbdadbeb-aa71-49d8-bffb-4977f2dbaa44"}]': finished 2024-04-25T00:15:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:05 smithi042 ceph-mon[28130]: osdmap e78: 12 total, 11 up, 12 in 2024-04-25T00:15:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:06.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:06 smithi106 ceph-mon[32510]: from='client.? 172.21.15.136:0/1075078923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:15:06.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:06 smithi042 ceph-mon[28130]: from='client.? 172.21.15.136:0/1075078923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:15:06.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:06 smithi136 ceph-mon[32923]: from='client.? 172.21.15.136:0/1075078923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T00:15:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:07 smithi106 ceph-mon[32510]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-25T00:15:07.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:07 smithi042 ceph-mon[28130]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-25T00:15:07.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:07 smithi136 ceph-mon[32923]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 124 KiB/s, 0 objects/s recovering 2024-04-25T00:15:09.441 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:09 smithi136 ceph-mon[32923]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:15:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:09 smithi106 ceph-mon[32510]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:15:09.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:09 smithi042 ceph-mon[28130]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T00:15:11.464 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:11 smithi136 ceph-mon[32923]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T00:15:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:11 smithi106 ceph-mon[32510]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T00:15:11.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:11 smithi042 ceph-mon[28130]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-04-25T00:15:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:13 smithi136 ceph-mon[32923]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T00:15:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:15:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:13 smithi136 ceph-mon[32923]: Deploying daemon osd.11 on smithi136 2024-04-25T00:15:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:13 smithi106 ceph-mon[32510]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T00:15:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:15:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:13 smithi106 ceph-mon[32510]: Deploying daemon osd.11 on smithi136 2024-04-25T00:15:13.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:13 smithi042 ceph-mon[28130]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-04-25T00:15:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T00:15:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:13 smithi042 ceph-mon[28130]: Deploying daemon osd.11 on smithi136 2024-04-25T00:15:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:15 smithi136 ceph-mon[32923]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:15:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:15 smithi106 ceph-mon[32510]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:15:15.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:15 smithi042 ceph-mon[28130]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-25T00:15:17.559 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:17 smithi136 ceph-mon[32923]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T00:15:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:17 smithi106 ceph-mon[32510]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T00:15:17.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:17 smithi042 ceph-mon[28130]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-04-25T00:15:19.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:19 smithi136 ceph-mon[32923]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:19.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:19.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:19.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:19 smithi106 ceph-mon[32510]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:19.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:19.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:19 smithi042 ceph-mon[28130]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:21.470 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:21 smithi136 ceph-mon[32923]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:21.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:21.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:21.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:21.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:21.471 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:21 smithi106 ceph-mon[32510]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:21 smithi042 ceph-mon[28130]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:21.874 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 11 on host 'smithi136' 2024-04-25T00:15:21.875 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.873+0000 7f52d4d99700 1 -- 172.21.15.136:0/1916940976 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f52d8062670 con 0x7f52c40a2c20 2024-04-25T00:15:21.876 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.875+0000 7f52dd24b700 1 -- 172.21.15.136:0/1916940976 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f52c40a2c20 msgr2=0x7f52c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.875+0000 7f52dd24b700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f52c40a2c20 0x7f52c40a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f52c8005fd0 tx=0x7f52c800a560 comp rx=0 tx=0).stop 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.875+0000 7f52dd24b700 1 -- 172.21.15.136:0/1916940976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8105650 msgr2=0x7f52d81989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.875+0000 7f52dd24b700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8105650 0x7f52d81989e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f52c000b560 tx=0x7f52c000b920 comp rx=0 tx=0).stop 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.876+0000 7f52dd24b700 1 -- 172.21.15.136:0/1916940976 shutdown_connections 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.876+0000 7f52dd24b700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f52c40a2c20 0x7f52c40a50e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.876+0000 7f52dd24b700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f52d8107f80 0x7f52d819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.876+0000 7f52dd24b700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52d8102d20 0x7f52d81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.876+0000 7f52dd24b700 1 --2- 172.21.15.136:0/1916940976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52d8105650 0x7f52d81989e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.876+0000 7f52dd24b700 1 -- 172.21.15.136:0/1916940976 >> 172.21.15.136:0/1916940976 conn(0x7f52d80fc750 msgr2=0x7f52d8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.876+0000 7f52dd24b700 1 -- 172.21.15.136:0/1916940976 shutdown_connections 2024-04-25T00:15:21.877 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:21.876+0000 7f52dd24b700 1 -- 172.21.15.136:0/1916940976 wait complete. 2024-04-25T00:15:22.476 DEBUG:teuthology.orchestra.run.smithi136:osd.11> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T00:15:22.478 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T00:15:22.478 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T00:15:22.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:22 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:22.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:22 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:22.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:22 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:22.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:22 smithi042 ceph-mon[28130]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:22.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:22 smithi042 ceph-mon[28130]: from='osd.11 [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:15:22.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:22 smithi042 ceph-mon[28130]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:15:22.945 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:23.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:22 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:23.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:22 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:23.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:22 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:23.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:22 smithi106 ceph-mon[32510]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:23.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:22 smithi106 ceph-mon[32510]: from='osd.11 [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:15:23.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:22 smithi106 ceph-mon[32510]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:15:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:22 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:22 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:22 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:22 smithi136 ceph-mon[32923]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:23.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:22 smithi136 ceph-mon[32923]: from='osd.11 [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:15:23.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:22 smithi136 ceph-mon[32923]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T00:15:23.899 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:23 smithi136 ceph-mon[32923]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:15:23.899 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:23 smithi136 ceph-mon[32923]: from='osd.11 [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:15:23.899 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:23 smithi136 ceph-mon[32923]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:15:23.899 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:23.899 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:23 smithi136 ceph-mon[32923]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:15:23.899 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:15:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:15:23.832+0000 7f3e149f7700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T00:15:23.899 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:15:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:15:23.846+0000 7f3e10feb700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T00:15:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:23 smithi106 ceph-mon[32510]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:15:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:23 smithi106 ceph-mon[32510]: from='osd.11 [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:15:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:23 smithi106 ceph-mon[32510]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:15:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:23 smithi106 ceph-mon[32510]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:15:24.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:23 smithi042 ceph-mon[28130]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T00:15:24.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:23 smithi042 ceph-mon[28130]: from='osd.11 [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:15:24.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:23 smithi042 ceph-mon[28130]: osdmap e79: 12 total, 11 up, 12 in 2024-04-25T00:15:24.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:24.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:23 smithi042 ceph-mon[28130]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-04-25T00:15:24.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:24 smithi136 ceph-mon[32923]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:15:24.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:24 smithi136 ceph-mon[32923]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T00:15:24.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:24 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:24.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:24 smithi136 ceph-mon[32923]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:24.993 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:24 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:25.098 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:24 smithi042 ceph-mon[28130]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:15:25.098 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:24 smithi042 ceph-mon[28130]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T00:15:25.098 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:24 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:25.099 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:24 smithi042 ceph-mon[28130]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:25.099 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:24 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:25.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.123+0000 7f701b000700 1 -- 172.21.15.42:0/224777099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7014105ba0 msgr2=0x7f7014105fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.123+0000 7f701b000700 1 --2- 172.21.15.42:0/224777099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7014105ba0 0x7f7014105fc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7004009990 tx=0x7f7004009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.123+0000 7f701b000700 1 -- 172.21.15.42:0/224777099 shutdown_connections 2024-04-25T00:15:25.124 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.123+0000 7f701b000700 1 --2- 172.21.15.42:0/224777099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7014104a80 0x7f701410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.123+0000 7f701b000700 1 --2- 172.21.15.42:0/224777099 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7014105ba0 0x7f7014105fc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.123+0000 7f701b000700 1 --2- 172.21.15.42:0/224777099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70141040e0 0x7f7014104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.123+0000 7f701b000700 1 -- 172.21.15.42:0/224777099 >> 172.21.15.42:0/224777099 conn(0x7f70140ffa80 msgr2=0x7f7014101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:25.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.123+0000 7f701b000700 1 -- 172.21.15.42:0/224777099 shutdown_connections 2024-04-25T00:15:25.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.123+0000 7f701b000700 1 -- 172.21.15.42:0/224777099 wait complete. 2024-04-25T00:15:25.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.124+0000 7f701b000700 1 Processor -- start 2024-04-25T00:15:25.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.124+0000 7f701b000700 1 -- start start 2024-04-25T00:15:25.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f701b000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70141040e0 0x7f701419a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f701b000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7014104a80 0x7f701419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f701b000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7014105ba0 0x7f70141a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f701b000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f70141101d0 con 0x7f7014105ba0 2024-04-25T00:15:25.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f701b000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7014110050 con 0x7f70141040e0 2024-04-25T00:15:25.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f701b000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7014110350 con 0x7f7014104a80 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f7013e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70141040e0 0x7f701419a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f7019000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7014105ba0 0x7f70141a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f7013e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70141040e0 0x7f701419a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49938/0 (socket says 172.21.15.42:49938) 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f7013e00700 1 -- 172.21.15.42:0/3197797922 learned_addr learned my addr 172.21.15.42:0/3197797922 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f7019000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7014105ba0 0x7f70141a1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34358/0 (socket says 172.21.15.42:34358) 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f7013400700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7014104a80 0x7f701419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.125+0000 7f7013e00700 1 -- 172.21.15.42:0/3197797922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7014104a80 msgr2=0x7f701419ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f7013e00700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7014104a80 0x7f701419ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f7013e00700 1 -- 172.21.15.42:0/3197797922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7014105ba0 msgr2=0x7f70141a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f7013e00700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7014105ba0 0x7f70141a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f7013e00700 1 -- 172.21.15.42:0/3197797922 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70141a2440 con 0x7f70141040e0 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f7019000700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7014105ba0 0x7f70141a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:25.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f7013400700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7014104a80 0x7f701419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:25.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f7013e00700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70141040e0 0x7f701419a7e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7004000c00 tx=0x7f7004009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f7010c00700 1 -- 172.21.15.42:0/3197797922 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004026070 con 0x7f70141040e0 2024-04-25T00:15:25.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f701b000700 1 -- 172.21.15.42:0/3197797922 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70141a26d0 con 0x7f70141040e0 2024-04-25T00:15:25.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f7010c00700 1 -- 172.21.15.42:0/3197797922 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f700400fde0 con 0x7f70141040e0 2024-04-25T00:15:25.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f7010c00700 1 -- 172.21.15.42:0/3197797922 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004017650 con 0x7f70141040e0 2024-04-25T00:15:25.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.126+0000 7f701b000700 1 -- 172.21.15.42:0/3197797922 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70141a2b20 con 0x7f70141040e0 2024-04-25T00:15:25.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.127+0000 7f701b000700 1 -- 172.21.15.42:0/3197797922 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7014068f20 con 0x7f70141040e0 2024-04-25T00:15:25.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.130+0000 7f7010c00700 1 -- 172.21.15.42:0/3197797922 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7004029070 con 0x7f70141040e0 2024-04-25T00:15:25.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.131+0000 7f7010c00700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6ff80a2c20 0x7f6ff80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:25.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.131+0000 7f7010c00700 1 -- 172.21.15.42:0/3197797922 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f70040cfde0 con 0x7f70141040e0 2024-04-25T00:15:25.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.131+0000 7f7013400700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6ff80a2c20 0x7f6ff80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:25.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.132+0000 7f7013400700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6ff80a2c20 0x7f6ff80a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7000009680 tx=0x7f7000009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:25.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.133+0000 7f7010c00700 1 -- 172.21.15.42:0/3197797922 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7004030080 con 0x7f70141040e0 2024-04-25T00:15:25.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:24 smithi106 ceph-mon[32510]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-04-25T00:15:25.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:24 smithi106 ceph-mon[32510]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T00:15:25.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:24 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:25.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:24 smithi106 ceph-mon[32510]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-04-25T00:15:25.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:24 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:25.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.372+0000 7f701b000700 1 -- 172.21.15.42:0/3197797922 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7014062e40 con 0x7f70141040e0 2024-04-25T00:15:25.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.373+0000 7f7010c00700 1 -- 172.21.15.42:0/3197797922 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7f7004013070 con 0x7f70141040e0 2024-04-25T00:15:25.374 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.376+0000 7f701b000700 1 -- 172.21.15.42:0/3197797922 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6ff80a2c20 msgr2=0x7f6ff80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.376+0000 7f701b000700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6ff80a2c20 0x7f6ff80a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7000009680 tx=0x7f7000009260 comp rx=0 tx=0).stop 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.376+0000 7f701b000700 1 -- 172.21.15.42:0/3197797922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70141040e0 msgr2=0x7f701419a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.376+0000 7f701b000700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70141040e0 0x7f701419a7e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7004000c00 tx=0x7f7004009580 comp rx=0 tx=0).stop 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.376+0000 7f701b000700 1 -- 172.21.15.42:0/3197797922 shutdown_connections 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.376+0000 7f701b000700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6ff80a2c20 0x7f6ff80a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.376+0000 7f701b000700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7014105ba0 0x7f70141a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.376+0000 7f701b000700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70141040e0 0x7f701419a7e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.376+0000 7f701b000700 1 --2- 172.21.15.42:0/3197797922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7014104a80 0x7f701419ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.376+0000 7f701b000700 1 -- 172.21.15.42:0/3197797922 >> 172.21.15.42:0/3197797922 conn(0x7f70140ffa80 msgr2=0x7f7014108960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:25.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.377+0000 7f701b000700 1 -- 172.21.15.42:0/3197797922 shutdown_connections 2024-04-25T00:15:25.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:25.377+0000 7f701b000700 1 -- 172.21.15.42:0/3197797922 wait complete. 2024-04-25T00:15:26.104 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: purged_snaps scrub starts 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: purged_snaps scrub ok 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: osd.11 [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] boot 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: Detected new or changed devices on smithi136 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3197797922' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi136 to 4272M 2024-04-25T00:15:26.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:26.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:26.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.157 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714004124,"num_in_osds":12,"osd_in_since":1714004105,"num_remapped_pgs":0} 2024-04-25T00:15:26.157 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:15:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: purged_snaps scrub starts 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: purged_snaps scrub ok 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: osd.11 [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] boot 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: Detected new or changed devices on smithi136 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3197797922' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi136 to 4272M 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:26.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:26.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: purged_snaps scrub starts 2024-04-25T00:15:26.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: purged_snaps scrub ok 2024-04-25T00:15:26.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: osd.11 [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] boot 2024-04-25T00:15:26.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: osdmap e81: 12 total, 12 up, 12 in 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: Detected new or changed devices on smithi136 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3197797922' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi136 to 4272M 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:26.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:26.594 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:27.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:26 smithi042 ceph-mon[28130]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:15:27.106 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:26 smithi042 ceph-mon[28130]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:27.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:26 smithi136 ceph-mon[32923]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:15:27.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:26 smithi136 ceph-mon[32923]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:27.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:26 smithi106 ceph-mon[32510]: osdmap e82: 12 total, 12 up, 12 in 2024-04-25T00:15:27.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:26 smithi106 ceph-mon[32510]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:28.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.883+0000 7efc36600700 1 -- 172.21.15.42:0/1483714723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc30113150 msgr2=0x7efc30115540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.883+0000 7efc36600700 1 --2- 172.21.15.42:0/1483714723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc30113150 0x7efc30115540 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b0a0 tx=0x7efc2800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.883+0000 7efc36600700 1 -- 172.21.15.42:0/1483714723 shutdown_connections 2024-04-25T00:15:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.883+0000 7efc36600700 1 --2- 172.21.15.42:0/1483714723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc30113150 0x7efc30115540 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.883+0000 7efc36600700 1 --2- 172.21.15.42:0/1483714723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc300745d0 0x7efc300749b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.883+0000 7efc36600700 1 --2- 172.21.15.42:0/1483714723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc30074ef0 0x7efc30112c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.883+0000 7efc36600700 1 -- 172.21.15.42:0/1483714723 >> 172.21.15.42:0/1483714723 conn(0x7efc300fdc50 msgr2=0x7efc30100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:28.885 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.883+0000 7efc36600700 1 -- 172.21.15.42:0/1483714723 shutdown_connections 2024-04-25T00:15:28.886 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.883+0000 7efc36600700 1 -- 172.21.15.42:0/1483714723 wait complete. 2024-04-25T00:15:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.885+0000 7efc36600700 1 Processor -- start 2024-04-25T00:15:28.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.885+0000 7efc36600700 1 -- start start 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.885+0000 7efc36600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc300745d0 0x7efc3019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.885+0000 7efc36600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc30074ef0 0x7efc3019f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.885+0000 7efc36600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc30113150 0x7efc301a3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.885+0000 7efc36600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efc30118a60 con 0x7efc30113150 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.885+0000 7efc36600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efc301188e0 con 0x7efc30074ef0 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.885+0000 7efc36600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc30118be0 con 0x7efc300745d0 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc2fe00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc30074ef0 0x7efc3019f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc2fe00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc30074ef0 0x7efc3019f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46704/0 (socket says 172.21.15.42:46704) 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc2fe00700 1 -- 172.21.15.42:0/1111283237 learned_addr learned my addr 172.21.15.42:0/1111283237 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc35c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc30113150 0x7efc301a3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc35200700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc300745d0 0x7efc3019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc2fe00700 1 -- 172.21.15.42:0/1111283237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc300745d0 msgr2=0x7efc3019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc2fe00700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc300745d0 0x7efc3019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc2fe00700 1 -- 172.21.15.42:0/1111283237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc30113150 msgr2=0x7efc301a3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc2fe00700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc30113150 0x7efc301a3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc2fe00700 1 -- 172.21.15.42:0/1111283237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc301a3f80 con 0x7efc30074ef0 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.886+0000 7efc35200700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc300745d0 0x7efc3019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.887+0000 7efc2fe00700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc30074ef0 0x7efc3019f4b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e9c0 tx=0x7efc1c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.887+0000 7efc2d600700 1 -- 172.21.15.42:0/1111283237 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c00cbd0 con 0x7efc30074ef0 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.887+0000 7efc2d600700 1 -- 172.21.15.42:0/1111283237 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c00cd70 con 0x7efc30074ef0 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.887+0000 7efc2d600700 1 -- 172.21.15.42:0/1111283237 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c010430 con 0x7efc30074ef0 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.887+0000 7efc36600700 1 -- 172.21.15.42:0/1111283237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc301a4270 con 0x7efc30074ef0 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.887+0000 7efc36600700 1 -- 172.21.15.42:0/1111283237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc301abaf0 con 0x7efc30074ef0 2024-04-25T00:15:28.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.888+0000 7efc36600700 1 -- 172.21.15.42:0/1111283237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc30068f20 con 0x7efc30074ef0 2024-04-25T00:15:28.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.890+0000 7efc2d600700 1 -- 172.21.15.42:0/1111283237 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efc1c004750 con 0x7efc30074ef0 2024-04-25T00:15:28.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.892+0000 7efc2d600700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc180a2ca0 0x7efc180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.892+0000 7efc2d600700 1 -- 172.21.15.42:0/1111283237 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7efc1c0cf000 con 0x7efc30074ef0 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.892+0000 7efc35200700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc180a2ca0 0x7efc180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:28.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.893+0000 7efc35200700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc180a2ca0 0x7efc180a5160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efc24009cb0 tx=0x7efc24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:28.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:28.894+0000 7efc2d600700 1 -- 172.21.15.42:0/1111283237 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7efc1c053740 con 0x7efc30074ef0 2024-04-25T00:15:29.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.126+0000 7efc36600700 1 -- 172.21.15.42:0/1111283237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc301a02d0 con 0x7efc30074ef0 2024-04-25T00:15:29.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:28 smithi042 ceph-mon[28130]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:29.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.127+0000 7efc2d600700 1 -- 172.21.15.42:0/1111283237 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18454 (secure 0 0 0) 0x7efc1c09c360 con 0x7efc30074ef0 2024-04-25T00:15:29.128 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:29.128 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":82,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:15:25.846529+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:29.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 -- 172.21.15.42:0/1111283237 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc180a2ca0 msgr2=0x7efc180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc180a2ca0 0x7efc180a5160 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efc24009cb0 tx=0x7efc24009260 comp rx=0 tx=0).stop 2024-04-25T00:15:29.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 -- 172.21.15.42:0/1111283237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc30074ef0 msgr2=0x7efc3019f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:29.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc30074ef0 0x7efc3019f4b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00e9c0 tx=0x7efc1c00ecd0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.131 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 -- 172.21.15.42:0/1111283237 shutdown_connections 2024-04-25T00:15:29.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc180a2ca0 0x7efc180a5160 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc30113150 0x7efc301a3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc300745d0 0x7efc3019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 --2- 172.21.15.42:0/1111283237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc30074ef0 0x7efc3019f4b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:29.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 -- 172.21.15.42:0/1111283237 >> 172.21.15.42:0/1111283237 conn(0x7efc300fdc50 msgr2=0x7efc300fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:29.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 -- 172.21.15.42:0/1111283237 shutdown_connections 2024-04-25T00:15:29.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:29.131+0000 7efc36600700 1 -- 172.21.15.42:0/1111283237 wait complete. 2024-04-25T00:15:29.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:28 smithi136 ceph-mon[32923]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:29.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:28 smithi106 ceph-mon[32510]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:29.911 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:11:14.472869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:15:29.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T00:15:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:29 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1111283237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:29 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1111283237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:29 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1111283237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:30.377 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:15:31.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:30 smithi042 ceph-mon[28130]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:31.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:30 smithi136 ceph-mon[32923]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:31.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:30 smithi106 ceph-mon[32510]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.624+0000 7f1d79000700 1 -- 172.21.15.42:0/804919083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d741022e0 msgr2=0x7f1d74102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.625+0000 7f1d79000700 1 --2- 172.21.15.42:0/804919083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d741022e0 0x7f1d74102760 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1d60009990 tx=0x7f1d60009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.625 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.625+0000 7f1d79000700 1 -- 172.21.15.42:0/804919083 shutdown_connections 2024-04-25T00:15:32.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.625+0000 7f1d79000700 1 --2- 172.21.15.42:0/804919083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d74102ca0 0x7f1d7410f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.625+0000 7f1d79000700 1 --2- 172.21.15.42:0/804919083 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d741082f0 0x7f1d741086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.625+0000 7f1d79000700 1 --2- 172.21.15.42:0/804919083 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d741022e0 0x7f1d74102760 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.625+0000 7f1d79000700 1 -- 172.21.15.42:0/804919083 >> 172.21.15.42:0/804919083 conn(0x7f1d74079390 msgr2=0x7f1d740797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.625+0000 7f1d79000700 1 -- 172.21.15.42:0/804919083 shutdown_connections 2024-04-25T00:15:32.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.625+0000 7f1d79000700 1 -- 172.21.15.42:0/804919083 wait complete. 2024-04-25T00:15:32.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.626+0000 7f1d79000700 1 Processor -- start 2024-04-25T00:15:32.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.626+0000 7f1d79000700 1 -- start start 2024-04-25T00:15:32.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d79000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d741022e0 0x7f1d741a0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d79000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d74102ca0 0x7f1d741a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d79000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d741082f0 0x7f1d7419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d79000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741125f0 con 0x7f1d74102ca0 2024-04-25T00:15:32.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d79000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74112470 con 0x7f1d741022e0 2024-04-25T00:15:32.627 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d79000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1d74112770 con 0x7f1d741082f0 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d71e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d741022e0 0x7f1d741a0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d71400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d74102ca0 0x7f1d741a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d71e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d741022e0 0x7f1d741a0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46718/0 (socket says 172.21.15.42:46718) 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d71e00700 1 -- 172.21.15.42:0/3831818923 learned_addr learned my addr 172.21.15.42:0/3831818923 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.627+0000 7f1d72800700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d741082f0 0x7f1d7419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d71e00700 1 -- 172.21.15.42:0/3831818923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d741082f0 msgr2=0x7f1d7419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d71e00700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d741082f0 0x7f1d7419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d71e00700 1 -- 172.21.15.42:0/3831818923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d74102ca0 msgr2=0x7f1d741a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d71e00700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d74102ca0 0x7f1d741a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d71e00700 1 -- 172.21.15.42:0/3831818923 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d7419b190 con 0x7f1d741022e0 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d71400700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d74102ca0 0x7f1d741a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:32.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d72800700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d741082f0 0x7f1d7419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d71e00700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d741022e0 0x7f1d741a0870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400e9a0 tx=0x7f1d6400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d6aa00700 1 -- 172.21.15.42:0/3831818923 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d6400ca10 con 0x7f1d741022e0 2024-04-25T00:15:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d79000700 1 -- 172.21.15.42:0/3831818923 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d7419b480 con 0x7f1d741022e0 2024-04-25T00:15:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d6aa00700 1 -- 172.21.15.42:0/3831818923 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d6400cbb0 con 0x7f1d741022e0 2024-04-25T00:15:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.628+0000 7f1d6aa00700 1 -- 172.21.15.42:0/3831818923 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64018680 con 0x7f1d741022e0 2024-04-25T00:15:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.629+0000 7f1d79000700 1 -- 172.21.15.42:0/3831818923 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d741a7740 con 0x7f1d741022e0 2024-04-25T00:15:32.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.630+0000 7f1d79000700 1 -- 172.21.15.42:0/3831818923 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d74068f20 con 0x7f1d741022e0 2024-04-25T00:15:32.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.632+0000 7f1d6aa00700 1 -- 172.21.15.42:0/3831818923 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1d64010430 con 0x7f1d741022e0 2024-04-25T00:15:32.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.633+0000 7f1d6aa00700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d580a2c20 0x7f1d580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:32.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.633+0000 7f1d6aa00700 1 -- 172.21.15.42:0/3831818923 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f1d640cf170 con 0x7f1d741022e0 2024-04-25T00:15:32.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.633+0000 7f1d71400700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d580a2c20 0x7f1d580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:32.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.634+0000 7f1d71400700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d580a2c20 0x7f1d580a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d74079c60 tx=0x7f1d6000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:32.635 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.635+0000 7f1d6aa00700 1 -- 172.21.15.42:0/3831818923 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d6409c4d0 con 0x7f1d741022e0 2024-04-25T00:15:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.860+0000 7f1d79000700 1 -- 172.21.15.42:0/3831818923 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1d7419bf60 con 0x7f1d741022e0 2024-04-25T00:15:32.861 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.861+0000 7f1d6aa00700 1 -- 172.21.15.42:0/3831818923 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f1d7419bf60 con 0x7f1d741022e0 2024-04-25T00:15:32.861 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-25T00:15:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 -- 172.21.15.42:0/3831818923 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d580a2c20 msgr2=0x7f1d580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d580a2c20 0x7f1d580a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1d74079c60 tx=0x7f1d6000b260 comp rx=0 tx=0).stop 2024-04-25T00:15:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 -- 172.21.15.42:0/3831818923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d741022e0 msgr2=0x7f1d741a0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:32.864 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d741022e0 0x7f1d741a0870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1d6400e9a0 tx=0x7f1d6400ecb0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 -- 172.21.15.42:0/3831818923 shutdown_connections 2024-04-25T00:15:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d580a2c20 0x7f1d580a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d741082f0 0x7f1d7419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d741022e0 0x7f1d741a0870 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 --2- 172.21.15.42:0/3831818923 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d74102ca0 0x7f1d741a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 -- 172.21.15.42:0/3831818923 >> 172.21.15.42:0/3831818923 conn(0x7f1d74079390 msgr2=0x7f1d7410d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 -- 172.21.15.42:0/3831818923 shutdown_connections 2024-04-25T00:15:32.865 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:32.864+0000 7f1d79000700 1 -- 172.21.15.42:0/3831818923 wait complete. 2024-04-25T00:15:33.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:32 smithi042 ceph-mon[28130]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:33.238 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:32 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3831818923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:33.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:32 smithi136 ceph-mon[32923]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:33.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:32 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3831818923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:33.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:32 smithi106 ceph-mon[32510]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:33.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:32 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3831818923' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:33.651 INFO:tasks.cephadm:Adding mds.a on smithi042 2024-04-25T00:15:33.651 INFO:tasks.cephadm:Adding mds.d on smithi042 2024-04-25T00:15:33.651 INFO:tasks.cephadm:Adding mds.g on smithi042 2024-04-25T00:15:33.652 INFO:tasks.cephadm:Adding mds.j on smithi042 2024-04-25T00:15:33.652 INFO:tasks.cephadm:Adding mds.b on smithi106 2024-04-25T00:15:33.652 INFO:tasks.cephadm:Adding mds.e on smithi106 2024-04-25T00:15:33.652 INFO:tasks.cephadm:Adding mds.h on smithi106 2024-04-25T00:15:33.652 INFO:tasks.cephadm:Adding mds.k on smithi106 2024-04-25T00:15:33.652 INFO:tasks.cephadm:Adding mds.c on smithi136 2024-04-25T00:15:33.652 INFO:tasks.cephadm:Adding mds.f on smithi136 2024-04-25T00:15:33.652 INFO:tasks.cephadm:Adding mds.i on smithi136 2024-04-25T00:15:33.652 INFO:tasks.cephadm:Adding mds.l on smithi136 2024-04-25T00:15:33.652 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l' 2024-04-25T00:15:33.901 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:15:35.253 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:34 smithi136 ceph-mon[32923]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:35.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:34 smithi106 ceph-mon[32510]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:35.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:34 smithi042 ceph-mon[28130]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:35.447 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.447+0000 7f08b9dde700 1 -- 172.21.15.136:0/3148130367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08b4101180 msgr2=0x7f08b41015a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:35.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.447+0000 7f08b9dde700 1 --2- 172.21.15.136:0/3148130367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08b4101180 0x7f08b41015a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f08a8009990 tx=0x7f08a8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:15:35.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.447+0000 7f08b9dde700 1 -- 172.21.15.136:0/3148130367 shutdown_connections 2024-04-25T00:15:35.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.447+0000 7f08b9dde700 1 --2- 172.21.15.136:0/3148130367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b4102d20 0x7f08b41095d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:35.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.447+0000 7f08b9dde700 1 --2- 172.21.15.136:0/3148130367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08b4101180 0x7f08b41015a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:35.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.447+0000 7f08b9dde700 1 --2- 172.21.15.136:0/3148130367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08b4102380 0x7f08b41027e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:35.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.447+0000 7f08b9dde700 1 -- 172.21.15.136:0/3148130367 >> 172.21.15.136:0/3148130367 conn(0x7f08b40fc700 msgr2=0x7f08b40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:35.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.447+0000 7f08b9dde700 1 -- 172.21.15.136:0/3148130367 shutdown_connections 2024-04-25T00:15:35.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.447+0000 7f08b9dde700 1 -- 172.21.15.136:0/3148130367 wait complete. 2024-04-25T00:15:35.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b9dde700 1 Processor -- start 2024-04-25T00:15:35.448 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b9dde700 1 -- start start 2024-04-25T00:15:35.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b9dde700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b4101180 0x7f08b4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:35.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b9dde700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08b4102380 0x7f08b4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:35.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b9dde700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08b4102d20 0x7f08b419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:35.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b9dde700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f08b410ce40 con 0x7f08b4101180 2024-04-25T00:15:35.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b9dde700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f08b410ccc0 con 0x7f08b4102d20 2024-04-25T00:15:35.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b9dde700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f08b410cfc0 con 0x7f08b4102380 2024-04-25T00:15:35.450 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b2ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08b4102380 0x7f08b4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b2ffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08b4102380 0x7f08b4198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:34434/0 (socket says 172.21.15.136:34434) 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b2ffd700 1 -- 172.21.15.136:0/4251515000 learned_addr learned my addr 172.21.15.136:0/4251515000 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b3fff700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08b4102d20 0x7f08b419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b2ffd700 1 -- 172.21.15.136:0/4251515000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08b4102d20 msgr2=0x7f08b419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b2ffd700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08b4102d20 0x7f08b419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b2ffd700 1 -- 172.21.15.136:0/4251515000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b4101180 msgr2=0x7f08b4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b2ffd700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b4101180 0x7f08b4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b2ffd700 1 -- 172.21.15.136:0/4251515000 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b41a0280 con 0x7f08b4102380 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b3fff700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08b4102d20 0x7f08b419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.448+0000 7f08b2ffd700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08b4102380 0x7f08b4198b60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f08a400b560 tx=0x7f08a400b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.449+0000 7f08b0ff9700 1 -- 172.21.15.136:0/4251515000 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a4015020 con 0x7f08b4102380 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.449+0000 7f08b9dde700 1 -- 172.21.15.136:0/4251515000 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b41a0570 con 0x7f08b4102380 2024-04-25T00:15:35.451 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.449+0000 7f08b0ff9700 1 -- 172.21.15.136:0/4251515000 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08a4011d10 con 0x7f08b4102380 2024-04-25T00:15:35.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.449+0000 7f08b0ff9700 1 -- 172.21.15.136:0/4251515000 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a400f2e0 con 0x7f08b4102380 2024-04-25T00:15:35.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.449+0000 7f08b9dde700 1 -- 172.21.15.136:0/4251515000 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b41a0b30 con 0x7f08b4102380 2024-04-25T00:15:35.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.450+0000 7f08b9dde700 1 -- 172.21.15.136:0/4251515000 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b4068f20 con 0x7f08b4102380 2024-04-25T00:15:35.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.451+0000 7f08b0ff9700 1 -- 172.21.15.136:0/4251515000 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f08a4018020 con 0x7f08b4102380 2024-04-25T00:15:35.452 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.452+0000 7f08b0ff9700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f089c0a2ca0 0x7f089c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:35.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.452+0000 7f08b0ff9700 1 -- 172.21.15.136:0/4251515000 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f08a40ce940 con 0x7f08b4102380 2024-04-25T00:15:35.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.452+0000 7f08b37fe700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f089c0a2ca0 0x7f089c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:35.453 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.453+0000 7f08b37fe700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f089c0a2ca0 0x7f089c0a5160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f08a8005f50 tx=0x7f08a8005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:35.455 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.455+0000 7f08b0ff9700 1 -- 172.21.15.136:0/4251515000 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08a401f020 con 0x7f08b4102380 2024-04-25T00:15:35.662 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.662+0000 7f08b9dde700 1 -- 172.21.15.136:0/4251515000 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}) -- 0x7f08b406b210 con 0x7f089c0a2ca0 2024-04-25T00:15:35.679 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.679+0000 7f08b0ff9700 1 -- 172.21.15.136:0/4251515000 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f08b406b210 con 0x7f089c0a2ca0 2024-04-25T00:15:35.679 INFO:teuthology.orchestra.run.smithi136.stdout:Scheduled mds.all update... 2024-04-25T00:15:35.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 -- 172.21.15.136:0/4251515000 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f089c0a2ca0 msgr2=0x7f089c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:35.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f089c0a2ca0 0x7f089c0a5160 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f08a8005f50 tx=0x7f08a8005e80 comp rx=0 tx=0).stop 2024-04-25T00:15:35.681 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 -- 172.21.15.136:0/4251515000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08b4102380 msgr2=0x7f08b4198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:35.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08b4102380 0x7f08b4198b60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f08a400b560 tx=0x7f08a400b920 comp rx=0 tx=0).stop 2024-04-25T00:15:35.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 -- 172.21.15.136:0/4251515000 shutdown_connections 2024-04-25T00:15:35.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f089c0a2ca0 0x7f089c0a5160 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:35.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08b4102d20 0x7f08b419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:35.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08b4101180 0x7f08b4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:35.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 --2- 172.21.15.136:0/4251515000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08b4102380 0x7f08b4198b60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:35.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 -- 172.21.15.136:0/4251515000 >> 172.21.15.136:0/4251515000 conn(0x7f08b40fc700 msgr2=0x7f08b406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:35.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 -- 172.21.15.136:0/4251515000 shutdown_connections 2024-04-25T00:15:35.682 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-25T00:15:35.681+0000 7f08b9dde700 1 -- 172.21.15.136:0/4251515000 wait complete. 2024-04-25T00:15:36.319 DEBUG:teuthology.orchestra.run.smithi042:mds.a> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T00:15:36.327 DEBUG:teuthology.orchestra.run.smithi042:mds.d> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T00:15:36.373 DEBUG:teuthology.orchestra.run.smithi042:mds.g> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T00:15:36.419 DEBUG:teuthology.orchestra.run.smithi042:mds.j> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T00:15:36.464 DEBUG:teuthology.orchestra.run.smithi106:mds.b> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T00:15:36.466 DEBUG:teuthology.orchestra.run.smithi106:mds.e> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T00:15:36.509 DEBUG:teuthology.orchestra.run.smithi106:mds.h> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T00:15:36.552 DEBUG:teuthology.orchestra.run.smithi106:mds.k> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T00:15:36.595 DEBUG:teuthology.orchestra.run.smithi136:mds.c> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T00:15:36.598 DEBUG:teuthology.orchestra.run.smithi136:mds.f> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T00:15:36.641 DEBUG:teuthology.orchestra.run.smithi136:mds.i> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T00:15:36.686 DEBUG:teuthology.orchestra.run.smithi136:mds.l> sudo journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T00:15:36.728 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T00:15:36.729 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:15:36.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:36 smithi106 ceph-mon[32510]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:36.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:36 smithi106 ceph-mon[32510]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-04-25T00:15:36.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:36.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:36 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:36.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.950+0000 7fd97be00640 1 -- 172.21.15.42:0/1936326485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd97c1049b0 msgr2=0x7fd97c104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.950+0000 7fd97be00640 1 --2- 172.21.15.42:0/1936326485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd97c1049b0 0x7fd97c104e30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd964009900 tx=0x7fd96402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.950+0000 7fd97be00640 1 -- 172.21.15.42:0/1936326485 shutdown_connections 2024-04-25T00:15:36.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.950+0000 7fd97be00640 1 --2- 172.21.15.42:0/1936326485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd97c105370 0x7fd97c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.950+0000 7fd97be00640 1 --2- 172.21.15.42:0/1936326485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd97c1049b0 0x7fd97c104e30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.950+0000 7fd97be00640 1 --2- 172.21.15.42:0/1936326485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd97c1037c0 0x7fd97c103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.950+0000 7fd97be00640 1 -- 172.21.15.42:0/1936326485 >> 172.21.15.42:0/1936326485 conn(0x7fd97c0fed50 msgr2=0x7fd97c101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:36.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.950+0000 7fd97be00640 1 -- 172.21.15.42:0/1936326485 shutdown_connections 2024-04-25T00:15:36.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.950+0000 7fd97be00640 1 -- 172.21.15.42:0/1936326485 wait complete. 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.951+0000 7fd97be00640 1 Processor -- start 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.951+0000 7fd97be00640 1 -- start start 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.951+0000 7fd97be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd97c1037c0 0x7fd97c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.951+0000 7fd97be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd97c1049b0 0x7fd97c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.951+0000 7fd97be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd97c105370 0x7fd97c1a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.951+0000 7fd97be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c10f470 con 0x7fd97c1049b0 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.951+0000 7fd97be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c10f2f0 con 0x7fd97c1037c0 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.951+0000 7fd97be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c10f5f0 con 0x7fd97c105370 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd97c105370 0x7fd97c1a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd97c1037c0 0x7fd97c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd97c1037c0 0x7fd97c199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44538/0 (socket says 172.21.15.42:44538) 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd97c105370 0x7fd97c1a10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57056/0 (socket says 172.21.15.42:57056) 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97aa00640 1 -- 172.21.15.42:0/1894775865 learned_addr learned my addr 172.21.15.42:0/1894775865 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd973e00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd97c1049b0 0x7fd97c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97aa00640 1 -- 172.21.15.42:0/1894775865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd97c105370 msgr2=0x7fd97c1a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97aa00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd97c105370 0x7fd97c1a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97aa00640 1 -- 172.21.15.42:0/1894775865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd97c1049b0 msgr2=0x7fd97c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97aa00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd97c1049b0 0x7fd97c19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97aa00640 1 -- 172.21.15.42:0/1894775865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c1a1710 con 0x7fd97c1037c0 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.952+0000 7fd97aa00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd97c1037c0 0x7fd97c199b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00e9c0 tx=0x7fd96c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.953+0000 7fd980a00640 1 -- 172.21.15.42:0/1894775865 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c00cd80 con 0x7fd97c1037c0 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.953+0000 7fd980a00640 1 -- 172.21.15.42:0/1894775865 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd96c0035f0 con 0x7fd97c1037c0 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.953+0000 7fd980a00640 1 -- 172.21.15.42:0/1894775865 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96c010690 con 0x7fd97c1037c0 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.953+0000 7fd97be00640 1 -- 172.21.15.42:0/1894775865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c1a1a00 con 0x7fd97c1037c0 2024-04-25T00:15:36.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.953+0000 7fd97be00640 1 -- 172.21.15.42:0/1894775865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c1a1ed0 con 0x7fd97c1037c0 2024-04-25T00:15:36.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.954+0000 7fd97be00640 1 -- 172.21.15.42:0/1894775865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c10aae0 con 0x7fd97c1037c0 2024-04-25T00:15:36.955 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.956+0000 7fd980a00640 1 -- 172.21.15.42:0/1894775865 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd96c0037a0 con 0x7fd97c1037c0 2024-04-25T00:15:36.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.957+0000 7fd980a00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd9600a2c10 0x7fd9600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:36.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.957+0000 7fd980a00640 1 -- 172.21.15.42:0/1894775865 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd96c0cefa0 con 0x7fd97c1037c0 2024-04-25T00:15:36.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.957+0000 7fd973e00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd9600a2c10 0x7fd9600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:36.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.958+0000 7fd973e00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd9600a2c10 0x7fd9600a50d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd964031440 tx=0x7fd9640315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:36.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:36.961+0000 7fd980a00640 1 -- 172.21.15.42:0/1894775865 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd96c09c300 con 0x7fd97c1037c0 2024-04-25T00:15:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:36 smithi136 ceph-mon[32923]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:36 smithi136 ceph-mon[32923]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-04-25T00:15:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:37.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:36 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:37.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:36 smithi042 ceph-mon[28130]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:15:37.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:36 smithi042 ceph-mon[28130]: Saving service mds.all spec with placement smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-04-25T00:15:37.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:37.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:36 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:15:37.169 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.169+0000 7fd97be00640 1 -- 172.21.15.42:0/1894775865 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd97c062340 con 0x7fd97c1037c0 2024-04-25T00:15:37.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.170+0000 7fd980a00640 1 -- 172.21.15.42:0/1894775865 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18454 (secure 0 0 0) 0x7fd96c09bc00 con 0x7fd97c1037c0 2024-04-25T00:15:37.171 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:37.171 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":82,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:15:25.846529+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:15:37.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 -- 172.21.15.42:0/1894775865 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd9600a2c10 msgr2=0x7fd9600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd9600a2c10 0x7fd9600a50d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd964031440 tx=0x7fd9640315b0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 -- 172.21.15.42:0/1894775865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd97c1037c0 msgr2=0x7fd97c199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd97c1037c0 0x7fd97c199b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd96c00e9c0 tx=0x7fd96c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:15:37.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 -- 172.21.15.42:0/1894775865 shutdown_connections 2024-04-25T00:15:37.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd9600a2c10 0x7fd9600a50d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd97c105370 0x7fd97c1a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd97c1049b0 0x7fd97c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 --2- 172.21.15.42:0/1894775865 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd97c1037c0 0x7fd97c199b40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 -- 172.21.15.42:0/1894775865 >> 172.21.15.42:0/1894775865 conn(0x7fd97c0fed50 msgr2=0x7fd97c107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:37.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 -- 172.21.15.42:0/1894775865 shutdown_connections 2024-04-25T00:15:37.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.173+0000 7fd97be00640 1 -- 172.21.15.42:0/1894775865 wait complete. 2024-04-25T00:15:37.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:11:14.472869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:15:37.197 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:15:37.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.423+0000 7fe521400640 1 -- 172.21.15.42:0/558252445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe51c107c80 msgr2=0x7fe51c1080a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.423+0000 7fe521400640 1 --2- 172.21.15.42:0/558252445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe51c107c80 0x7fe51c1080a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009900 tx=0x7fe50c02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.423+0000 7fe521400640 1 -- 172.21.15.42:0/558252445 shutdown_connections 2024-04-25T00:15:37.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.423+0000 7fe521400640 1 --2- 172.21.15.42:0/558252445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c109830 0x7fe51c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.423+0000 7fe521400640 1 --2- 172.21.15.42:0/558252445 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe51c108e70 0x7fe51c1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.423+0000 7fe521400640 1 --2- 172.21.15.42:0/558252445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe51c107c80 0x7fe51c1080a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.423+0000 7fe521400640 1 -- 172.21.15.42:0/558252445 >> 172.21.15.42:0/558252445 conn(0x7fe51c076c10 msgr2=0x7fe51c079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:37.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.423+0000 7fe521400640 1 -- 172.21.15.42:0/558252445 shutdown_connections 2024-04-25T00:15:37.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.423+0000 7fe521400640 1 -- 172.21.15.42:0/558252445 wait complete. 2024-04-25T00:15:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.424+0000 7fe521400640 1 Processor -- start 2024-04-25T00:15:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe521400640 1 -- start start 2024-04-25T00:15:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe521400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c107c80 0x7fe51c19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe521400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe51c108e70 0x7fe51c19e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe521400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe51c109830 0x7fe51c1a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe521400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c113950 con 0x7fe51c108e70 2024-04-25T00:15:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe521400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c1137d0 con 0x7fe51c107c80 2024-04-25T00:15:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe521400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe51c113ad0 con 0x7fe51c109830 2024-04-25T00:15:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe513e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe51c108e70 0x7fe51c19e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe513e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe51c108e70 0x7fe51c19e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49634/0 (socket says 172.21.15.42:49634) 2024-04-25T00:15:37.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe513e00640 1 -- 172.21.15.42:0/509818531 learned_addr learned my addr 172.21.15.42:0/509818531 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:37.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe51be00640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c107c80 0x7fe51c19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.425+0000 7fe520a00640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe51c109830 0x7fe51c1a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.426+0000 7fe513e00640 1 -- 172.21.15.42:0/509818531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe51c109830 msgr2=0x7fe51c1a5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.426+0000 7fe513e00640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe51c109830 0x7fe51c1a5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.426+0000 7fe513e00640 1 -- 172.21.15.42:0/509818531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c107c80 msgr2=0x7fe51c19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.426+0000 7fe513e00640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c107c80 0x7fe51c19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.426+0000 7fe513e00640 1 -- 172.21.15.42:0/509818531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe51c1a5b80 con 0x7fe51c108e70 2024-04-25T00:15:37.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.426+0000 7fe513e00640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe51c108e70 0x7fe51c19e4a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe50400e9f0 tx=0x7fe50400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:37.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.426+0000 7fe519600640 1 -- 172.21.15.42:0/509818531 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50400cd80 con 0x7fe51c108e70 2024-04-25T00:15:37.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.427+0000 7fe521400640 1 -- 172.21.15.42:0/509818531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe51c1a5e70 con 0x7fe51c108e70 2024-04-25T00:15:37.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.427+0000 7fe519600640 1 -- 172.21.15.42:0/509818531 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe504010910 con 0x7fe51c108e70 2024-04-25T00:15:37.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.427+0000 7fe521400640 1 -- 172.21.15.42:0/509818531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe51c1a6330 con 0x7fe51c108e70 2024-04-25T00:15:37.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.427+0000 7fe519600640 1 -- 172.21.15.42:0/509818531 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504018910 con 0x7fe51c108e70 2024-04-25T00:15:37.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.428+0000 7fe521400640 1 -- 172.21.15.42:0/509818531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe51c10ef40 con 0x7fe51c108e70 2024-04-25T00:15:37.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.430+0000 7fe519600640 1 -- 172.21.15.42:0/509818531 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe504010ac0 con 0x7fe51c108e70 2024-04-25T00:15:37.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.432+0000 7fe519600640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5000a2b90 0x7fe5000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.432+0000 7fe519600640 1 -- 172.21.15.42:0/509818531 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe5040cefc0 con 0x7fe51c108e70 2024-04-25T00:15:37.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.432+0000 7fe51be00640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5000a2b90 0x7fe5000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.433+0000 7fe51be00640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5000a2b90 0x7fe5000a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe50c02a9e0 tx=0x7fe50c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:37.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.436+0000 7fe519600640 1 -- 172.21.15.42:0/509818531 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe50409bcd0 con 0x7fe51c108e70 2024-04-25T00:15:37.638 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.638+0000 7fe521400640 1 -- 172.21.15.42:0/509818531 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe51c062340 con 0x7fe51c108e70 2024-04-25T00:15:37.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.639+0000 7fe519600640 1 -- 172.21.15.42:0/509818531 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fe50409ffd0 con 0x7fe51c108e70 2024-04-25T00:15:37.639 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-25T00:15:37.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 -- 172.21.15.42:0/509818531 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5000a2b90 msgr2=0x7fe5000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5000a2b90 0x7fe5000a5050 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe50c02a9e0 tx=0x7fe50c0315b0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 -- 172.21.15.42:0/509818531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe51c108e70 msgr2=0x7fe51c19e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe51c108e70 0x7fe51c19e4a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe50400e9f0 tx=0x7fe50400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 -- 172.21.15.42:0/509818531 shutdown_connections 2024-04-25T00:15:37.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5000a2b90 0x7fe5000a5050 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe51c109830 0x7fe51c1a5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe51c108e70 0x7fe51c19e4a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 --2- 172.21.15.42:0/509818531 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe51c107c80 0x7fe51c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 -- 172.21.15.42:0/509818531 >> 172.21.15.42:0/509818531 conn(0x7fe51c076c10 msgr2=0x7fe51c10c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:37.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 -- 172.21.15.42:0/509818531 shutdown_connections 2024-04-25T00:15:37.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.642+0000 7fe521400640 1 -- 172.21.15.42:0/509818531 wait complete. 2024-04-25T00:15:37.663 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:37.811 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:37 smithi136 ceph-mon[32923]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:37.811 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:37.811 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:37.811 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:37.811 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:37.811 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:37.811 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:37.811 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:37 smithi136 ceph-mon[32923]: Deploying daemon mds.l on smithi136 2024-04-25T00:15:37.811 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:37 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1894775865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:37.811 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:37 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/509818531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:37.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.872+0000 7f569fe00640 1 -- 172.21.15.42:0/1004822747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56a0077340 msgr2=0x7f56a00777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.872+0000 7f569fe00640 1 --2- 172.21.15.42:0/1004822747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56a0077340 0x7f56a00777a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5688009900 tx=0x7f568802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.872+0000 7f569fe00640 1 -- 172.21.15.42:0/1004822747 shutdown_connections 2024-04-25T00:15:37.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.872+0000 7f569fe00640 1 --2- 172.21.15.42:0/1004822747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56a0077ce0 0x7f56a0078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.872+0000 7f569fe00640 1 --2- 172.21.15.42:0/1004822747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56a0077340 0x7f56a00777a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.872+0000 7f569fe00640 1 --2- 172.21.15.42:0/1004822747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a0076100 0x7f56a0076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.872+0000 7f569fe00640 1 -- 172.21.15.42:0/1004822747 >> 172.21.15.42:0/1004822747 conn(0x7f56a00fed50 msgr2=0x7f56a0101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:37.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.872+0000 7f569fe00640 1 -- 172.21.15.42:0/1004822747 shutdown_connections 2024-04-25T00:15:37.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.872+0000 7f569fe00640 1 -- 172.21.15.42:0/1004822747 wait complete. 2024-04-25T00:15:37.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.873+0000 7f569fe00640 1 Processor -- start 2024-04-25T00:15:37.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.873+0000 7f569fe00640 1 -- start start 2024-04-25T00:15:37.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56a0076100 0x7f56a019de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56a0077340 0x7f56a019e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a0077ce0 0x7f56a01a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0113890 con 0x7f56a0077340 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0113710 con 0x7f56a0076100 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56a0113a10 con 0x7f56a0077ce0 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f5696000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56a0077340 0x7f56a019e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56a0076100 0x7f56a019de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f5696000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56a0077340 0x7f56a019e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49662/0 (socket says 172.21.15.42:49662) 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f5696000640 1 -- 172.21.15.42:0/4165646135 learned_addr learned my addr 172.21.15.42:0/4165646135 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569f400640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a0077ce0 0x7f56a01a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569ea00640 1 -- 172.21.15.42:0/4165646135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a0077ce0 msgr2=0x7f56a01a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569ea00640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a0077ce0 0x7f56a01a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569ea00640 1 -- 172.21.15.42:0/4165646135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56a0077340 msgr2=0x7f56a019e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569ea00640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56a0077340 0x7f56a019e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.874+0000 7f569ea00640 1 -- 172.21.15.42:0/4165646135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a01a5ab0 con 0x7f56a0076100 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.875+0000 7f569ea00640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56a0076100 0x7f56a019de90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5690005fd0 tx=0x7f569000dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:37.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.875+0000 7f5697e00640 1 -- 172.21.15.42:0/4165646135 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690009c30 con 0x7f56a0076100 2024-04-25T00:15:37.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.875+0000 7f5697e00640 1 -- 172.21.15.42:0/4165646135 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5690009dd0 con 0x7f56a0076100 2024-04-25T00:15:37.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.875+0000 7f5697e00640 1 -- 172.21.15.42:0/4165646135 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5690010460 con 0x7f56a0076100 2024-04-25T00:15:37.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.875+0000 7f569fe00640 1 -- 172.21.15.42:0/4165646135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a01a5da0 con 0x7f56a0076100 2024-04-25T00:15:37.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.875+0000 7f569fe00640 1 -- 172.21.15.42:0/4165646135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a01a61b0 con 0x7f56a0076100 2024-04-25T00:15:37.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.876+0000 7f569fe00640 1 -- 172.21.15.42:0/4165646135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a01102a0 con 0x7f56a0076100 2024-04-25T00:15:37.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.878+0000 7f5697e00640 1 -- 172.21.15.42:0/4165646135 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5690003f50 con 0x7f56a0076100 2024-04-25T00:15:37.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.879+0000 7f5697e00640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56840a2b90 0x7f56840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:37.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.880+0000 7f5697e00640 1 -- 172.21.15.42:0/4165646135 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f56900ceb70 con 0x7f56a0076100 2024-04-25T00:15:37.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.880+0000 7f5696000640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56840a2b90 0x7f56840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:37.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.880+0000 7f5696000640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56840a2b90 0x7f56840a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f56a019f3a0 tx=0x7f56880315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:37.884 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:37.884+0000 7f5697e00640 1 -- 172.21.15.42:0/4165646135 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5690017020 con 0x7f56a0076100 2024-04-25T00:15:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:37 smithi106 ceph-mon[32510]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:38.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:37 smithi106 ceph-mon[32510]: Deploying daemon mds.l on smithi136 2024-04-25T00:15:38.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:37 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1894775865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:38.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:37 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/509818531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:37 smithi042 ceph-mon[28130]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:15:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:37 smithi042 ceph-mon[28130]: Deploying daemon mds.l on smithi136 2024-04-25T00:15:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:37 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1894775865' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:15:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:37 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/509818531' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:15:38.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.129+0000 7f569fe00640 1 -- 172.21.15.42:0/4165646135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56a002d170 con 0x7f56a0076100 2024-04-25T00:15:38.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.129+0000 7f5697e00640 1 -- 172.21.15.42:0/4165646135 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7f56a002d170 con 0x7f56a0076100 2024-04-25T00:15:38.130 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:38.130 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T00:15:38.130 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 1 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 -- 172.21.15.42:0/4165646135 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56840a2b90 msgr2=0x7f56840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56840a2b90 0x7f56840a5050 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f56a019f3a0 tx=0x7f56880315b0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 -- 172.21.15.42:0/4165646135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56a0076100 msgr2=0x7f56a019de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56a0076100 0x7f56a019de90 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5690005fd0 tx=0x7f569000dde0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 -- 172.21.15.42:0/4165646135 shutdown_connections 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56840a2b90 0x7f56840a5050 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56a0077ce0 0x7f56a01a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56a0077340 0x7f56a019e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 --2- 172.21.15.42:0/4165646135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56a0076100 0x7f56a019de90 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 -- 172.21.15.42:0/4165646135 >> 172.21.15.42:0/4165646135 conn(0x7f56a00fed50 msgr2=0x7f56a01003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 -- 172.21.15.42:0/4165646135 shutdown_connections 2024-04-25T00:15:38.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:38.132+0000 7f569fe00640 1 -- 172.21.15.42:0/4165646135 wait complete. 2024-04-25T00:15:38.807 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:38 smithi136 ceph-mon[32923]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:38.807 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:38 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/4165646135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:38 smithi106 ceph-mon[32510]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:38 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/4165646135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:39.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:38 smithi042 ceph-mon[28130]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:38 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/4165646135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:39.211 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:15:38 smithi136 systemd[1]: Starting Ceph mds.l for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:15:39.511 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:15:39 smithi136 podman[62677]: 2024-04-25 00:15:39.114310604 +0000 UTC m=+0.018202292 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:15:39.511 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:15:39 smithi136 podman[62677]: 2024-04-25 00:15:39.23363328 +0000 UTC m=+0.137524972 container create 0b8ea76fe11339e92d4ab09553bfebe3a6b3f75fb644465d73d2a8f4a8b4301c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T00:15:39.511 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:15:39 smithi136 podman[62677]: 2024-04-25 00:15:39.419350836 +0000 UTC m=+0.323242524 container init 0b8ea76fe11339e92d4ab09553bfebe3a6b3f75fb644465d73d2a8f4a8b4301c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T00:15:39.511 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:15:39 smithi136 podman[62677]: 2024-04-25 00:15:39.421906776 +0000 UTC m=+0.325798464 container start 0b8ea76fe11339e92d4ab09553bfebe3a6b3f75fb644465d73d2a8f4a8b4301c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-l, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T00:15:39.511 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:15:39 smithi136 ceph-mds[62696]: -- 172.21.15.136:0/1812585001 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f59179ba40 con 0x55f59176f000 2024-04-25T00:15:39.511 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:15:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-l[62692]: starting mds.l at 2024-04-25T00:15:39.511 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:15:39 smithi136 bash[62677]: 0b8ea76fe11339e92d4ab09553bfebe3a6b3f75fb644465d73d2a8f4a8b4301c 2024-04-25T00:15:39.763 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:15:39 smithi136 systemd[1]: Started Ceph mds.l for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:15:40.155 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.381+0000 7ff245400640 1 -- 172.21.15.42:0/2027785141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2401015e0 msgr2=0x7ff240101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.381+0000 7ff245400640 1 --2- 172.21.15.42:0/2027785141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2401015e0 0x7ff240101a00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff230009900 tx=0x7ff23002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.381+0000 7ff245400640 1 -- 172.21.15.42:0/2027785141 shutdown_connections 2024-04-25T00:15:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.381+0000 7ff245400640 1 --2- 172.21.15.42:0/2027785141 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff240103190 0x7ff240109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.381+0000 7ff245400640 1 --2- 172.21.15.42:0/2027785141 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2401027d0 0x7ff240102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.381+0000 7ff245400640 1 --2- 172.21.15.42:0/2027785141 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2401015e0 0x7ff240101a00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.381+0000 7ff245400640 1 -- 172.21.15.42:0/2027785141 >> 172.21.15.42:0/2027785141 conn(0x7ff2400fcb50 msgr2=0x7ff2400fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.381+0000 7ff245400640 1 -- 172.21.15.42:0/2027785141 shutdown_connections 2024-04-25T00:15:40.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.381+0000 7ff245400640 1 -- 172.21.15.42:0/2027785141 wait complete. 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.382+0000 7ff245400640 1 Processor -- start 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.382+0000 7ff245400640 1 -- start start 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff245400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2401015e0 0x7ff240199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff245400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2401027d0 0x7ff24019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff245400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff240103190 0x7ff2401a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff245400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff24010d250 con 0x7ff2401027d0 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff245400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff24010d0d0 con 0x7ff2401015e0 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff245400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff24010d3d0 con 0x7ff240103190 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff23fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2401015e0 0x7ff240199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff23fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2401015e0 0x7ff240199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44598/0 (socket says 172.21.15.42:44598) 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff23fe00640 1 -- 172.21.15.42:0/588354580 learned_addr learned my addr 172.21.15.42:0/588354580 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff23f400640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2401027d0 0x7ff24019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.383+0000 7ff244a00640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff240103190 0x7ff2401a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff23fe00640 1 -- 172.21.15.42:0/588354580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff240103190 msgr2=0x7ff2401a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff23fe00640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff240103190 0x7ff2401a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff23fe00640 1 -- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2401027d0 msgr2=0x7ff24019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff23fe00640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2401027d0 0x7ff24019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff23fe00640 1 -- 172.21.15.42:0/588354580 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2401a1780 con 0x7ff2401015e0 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff23f400640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2401027d0 0x7ff24019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff23fe00640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2401015e0 0x7ff240199bb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff230005ab0 tx=0x7ff230004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff23cc00640 1 -- 172.21.15.42:0/588354580 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23003f030 con 0x7ff2401015e0 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff23cc00640 1 -- 172.21.15.42:0/588354580 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff230003ad0 con 0x7ff2401015e0 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff23cc00640 1 -- 172.21.15.42:0/588354580 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230035070 con 0x7ff2401015e0 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff245400640 1 -- 172.21.15.42:0/588354580 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2401a1a10 con 0x7ff2401015e0 2024-04-25T00:15:40.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.384+0000 7ff245400640 1 -- 172.21.15.42:0/588354580 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2401a1f00 con 0x7ff2401015e0 2024-04-25T00:15:40.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.386+0000 7ff245400640 1 -- 172.21.15.42:0/588354580 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2401a20d0 con 0x7ff2401015e0 2024-04-25T00:15:40.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.387+0000 7ff23cc00640 1 -- 172.21.15.42:0/588354580 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff230042030 con 0x7ff2401015e0 2024-04-25T00:15:40.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.389+0000 7ff23cc00640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2240a2b90 0x7ff2240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:40.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.389+0000 7ff23cc00640 1 -- 172.21.15.42:0/588354580 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff2300eef10 con 0x7ff2401015e0 2024-04-25T00:15:40.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.389+0000 7ff23f400640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2240a2b90 0x7ff2240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:40.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.390+0000 7ff23f400640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2240a2b90 0x7ff2240a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff24019b070 tx=0x7ff22c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:40.394 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.394+0000 7ff23cc00640 1 -- 172.21.15.42:0/588354580 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff2300bbc20 con 0x7ff2401015e0 2024-04-25T00:15:40.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:40 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6832/756794270,v1:172.21.15.136:6833/756794270] up:boot 2024-04-25T00:15:40.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:40 smithi106 ceph-mon[32510]: fsmap 1 up:standby 2024-04-25T00:15:40.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:15:40.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:40.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:40.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:40.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:40.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:40.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:40 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:40.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:40 smithi106 ceph-mon[32510]: Deploying daemon mds.k on smithi106 2024-04-25T00:15:40.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:40 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6832/756794270,v1:172.21.15.136:6833/756794270] up:boot 2024-04-25T00:15:40.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:40 smithi042 ceph-mon[28130]: fsmap 1 up:standby 2024-04-25T00:15:40.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:15:40.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:40.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:40.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:40.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:40.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:40.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:40 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:40.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:40 smithi042 ceph-mon[28130]: Deploying daemon mds.k on smithi106 2024-04-25T00:15:40.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.644+0000 7ff245400640 1 -- 172.21.15.42:0/588354580 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2401a22a0 con 0x7ff2401015e0 2024-04-25T00:15:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.646+0000 7ff23cc00640 1 -- 172.21.15.42:0/588354580 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7ff2300a6590 con 0x7ff2401015e0 2024-04-25T00:15:40.646 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:40.646 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T00:15:40.646 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 2 2024-04-25T00:15:40.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.648+0000 7ff245400640 1 -- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2240a2b90 msgr2=0x7ff2240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.648+0000 7ff245400640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2240a2b90 0x7ff2240a5050 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff24019b070 tx=0x7ff22c005f90 comp rx=0 tx=0).stop 2024-04-25T00:15:40.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.648+0000 7ff245400640 1 -- 172.21.15.42:0/588354580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2401015e0 msgr2=0x7ff240199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:40.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.648+0000 7ff245400640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2401015e0 0x7ff240199bb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff230005ab0 tx=0x7ff230004270 comp rx=0 tx=0).stop 2024-04-25T00:15:40.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.649+0000 7ff245400640 1 -- 172.21.15.42:0/588354580 shutdown_connections 2024-04-25T00:15:40.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.649+0000 7ff245400640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2240a2b90 0x7ff2240a5050 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.649+0000 7ff245400640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff240103190 0x7ff2401a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.649+0000 7ff245400640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2401027d0 0x7ff24019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.649+0000 7ff245400640 1 --2- 172.21.15.42:0/588354580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2401015e0 0x7ff240199bb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:40.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.649+0000 7ff245400640 1 -- 172.21.15.42:0/588354580 >> 172.21.15.42:0/588354580 conn(0x7ff2400fcb50 msgr2=0x7ff240073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:40.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.649+0000 7ff245400640 1 -- 172.21.15.42:0/588354580 shutdown_connections 2024-04-25T00:15:40.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:40.649+0000 7ff245400640 1 -- 172.21.15.42:0/588354580 wait complete. 2024-04-25T00:15:40.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:40 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6832/756794270,v1:172.21.15.136:6833/756794270] up:boot 2024-04-25T00:15:40.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:40 smithi136 ceph-mon[32923]: fsmap 1 up:standby 2024-04-25T00:15:40.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T00:15:40.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:40.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:40.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:40.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:40.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:40.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:40 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:40.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:40 smithi136 ceph-mon[32923]: Deploying daemon mds.k on smithi106 2024-04-25T00:15:41.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:41 smithi042 ceph-mon[28130]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:41.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:41 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/588354580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:41 smithi136 ceph-mon[32923]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:41 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/588354580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:42.069 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:41 smithi106 ceph-mon[32510]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:42.069 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:41 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/588354580' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:42.069 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:15:41 smithi106 systemd[1]: Starting Ceph mds.k for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:15:42.337 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:15:42 smithi106 podman[62432]: 2024-04-25 00:15:41.971566992 +0000 UTC m=+0.017924254 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:15:42.337 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:15:42 smithi106 podman[62432]: 2024-04-25 00:15:42.076556324 +0000 UTC m=+0.122913613 container create 9af6b9f04f64e1956c88d9281b468db6d238ca97912d3124a13ded533bce4f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:15:42.337 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:15:42 smithi106 podman[62432]: 2024-04-25 00:15:42.337181732 +0000 UTC m=+0.383538979 container init 9af6b9f04f64e1956c88d9281b468db6d238ca97912d3124a13ded533bce4f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-k, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-25T00:15:42.588 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:15:42 smithi106 podman[62432]: 2024-04-25 00:15:42.340200878 +0000 UTC m=+0.386558135 container start 9af6b9f04f64e1956c88d9281b468db6d238ca97912d3124a13ded533bce4f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True) 2024-04-25T00:15:42.588 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:15:42 smithi106 ceph-mds[62451]: -- 172.21.15.106:0/989220358 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55831efb1a40 con 0x55831ef85000 2024-04-25T00:15:42.588 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:15:42 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-k[62447]: starting mds.k at 2024-04-25T00:15:42.588 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:15:42 smithi106 bash[62432]: 9af6b9f04f64e1956c88d9281b468db6d238ca97912d3124a13ded533bce4f6e 2024-04-25T00:15:42.588 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:15:42 smithi106 systemd[1]: Started Ceph mds.k for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:15:42.669 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:42.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.872+0000 7fb95ea00640 1 -- 172.21.15.42:0/1137657721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9581006c0 msgr2=0x7fb958100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.872+0000 7fb95ea00640 1 --2- 172.21.15.42:0/1137657721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9581006c0 0x7fb958100ae0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb948009900 tx=0x7fb94802a400 comp rx=0 tx=0).stop 2024-04-25T00:15:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.872+0000 7fb95ea00640 1 -- 172.21.15.42:0/1137657721 shutdown_connections 2024-04-25T00:15:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.872+0000 7fb95ea00640 1 --2- 172.21.15.42:0/1137657721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb958102270 0x7fb958108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.872+0000 7fb95ea00640 1 --2- 172.21.15.42:0/1137657721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9581018b0 0x7fb958101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.872+0000 7fb95ea00640 1 --2- 172.21.15.42:0/1137657721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9581006c0 0x7fb958100ae0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.872+0000 7fb95ea00640 1 -- 172.21.15.42:0/1137657721 >> 172.21.15.42:0/1137657721 conn(0x7fb9580fbc30 msgr2=0x7fb9580fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.872+0000 7fb95ea00640 1 -- 172.21.15.42:0/1137657721 shutdown_connections 2024-04-25T00:15:42.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.872+0000 7fb95ea00640 1 -- 172.21.15.42:0/1137657721 wait complete. 2024-04-25T00:15:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.873+0000 7fb95ea00640 1 Processor -- start 2024-04-25T00:15:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.873+0000 7fb95ea00640 1 -- start start 2024-04-25T00:15:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.873+0000 7fb95ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9581006c0 0x7fb958199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.873+0000 7fb95ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9581018b0 0x7fb95819a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.873+0000 7fb95ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb958102270 0x7fb9581a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.873+0000 7fb95ea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb95810c380 con 0x7fb9581006c0 2024-04-25T00:15:42.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.873+0000 7fb95ea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb95810c200 con 0x7fb958102270 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.873+0000 7fb95ea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb95810c500 con 0x7fb9581018b0 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb957e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9581006c0 0x7fb958199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9581018b0 0x7fb95819a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb957e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9581006c0 0x7fb958199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49702/0 (socket says 172.21.15.42:49702) 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb957e00640 1 -- 172.21.15.42:0/2674471215 learned_addr learned my addr 172.21.15.42:0/2674471215 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb95ca00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb958102270 0x7fb9581a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb95ca00640 1 -- 172.21.15.42:0/2674471215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9581018b0 msgr2=0x7fb95819a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb95ca00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9581018b0 0x7fb95819a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb95ca00640 1 -- 172.21.15.42:0/2674471215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9581006c0 msgr2=0x7fb958199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb95ca00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9581006c0 0x7fb958199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb95ca00640 1 -- 172.21.15.42:0/2674471215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9581a1960 con 0x7fb958102270 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb94fe00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9581018b0 0x7fb95819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.874+0000 7fb95ca00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb958102270 0x7fb9581a12f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb95000e940 tx=0x7fb95000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.875+0000 7fb955600640 1 -- 172.21.15.42:0/2674471215 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950004d20 con 0x7fb958102270 2024-04-25T00:15:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.875+0000 7fb95ea00640 1 -- 172.21.15.42:0/2674471215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9581a1c50 con 0x7fb958102270 2024-04-25T00:15:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.875+0000 7fb955600640 1 -- 172.21.15.42:0/2674471215 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb950005020 con 0x7fb958102270 2024-04-25T00:15:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.875+0000 7fb955600640 1 -- 172.21.15.42:0/2674471215 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9500116b0 con 0x7fb958102270 2024-04-25T00:15:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.875+0000 7fb95ea00640 1 -- 172.21.15.42:0/2674471215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9581a21a0 con 0x7fb958102270 2024-04-25T00:15:42.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.876+0000 7fb95ea00640 1 -- 172.21.15.42:0/2674471215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9581a23d0 con 0x7fb958102270 2024-04-25T00:15:42.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.878+0000 7fb955600640 1 -- 172.21.15.42:0/2674471215 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb9500054d0 con 0x7fb958102270 2024-04-25T00:15:42.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.879+0000 7fb955600640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb93c0a2b90 0x7fb93c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:42.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.879+0000 7fb955600640 1 -- 172.21.15.42:0/2674471215 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb9500ceca0 con 0x7fb958102270 2024-04-25T00:15:42.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.880+0000 7fb957e00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb93c0a2b90 0x7fb93c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:42.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.880+0000 7fb957e00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb93c0a2b90 0x7fb93c0a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb948000c00 tx=0x7fb948005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:42.883 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:42.884+0000 7fb955600640 1 -- 172.21.15.42:0/2674471215 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb950016030 con 0x7fb958102270 2024-04-25T00:15:43.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.132+0000 7fb95ea00640 1 -- 172.21.15.42:0/2674471215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9581a25a0 con 0x7fb958102270 2024-04-25T00:15:43.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.132+0000 7fb955600640 1 -- 172.21.15.42:0/2674471215 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fb95009b9b0 con 0x7fb958102270 2024-04-25T00:15:43.133 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:43.133 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T00:15:43.133 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 3 2024-04-25T00:15:43.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 -- 172.21.15.42:0/2674471215 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb93c0a2b90 msgr2=0x7fb93c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:43.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb93c0a2b90 0x7fb93c0a5050 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb948000c00 tx=0x7fb948005a20 comp rx=0 tx=0).stop 2024-04-25T00:15:43.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 -- 172.21.15.42:0/2674471215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb958102270 msgr2=0x7fb9581a12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:43.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb958102270 0x7fb9581a12f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb95000e940 tx=0x7fb95000ee00 comp rx=0 tx=0).stop 2024-04-25T00:15:43.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 -- 172.21.15.42:0/2674471215 shutdown_connections 2024-04-25T00:15:43.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb93c0a2b90 0x7fb93c0a5050 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:43.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb958102270 0x7fb9581a12f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:43.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9581018b0 0x7fb95819a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:43.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 --2- 172.21.15.42:0/2674471215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9581006c0 0x7fb958199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:43.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 -- 172.21.15.42:0/2674471215 >> 172.21.15.42:0/2674471215 conn(0x7fb9580fbc30 msgr2=0x7fb9580fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:43.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.135+0000 7fb95ea00640 1 -- 172.21.15.42:0/2674471215 shutdown_connections 2024-04-25T00:15:43.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:43.136+0000 7fb95ea00640 1 -- 172.21.15.42:0/2674471215 wait complete. 2024-04-25T00:15:43.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:43.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:43.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:43.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:43.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:43.763 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:43.764 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:43.764 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: Deploying daemon mds.j on smithi042 2024-04-25T00:15:43.764 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.106:6832/1706914889,v1:172.21.15.106:6833/1706914889] up:boot 2024-04-25T00:15:43.764 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: fsmap 2 up:standby 2024-04-25T00:15:43.764 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:15:43.764 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:43 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2674471215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:43.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:43.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:43.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:43.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:43.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:43.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:43.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:43.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: Deploying daemon mds.j on smithi042 2024-04-25T00:15:43.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.106:6832/1706914889,v1:172.21.15.106:6833/1706914889] up:boot 2024-04-25T00:15:43.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: fsmap 2 up:standby 2024-04-25T00:15:43.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:15:43.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:43 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2674471215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:43.807 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:43.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:43.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:43.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:43.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:43.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:43.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:43.808 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: Deploying daemon mds.j on smithi042 2024-04-25T00:15:43.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.106:6832/1706914889,v1:172.21.15.106:6833/1706914889] up:boot 2024-04-25T00:15:43.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: fsmap 2 up:standby 2024-04-25T00:15:43.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T00:15:43.809 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:43 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2674471215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:45.158 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.387+0000 7fd4cdc00640 1 -- 172.21.15.42:0/340773256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4c810b0b0 msgr2=0x7fd4c810d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.387+0000 7fd4cdc00640 1 --2- 172.21.15.42:0/340773256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4c810b0b0 0x7fd4c810d540 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009ca0 tx=0x7fd4c002a8e0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.388+0000 7fd4cdc00640 1 -- 172.21.15.42:0/340773256 shutdown_connections 2024-04-25T00:15:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.388+0000 7fd4cdc00640 1 --2- 172.21.15.42:0/340773256 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4c810da80 0x7fd4c810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.388+0000 7fd4cdc00640 1 --2- 172.21.15.42:0/340773256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4c810b0b0 0x7fd4c810d540 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.388+0000 7fd4cdc00640 1 --2- 172.21.15.42:0/340773256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c8108740 0x7fd4c810ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.388+0000 7fd4cdc00640 1 -- 172.21.15.42:0/340773256 >> 172.21.15.42:0/340773256 conn(0x7fd4c806e5e0 msgr2=0x7fd4c8070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:45.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.388+0000 7fd4cdc00640 1 -- 172.21.15.42:0/340773256 shutdown_connections 2024-04-25T00:15:45.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.388+0000 7fd4cdc00640 1 -- 172.21.15.42:0/340773256 wait complete. 2024-04-25T00:15:45.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.389+0000 7fd4cdc00640 1 Processor -- start 2024-04-25T00:15:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.390+0000 7fd4cdc00640 1 -- start start 2024-04-25T00:15:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.390+0000 7fd4cdc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c8108740 0x7fd4c81a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.390+0000 7fd4cdc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4c810b0b0 0x7fd4c81a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.390+0000 7fd4cdc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4c810da80 0x7fd4c81ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.390+0000 7fd4c6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c8108740 0x7fd4c81a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.390+0000 7fd4c6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c8108740 0x7fd4c81a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44638/0 (socket says 172.21.15.42:44638) 2024-04-25T00:15:45.391 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4c6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4c810b0b0 0x7fd4c81a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4c6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4c810b0b0 0x7fd4c81a6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49720/0 (socket says 172.21.15.42:49720) 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4cdc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c811c270 con 0x7fd4c810b0b0 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4cdc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c811c0f0 con 0x7fd4c8108740 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4cdc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c811c3f0 con 0x7fd4c810da80 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4c6a00640 1 -- 172.21.15.42:0/680443150 learned_addr learned my addr 172.21.15.42:0/680443150 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4c7400640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4c810da80 0x7fd4c81ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4c6a00640 1 -- 172.21.15.42:0/680443150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4c810da80 msgr2=0x7fd4c81ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4c6a00640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4c810da80 0x7fd4c81ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4c6a00640 1 -- 172.21.15.42:0/680443150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4c810b0b0 msgr2=0x7fd4c81a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4c6a00640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4c810b0b0 0x7fd4c81a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.392 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.391+0000 7fd4c6a00640 1 -- 172.21.15.42:0/680443150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c81ae4c0 con 0x7fd4c8108740 2024-04-25T00:15:45.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.392+0000 7fd4c6a00640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c8108740 0x7fd4c81a68f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800c920 tx=0x7fd4b800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:45.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.392+0000 7fd4bf400640 1 -- 172.21.15.42:0/680443150 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8007c70 con 0x7fd4c8108740 2024-04-25T00:15:45.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.392+0000 7fd4bf400640 1 -- 172.21.15.42:0/680443150 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4b8007e10 con 0x7fd4c8108740 2024-04-25T00:15:45.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.392+0000 7fd4bf400640 1 -- 172.21.15.42:0/680443150 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8018890 con 0x7fd4c8108740 2024-04-25T00:15:45.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.393+0000 7fd4cdc00640 1 -- 172.21.15.42:0/680443150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c81ae7b0 con 0x7fd4c8108740 2024-04-25T00:15:45.393 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.393+0000 7fd4cdc00640 1 -- 172.21.15.42:0/680443150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c81aed00 con 0x7fd4c8108740 2024-04-25T00:15:45.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.395+0000 7fd4cdc00640 1 -- 172.21.15.42:0/680443150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4c8068560 con 0x7fd4c8108740 2024-04-25T00:15:45.396 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.397+0000 7fd4bf400640 1 -- 172.21.15.42:0/680443150 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd4b8018a50 con 0x7fd4c8108740 2024-04-25T00:15:45.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.399+0000 7fd4bf400640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd4ac0a2c10 0x7fd4ac0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:45.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.399+0000 7fd4bf400640 1 -- 172.21.15.42:0/680443150 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd4b80ced30 con 0x7fd4c8108740 2024-04-25T00:15:45.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.400+0000 7fd4c6000640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd4ac0a2c10 0x7fd4ac0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:45.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.401+0000 7fd4c6000640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd4ac0a2c10 0x7fd4ac0a50d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0006010 tx=0x7fd4c0009850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:45.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.402+0000 7fd4bf400640 1 -- 172.21.15.42:0/680443150 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd4b8014030 con 0x7fd4c8108740 2024-04-25T00:15:45.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.653+0000 7fd4cdc00640 1 -- 172.21.15.42:0/680443150 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4c8062340 con 0x7fd4c8108740 2024-04-25T00:15:45.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.654+0000 7fd4bf400640 1 -- 172.21.15.42:0/680443150 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fd4b809c090 con 0x7fd4c8108740 2024-04-25T00:15:45.654 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:45.655 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T00:15:45.655 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 3 2024-04-25T00:15:45.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 -- 172.21.15.42:0/680443150 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd4ac0a2c10 msgr2=0x7fd4ac0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:45.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd4ac0a2c10 0x7fd4ac0a50d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0006010 tx=0x7fd4c0009850 comp rx=0 tx=0).stop 2024-04-25T00:15:45.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 -- 172.21.15.42:0/680443150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c8108740 msgr2=0x7fd4c81a68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:45.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c8108740 0x7fd4c81a68f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800c920 tx=0x7fd4b800cde0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 -- 172.21.15.42:0/680443150 shutdown_connections 2024-04-25T00:15:45.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd4ac0a2c10 0x7fd4ac0a50d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd4c810da80 0x7fd4c81ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd4c810b0b0 0x7fd4c81a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 --2- 172.21.15.42:0/680443150 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c8108740 0x7fd4c81a68f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:45.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 -- 172.21.15.42:0/680443150 >> 172.21.15.42:0/680443150 conn(0x7fd4c806e5e0 msgr2=0x7fd4c8109cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:45.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 -- 172.21.15.42:0/680443150 shutdown_connections 2024-04-25T00:15:45.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:45.657+0000 7fd4cdc00640 1 -- 172.21.15.42:0/680443150 wait complete. 2024-04-25T00:15:45.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:45 smithi136 ceph-mon[32923]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:45.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:45 smithi106 ceph-mon[32510]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:45.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:45 smithi042 ceph-mon[28130]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:46.375 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:15:46 smithi042 podman[68040]: 2024-04-25 00:15:46.018744258 +0000 UTC m=+0.156885888 container create cc36f7531251f245b5801273ceb2c04256c317476261b1b4057caa52e92de253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T00:15:46.375 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:15:46 smithi042 podman[68040]: 2024-04-25 00:15:46.37284664 +0000 UTC m=+0.510988294 container init cc36f7531251f245b5801273ceb2c04256c317476261b1b4057caa52e92de253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-j, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T00:15:46.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:46 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/680443150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:46.627 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:15:46 smithi042 podman[68040]: 2024-04-25 00:15:46.3846307 +0000 UTC m=+0.522772353 container start cc36f7531251f245b5801273ceb2c04256c317476261b1b4057caa52e92de253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:15:46.627 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:15:46 smithi042 ceph-mds[68059]: -- 172.21.15.42:0/1089698136 <== mon.0 v2:172.21.15.42:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff0c28fa40 con 0x55ff0c262c00 2024-04-25T00:15:46.627 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:15:46 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-j[68055]: starting mds.j at 2024-04-25T00:15:46.627 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:15:46 smithi042 bash[68040]: cc36f7531251f245b5801273ceb2c04256c317476261b1b4057caa52e92de253 2024-04-25T00:15:46.627 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:15:46 smithi042 systemd[1]: Started Ceph mds.j for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:15:46.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:46 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/680443150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:47.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:46 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/680443150' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:47.677 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.42:6834/1318501714,v1:172.21.15.42:6835/1318501714] up:boot 2024-04-25T00:15:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: fsmap 3 up:standby 2024-04-25T00:15:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:15:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:47.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:47.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:47.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:47.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:47.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:47.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:47 smithi106 ceph-mon[32510]: Deploying daemon mds.d on smithi042 2024-04-25T00:15:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.42:6834/1318501714,v1:172.21.15.42:6835/1318501714] up:boot 2024-04-25T00:15:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: fsmap 3 up:standby 2024-04-25T00:15:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:15:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:47.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:47.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:47.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:47 smithi042 ceph-mon[28130]: Deploying daemon mds.d on smithi042 2024-04-25T00:15:47.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.893+0000 7fe7c7e00640 1 -- 172.21.15.42:0/3729545992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7c81016b0 msgr2=0x7fe7c8101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:47.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.893+0000 7fe7c7e00640 1 --2- 172.21.15.42:0/3729545992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7c81016b0 0x7fe7c8101b30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0009900 tx=0x7fe7b002a410 comp rx=0 tx=0).stop 2024-04-25T00:15:47.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.893+0000 7fe7c7e00640 1 -- 172.21.15.42:0/3729545992 shutdown_connections 2024-04-25T00:15:47.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.893+0000 7fe7c7e00640 1 --2- 172.21.15.42:0/3729545992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7c8102070 0x7fe7c8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.893+0000 7fe7c7e00640 1 --2- 172.21.15.42:0/3729545992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7c81016b0 0x7fe7c8101b30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.893+0000 7fe7c7e00640 1 --2- 172.21.15.42:0/3729545992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c81004c0 0x7fe7c81008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.893+0000 7fe7c7e00640 1 -- 172.21.15.42:0/3729545992 >> 172.21.15.42:0/3729545992 conn(0x7fe7c80fba10 msgr2=0x7fe7c80fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:47.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.893+0000 7fe7c7e00640 1 -- 172.21.15.42:0/3729545992 shutdown_connections 2024-04-25T00:15:47.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.893+0000 7fe7c7e00640 1 -- 172.21.15.42:0/3729545992 wait complete. 2024-04-25T00:15:47.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.894+0000 7fe7c7e00640 1 Processor -- start 2024-04-25T00:15:47.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.894+0000 7fe7c7e00640 1 -- start start 2024-04-25T00:15:47.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c81004c0 0x7fe7c810d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:47.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7c81016b0 0x7fe7c810bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:47.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7c8102070 0x7fe7c810c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:47.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8068ea0 con 0x7fe7c8102070 2024-04-25T00:15:47.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c7e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8068d20 con 0x7fe7c81004c0 2024-04-25T00:15:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c8069020 con 0x7fe7c81016b0 2024-04-25T00:15:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7c8102070 0x7fe7c810c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c81004c0 0x7fe7c810d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7c8102070 0x7fe7c810c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52718/0 (socket says 172.21.15.42:52718) 2024-04-25T00:15:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c7400640 1 -- 172.21.15.42:0/1320050195 learned_addr learned my addr 172.21.15.42:0/1320050195 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:47.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7be000640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7c81016b0 0x7fe7c810bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c6a00640 1 -- 172.21.15.42:0/1320050195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7c81016b0 msgr2=0x7fe7c810bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c6a00640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7c81016b0 0x7fe7c810bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c6a00640 1 -- 172.21.15.42:0/1320050195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7c8102070 msgr2=0x7fe7c810c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c6a00640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7c8102070 0x7fe7c810c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c6a00640 1 -- 172.21.15.42:0/1320050195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c810cb60 con 0x7fe7c81004c0 2024-04-25T00:15:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.895+0000 7fe7c7400640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7c8102070 0x7fe7c810c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.896+0000 7fe7c6a00640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c81004c0 0x7fe7c810d750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e9c0 tx=0x7fe7b800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:47.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.896+0000 7fe7bfe00640 1 -- 172.21.15.42:0/1320050195 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800ccc0 con 0x7fe7c81004c0 2024-04-25T00:15:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.896+0000 7fe7bfe00640 1 -- 172.21.15.42:0/1320050195 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe7b80035f0 con 0x7fe7c81004c0 2024-04-25T00:15:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.896+0000 7fe7c7e00640 1 -- 172.21.15.42:0/1320050195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c81a19f0 con 0x7fe7c81004c0 2024-04-25T00:15:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.896+0000 7fe7bfe00640 1 -- 172.21.15.42:0/1320050195 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b800ce60 con 0x7fe7c81004c0 2024-04-25T00:15:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.896+0000 7fe7c7e00640 1 -- 172.21.15.42:0/1320050195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c81a1f10 con 0x7fe7c81004c0 2024-04-25T00:15:47.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.897+0000 7fe7c7e00640 1 -- 172.21.15.42:0/1320050195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7c8068480 con 0x7fe7c81004c0 2024-04-25T00:15:47.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.899+0000 7fe7bfe00640 1 -- 172.21.15.42:0/1320050195 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe7b80037a0 con 0x7fe7c81004c0 2024-04-25T00:15:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.901+0000 7fe7bfe00640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe7ac0a2b90 0x7fe7ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.901+0000 7fe7bfe00640 1 -- 172.21.15.42:0/1320050195 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe7b80cefd0 con 0x7fe7c81004c0 2024-04-25T00:15:47.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.901+0000 7fe7be000640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe7ac0a2b90 0x7fe7ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:47.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.902+0000 7fe7be000640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe7ac0a2b90 0x7fe7ac0a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0031440 tx=0x7fe7b00315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:47.905 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:47.905+0000 7fe7bfe00640 1 -- 172.21.15.42:0/1320050195 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe7b809c370 con 0x7fe7c81004c0 2024-04-25T00:15:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.42:6834/1318501714,v1:172.21.15.42:6835/1318501714] up:boot 2024-04-25T00:15:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: fsmap 3 up:standby 2024-04-25T00:15:48.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T00:15:48.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:48.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:48.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:48.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:48.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:48.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:48.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:47 smithi136 ceph-mon[32923]: Deploying daemon mds.d on smithi042 2024-04-25T00:15:48.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.162+0000 7fe7c7e00640 1 -- 172.21.15.42:0/1320050195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7c8107710 con 0x7fe7c81004c0 2024-04-25T00:15:48.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.163+0000 7fe7bfe00640 1 -- 172.21.15.42:0/1320050195 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fe7b809bc70 con 0x7fe7c81004c0 2024-04-25T00:15:48.163 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:48.164 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T00:15:48.164 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 4 2024-04-25T00:15:48.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 -- 172.21.15.42:0/1320050195 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe7ac0a2b90 msgr2=0x7fe7ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:48.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe7ac0a2b90 0x7fe7ac0a5050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fe7b0031440 tx=0x7fe7b00315b0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 -- 172.21.15.42:0/1320050195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c81004c0 msgr2=0x7fe7c810d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:48.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c81004c0 0x7fe7c810d750 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe7b800e9c0 tx=0x7fe7b800ee80 comp rx=0 tx=0).stop 2024-04-25T00:15:48.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 -- 172.21.15.42:0/1320050195 shutdown_connections 2024-04-25T00:15:48.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe7ac0a2b90 0x7fe7ac0a5050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe7c8102070 0x7fe7c810c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7c81016b0 0x7fe7c810bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 --2- 172.21.15.42:0/1320050195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe7c81004c0 0x7fe7c810d750 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:48.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 -- 172.21.15.42:0/1320050195 >> 172.21.15.42:0/1320050195 conn(0x7fe7c80fba10 msgr2=0x7fe7c81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:48.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 -- 172.21.15.42:0/1320050195 shutdown_connections 2024-04-25T00:15:48.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:48.166+0000 7fe7c7e00640 1 -- 172.21.15.42:0/1320050195 wait complete. 2024-04-25T00:15:48.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:48 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1320050195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:48.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:48 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1320050195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:49.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:48 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1320050195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:49.701 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:49 smithi042 ceph-mon[28130]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:49.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:49 smithi106 ceph-mon[32510]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:49 smithi136 ceph-mon[32923]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:50.073 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:15:49 smithi042 systemd[1]: Starting Ceph mds.d for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:15:50.186 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:50.377 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:15:50 smithi042 podman[68384]: 2024-04-25 00:15:49.977449071 +0000 UTC m=+0.040815803 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:15:50.377 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:15:50 smithi042 podman[68384]: 2024-04-25 00:15:50.122524008 +0000 UTC m=+0.185890777 container create 2e34cbe51038388e38e7f8777db54d55fa138be2505fc0b61a1523d5ef8e44bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T00:15:50.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.400+0000 7fd5c3000640 1 -- 172.21.15.42:0/801184106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5bc104b90 msgr2=0x7fd5bc106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.400+0000 7fd5c3000640 1 --2- 172.21.15.42:0/801184106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5bc104b90 0x7fd5bc106f80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4009900 tx=0x7fd5a402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.400+0000 7fd5c3000640 1 -- 172.21.15.42:0/801184106 shutdown_connections 2024-04-25T00:15:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.400+0000 7fd5c3000640 1 --2- 172.21.15.42:0/801184106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5bc1074c0 0x7fd5bc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.400+0000 7fd5c3000640 1 --2- 172.21.15.42:0/801184106 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5bc104b90 0x7fd5bc106f80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.400+0000 7fd5c3000640 1 --2- 172.21.15.42:0/801184106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5bc102260 0x7fd5bc104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.400+0000 7fd5c3000640 1 -- 172.21.15.42:0/801184106 >> 172.21.15.42:0/801184106 conn(0x7fd5bc0fbbf0 msgr2=0x7fd5bc0fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.400+0000 7fd5c3000640 1 -- 172.21.15.42:0/801184106 shutdown_connections 2024-04-25T00:15:50.401 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.400+0000 7fd5c3000640 1 -- 172.21.15.42:0/801184106 wait complete. 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.401+0000 7fd5c3000640 1 Processor -- start 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.401+0000 7fd5c3000640 1 -- start start 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5c3000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5bc102260 0x7fd5bc199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5c3000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5bc104b90 0x7fd5bc19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5c3000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5bc1074c0 0x7fd5bc1a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5c3000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc10d420 con 0x7fd5bc1074c0 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5c3000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc10d2a0 con 0x7fd5bc102260 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5c3000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc10d5a0 con 0x7fd5bc104b90 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5bbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5bc102260 0x7fd5bc199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5c1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5bc1074c0 0x7fd5bc1a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5bbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5bc102260 0x7fd5bc199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:36604/0 (socket says 172.21.15.42:36604) 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5bbe00640 1 -- 172.21.15.42:0/1661385793 learned_addr learned my addr 172.21.15.42:0/1661385793 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5b3e00640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5bc104b90 0x7fd5bc19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5bbe00640 1 -- 172.21.15.42:0/1661385793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5bc104b90 msgr2=0x7fd5bc19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5bbe00640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5bc104b90 0x7fd5bc19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5bbe00640 1 -- 172.21.15.42:0/1661385793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5bc1074c0 msgr2=0x7fd5bc1a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5bbe00640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5bc1074c0 0x7fd5bc1a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.402+0000 7fd5bbe00640 1 -- 172.21.15.42:0/1661385793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc1a1900 con 0x7fd5bc102260 2024-04-25T00:15:50.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.403+0000 7fd5bbe00640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5bc102260 0x7fd5bc199cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00e9f0 tx=0x7fd5ac00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:50.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.403+0000 7fd5b9600640 1 -- 172.21.15.42:0/1661385793 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac00ccc0 con 0x7fd5bc102260 2024-04-25T00:15:50.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.403+0000 7fd5b9600640 1 -- 172.21.15.42:0/1661385793 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5ac010910 con 0x7fd5bc102260 2024-04-25T00:15:50.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.403+0000 7fd5c3000640 1 -- 172.21.15.42:0/1661385793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc1a1bf0 con 0x7fd5bc102260 2024-04-25T00:15:50.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.403+0000 7fd5b9600640 1 -- 172.21.15.42:0/1661385793 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0188a0 con 0x7fd5bc102260 2024-04-25T00:15:50.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.403+0000 7fd5c3000640 1 -- 172.21.15.42:0/1661385793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc1a2140 con 0x7fd5bc102260 2024-04-25T00:15:50.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.404+0000 7fd5c3000640 1 -- 172.21.15.42:0/1661385793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5bc108fb0 con 0x7fd5bc102260 2024-04-25T00:15:50.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.406+0000 7fd5b9600640 1 -- 172.21.15.42:0/1661385793 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd5ac010ac0 con 0x7fd5bc102260 2024-04-25T00:15:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.408+0000 7fd5b9600640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5a00a2b90 0x7fd5a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.408+0000 7fd5b9600640 1 -- 172.21.15.42:0/1661385793 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd5ac0cf0c0 con 0x7fd5bc102260 2024-04-25T00:15:50.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.408+0000 7fd5b3e00640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5a00a2b90 0x7fd5a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:50.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.409+0000 7fd5b3e00640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5a00a2b90 0x7fd5a00a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5a402a9e0 tx=0x7fd5a40315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:50.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.412+0000 7fd5b9600640 1 -- 172.21.15.42:0/1661385793 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd5ac09c420 con 0x7fd5bc102260 2024-04-25T00:15:50.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.679+0000 7fd5c3000640 1 -- 172.21.15.42:0/1661385793 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5bc062370 con 0x7fd5bc102260 2024-04-25T00:15:50.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.680+0000 7fd5b9600640 1 -- 172.21.15.42:0/1661385793 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7fd5ac09bd20 con 0x7fd5bc102260 2024-04-25T00:15:50.681 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:50.681 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T00:15:50.681 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 5 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 -- 172.21.15.42:0/1661385793 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5a00a2b90 msgr2=0x7fd5a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5a00a2b90 0x7fd5a00a5050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd5a402a9e0 tx=0x7fd5a40315b0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 -- 172.21.15.42:0/1661385793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5bc102260 msgr2=0x7fd5bc199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5bc102260 0x7fd5bc199cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00e9f0 tx=0x7fd5ac00edf0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 -- 172.21.15.42:0/1661385793 shutdown_connections 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5a00a2b90 0x7fd5a00a5050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5bc1074c0 0x7fd5bc1a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5bc104b90 0x7fd5bc19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 --2- 172.21.15.42:0/1661385793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5bc102260 0x7fd5bc199cf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 -- 172.21.15.42:0/1661385793 >> 172.21.15.42:0/1661385793 conn(0x7fd5bc0fbbf0 msgr2=0x7fd5bc101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 -- 172.21.15.42:0/1661385793 shutdown_connections 2024-04-25T00:15:50.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:50.683+0000 7fd5c3000640 1 -- 172.21.15.42:0/1661385793 wait complete. 2024-04-25T00:15:50.700 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:15:50 smithi042 podman[68384]: 2024-04-25 00:15:50.385257633 +0000 UTC m=+0.448624408 container init 2e34cbe51038388e38e7f8777db54d55fa138be2505fc0b61a1523d5ef8e44bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-d, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T00:15:50.701 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:15:50 smithi042 podman[68384]: 2024-04-25 00:15:50.398843453 +0000 UTC m=+0.462210203 container start 2e34cbe51038388e38e7f8777db54d55fa138be2505fc0b61a1523d5ef8e44bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-d, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:15:50.701 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:15:50 smithi042 ceph-mds[68433]: -- 172.21.15.42:0/3171953603 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d01181fa40 con 0x55d0117f2800 2024-04-25T00:15:50.701 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:15:50 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-d[68415]: starting mds.d at 2024-04-25T00:15:50.701 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:15:50 smithi042 bash[68384]: 2e34cbe51038388e38e7f8777db54d55fa138be2505fc0b61a1523d5ef8e44bf 2024-04-25T00:15:50.701 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:15:50 smithi042 systemd[1]: Started Ceph mds.d for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:15:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.42:6836/3589701125,v1:172.21.15.42:6837/3589701125] up:boot 2024-04-25T00:15:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: fsmap 4 up:standby 2024-04-25T00:15:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:15:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: Deploying daemon mds.e on smithi106 2024-04-25T00:15:51.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:51 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1661385793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:51.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:51.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.42:6836/3589701125,v1:172.21.15.42:6837/3589701125] up:boot 2024-04-25T00:15:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: fsmap 4 up:standby 2024-04-25T00:15:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:15:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: Deploying daemon mds.e on smithi106 2024-04-25T00:15:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:51 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1661385793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.42:6836/3589701125,v1:172.21.15.42:6837/3589701125] up:boot 2024-04-25T00:15:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: fsmap 4 up:standby 2024-04-25T00:15:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T00:15:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: Deploying daemon mds.e on smithi106 2024-04-25T00:15:52.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:51 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1661385793' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:52.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:52.838 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:15:52 smithi106 systemd[1]: Starting Ceph mds.e for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:15:52.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.933+0000 7f56e1000640 1 -- 172.21.15.42:0/2061998653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc1072a0 msgr2=0x7f56dc1096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.933+0000 7f56e1000640 1 --2- 172.21.15.42:0/2061998653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc1072a0 0x7f56dc1096f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009be0 tx=0x7f56d402a6d0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.933+0000 7f56e1000640 1 -- 172.21.15.42:0/2061998653 shutdown_connections 2024-04-25T00:15:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.933+0000 7f56e1000640 1 --2- 172.21.15.42:0/2061998653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc1072a0 0x7f56dc1096f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.933+0000 7f56e1000640 1 --2- 172.21.15.42:0/2061998653 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56dc104970 0x7f56dc106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.933+0000 7f56e1000640 1 --2- 172.21.15.42:0/2061998653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56dc102040 0x7f56dc104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.933+0000 7f56e1000640 1 -- 172.21.15.42:0/2061998653 >> 172.21.15.42:0/2061998653 conn(0x7f56dc0fba10 msgr2=0x7f56dc0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.933+0000 7f56e1000640 1 -- 172.21.15.42:0/2061998653 shutdown_connections 2024-04-25T00:15:52.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.933+0000 7f56e1000640 1 -- 172.21.15.42:0/2061998653 wait complete. 2024-04-25T00:15:52.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.935+0000 7f56e1000640 1 Processor -- start 2024-04-25T00:15:52.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.935+0000 7f56e1000640 1 -- start start 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.935+0000 7f56e1000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56dc102040 0x7f56dc1019b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.935+0000 7f56e1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56dc104970 0x7f56dc0ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.935+0000 7f56e1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc1072a0 0x7f56dc100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.935+0000 7f56e1000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc10d250 con 0x7f56dc104970 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.935+0000 7f56e1000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc10d0d0 con 0x7f56dc102040 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.935+0000 7f56e1000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc10d3d0 con 0x7f56dc1072a0 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc1072a0 0x7f56dc100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc1072a0 0x7f56dc100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44346/0 (socket says 172.21.15.42:44346) 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56dbe00640 1 -- 172.21.15.42:0/4267726482 learned_addr learned my addr 172.21.15.42:0/4267726482 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56db400640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56dc102040 0x7f56dc1019b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56daa00640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56dc104970 0x7f56dc0ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56dbe00640 1 -- 172.21.15.42:0/4267726482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56dc102040 msgr2=0x7f56dc1019b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56dbe00640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56dc102040 0x7f56dc1019b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56dbe00640 1 -- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56dc104970 msgr2=0x7f56dc0ffff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56dbe00640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56dc104970 0x7f56dc0ffff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56dbe00640 1 -- 172.21.15.42:0/4267726482 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc100dc0 con 0x7f56dc1072a0 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56daa00640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56dc104970 0x7f56dc0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.936+0000 7f56dbe00640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc1072a0 0x7f56dc100530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009b30 tx=0x7f56d40305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.937+0000 7f56d3e00640 1 -- 172.21.15.42:0/4267726482 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4041020 con 0x7f56dc1072a0 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.937+0000 7f56d3e00640 1 -- 172.21.15.42:0/4267726482 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56d4030740 con 0x7f56dc1072a0 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.937+0000 7f56d3e00640 1 -- 172.21.15.42:0/4267726482 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4036040 con 0x7f56dc1072a0 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.937+0000 7f56e1000640 1 -- 172.21.15.42:0/4267726482 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56dc1a1930 con 0x7f56dc1072a0 2024-04-25T00:15:52.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.937+0000 7f56e1000640 1 -- 172.21.15.42:0/4267726482 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56dc1a1ed0 con 0x7f56dc1072a0 2024-04-25T00:15:52.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.938+0000 7f56e1000640 1 -- 172.21.15.42:0/4267726482 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56dc0fd500 con 0x7f56dc1072a0 2024-04-25T00:15:52.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.940+0000 7f56d3e00640 1 -- 172.21.15.42:0/4267726482 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f56d4044020 con 0x7f56dc1072a0 2024-04-25T00:15:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.941+0000 7f56d3e00640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56c00a2c10 0x7f56c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:52.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.941+0000 7f56d3e00640 1 -- 172.21.15.42:0/4267726482 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f56d40efaa0 con 0x7f56dc1072a0 2024-04-25T00:15:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.941+0000 7f56db400640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56c00a2c10 0x7f56c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:52.942 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.942+0000 7f56db400640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56c00a2c10 0x7f56c00a50d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009b60 tx=0x7f56cc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:52.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:52.945+0000 7f56d3e00640 1 -- 172.21.15.42:0/4267726482 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f56d404b020 con 0x7f56dc1072a0 2024-04-25T00:15:53.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.197+0000 7f56e1000640 1 -- 172.21.15.42:0/4267726482 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56dc02d170 con 0x7f56dc1072a0 2024-04-25T00:15:53.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.198+0000 7f56d3e00640 1 -- 172.21.15.42:0/4267726482 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f56d403b3d0 con 0x7f56dc1072a0 2024-04-25T00:15:53.198 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:53.198 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T00:15:53.199 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 5 2024-04-25T00:15:53.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 -- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56c00a2c10 msgr2=0x7f56c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56c00a2c10 0x7f56c00a50d0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f56cc009b60 tx=0x7f56cc005f90 comp rx=0 tx=0).stop 2024-04-25T00:15:53.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 -- 172.21.15.42:0/4267726482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc1072a0 msgr2=0x7f56dc100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:53.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc1072a0 0x7f56dc100530 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f56d4009b30 tx=0x7f56d40305d0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 -- 172.21.15.42:0/4267726482 shutdown_connections 2024-04-25T00:15:53.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56c00a2c10 0x7f56c00a50d0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56dc1072a0 0x7f56dc100530 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f56dc104970 0x7f56dc0ffff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 --2- 172.21.15.42:0/4267726482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56dc102040 0x7f56dc1019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:53.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 -- 172.21.15.42:0/4267726482 >> 172.21.15.42:0/4267726482 conn(0x7f56dc0fba10 msgr2=0x7f56dc1029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:53.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 -- 172.21.15.42:0/4267726482 shutdown_connections 2024-04-25T00:15:53.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:53.201+0000 7f56e1000640 1 -- 172.21.15.42:0/4267726482 wait complete. 2024-04-25T00:15:53.203 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:15:52 smithi106 podman[62750]: 2024-04-25 00:15:52.845945328 +0000 UTC m=+0.017926028 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:15:53.203 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:15:52 smithi106 podman[62750]: 2024-04-25 00:15:52.967584164 +0000 UTC m=+0.139564845 container create 6b97b95500167e021bbf1385626bff6db9f458ede69957448ef7069a09af4ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2) 2024-04-25T00:15:53.203 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:15:53 smithi106 podman[62750]: 2024-04-25 00:15:53.203537054 +0000 UTC m=+0.375517747 container init 6b97b95500167e021bbf1385626bff6db9f458ede69957448ef7069a09af4ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T00:15:53.551 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:15:53 smithi106 podman[62750]: 2024-04-25 00:15:53.20651401 +0000 UTC m=+0.378494693 container start 6b97b95500167e021bbf1385626bff6db9f458ede69957448ef7069a09af4ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T00:15:53.551 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:15:53 smithi106 ceph-mds[62769]: -- 172.21.15.106:0/3461195734 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5579dc679a40 con 0x5579dc64cc00 2024-04-25T00:15:53.552 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:15:53 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-e[62765]: starting mds.e at 2024-04-25T00:15:53.552 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:15:53 smithi106 bash[62750]: 6b97b95500167e021bbf1385626bff6db9f458ede69957448ef7069a09af4ecb 2024-04-25T00:15:53.552 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:15:53 smithi106 systemd[1]: Started Ceph mds.e for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:15:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:53 smithi106 ceph-mon[32510]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:53 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/4267726482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:53 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:53.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:53 smithi042 ceph-mon[28130]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:53.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:53 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/4267726482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:53.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:53.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:53.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:53.854 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:53.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:53.855 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:53 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:53 smithi136 ceph-mon[32923]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:53 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/4267726482' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:54.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:54.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:54.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:53 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:54.582 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:54 smithi136 ceph-mon[32923]: Deploying daemon mds.f on smithi136 2024-04-25T00:15:54.582 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:54 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.106:6834/1507277236,v1:172.21.15.106:6835/1507277236] up:boot 2024-04-25T00:15:54.582 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:54 smithi136 ceph-mon[32923]: fsmap 5 up:standby 2024-04-25T00:15:54.583 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:15:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:54 smithi106 ceph-mon[32510]: Deploying daemon mds.f on smithi136 2024-04-25T00:15:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:54 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.106:6834/1507277236,v1:172.21.15.106:6835/1507277236] up:boot 2024-04-25T00:15:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:54 smithi106 ceph-mon[32510]: fsmap 5 up:standby 2024-04-25T00:15:54.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:15:54.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:54 smithi042 ceph-mon[28130]: Deploying daemon mds.f on smithi136 2024-04-25T00:15:54.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:54 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.106:6834/1507277236,v1:172.21.15.106:6835/1507277236] up:boot 2024-04-25T00:15:54.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:54 smithi042 ceph-mon[28130]: fsmap 5 up:standby 2024-04-25T00:15:54.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T00:15:55.223 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:55.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.445+0000 7f01aca00640 1 -- 172.21.15.42:0/833041644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a8105370 msgr2=0x7f01a810bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:55.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.445+0000 7f01aca00640 1 --2- 172.21.15.42:0/833041644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a8105370 0x7f01a810bc00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f01a0009ca0 tx=0x7f01a002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.446+0000 7f01aca00640 1 -- 172.21.15.42:0/833041644 shutdown_connections 2024-04-25T00:15:55.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.446+0000 7f01aca00640 1 --2- 172.21.15.42:0/833041644 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a8105370 0x7f01a810bc00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.446+0000 7f01aca00640 1 --2- 172.21.15.42:0/833041644 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01a81049b0 0x7f01a8104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.446+0000 7f01aca00640 1 --2- 172.21.15.42:0/833041644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a81037c0 0x7f01a8103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.446+0000 7f01aca00640 1 -- 172.21.15.42:0/833041644 >> 172.21.15.42:0/833041644 conn(0x7f01a80fed50 msgr2=0x7f01a8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:55.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.446+0000 7f01aca00640 1 -- 172.21.15.42:0/833041644 shutdown_connections 2024-04-25T00:15:55.446 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.446+0000 7f01aca00640 1 -- 172.21.15.42:0/833041644 wait complete. 2024-04-25T00:15:55.448 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01aca00640 1 Processor -- start 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01aca00640 1 -- start start 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01aca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01a81037c0 0x7f01a8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01aca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a81049b0 0x7f01a8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01aca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a8105370 0x7f01a81a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01aca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f01a810f420 con 0x7f01a81049b0 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01aca00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f01a810f2a0 con 0x7f01a81037c0 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01aca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01a810f5a0 con 0x7f01a8105370 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01a6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a81049b0 0x7f01a8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01a7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a8105370 0x7f01a81a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01a7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a8105370 0x7f01a81a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44372/0 (socket says 172.21.15.42:44372) 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01a7e00640 1 -- 172.21.15.42:0/607454911 learned_addr learned my addr 172.21.15.42:0/607454911 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.447+0000 7f01a7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01a81037c0 0x7f01a8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f01a7e00640 1 -- 172.21.15.42:0/607454911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01a81037c0 msgr2=0x7f01a8199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:55.449 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f01a7e00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01a81037c0 0x7f01a8199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f01a7e00640 1 -- 172.21.15.42:0/607454911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a81049b0 msgr2=0x7f01a8199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f01a7e00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a81049b0 0x7f01a8199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f01a7e00640 1 -- 172.21.15.42:0/607454911 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a81a1650 con 0x7f01a8105370 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f01a7400640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01a81037c0 0x7f01a8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f01a6a00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a81049b0 0x7f01a8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f01a7e00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a8105370 0x7f01a81a0fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01a0031ae0 tx=0x7f01a0031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f019fe00640 1 -- 172.21.15.42:0/607454911 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a0003b70 con 0x7f01a8105370 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f019fe00640 1 -- 172.21.15.42:0/607454911 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01a0003d10 con 0x7f01a8105370 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f01aca00640 1 -- 172.21.15.42:0/607454911 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a81a18e0 con 0x7f01a8105370 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.448+0000 7f019fe00640 1 -- 172.21.15.42:0/607454911 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a002fc40 con 0x7f01a8105370 2024-04-25T00:15:55.450 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.449+0000 7f01aca00640 1 -- 172.21.15.42:0/607454911 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a81a1e30 con 0x7f01a8105370 2024-04-25T00:15:55.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.450+0000 7f01aca00640 1 -- 172.21.15.42:0/607454911 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a810aae0 con 0x7f01a8105370 2024-04-25T00:15:55.451 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.452+0000 7f019fe00640 1 -- 172.21.15.42:0/607454911 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f01a0070b30 con 0x7f01a8105370 2024-04-25T00:15:55.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.453+0000 7f019fe00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f018c0a2b90 0x7f018c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:55.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.453+0000 7f019fe00640 1 -- 172.21.15.42:0/607454911 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f01a00efe70 con 0x7f01a8105370 2024-04-25T00:15:55.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.453+0000 7f01a7400640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f018c0a2b90 0x7f018c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:55.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.454+0000 7f01a7400640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f018c0a2b90 0x7f018c0a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01a8104810 tx=0x7f0198005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:55.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.456+0000 7f019fe00640 1 -- 172.21.15.42:0/607454911 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f01a0070e20 con 0x7f01a8105370 2024-04-25T00:15:55.513 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:15:55 smithi136 systemd[1]: Starting Ceph mds.f for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:15:55.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.703+0000 7f01aca00640 1 -- 172.21.15.42:0/607454911 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01a802d290 con 0x7f01a8105370 2024-04-25T00:15:55.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.704+0000 7f019fe00640 1 -- 172.21.15.42:0/607454911 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f01a00bd1d0 con 0x7f01a8105370 2024-04-25T00:15:55.705 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:55.705 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T00:15:55.705 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 6 2024-04-25T00:15:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 -- 172.21.15.42:0/607454911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f018c0a2b90 msgr2=0x7f018c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f018c0a2b90 0x7f018c0a5050 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f01a8104810 tx=0x7f0198005f90 comp rx=0 tx=0).stop 2024-04-25T00:15:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 -- 172.21.15.42:0/607454911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a8105370 msgr2=0x7f01a81a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a8105370 0x7f01a81a0fe0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f01a0031ae0 tx=0x7f01a0031ee0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 -- 172.21.15.42:0/607454911 shutdown_connections 2024-04-25T00:15:55.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f018c0a2b90 0x7f018c0a5050 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01a8105370 0x7f01a81a0fe0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f01a81049b0 0x7f01a8199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 --2- 172.21.15.42:0/607454911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01a81037c0 0x7f01a8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 -- 172.21.15.42:0/607454911 >> 172.21.15.42:0/607454911 conn(0x7f01a80fed50 msgr2=0x7f01a8107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 -- 172.21.15.42:0/607454911 shutdown_connections 2024-04-25T00:15:55.709 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:55.708+0000 7f01aca00640 1 -- 172.21.15.42:0/607454911 wait complete. 2024-04-25T00:15:55.828 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:55 smithi136 ceph-mon[32923]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:55.828 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:15:55 smithi136 podman[62981]: 2024-04-25 00:15:55.522443664 +0000 UTC m=+0.017775754 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:15:55.828 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:15:55 smithi136 podman[62981]: 2024-04-25 00:15:55.643985547 +0000 UTC m=+0.139317624 container create 2c248d79ecf85a2ae0cb57a5cbb975f950fcf2517792ccf7792b02e14856c577 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-25T00:15:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:55 smithi106 ceph-mon[32510]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:55 smithi042 ceph-mon[28130]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:56.263 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:15:55 smithi136 podman[62981]: 2024-04-25 00:15:55.921802461 +0000 UTC m=+0.417134539 container init 2c248d79ecf85a2ae0cb57a5cbb975f950fcf2517792ccf7792b02e14856c577 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T00:15:56.263 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:15:55 smithi136 podman[62981]: 2024-04-25 00:15:55.924295367 +0000 UTC m=+0.419627445 container start 2c248d79ecf85a2ae0cb57a5cbb975f950fcf2517792ccf7792b02e14856c577 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T00:15:56.263 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:15:55 smithi136 ceph-mds[63000]: -- 172.21.15.136:0/2779480185 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564a2f05da40 con 0x564a2f030800 2024-04-25T00:15:56.263 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:15:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f[62996]: starting mds.f at 2024-04-25T00:15:56.264 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:15:56 smithi136 bash[62981]: 2c248d79ecf85a2ae0cb57a5cbb975f950fcf2517792ccf7792b02e14856c577 2024-04-25T00:15:56.264 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:15:56 smithi136 systemd[1]: Started Ceph mds.f for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:15:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:56 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/607454911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:56.866 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:56 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/607454911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:56.866 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:56.866 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:56.866 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:56.866 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:56.866 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:56.866 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:56 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/607454911' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:15:57.730 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:15:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:57 smithi106 ceph-mon[32510]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:57 smithi106 ceph-mon[32510]: Deploying daemon mds.c on smithi136 2024-04-25T00:15:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:57 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6834/12913503,v1:172.21.15.136:6835/12913503] up:boot 2024-04-25T00:15:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:57 smithi106 ceph-mon[32510]: fsmap 6 up:standby 2024-04-25T00:15:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:15:57.846 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:57 smithi136 ceph-mon[32923]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:57.846 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:57 smithi136 ceph-mon[32923]: Deploying daemon mds.c on smithi136 2024-04-25T00:15:57.846 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:57 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6834/12913503,v1:172.21.15.136:6835/12913503] up:boot 2024-04-25T00:15:57.847 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:57 smithi136 ceph-mon[32923]: fsmap 6 up:standby 2024-04-25T00:15:57.847 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:15:57.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:57 smithi042 ceph-mon[28130]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:57.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:57 smithi042 ceph-mon[28130]: Deploying daemon mds.c on smithi136 2024-04-25T00:15:57.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:57 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6834/12913503,v1:172.21.15.136:6835/12913503] up:boot 2024-04-25T00:15:57.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:57 smithi042 ceph-mon[28130]: fsmap 6 up:standby 2024-04-25T00:15:57.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:15:57.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.957+0000 7f8d93000640 1 -- 172.21.15.42:0/2014176307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d8c107bc0 msgr2=0x7f8d8c107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:57.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.957+0000 7f8d93000640 1 --2- 172.21.15.42:0/2014176307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d8c107bc0 0x7f8d8c107fe0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d80009900 tx=0x7f8d8002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:15:57.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.957+0000 7f8d93000640 1 -- 172.21.15.42:0/2014176307 shutdown_connections 2024-04-25T00:15:57.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.957+0000 7f8d93000640 1 --2- 172.21.15.42:0/2014176307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c109770 0x7f8d8c110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:57.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.957+0000 7f8d93000640 1 --2- 172.21.15.42:0/2014176307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d8c108db0 0x7f8d8c109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:57.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.957+0000 7f8d93000640 1 --2- 172.21.15.42:0/2014176307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d8c107bc0 0x7f8d8c107fe0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:57.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.957+0000 7f8d93000640 1 -- 172.21.15.42:0/2014176307 >> 172.21.15.42:0/2014176307 conn(0x7f8d8c076b80 msgr2=0x7f8d8c078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:57.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.957+0000 7f8d93000640 1 -- 172.21.15.42:0/2014176307 shutdown_connections 2024-04-25T00:15:57.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.957+0000 7f8d93000640 1 -- 172.21.15.42:0/2014176307 wait complete. 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d93000640 1 Processor -- start 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d93000640 1 -- start start 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d93000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c107bc0 0x7f8d8c19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d93000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d8c108db0 0x7f8d8c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d93000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d8c109770 0x7f8d8c1a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d93000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c113860 con 0x7f8d8c109770 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d93000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c1136e0 con 0x7f8d8c108db0 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d93000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8d8c1139e0 con 0x7f8d8c107bc0 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d91c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c107bc0 0x7f8d8c19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d91c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c107bc0 0x7f8d8c19deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:52726/0 (socket says 172.21.15.42:52726) 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d91c00640 1 -- 172.21.15.42:0/4074048170 learned_addr learned my addr 172.21.15.42:0/4074048170 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:15:57.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.959+0000 7f8d92600640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d8c109770 0x7f8d8c1a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.960+0000 7f8d91200640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d8c108db0 0x7f8d8c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.960+0000 7f8d91c00640 1 -- 172.21.15.42:0/4074048170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d8c108db0 msgr2=0x7f8d8c19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.960+0000 7f8d91c00640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d8c108db0 0x7f8d8c19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.960+0000 7f8d91c00640 1 -- 172.21.15.42:0/4074048170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d8c109770 msgr2=0x7f8d8c1a5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.960+0000 7f8d91c00640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d8c109770 0x7f8d8c1a5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.960+0000 7f8d91c00640 1 -- 172.21.15.42:0/4074048170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8c1a5ad0 con 0x7f8d8c107bc0 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.960+0000 7f8d91c00640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c107bc0 0x7f8d8c19deb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d80031440 tx=0x7f8d80032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.960+0000 7f8d8a000640 1 -- 172.21.15.42:0/4074048170 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d8003f030 con 0x7f8d8c107bc0 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.961+0000 7f8d93000640 1 -- 172.21.15.42:0/4074048170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8c1a5d60 con 0x7f8d8c107bc0 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.961+0000 7f8d8a000640 1 -- 172.21.15.42:0/4074048170 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8d80003ce0 con 0x7f8d8c107bc0 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.961+0000 7f8d8a000640 1 -- 172.21.15.42:0/4074048170 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d80035070 con 0x7f8d8c107bc0 2024-04-25T00:15:57.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.961+0000 7f8d93000640 1 -- 172.21.15.42:0/4074048170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d8c1a6190 con 0x7f8d8c107bc0 2024-04-25T00:15:57.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.962+0000 7f8d93000640 1 -- 172.21.15.42:0/4074048170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d8c10efb0 con 0x7f8d8c107bc0 2024-04-25T00:15:57.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.964+0000 7f8d8a000640 1 -- 172.21.15.42:0/4074048170 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8d80042030 con 0x7f8d8c107bc0 2024-04-25T00:15:57.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.966+0000 7f8d8a000640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d740a2b90 0x7f8d740a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:15:57.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.966+0000 7f8d8a000640 1 -- 172.21.15.42:0/4074048170 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8d800ef0c0 con 0x7f8d8c107bc0 2024-04-25T00:15:57.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.966+0000 7f8d91200640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d740a2b90 0x7f8d740a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:15:57.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.966+0000 7f8d91200640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d740a2b90 0x7f8d740a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c19f3c0 tx=0x7f8d78005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:15:57.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:57.974+0000 7f8d8a000640 1 -- 172.21.15.42:0/4074048170 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8d800bc420 con 0x7f8d8c107bc0 2024-04-25T00:15:58.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.222+0000 7f8d93000640 1 -- 172.21.15.42:0/4074048170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8d8c062440 con 0x7f8d8c107bc0 2024-04-25T00:15:58.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.223+0000 7f8d8a000640 1 -- 172.21.15.42:0/4074048170 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f8d800bbd20 con 0x7f8d8c107bc0 2024-04-25T00:15:58.223 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:15:58.224 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-25T00:15:58.224 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 7 2024-04-25T00:15:58.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.227+0000 7f8d93000640 1 -- 172.21.15.42:0/4074048170 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d740a2b90 msgr2=0x7f8d740a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:58.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.227+0000 7f8d93000640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d740a2b90 0x7f8d740a5050 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c19f3c0 tx=0x7f8d78005f90 comp rx=0 tx=0).stop 2024-04-25T00:15:58.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.227+0000 7f8d93000640 1 -- 172.21.15.42:0/4074048170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c107bc0 msgr2=0x7f8d8c19deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:15:58.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.227+0000 7f8d93000640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c107bc0 0x7f8d8c19deb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8d80031440 tx=0x7f8d80032ce0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.227+0000 7f8d93000640 1 -- 172.21.15.42:0/4074048170 shutdown_connections 2024-04-25T00:15:58.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.227+0000 7f8d93000640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8d740a2b90 0x7f8d740a5050 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.227+0000 7f8d93000640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8d8c109770 0x7f8d8c1a5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.227+0000 7f8d93000640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d8c108db0 0x7f8d8c19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.227+0000 7f8d93000640 1 --2- 172.21.15.42:0/4074048170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8d8c107bc0 0x7f8d8c19deb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:15:58.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.227+0000 7f8d93000640 1 -- 172.21.15.42:0/4074048170 >> 172.21.15.42:0/4074048170 conn(0x7f8d8c076b80 msgr2=0x7f8d8c10c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:15:58.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.228+0000 7f8d93000640 1 -- 172.21.15.42:0/4074048170 shutdown_connections 2024-04-25T00:15:58.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:15:58.228+0000 7f8d93000640 1 -- 172.21.15.42:0/4074048170 wait complete. 2024-04-25T00:15:58.253 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:15:58 smithi136 systemd[1]: Starting Ceph mds.c for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:15:58.513 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:15:58 smithi136 podman[63282]: 2024-04-25 00:15:58.15651512 +0000 UTC m=+0.018192091 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:15:58.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:15:58 smithi136 podman[63282]: 2024-04-25 00:15:58.26414054 +0000 UTC m=+0.125817541 container create dc40c9ab2d5391f155a497e3b460fe7972a396e6bbbc5e21574aa76bf428d74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True) 2024-04-25T00:15:58.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:15:58 smithi136 podman[63282]: 2024-04-25 00:15:58.466455312 +0000 UTC m=+0.328132267 container init dc40c9ab2d5391f155a497e3b460fe7972a396e6bbbc5e21574aa76bf428d74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T00:15:58.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:15:58 smithi136 podman[63282]: 2024-04-25 00:15:58.468991869 +0000 UTC m=+0.330668827 container start dc40c9ab2d5391f155a497e3b460fe7972a396e6bbbc5e21574aa76bf428d74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:15:58.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:15:58 smithi136 ceph-mds[63300]: -- 172.21.15.136:0/767726726 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559a7b19ba40 con 0x559a7b16e800 2024-04-25T00:15:58.514 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:15:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c[63296]: starting mds.c at 2024-04-25T00:15:58.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:58 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/4074048170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:59.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:58 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/4074048170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:59.013 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:15:58 smithi136 bash[63282]: dc40c9ab2d5391f155a497e3b460fe7972a396e6bbbc5e21574aa76bf428d74a 2024-04-25T00:15:59.013 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:15:58 smithi136 systemd[1]: Started Ceph mds.c for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:15:59.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:58 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/4074048170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:15:59.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:15:59.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] up:boot 2024-04-25T00:15:59.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: fsmap 7 up:standby 2024-04-25T00:15:59.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:15:59.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:15:59.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:59.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:59.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:15:59.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:15:59.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:15:59.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:15:59 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] up:boot 2024-04-25T00:16:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: fsmap 7 up:standby 2024-04-25T00:16:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:16:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:16:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:00.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:00.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:00.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:00.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:00.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:15:59 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] up:boot 2024-04-25T00:16:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: fsmap 7 up:standby 2024-04-25T00:16:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:16:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:16:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:00.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:00.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:00.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:00.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:00.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:15:59 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:00.248 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:00.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.472+0000 7fc8f5c00640 1 -- 172.21.15.42:0/2437136726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8f01039c0 msgr2=0x7fc8f0103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:00.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.472+0000 7fc8f5c00640 1 --2- 172.21.15.42:0/2437136726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8f01039c0 0x7fc8f0103de0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0009900 tx=0x7fc8e002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.472+0000 7fc8f5c00640 1 -- 172.21.15.42:0/2437136726 shutdown_connections 2024-04-25T00:16:00.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.472+0000 7fc8f5c00640 1 --2- 172.21.15.42:0/2437136726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8f0105570 0x7fc8f010be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.472+0000 7fc8f5c00640 1 --2- 172.21.15.42:0/2437136726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8f0104bb0 0x7fc8f0105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.472+0000 7fc8f5c00640 1 --2- 172.21.15.42:0/2437136726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8f01039c0 0x7fc8f0103de0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.472+0000 7fc8f5c00640 1 -- 172.21.15.42:0/2437136726 >> 172.21.15.42:0/2437136726 conn(0x7fc8f00fef30 msgr2=0x7fc8f0101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:00.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.472+0000 7fc8f5c00640 1 -- 172.21.15.42:0/2437136726 shutdown_connections 2024-04-25T00:16:00.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.472+0000 7fc8f5c00640 1 -- 172.21.15.42:0/2437136726 wait complete. 2024-04-25T00:16:00.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.473+0000 7fc8f5c00640 1 Processor -- start 2024-04-25T00:16:00.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8f5c00640 1 -- start start 2024-04-25T00:16:00.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8f5c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8f01039c0 0x7fc8f0199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8f5c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8f0104bb0 0x7fc8f019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8f5c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8f0105570 0x7fc8f01a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8f5c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f010f680 con 0x7fc8f0104bb0 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8f5c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f010f500 con 0x7fc8f0105570 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8f5c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f010f800 con 0x7fc8f01039c0 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8f01039c0 0x7fc8f0199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8ef400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8f0105570 0x7fc8f01a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8eea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8f01039c0 0x7fc8f0199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:52752/0 (socket says 172.21.15.42:52752) 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8eea00640 1 -- 172.21.15.42:0/1751836520 learned_addr learned my addr 172.21.15.42:0/1751836520 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.474+0000 7fc8ee000640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8f0104bb0 0x7fc8f019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8eea00640 1 -- 172.21.15.42:0/1751836520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8f0105570 msgr2=0x7fc8f01a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8eea00640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8f0105570 0x7fc8f01a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8eea00640 1 -- 172.21.15.42:0/1751836520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8f0104bb0 msgr2=0x7fc8f019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:00.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8eea00640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8f0104bb0 0x7fc8f019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8eea00640 1 -- 172.21.15.42:0/1751836520 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f01a1970 con 0x7fc8f01039c0 2024-04-25T00:16:00.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8ef400640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8f0105570 0x7fc8f01a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:00.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8eea00640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8f01039c0 0x7fc8f0199d50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0031440 tx=0x7fc8e0032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:00.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8e7400640 1 -- 172.21.15.42:0/1751836520 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e003f030 con 0x7fc8f01039c0 2024-04-25T00:16:00.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8f5c00640 1 -- 172.21.15.42:0/1751836520 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f01a1c00 con 0x7fc8f01039c0 2024-04-25T00:16:00.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8e7400640 1 -- 172.21.15.42:0/1751836520 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8e0003ce0 con 0x7fc8f01039c0 2024-04-25T00:16:00.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.475+0000 7fc8e7400640 1 -- 172.21.15.42:0/1751836520 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e0035070 con 0x7fc8f01039c0 2024-04-25T00:16:00.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.476+0000 7fc8f5c00640 1 -- 172.21.15.42:0/1751836520 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f01a2140 con 0x7fc8f01039c0 2024-04-25T00:16:00.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.476+0000 7fc8f5c00640 1 -- 172.21.15.42:0/1751836520 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f010ad30 con 0x7fc8f01039c0 2024-04-25T00:16:00.479 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.479+0000 7fc8e7400640 1 -- 172.21.15.42:0/1751836520 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc8e0042030 con 0x7fc8f01039c0 2024-04-25T00:16:00.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.480+0000 7fc8e7400640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc8d40a2c10 0x7fc8d40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:00.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.480+0000 7fc8e7400640 1 -- 172.21.15.42:0/1751836520 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc8e00ef450 con 0x7fc8f01039c0 2024-04-25T00:16:00.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.480+0000 7fc8ee000640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc8d40a2c10 0x7fc8d40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:00.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.481+0000 7fc8ee000640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc8d40a2c10 0x7fc8d40a50d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc8f019b260 tx=0x7fc8d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:00.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.482+0000 7fc8e7400640 1 -- 172.21.15.42:0/1751836520 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc8e00bc7b0 con 0x7fc8f01039c0 2024-04-25T00:16:00.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.729+0000 7fc8f5c00640 1 -- 172.21.15.42:0/1751836520 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8f002d320 con 0x7fc8f01039c0 2024-04-25T00:16:00.730 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.730+0000 7fc8e7400640 1 -- 172.21.15.42:0/1751836520 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7fc8e00bc0b0 con 0x7fc8f01039c0 2024-04-25T00:16:00.731 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:00.731 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T00:16:00.731 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 8 2024-04-25T00:16:00.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.733+0000 7fc8f5c00640 1 -- 172.21.15.42:0/1751836520 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc8d40a2c10 msgr2=0x7fc8d40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:00.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.733+0000 7fc8f5c00640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc8d40a2c10 0x7fc8d40a50d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc8f019b260 tx=0x7fc8d8005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:00.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.733+0000 7fc8f5c00640 1 -- 172.21.15.42:0/1751836520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8f01039c0 msgr2=0x7fc8f0199d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:00.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.733+0000 7fc8f5c00640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8f01039c0 0x7fc8f0199d50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0031440 tx=0x7fc8e0032ce0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.733+0000 7fc8f5c00640 1 -- 172.21.15.42:0/1751836520 shutdown_connections 2024-04-25T00:16:00.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.733+0000 7fc8f5c00640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc8d40a2c10 0x7fc8d40a50d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.733+0000 7fc8f5c00640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8f0105570 0x7fc8f01a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.733+0000 7fc8f5c00640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc8f0104bb0 0x7fc8f019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.734 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.733+0000 7fc8f5c00640 1 --2- 172.21.15.42:0/1751836520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8f01039c0 0x7fc8f0199d50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:00.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.733+0000 7fc8f5c00640 1 -- 172.21.15.42:0/1751836520 >> 172.21.15.42:0/1751836520 conn(0x7fc8f00fef30 msgr2=0x7fc8f0107f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:00.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.734+0000 7fc8f5c00640 1 -- 172.21.15.42:0/1751836520 shutdown_connections 2024-04-25T00:16:00.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:00.734+0000 7fc8f5c00640 1 -- 172.21.15.42:0/1751836520 wait complete. 2024-04-25T00:16:00.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:00 smithi106 ceph-mon[32510]: Deploying daemon mds.h on smithi106 2024-04-25T00:16:00.839 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:16:00 smithi106 systemd[1]: Starting Ceph mds.h for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:16:00.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:00 smithi042 ceph-mon[28130]: Deploying daemon mds.h on smithi106 2024-04-25T00:16:01.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:00 smithi136 ceph-mon[32923]: Deploying daemon mds.h on smithi106 2024-04-25T00:16:01.241 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:16:00 smithi106 podman[63063]: 2024-04-25 00:16:00.886247901 +0000 UTC m=+0.017928729 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:16:01.241 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:16:00 smithi106 podman[63063]: 2024-04-25 00:16:00.995809573 +0000 UTC m=+0.127490430 container create b047dea94f74c5c6c64d447916d94bd1cf97d3b7d4915181808ae654a8e99187 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T00:16:01.241 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:16:01 smithi106 podman[63063]: 2024-04-25 00:16:01.214857685 +0000 UTC m=+0.346538501 container init b047dea94f74c5c6c64d447916d94bd1cf97d3b7d4915181808ae654a8e99187 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T00:16:01.241 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:16:01 smithi106 podman[63063]: 2024-04-25 00:16:01.2172638 +0000 UTC m=+0.348944622 container start b047dea94f74c5c6c64d447916d94bd1cf97d3b7d4915181808ae654a8e99187 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:16:01.241 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:16:01 smithi106 ceph-mds[63082]: -- 172.21.15.106:0/3763943072 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9a1fc3a40 con 0x55d9a1f96c00 2024-04-25T00:16:01.588 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:16:01 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h[63078]: starting mds.h at 2024-04-25T00:16:01.588 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:16:01 smithi106 bash[63063]: b047dea94f74c5c6c64d447916d94bd1cf97d3b7d4915181808ae654a8e99187 2024-04-25T00:16:01.588 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:16:01 smithi106 systemd[1]: Started Ceph mds.h for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:16:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:01 smithi042 ceph-mon[28130]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:01 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1751836520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:01.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:01.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:01.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:01.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:01.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:01 smithi136 ceph-mon[32923]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:01 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1751836520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:01 smithi106 ceph-mon[32510]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:01 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1751836520' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:02.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:02.756 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:02.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:02 smithi136 ceph-mon[32923]: Deploying daemon mds.i on smithi136 2024-04-25T00:16:02.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:02 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.106:6836/1331229112,v1:172.21.15.106:6837/1331229112] up:boot 2024-04-25T00:16:02.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:02 smithi136 ceph-mon[32923]: fsmap 8 up:standby 2024-04-25T00:16:02.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:02 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:16:02.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:02 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:16:02.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.979+0000 7f3bba600640 1 -- 172.21.15.42:0/1045262814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb41037c0 msgr2=0x7f3bb4103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.979+0000 7f3bba600640 1 --2- 172.21.15.42:0/1045262814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb41037c0 0x7f3bb4103be0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009ca0 tx=0x7f3ba402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.979+0000 7f3bba600640 1 -- 172.21.15.42:0/1045262814 shutdown_connections 2024-04-25T00:16:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.979+0000 7f3bba600640 1 --2- 172.21.15.42:0/1045262814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bb4105370 0x7f3bb410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.979+0000 7f3bba600640 1 --2- 172.21.15.42:0/1045262814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb41049b0 0x7f3bb4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.979+0000 7f3bba600640 1 --2- 172.21.15.42:0/1045262814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb41037c0 0x7f3bb4103be0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.979+0000 7f3bba600640 1 -- 172.21.15.42:0/1045262814 >> 172.21.15.42:0/1045262814 conn(0x7f3bb40fed50 msgr2=0x7f3bb4101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.979+0000 7f3bba600640 1 -- 172.21.15.42:0/1045262814 shutdown_connections 2024-04-25T00:16:02.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.979+0000 7f3bba600640 1 -- 172.21.15.42:0/1045262814 wait complete. 2024-04-25T00:16:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.980+0000 7f3bba600640 1 Processor -- start 2024-04-25T00:16:02.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bba600640 1 -- start start 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bba600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb41037c0 0x7f3bb4199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bba600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bb41049b0 0x7f3bb4199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bba600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb4105370 0x7f3bb41a1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bba600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb410f490 con 0x7f3bb4105370 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bba600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb410f310 con 0x7f3bb41049b0 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bba600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb410f610 con 0x7f3bb41037c0 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bb9200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb41037c0 0x7f3bb4199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bb9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb4105370 0x7f3bb41a1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bb9200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb41037c0 0x7f3bb4199aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:52772/0 (socket says 172.21.15.42:52772) 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bb9200640 1 -- 172.21.15.42:0/3409874725 learned_addr learned my addr 172.21.15.42:0/3409874725 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:02.982 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.981+0000 7f3bb3e00640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bb41049b0 0x7f3bb4199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bb9c00640 1 -- 172.21.15.42:0/3409874725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb41037c0 msgr2=0x7f3bb4199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:02.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bb9c00640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb41037c0 0x7f3bb4199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bb9c00640 1 -- 172.21.15.42:0/3409874725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bb41049b0 msgr2=0x7f3bb4199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bb9c00640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bb41049b0 0x7f3bb4199fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bb9c00640 1 -- 172.21.15.42:0/3409874725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb41a16c0 con 0x7f3bb4105370 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bb9200640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb41037c0 0x7f3bb4199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bb9c00640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb4105370 0x7f3bb41a1050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00e970 tx=0x7f3bac00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bb1600640 1 -- 172.21.15.42:0/3409874725 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac00ccc0 con 0x7f3bb4105370 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bba600640 1 -- 172.21.15.42:0/3409874725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb41a19b0 con 0x7f3bb4105370 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bb1600640 1 -- 172.21.15.42:0/3409874725 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3bac005020 con 0x7f3bb4105370 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.982+0000 7f3bb1600640 1 -- 172.21.15.42:0/3409874725 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac004a00 con 0x7f3bb4105370 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.983+0000 7f3bba600640 1 -- 172.21.15.42:0/3409874725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb41a1f00 con 0x7f3bb4105370 2024-04-25T00:16:02.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.984+0000 7f3bba600640 1 -- 172.21.15.42:0/3409874725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb410aae0 con 0x7f3bb4105370 2024-04-25T00:16:02.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.986+0000 7f3bb1600640 1 -- 172.21.15.42:0/3409874725 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3bac0051d0 con 0x7f3bb4105370 2024-04-25T00:16:02.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.987+0000 7f3bb1600640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3b980a2b90 0x7f3b980a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:02.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.987+0000 7f3bb1600640 1 -- 172.21.15.42:0/3409874725 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3bac0ceb70 con 0x7f3bb4105370 2024-04-25T00:16:02.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.987+0000 7f3bb9200640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3b980a2b90 0x7f3b980a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:02.988 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.988+0000 7f3bb9200640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3b980a2b90 0x7f3b980a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4004ab0 tx=0x7f3ba4004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:02.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:02.992+0000 7f3bb1600640 1 -- 172.21.15.42:0/3409874725 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3bac09bed0 con 0x7f3bb4105370 2024-04-25T00:16:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:02 smithi106 ceph-mon[32510]: Deploying daemon mds.i on smithi136 2024-04-25T00:16:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:02 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.106:6836/1331229112,v1:172.21.15.106:6837/1331229112] up:boot 2024-04-25T00:16:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:02 smithi106 ceph-mon[32510]: fsmap 8 up:standby 2024-04-25T00:16:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:02 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:16:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:02 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:16:03.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:02 smithi042 ceph-mon[28130]: Deploying daemon mds.i on smithi136 2024-04-25T00:16:03.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:02 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.106:6836/1331229112,v1:172.21.15.106:6837/1331229112] up:boot 2024-04-25T00:16:03.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:02 smithi042 ceph-mon[28130]: fsmap 8 up:standby 2024-04-25T00:16:03.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:02 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:16:03.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:02 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:16:03.239 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.239+0000 7f3bba600640 1 -- 172.21.15.42:0/3409874725 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3bb402d170 con 0x7f3bb4105370 2024-04-25T00:16:03.240 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.240+0000 7f3bb1600640 1 -- 172.21.15.42:0/3409874725 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7f3bac09b7d0 con 0x7f3bb4105370 2024-04-25T00:16:03.241 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:03.241 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T00:16:03.241 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 9 2024-04-25T00:16:03.243 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 -- 172.21.15.42:0/3409874725 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3b980a2b90 msgr2=0x7f3b980a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3b980a2b90 0x7f3b980a5050 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4004ab0 tx=0x7f3ba4004a40 comp rx=0 tx=0).stop 2024-04-25T00:16:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 -- 172.21.15.42:0/3409874725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb4105370 msgr2=0x7f3bb41a1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb4105370 0x7f3bb41a1050 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00e970 tx=0x7f3bac00ee30 comp rx=0 tx=0).stop 2024-04-25T00:16:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 -- 172.21.15.42:0/3409874725 shutdown_connections 2024-04-25T00:16:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3b980a2b90 0x7f3b980a5050 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3bb4105370 0x7f3bb41a1050 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bb41049b0 0x7f3bb4199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 --2- 172.21.15.42:0/3409874725 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bb41037c0 0x7f3bb4199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 -- 172.21.15.42:0/3409874725 >> 172.21.15.42:0/3409874725 conn(0x7f3bb40fed50 msgr2=0x7f3bb4107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 -- 172.21.15.42:0/3409874725 shutdown_connections 2024-04-25T00:16:03.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:03.243+0000 7f3bba600640 1 -- 172.21.15.42:0/3409874725 wait complete. 2024-04-25T00:16:03.635 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:16:03 smithi136 systemd[1]: Starting Ceph mds.i for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:16:03.901 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:03 smithi136 ceph-mon[32923]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:03.901 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:03 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3409874725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:03.901 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:16:03 smithi136 podman[63588]: 2024-04-25 00:16:03.678937743 +0000 UTC m=+0.017803649 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:16:03.901 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:16:03 smithi136 podman[63588]: 2024-04-25 00:16:03.784755876 +0000 UTC m=+0.123621766 container create 29203519efc76980b0f39345291f22fcc65b564f664204eea6106cbd21d4d6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-04-25T00:16:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:03 smithi106 ceph-mon[32510]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:03 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3409874725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:03 smithi042 ceph-mon[28130]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:04.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:03 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3409874725' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:04.263 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:16:04 smithi136 podman[63588]: 2024-04-25 00:16:04.004062551 +0000 UTC m=+0.342928440 container init 29203519efc76980b0f39345291f22fcc65b564f664204eea6106cbd21d4d6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T00:16:04.263 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:16:04 smithi136 podman[63588]: 2024-04-25 00:16:04.006714713 +0000 UTC m=+0.345580604 container start 29203519efc76980b0f39345291f22fcc65b564f664204eea6106cbd21d4d6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T00:16:04.263 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:16:04 smithi136 ceph-mds[63607]: -- 172.21.15.136:0/2308002835 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fef7c3ba40 con 0x55fef7c0e800 2024-04-25T00:16:04.263 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:16:04 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: starting mds.i at 2024-04-25T00:16:04.264 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:16:04 smithi136 bash[63588]: 29203519efc76980b0f39345291f22fcc65b564f664204eea6106cbd21d4d6ca 2024-04-25T00:16:04.264 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:16:04 smithi136 systemd[1]: Started Ceph mds.i for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:16:05.266 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:05.283 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:05.284 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:05.284 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:05.284 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:05.284 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:05.284 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:05.284 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:05.284 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: Deploying daemon mds.g on smithi042 2024-04-25T00:16:05.284 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6838/501637536,v1:172.21.15.136:6839/501637536] up:boot 2024-04-25T00:16:05.284 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: fsmap 9 up:standby 2024-04-25T00:16:05.284 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:16:05.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.497+0000 7fa829600640 1 -- 172.21.15.42:0/3854147460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8240741a0 msgr2=0x7fa824074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:05.500 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.497+0000 7fa829600640 1 --2- 172.21.15.42:0/3854147460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8240741a0 0x7fa824074620 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa810009900 tx=0x7fa81002a400 comp rx=0 tx=0).stop 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.497+0000 7fa829600640 1 -- 172.21.15.42:0/3854147460 shutdown_connections 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.497+0000 7fa829600640 1 --2- 172.21.15.42:0/3854147460 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8241094a0 0x7fa82410b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.497+0000 7fa829600640 1 --2- 172.21.15.42:0/3854147460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8240741a0 0x7fa824074620 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.497+0000 7fa829600640 1 --2- 172.21.15.42:0/3854147460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa824075780 0x7fa824073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.497+0000 7fa829600640 1 -- 172.21.15.42:0/3854147460 >> 172.21.15.42:0/3854147460 conn(0x7fa8240fcb50 msgr2=0x7fa8240fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.497+0000 7fa829600640 1 -- 172.21.15.42:0/3854147460 shutdown_connections 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.497+0000 7fa829600640 1 -- 172.21.15.42:0/3854147460 wait complete. 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.498+0000 7fa829600640 1 Processor -- start 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.498+0000 7fa829600640 1 -- start start 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.498+0000 7fa829600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8240741a0 0x7fa824199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.498+0000 7fa829600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa824075780 0x7fa82419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.498+0000 7fa829600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8241094a0 0x7fa8241a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:05.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.498+0000 7fa829600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410f490 con 0x7fa8241094a0 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.498+0000 7fa829600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410f310 con 0x7fa824075780 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.498+0000 7fa829600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa82410f610 con 0x7fa8240741a0 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa828c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8241094a0 0x7fa8241a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa828c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8241094a0 0x7fa8241a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36186/0 (socket says 172.21.15.42:36186) 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa823e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8240741a0 0x7fa824199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa828c00640 1 -- 172.21.15.42:0/3842402939 learned_addr learned my addr 172.21.15.42:0/3842402939 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa823400640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa824075780 0x7fa82419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa823e00640 1 -- 172.21.15.42:0/3842402939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa824075780 msgr2=0x7fa82419a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa823e00640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa824075780 0x7fa82419a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa823e00640 1 -- 172.21.15.42:0/3842402939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8241094a0 msgr2=0x7fa8241a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa823e00640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8241094a0 0x7fa8241a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.502 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa823e00640 1 -- 172.21.15.42:0/3842402939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8241a1720 con 0x7fa8240741a0 2024-04-25T00:16:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.499+0000 7fa828c00640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8241094a0 0x7fa8241a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.500+0000 7fa823e00640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8240741a0 0x7fa824199b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa81400e9f0 tx=0x7fa81400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.500+0000 7fa820c00640 1 -- 172.21.15.42:0/3842402939 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa81400ccc0 con 0x7fa8240741a0 2024-04-25T00:16:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.500+0000 7fa820c00640 1 -- 172.21.15.42:0/3842402939 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa814005020 con 0x7fa8240741a0 2024-04-25T00:16:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.500+0000 7fa829600640 1 -- 172.21.15.42:0/3842402939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8241a1a10 con 0x7fa8240741a0 2024-04-25T00:16:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.500+0000 7fa820c00640 1 -- 172.21.15.42:0/3842402939 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8140049e0 con 0x7fa8240741a0 2024-04-25T00:16:05.503 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.500+0000 7fa829600640 1 -- 172.21.15.42:0/3842402939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8241a1ee0 con 0x7fa8240741a0 2024-04-25T00:16:05.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.501+0000 7fa829600640 1 -- 172.21.15.42:0/3842402939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8241081d0 con 0x7fa8240741a0 2024-04-25T00:16:05.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.503+0000 7fa820c00640 1 -- 172.21.15.42:0/3842402939 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa814005560 con 0x7fa8240741a0 2024-04-25T00:16:05.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.504+0000 7fa820c00640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8080a2b90 0x7fa8080a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.505+0000 7fa820c00640 1 -- 172.21.15.42:0/3842402939 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa8140ceb90 con 0x7fa8240741a0 2024-04-25T00:16:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.505+0000 7fa823400640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8080a2b90 0x7fa8080a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:05.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.505+0000 7fa823400640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8080a2b90 0x7fa8080a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa810031440 tx=0x7fa8100315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:05.509 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.509+0000 7fa820c00640 1 -- 172.21.15.42:0/3842402939 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa8140051d0 con 0x7fa8240741a0 2024-04-25T00:16:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: Deploying daemon mds.g on smithi042 2024-04-25T00:16:05.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6838/501637536,v1:172.21.15.136:6839/501637536] up:boot 2024-04-25T00:16:05.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: fsmap 9 up:standby 2024-04-25T00:16:05.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:16:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:05.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:05.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:05.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:05.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: Deploying daemon mds.g on smithi042 2024-04-25T00:16:05.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6838/501637536,v1:172.21.15.136:6839/501637536] up:boot 2024-04-25T00:16:05.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: fsmap 9 up:standby 2024-04-25T00:16:05.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:16:05.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.758+0000 7fa829600640 1 -- 172.21.15.42:0/3842402939 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa82402d170 con 0x7fa8240741a0 2024-04-25T00:16:05.759 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.759+0000 7fa820c00640 1 -- 172.21.15.42:0/3842402939 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7fa81409bef0 con 0x7fa8240741a0 2024-04-25T00:16:05.760 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:05.760 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:16:05.760 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 10 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 -- 172.21.15.42:0/3842402939 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8080a2b90 msgr2=0x7fa8080a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8080a2b90 0x7fa8080a5050 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa810031440 tx=0x7fa8100315b0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 -- 172.21.15.42:0/3842402939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8240741a0 msgr2=0x7fa824199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8240741a0 0x7fa824199b00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa81400e9f0 tx=0x7fa81400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 -- 172.21.15.42:0/3842402939 shutdown_connections 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8080a2b90 0x7fa8080a5050 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa8241094a0 0x7fa8241a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa824075780 0x7fa82419a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 --2- 172.21.15.42:0/3842402939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa8240741a0 0x7fa824199b00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 -- 172.21.15.42:0/3842402939 >> 172.21.15.42:0/3842402939 conn(0x7fa8240fcb50 msgr2=0x7fa8241053a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 -- 172.21.15.42:0/3842402939 shutdown_connections 2024-04-25T00:16:05.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:05.762+0000 7fa829600640 1 -- 172.21.15.42:0/3842402939 wait complete. 2024-04-25T00:16:06.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:06 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3842402939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:06.562 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:06 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3842402939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:06.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:06 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3842402939' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:07.364 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:07 smithi042 ceph-mon[28130]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:07.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:07 smithi136 ceph-mon[32923]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:07 smithi106 ceph-mon[32510]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:07.627 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:16:07 smithi042 systemd[1]: Starting Ceph mds.g for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:16:07.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:07.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.997+0000 7ff27fe00640 1 -- 172.21.15.42:0/436641500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff280103190 msgr2=0x7ff280109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.997+0000 7ff27fe00640 1 --2- 172.21.15.42:0/436641500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff280103190 0x7ff280109a20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff278009be0 tx=0x7ff27802a6c0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.997+0000 7ff27fe00640 1 -- 172.21.15.42:0/436641500 shutdown_connections 2024-04-25T00:16:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.997+0000 7ff27fe00640 1 --2- 172.21.15.42:0/436641500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff280103190 0x7ff280109a20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.997+0000 7ff27fe00640 1 --2- 172.21.15.42:0/436641500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2801027d0 0x7ff280102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.997+0000 7ff27fe00640 1 --2- 172.21.15.42:0/436641500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2801015e0 0x7ff280101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.997+0000 7ff27fe00640 1 -- 172.21.15.42:0/436641500 >> 172.21.15.42:0/436641500 conn(0x7ff2800fcb50 msgr2=0x7ff2800fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.997+0000 7ff27fe00640 1 -- 172.21.15.42:0/436641500 shutdown_connections 2024-04-25T00:16:07.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.997+0000 7ff27fe00640 1 -- 172.21.15.42:0/436641500 wait complete. 2024-04-25T00:16:07.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.998+0000 7ff27fe00640 1 Processor -- start 2024-04-25T00:16:07.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.998+0000 7ff27fe00640 1 -- start start 2024-04-25T00:16:07.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2801015e0 0x7ff280199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:07.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2801027d0 0x7ff28019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:07.999 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280103190 0x7ff2801a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff28010d220 con 0x7ff280103190 2024-04-25T00:16:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff28010d0a0 con 0x7ff2801015e0 2024-04-25T00:16:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff28010d3a0 con 0x7ff2801027d0 2024-04-25T00:16:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2801015e0 0x7ff280199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280103190 0x7ff2801a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2801015e0 0x7ff280199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42304/0 (socket says 172.21.15.42:42304) 2024-04-25T00:16:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27ea00640 1 -- 172.21.15.42:0/1675062935 learned_addr learned my addr 172.21.15.42:0/1675062935 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:08.000 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:07.999+0000 7ff27e000640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2801027d0 0x7ff28019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff27ea00640 1 -- 172.21.15.42:0/1675062935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2801027d0 msgr2=0x7ff28019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff27ea00640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2801027d0 0x7ff28019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff27ea00640 1 -- 172.21.15.42:0/1675062935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280103190 msgr2=0x7ff2801a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff27ea00640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280103190 0x7ff2801a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff27ea00640 1 -- 172.21.15.42:0/1675062935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2801a16e0 con 0x7ff2801015e0 2024-04-25T00:16:08.001 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff27f400640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280103190 0x7ff2801a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff27ea00640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2801015e0 0x7ff280199b60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff27000e9a0 tx=0x7ff27000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff277400640 1 -- 172.21.15.42:0/1675062935 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27000cd80 con 0x7ff2801015e0 2024-04-25T00:16:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff277400640 1 -- 172.21.15.42:0/1675062935 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff270010910 con 0x7ff2801015e0 2024-04-25T00:16:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff27fe00640 1 -- 172.21.15.42:0/1675062935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2801a19d0 con 0x7ff2801015e0 2024-04-25T00:16:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.000+0000 7ff277400640 1 -- 172.21.15.42:0/1675062935 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff270018a30 con 0x7ff2801015e0 2024-04-25T00:16:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.001+0000 7ff27fe00640 1 -- 172.21.15.42:0/1675062935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801a1f20 con 0x7ff2801015e0 2024-04-25T00:16:08.002 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.002+0000 7ff27fe00640 1 -- 172.21.15.42:0/1675062935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff280068480 con 0x7ff2801015e0 2024-04-25T00:16:08.006 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:16:07 smithi042 podman[68985]: 2024-04-25 00:16:07.682065851 +0000 UTC m=+0.038571218 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:16:08.006 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:16:07 smithi042 podman[68985]: 2024-04-25 00:16:07.846884086 +0000 UTC m=+0.203389452 container create 2960e809835a209e6df18b3d9b1f603db91e7ed8a77cd7396fd5abc6f35fee96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:16:08.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.005+0000 7ff277400640 1 -- 172.21.15.42:0/1675062935 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff270010ac0 con 0x7ff2801015e0 2024-04-25T00:16:08.006 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.007+0000 7ff277400640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2640a2b90 0x7ff2640a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:08.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.007+0000 7ff277400640 1 -- 172.21.15.42:0/1675062935 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff2700cefe0 con 0x7ff2801015e0 2024-04-25T00:16:08.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.007+0000 7ff27e000640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2640a2b90 0x7ff2640a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:08.007 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.008+0000 7ff27e000640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2640a2b90 0x7ff2640a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff28019afd0 tx=0x7ff26c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:08.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.011+0000 7ff277400640 1 -- 172.21.15.42:0/1675062935 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff270053740 con 0x7ff2801015e0 2024-04-25T00:16:08.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.267+0000 7ff27fe00640 1 -- 172.21.15.42:0/1675062935 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2801a2810 con 0x7ff2801015e0 2024-04-25T00:16:08.268 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.268+0000 7ff277400640 1 -- 172.21.15.42:0/1675062935 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7ff27009c3e0 con 0x7ff2801015e0 2024-04-25T00:16:08.268 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:08.269 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:16:08.269 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 10 2024-04-25T00:16:08.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.270+0000 7ff27fe00640 1 -- 172.21.15.42:0/1675062935 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2640a2b90 msgr2=0x7ff2640a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:08.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.270+0000 7ff27fe00640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2640a2b90 0x7ff2640a5050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff28019afd0 tx=0x7ff26c005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:08.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.271+0000 7ff27fe00640 1 -- 172.21.15.42:0/1675062935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2801015e0 msgr2=0x7ff280199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:08.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.271+0000 7ff27fe00640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2801015e0 0x7ff280199b60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff27000e9a0 tx=0x7ff27000ee60 comp rx=0 tx=0).stop 2024-04-25T00:16:08.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.271+0000 7ff27fe00640 1 -- 172.21.15.42:0/1675062935 shutdown_connections 2024-04-25T00:16:08.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.271+0000 7ff27fe00640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2640a2b90 0x7ff2640a5050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:08.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.271+0000 7ff27fe00640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff280103190 0x7ff2801a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:08.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.271+0000 7ff27fe00640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2801027d0 0x7ff28019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:08.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.271+0000 7ff27fe00640 1 --2- 172.21.15.42:0/1675062935 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2801015e0 0x7ff280199b60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:08.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.271+0000 7ff27fe00640 1 -- 172.21.15.42:0/1675062935 >> 172.21.15.42:0/1675062935 conn(0x7ff2800fcb50 msgr2=0x7ff280073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:08.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.271+0000 7ff27fe00640 1 -- 172.21.15.42:0/1675062935 shutdown_connections 2024-04-25T00:16:08.272 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:08.271+0000 7ff27fe00640 1 -- 172.21.15.42:0/1675062935 wait complete. 2024-04-25T00:16:08.377 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:16:08 smithi042 podman[68985]: 2024-04-25 00:16:08.135724944 +0000 UTC m=+0.492230388 container init 2960e809835a209e6df18b3d9b1f603db91e7ed8a77cd7396fd5abc6f35fee96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T00:16:08.377 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:16:08 smithi042 podman[68985]: 2024-04-25 00:16:08.147114177 +0000 UTC m=+0.503619611 container start 2960e809835a209e6df18b3d9b1f603db91e7ed8a77cd7396fd5abc6f35fee96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:16:08.377 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:16:08 smithi042 ceph-mds[69050]: -- 172.21.15.42:0/3699619280 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586620eba40 con 0x5586620bec00 2024-04-25T00:16:08.377 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:16:08 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-g[69022]: starting mds.g at 2024-04-25T00:16:08.378 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:16:08 smithi042 bash[68985]: 2960e809835a209e6df18b3d9b1f603db91e7ed8a77cd7396fd5abc6f35fee96 2024-04-25T00:16:08.378 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:16:08 smithi042 systemd[1]: Started Ceph mds.g for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:16:09.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:09 smithi042 ceph-mon[28130]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:09.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:09 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1675062935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:09.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:09.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:09.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:09.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:09.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:09.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:09.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:09 smithi042 ceph-mon[28130]: Deploying daemon mds.a on smithi042 2024-04-25T00:16:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:09 smithi136 ceph-mon[32923]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:09 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1675062935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:09.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:09.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:09.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:09.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:09 smithi136 ceph-mon[32923]: Deploying daemon mds.a on smithi042 2024-04-25T00:16:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:09 smithi106 ceph-mon[32510]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:09 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1675062935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:09.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:09.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:09 smithi106 ceph-mon[32510]: Deploying daemon mds.a on smithi042 2024-04-25T00:16:10.291 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.512+0000 7ff2f1a00640 1 -- 172.21.15.42:0/2523616976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec102040 msgr2=0x7ff2ec104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.512+0000 7ff2f1a00640 1 --2- 172.21.15.42:0/2523616976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec102040 0x7ff2ec104430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc009ca0 tx=0x7ff2dc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.513+0000 7ff2f1a00640 1 -- 172.21.15.42:0/2523616976 shutdown_connections 2024-04-25T00:16:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.513+0000 7ff2f1a00640 1 --2- 172.21.15.42:0/2523616976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2ec1072a0 0x7ff2ec1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.513+0000 7ff2f1a00640 1 --2- 172.21.15.42:0/2523616976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ec104970 0x7ff2ec106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.513+0000 7ff2f1a00640 1 --2- 172.21.15.42:0/2523616976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec102040 0x7ff2ec104430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.513 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.513+0000 7ff2f1a00640 1 -- 172.21.15.42:0/2523616976 >> 172.21.15.42:0/2523616976 conn(0x7ff2ec0fba10 msgr2=0x7ff2ec0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:10.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.513+0000 7ff2f1a00640 1 -- 172.21.15.42:0/2523616976 shutdown_connections 2024-04-25T00:16:10.514 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.513+0000 7ff2f1a00640 1 -- 172.21.15.42:0/2523616976 wait complete. 2024-04-25T00:16:10.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:10 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.42:6838/3053633590,v1:172.21.15.42:6839/3053633590] up:boot 2024-04-25T00:16:10.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:10 smithi136 ceph-mon[32923]: fsmap 10 up:standby 2024-04-25T00:16:10.515 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:10 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:16:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.514+0000 7ff2f1a00640 1 Processor -- start 2024-04-25T00:16:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.514+0000 7ff2f1a00640 1 -- start start 2024-04-25T00:16:10.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2ec102040 0x7ff2ec199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ec104970 0x7ff2ec19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec1072a0 0x7ff2ec1a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec10d2a0 con 0x7ff2ec104970 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec10d120 con 0x7ff2ec1072a0 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec10d420 con 0x7ff2ec102040 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2eb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ec104970 0x7ff2ec19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec1072a0 0x7ff2ec1a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2eb400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ec104970 0x7ff2ec19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42134/0 (socket says 172.21.15.42:42134) 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2eb400640 1 -- 172.21.15.42:0/1961201486 learned_addr learned my addr 172.21.15.42:0/1961201486 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2ebe00640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2ec102040 0x7ff2ec199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1000640 1 -- 172.21.15.42:0/1961201486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2ec102040 msgr2=0x7ff2ec199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1000640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2ec102040 0x7ff2ec199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1000640 1 -- 172.21.15.42:0/1961201486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ec104970 msgr2=0x7ff2ec19a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1000640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ec104970 0x7ff2ec19a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.515+0000 7ff2f1000640 1 -- 172.21.15.42:0/1961201486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec1a1710 con 0x7ff2ec1072a0 2024-04-25T00:16:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.516+0000 7ff2f1000640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec1072a0 0x7ff2ec1a10a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400e970 tx=0x7ff2e400ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.516+0000 7ff2e8c00640 1 -- 172.21.15.42:0/1961201486 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e400ccc0 con 0x7ff2ec1072a0 2024-04-25T00:16:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.516+0000 7ff2e8c00640 1 -- 172.21.15.42:0/1961201486 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2e4005020 con 0x7ff2ec1072a0 2024-04-25T00:16:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.516+0000 7ff2e8c00640 1 -- 172.21.15.42:0/1961201486 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4004a00 con 0x7ff2ec1072a0 2024-04-25T00:16:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.516+0000 7ff2f1a00640 1 -- 172.21.15.42:0/1961201486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ec1a1a00 con 0x7ff2ec1072a0 2024-04-25T00:16:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.516+0000 7ff2f1a00640 1 -- 172.21.15.42:0/1961201486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ec1a1fa0 con 0x7ff2ec1072a0 2024-04-25T00:16:10.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.518+0000 7ff2f1a00640 1 -- 172.21.15.42:0/1961201486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ec108df0 con 0x7ff2ec1072a0 2024-04-25T00:16:10.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.519+0000 7ff2e8c00640 1 -- 172.21.15.42:0/1961201486 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff2e40051d0 con 0x7ff2ec1072a0 2024-04-25T00:16:10.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.521+0000 7ff2e8c00640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2d00a2c10 0x7ff2d00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:10.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.521+0000 7ff2e8c00640 1 -- 172.21.15.42:0/1961201486 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff2e40ceac0 con 0x7ff2ec1072a0 2024-04-25T00:16:10.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.521+0000 7ff2ebe00640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2d00a2c10 0x7ff2d00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:10.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.522+0000 7ff2ebe00640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2d00a2c10 0x7ff2d00a50d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005e30 tx=0x7ff2dc005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:10.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.525+0000 7ff2e8c00640 1 -- 172.21.15.42:0/1961201486 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff2e409be20 con 0x7ff2ec1072a0 2024-04-25T00:16:10.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:10 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.42:6838/3053633590,v1:172.21.15.42:6839/3053633590] up:boot 2024-04-25T00:16:10.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:10 smithi106 ceph-mon[32510]: fsmap 10 up:standby 2024-04-25T00:16:10.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:10 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:16:10.625 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:10 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.42:6838/3053633590,v1:172.21.15.42:6839/3053633590] up:boot 2024-04-25T00:16:10.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:10 smithi042 ceph-mon[28130]: fsmap 10 up:standby 2024-04-25T00:16:10.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:10 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T00:16:10.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.776+0000 7ff2f1a00640 1 -- 172.21.15.42:0/1961201486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2ec062340 con 0x7ff2ec1072a0 2024-04-25T00:16:10.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.777+0000 7ff2e8c00640 1 -- 172.21.15.42:0/1961201486 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7ff2e409b720 con 0x7ff2ec1072a0 2024-04-25T00:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:10.778 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T00:16:10.778 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 11 2024-04-25T00:16:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 -- 172.21.15.42:0/1961201486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2d00a2c10 msgr2=0x7ff2d00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2d00a2c10 0x7ff2d00a50d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc005e30 tx=0x7ff2dc005f50 comp rx=0 tx=0).stop 2024-04-25T00:16:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 -- 172.21.15.42:0/1961201486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec1072a0 msgr2=0x7ff2ec1a10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:10.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec1072a0 0x7ff2ec1a10a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400e970 tx=0x7ff2e400ee30 comp rx=0 tx=0).stop 2024-04-25T00:16:10.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 -- 172.21.15.42:0/1961201486 shutdown_connections 2024-04-25T00:16:10.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff2d00a2c10 0x7ff2d00a50d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2ec1072a0 0x7ff2ec1a10a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff2ec104970 0x7ff2ec19a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 --2- 172.21.15.42:0/1961201486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff2ec102040 0x7ff2ec199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:10.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 -- 172.21.15.42:0/1961201486 >> 172.21.15.42:0/1961201486 conn(0x7ff2ec0fba10 msgr2=0x7ff2ec1016a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:10.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 -- 172.21.15.42:0/1961201486 shutdown_connections 2024-04-25T00:16:10.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:10.780+0000 7ff2f1a00640 1 -- 172.21.15.42:0/1961201486 wait complete. 2024-04-25T00:16:11.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:11 smithi042 ceph-mon[28130]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:11.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:11 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1961201486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:11 smithi136 ceph-mon[32923]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:11 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1961201486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:11 smithi106 ceph-mon[32510]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:11 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1961201486' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:11.781 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:16:11 smithi042 systemd[1]: Starting Ceph mds.a for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:16:12.079 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:16:11 smithi042 podman[69384]: 2024-04-25 00:16:11.686213918 +0000 UTC m=+0.037486289 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:16:12.079 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:16:11 smithi042 podman[69384]: 2024-04-25 00:16:11.827473278 +0000 UTC m=+0.178745679 container create 640f6360ec465e117b397853204e118b227683875cd118cba1ad282a43080275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T00:16:12.079 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:16:12 smithi042 podman[69384]: 2024-04-25 00:16:12.078099195 +0000 UTC m=+0.429371603 container init 640f6360ec465e117b397853204e118b227683875cd118cba1ad282a43080275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:16:12.377 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:16:12 smithi042 podman[69384]: 2024-04-25 00:16:12.088679456 +0000 UTC m=+0.439951830 container start 640f6360ec465e117b397853204e118b227683875cd118cba1ad282a43080275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-25T00:16:12.377 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:16:12 smithi042 ceph-mds[69403]: -- 172.21.15.42:0/3874052371 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56115fc1da40 con 0x56115fbf0800 2024-04-25T00:16:12.377 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:16:12 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a[69399]: starting mds.a at 2024-04-25T00:16:12.377 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:16:12 smithi042 bash[69384]: 640f6360ec465e117b397853204e118b227683875cd118cba1ad282a43080275 2024-04-25T00:16:12.377 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:16:12 smithi042 systemd[1]: Started Ceph mds.a for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:16:12.801 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:13.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.021+0000 7fb3c1000640 1 -- 172.21.15.42:0/847635808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3bc108db0 msgr2=0x7fb3bc109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:13.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.021+0000 7fb3c1000640 1 --2- 172.21.15.42:0/847635808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3bc108db0 0x7fb3bc109230 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009900 tx=0x7fb3a802a400 comp rx=0 tx=0).stop 2024-04-25T00:16:13.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.021+0000 7fb3c1000640 1 -- 172.21.15.42:0/847635808 shutdown_connections 2024-04-25T00:16:13.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.021+0000 7fb3c1000640 1 --2- 172.21.15.42:0/847635808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc109770 0x7fb3bc110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.021+0000 7fb3c1000640 1 --2- 172.21.15.42:0/847635808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3bc108db0 0x7fb3bc109230 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.021+0000 7fb3c1000640 1 --2- 172.21.15.42:0/847635808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3bc107bc0 0x7fb3bc107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.021+0000 7fb3c1000640 1 -- 172.21.15.42:0/847635808 >> 172.21.15.42:0/847635808 conn(0x7fb3bc076b80 msgr2=0x7fb3bc078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:13.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.022+0000 7fb3c1000640 1 -- 172.21.15.42:0/847635808 shutdown_connections 2024-04-25T00:16:13.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.022+0000 7fb3c1000640 1 -- 172.21.15.42:0/847635808 wait complete. 2024-04-25T00:16:13.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.023+0000 7fb3c1000640 1 Processor -- start 2024-04-25T00:16:13.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.023+0000 7fb3c1000640 1 -- start start 2024-04-25T00:16:13.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.023+0000 7fb3c1000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3bc107bc0 0x7fb3bc19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:13.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.023+0000 7fb3c1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc108db0 0x7fb3bc19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:13.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.023+0000 7fb3c1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3bc109770 0x7fb3bc1a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:13.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3c1000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc113870 con 0x7fb3bc108db0 2024-04-25T00:16:13.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3c1000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1136f0 con 0x7fb3bc107bc0 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3c1000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc1139f0 con 0x7fb3bc109770 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3baa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc108db0 0x7fb3bc19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3bb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3bc107bc0 0x7fb3bc19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3baa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc108db0 0x7fb3bc19e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42150/0 (socket says 172.21.15.42:42150) 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3baa00640 1 -- 172.21.15.42:0/4262793987 learned_addr learned my addr 172.21.15.42:0/4262793987 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3bb400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3bc107bc0 0x7fb3bc19de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42398/0 (socket says 172.21.15.42:42398) 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3bbe00640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3bc109770 0x7fb3bc1a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3bb400640 1 -- 172.21.15.42:0/4262793987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3bc109770 msgr2=0x7fb3bc1a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3bb400640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3bc109770 0x7fb3bc1a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3bb400640 1 -- 172.21.15.42:0/4262793987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc108db0 msgr2=0x7fb3bc19e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3bb400640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc108db0 0x7fb3bc19e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3bb400640 1 -- 172.21.15.42:0/4262793987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc1a5a70 con 0x7fb3bc107bc0 2024-04-25T00:16:13.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.024+0000 7fb3baa00640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc108db0 0x7fb3bc19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:13.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.025+0000 7fb3bb400640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3bc107bc0 0x7fb3bc19de50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00c940 tx=0x7fb3ac00ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:13.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.025+0000 7fb3b3e00640 1 -- 172.21.15.42:0/4262793987 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac007bb0 con 0x7fb3bc107bc0 2024-04-25T00:16:13.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.025+0000 7fb3c1000640 1 -- 172.21.15.42:0/4262793987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc1a5d60 con 0x7fb3bc107bc0 2024-04-25T00:16:13.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.025+0000 7fb3b3e00640 1 -- 172.21.15.42:0/4262793987 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3ac007d50 con 0x7fb3bc107bc0 2024-04-25T00:16:13.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.025+0000 7fb3b3e00640 1 -- 172.21.15.42:0/4262793987 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac018710 con 0x7fb3bc107bc0 2024-04-25T00:16:13.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.025+0000 7fb3c1000640 1 -- 172.21.15.42:0/4262793987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc1a6210 con 0x7fb3bc107bc0 2024-04-25T00:16:13.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.026+0000 7fb3c1000640 1 -- 172.21.15.42:0/4262793987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3bc10ee10 con 0x7fb3bc107bc0 2024-04-25T00:16:13.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.028+0000 7fb3b3e00640 1 -- 172.21.15.42:0/4262793987 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb3ac018910 con 0x7fb3bc107bc0 2024-04-25T00:16:13.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.029+0000 7fb3b3e00640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3a00a2b90 0x7fb3a00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:13.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.030+0000 7fb3b3e00640 1 -- 172.21.15.42:0/4262793987 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb3ac0cec50 con 0x7fb3bc107bc0 2024-04-25T00:16:13.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.030+0000 7fb3baa00640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3a00a2b90 0x7fb3a00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:13.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.030+0000 7fb3baa00640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3a00a2b90 0x7fb3a00a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc19f360 tx=0x7fb3a80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:13.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.034+0000 7fb3b3e00640 1 -- 172.21.15.42:0/4262793987 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb3ac014030 con 0x7fb3bc107bc0 2024-04-25T00:16:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.278+0000 7fb3c1000640 1 -- 172.21.15.42:0/4262793987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3bc0623e0 con 0x7fb3bc107bc0 2024-04-25T00:16:13.278 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.278+0000 7fb3b3e00640 1 -- 172.21.15.42:0/4262793987 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7fb3ac09bfb0 con 0x7fb3bc107bc0 2024-04-25T00:16:13.279 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:13.279 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/4068466248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4068466248},{"type":"v1","addr":"172.21.15.42:6841","nonce":4068466248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-04-25T00:16:13.279 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 12 2024-04-25T00:16:13.281 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 -- 172.21.15.42:0/4262793987 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3a00a2b90 msgr2=0x7fb3a00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3a00a2b90 0x7fb3a00a5050 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc19f360 tx=0x7fb3a80315b0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 -- 172.21.15.42:0/4262793987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3bc107bc0 msgr2=0x7fb3bc19de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3bc107bc0 0x7fb3bc19de50 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00c940 tx=0x7fb3ac00ce00 comp rx=0 tx=0).stop 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 -- 172.21.15.42:0/4262793987 shutdown_connections 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb3a00a2b90 0x7fb3a00a5050 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3bc109770 0x7fb3bc1a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb3bc108db0 0x7fb3bc19e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 --2- 172.21.15.42:0/4262793987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3bc107bc0 0x7fb3bc19de50 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 -- 172.21.15.42:0/4262793987 >> 172.21.15.42:0/4262793987 conn(0x7fb3bc076b80 msgr2=0x7fb3bc10bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.281+0000 7fb3c1000640 1 -- 172.21.15.42:0/4262793987 shutdown_connections 2024-04-25T00:16:13.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:13.282+0000 7fb3c1000640 1 -- 172.21.15.42:0/4262793987 wait complete. 2024-04-25T00:16:13.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] up:boot 2024-04-25T00:16:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: fsmap 11 up:standby 2024-04-25T00:16:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:16:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:13 smithi106 ceph-mon[32510]: Deploying daemon mds.b on smithi106 2024-04-25T00:16:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:13.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] up:boot 2024-04-25T00:16:13.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: fsmap 11 up:standby 2024-04-25T00:16:13.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:16:13.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:13.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:13.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:13.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:13.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:13.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:13.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:13 smithi136 ceph-mon[32923]: Deploying daemon mds.b on smithi106 2024-04-25T00:16:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] up:boot 2024-04-25T00:16:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: fsmap 11 up:standby 2024-04-25T00:16:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:16:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T00:16:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T00:16:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:13.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:13 smithi042 ceph-mon[28130]: Deploying daemon mds.b on smithi106 2024-04-25T00:16:14.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:14 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/4262793987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:14.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:14 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/4262793987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:14.588 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:14 smithi106 systemd[1]: Starting Ceph mds.b for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:16:14.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:14 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/4262793987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:14.867 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:14 smithi106 podman[63373]: 2024-04-25 00:16:14.514857158 +0000 UTC m=+0.017858992 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:16:14.867 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:14 smithi106 podman[63373]: 2024-04-25 00:16:14.623296574 +0000 UTC m=+0.126298393 container create 132e3dfb6c2687f815f677db695f2825215fb011dcc2311f283344be59f4f295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T00:16:14.868 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:14 smithi106 podman[63373]: 2024-04-25 00:16:14.842348882 +0000 UTC m=+0.345350702 container init 132e3dfb6c2687f815f677db695f2825215fb011dcc2311f283344be59f4f295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T00:16:14.868 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:14 smithi106 podman[63373]: 2024-04-25 00:16:14.844913388 +0000 UTC m=+0.347915217 container start 132e3dfb6c2687f815f677db695f2825215fb011dcc2311f283344be59f4f295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:16:15.209 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:14 smithi106 ceph-mds[63392]: -- 172.21.15.106:0/2541550036 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561a819a7a40 con 0x561a8197b000 2024-04-25T00:16:15.210 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:14 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b[63388]: starting mds.b at 2024-04-25T00:16:15.210 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:14 smithi106 bash[63373]: 132e3dfb6c2687f815f677db695f2825215fb011dcc2311f283344be59f4f295 2024-04-25T00:16:15.210 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:14 smithi106 systemd[1]: Started Ceph mds.b for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:16:15.210 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:15 smithi106 ceph-mon[32510]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:15.210 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.303 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:15 smithi136 ceph-mon[32923]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.520+0000 7f4edb800640 1 -- 172.21.15.42:0/321301510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ed4075700 msgr2=0x7f4ed4073b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.520+0000 7f4edb800640 1 --2- 172.21.15.42:0/321301510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ed4075700 0x7f4ed4073b50 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009900 tx=0x7f4ec402a480 comp rx=0 tx=0).stop 2024-04-25T00:16:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.520+0000 7f4edb800640 1 -- 172.21.15.42:0/321301510 shutdown_connections 2024-04-25T00:16:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.520+0000 7f4edb800640 1 --2- 172.21.15.42:0/321301510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed4109640 0x7f4ed410bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.520+0000 7f4edb800640 1 --2- 172.21.15.42:0/321301510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ed4074120 0x7f4ed4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.520+0000 7f4edb800640 1 --2- 172.21.15.42:0/321301510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ed4075700 0x7f4ed4073b50 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.520+0000 7f4edb800640 1 -- 172.21.15.42:0/321301510 >> 172.21.15.42:0/321301510 conn(0x7f4ed40fcd40 msgr2=0x7f4ed40ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.520+0000 7f4edb800640 1 -- 172.21.15.42:0/321301510 shutdown_connections 2024-04-25T00:16:15.521 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.520+0000 7f4edb800640 1 -- 172.21.15.42:0/321301510 wait complete. 2024-04-25T00:16:15.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.521+0000 7f4edb800640 1 Processor -- start 2024-04-25T00:16:15.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.521+0000 7f4edb800640 1 -- start start 2024-04-25T00:16:15.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4edb800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ed4074120 0x7f4ed4199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4edb800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ed4075700 0x7f4ed419a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4edb800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed4109640 0x7f4ed41a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4edb800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed410f660 con 0x7f4ed4109640 2024-04-25T00:16:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4edb800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed410f4e0 con 0x7f4ed4075700 2024-04-25T00:16:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4edb800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed410f7e0 con 0x7f4ed4074120 2024-04-25T00:16:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4ed8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ed4074120 0x7f4ed4199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4ed9800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed4109640 0x7f4ed41a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4ed8e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ed4074120 0x7f4ed4199cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45132/0 (socket says 172.21.15.42:45132) 2024-04-25T00:16:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4ecbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ed4075700 0x7f4ed419a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.522+0000 7f4ed8e00640 1 -- 172.21.15.42:0/1422611372 learned_addr learned my addr 172.21.15.42:0/1422611372 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4ed8e00640 1 -- 172.21.15.42:0/1422611372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ed4075700 msgr2=0x7f4ed419a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4ed8e00640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ed4075700 0x7f4ed419a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4ed8e00640 1 -- 172.21.15.42:0/1422611372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed4109640 msgr2=0x7f4ed41a1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4ed8e00640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed4109640 0x7f4ed41a1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4ed8e00640 1 -- 172.21.15.42:0/1422611372 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed41a18f0 con 0x7f4ed4074120 2024-04-25T00:16:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4ecbe00640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ed4075700 0x7f4ed419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:15.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4ed8e00640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ed4074120 0x7f4ed4199cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4031440 tx=0x7f4ec4032e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:15.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4ed2000640 1 -- 172.21.15.42:0/1422611372 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec403f030 con 0x7f4ed4074120 2024-04-25T00:16:15.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4ed2000640 1 -- 172.21.15.42:0/1422611372 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ec4003ce0 con 0x7f4ed4074120 2024-04-25T00:16:15.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4edb800640 1 -- 172.21.15.42:0/1422611372 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed41a1b80 con 0x7f4ed4074120 2024-04-25T00:16:15.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4ed2000640 1 -- 172.21.15.42:0/1422611372 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4035070 con 0x7f4ed4074120 2024-04-25T00:16:15.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.523+0000 7f4edb800640 1 -- 172.21.15.42:0/1422611372 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed41a2070 con 0x7f4ed4074120 2024-04-25T00:16:15.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.525+0000 7f4edb800640 1 -- 172.21.15.42:0/1422611372 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed4108400 con 0x7f4ed4074120 2024-04-25T00:16:15.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.526+0000 7f4ed2000640 1 -- 172.21.15.42:0/1422611372 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4ec4042030 con 0x7f4ed4074120 2024-04-25T00:16:15.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.528+0000 7f4ed2000640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4eb80a2b90 0x7f4eb80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:15.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.528+0000 7f4ed2000640 1 -- 172.21.15.42:0/1422611372 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4ec40ef1b0 con 0x7f4ed4074120 2024-04-25T00:16:15.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.528+0000 7f4ecbe00640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4eb80a2b90 0x7f4eb80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:15.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.529+0000 7f4ecbe00640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4eb80a2b90 0x7f4eb80a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4ed419b1b0 tx=0x7f4ebc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:15.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.536+0000 7f4ed2000640 1 -- 172.21.15.42:0/1422611372 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4ec40bbec0 con 0x7f4ed4074120 2024-04-25T00:16:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:15 smithi042 ceph-mon[28130]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:15.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:15.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.785+0000 7f4edb800640 1 -- 172.21.15.42:0/1422611372 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ed4062340 con 0x7f4ed4074120 2024-04-25T00:16:15.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.786+0000 7f4ed2000640 1 -- 172.21.15.42:0/1422611372 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7f4ec40c01c0 con 0x7f4ed4074120 2024-04-25T00:16:15.786 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:15.787 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24505,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24517,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24521,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24538,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24568,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6841/4068466248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4068466248},{"type":"v1","addr":"172.21.15.42:6841","nonce":4068466248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6839/3349564106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3349564106},{"type":"v1","addr":"172.21.15.106:6839","nonce":3349564106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T00:16:15.787 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 13 2024-04-25T00:16:15.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 -- 172.21.15.42:0/1422611372 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4eb80a2b90 msgr2=0x7f4eb80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4eb80a2b90 0x7f4eb80a5050 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4ed419b1b0 tx=0x7f4ebc00b390 comp rx=0 tx=0).stop 2024-04-25T00:16:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 -- 172.21.15.42:0/1422611372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ed4074120 msgr2=0x7f4ed4199cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ed4074120 0x7f4ed4199cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4031440 tx=0x7f4ec4032e20 comp rx=0 tx=0).stop 2024-04-25T00:16:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 -- 172.21.15.42:0/1422611372 shutdown_connections 2024-04-25T00:16:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4eb80a2b90 0x7f4eb80a5050 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4ed4109640 0x7f4ed41a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ed4075700 0x7f4ed419a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 --2- 172.21.15.42:0/1422611372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ed4074120 0x7f4ed4199cb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 -- 172.21.15.42:0/1422611372 >> 172.21.15.42:0/1422611372 conn(0x7f4ed40fcd40 msgr2=0x7f4ed41055d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:15.790 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 -- 172.21.15.42:0/1422611372 shutdown_connections 2024-04-25T00:16:15.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:15.789+0000 7f4edb800640 1 -- 172.21.15.42:0/1422611372 wait complete. 2024-04-25T00:16:15.809 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.014+0000 7fdf3aa00640 1 -- 172.21.15.42:0/2253552185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf34077340 msgr2=0x7fdf340777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.014+0000 7fdf3aa00640 1 --2- 172.21.15.42:0/2253552185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf34077340 0x7fdf340777a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009900 tx=0x7fdf2002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.015+0000 7fdf3aa00640 1 -- 172.21.15.42:0/2253552185 shutdown_connections 2024-04-25T00:16:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.015+0000 7fdf3aa00640 1 --2- 172.21.15.42:0/2253552185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf34077ce0 0x7fdf34078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.015+0000 7fdf3aa00640 1 --2- 172.21.15.42:0/2253552185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf34077340 0x7fdf340777a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.015+0000 7fdf3aa00640 1 --2- 172.21.15.42:0/2253552185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf34076100 0x7fdf34076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.015+0000 7fdf3aa00640 1 -- 172.21.15.42:0/2253552185 >> 172.21.15.42:0/2253552185 conn(0x7fdf340fed50 msgr2=0x7fdf34101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.015+0000 7fdf3aa00640 1 -- 172.21.15.42:0/2253552185 shutdown_connections 2024-04-25T00:16:16.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.015+0000 7fdf3aa00640 1 -- 172.21.15.42:0/2253552185 wait complete. 2024-04-25T00:16:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf3aa00640 1 Processor -- start 2024-04-25T00:16:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf3aa00640 1 -- start start 2024-04-25T00:16:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf3aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf34076100 0x7fdf3419de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf3aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf34077340 0x7fdf3419e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf3aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf34077ce0 0x7fdf341a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.018 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf3aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34113890 con 0x7fdf34077340 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf3aa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34113710 con 0x7fdf34077ce0 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf3aa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdf34113a10 con 0x7fdf34076100 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf38c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf34077340 0x7fdf3419e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf38c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf34077340 0x7fdf3419e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42170/0 (socket says 172.21.15.42:42170) 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf38c00640 1 -- 172.21.15.42:0/3882274277 learned_addr learned my addr 172.21.15.42:0/3882274277 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf3a000640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf34077ce0 0x7fdf341a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.016+0000 7fdf39600640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf34076100 0x7fdf3419de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf38c00640 1 -- 172.21.15.42:0/3882274277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf34076100 msgr2=0x7fdf3419de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf38c00640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf34076100 0x7fdf3419de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf38c00640 1 -- 172.21.15.42:0/3882274277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf34077ce0 msgr2=0x7fdf341a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf38c00640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf34077ce0 0x7fdf341a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf38c00640 1 -- 172.21.15.42:0/3882274277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf341a5ab0 con 0x7fdf34077340 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf39600640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf34076100 0x7fdf3419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:16.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf3a000640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf34077ce0 0x7fdf341a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf38c00640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf34077340 0x7fdf3419e3d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf20031440 tx=0x7fdf20032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf32000640 1 -- 172.21.15.42:0/3882274277 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2003f030 con 0x7fdf34077340 2024-04-25T00:16:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf3aa00640 1 -- 172.21.15.42:0/3882274277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf341a5d40 con 0x7fdf34077340 2024-04-25T00:16:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf32000640 1 -- 172.21.15.42:0/3882274277 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf20003ce0 con 0x7fdf34077340 2024-04-25T00:16:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.017+0000 7fdf32000640 1 -- 172.21.15.42:0/3882274277 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf20035070 con 0x7fdf34077340 2024-04-25T00:16:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.018+0000 7fdf3aa00640 1 -- 172.21.15.42:0/3882274277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf341a6220 con 0x7fdf34077340 2024-04-25T00:16:16.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.019+0000 7fdf3aa00640 1 -- 172.21.15.42:0/3882274277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf34110280 con 0x7fdf34077340 2024-04-25T00:16:16.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.021+0000 7fdf32000640 1 -- 172.21.15.42:0/3882274277 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdf20042030 con 0x7fdf34077340 2024-04-25T00:16:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.022+0000 7fdf32000640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2b90 0x7fdf1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.022+0000 7fdf32000640 1 -- 172.21.15.42:0/3882274277 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdf200ef180 con 0x7fdf34077340 2024-04-25T00:16:16.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.022+0000 7fdf39600640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2b90 0x7fdf1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.023+0000 7fdf39600640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2b90 0x7fdf1c0a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdf28005fd0 tx=0x7fdf28009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.027+0000 7fdf32000640 1 -- 172.21.15.42:0/3882274277 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdf200bbe90 con 0x7fdf34077340 2024-04-25T00:16:16.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.256+0000 7fdf3aa00640 1 -- 172.21.15.42:0/3882274277 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdf34062340 con 0x7fdf34077340 2024-04-25T00:16:16.257 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.258+0000 7fdf32000640 1 -- 172.21.15.42:0/3882274277 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18454 (secure 0 0 0) 0x7fdf200c0190 con 0x7fdf34077340 2024-04-25T00:16:16.258 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:16.258 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":82,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:15:25.846529+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 -- 172.21.15.42:0/3882274277 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2b90 msgr2=0x7fdf1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2b90 0x7fdf1c0a5050 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fdf28005fd0 tx=0x7fdf28009500 comp rx=0 tx=0).stop 2024-04-25T00:16:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 -- 172.21.15.42:0/3882274277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf34077340 msgr2=0x7fdf3419e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf34077340 0x7fdf3419e3d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fdf20031440 tx=0x7fdf20032ce0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 -- 172.21.15.42:0/3882274277 shutdown_connections 2024-04-25T00:16:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdf1c0a2b90 0x7fdf1c0a5050 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdf34077ce0 0x7fdf341a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdf34077340 0x7fdf3419e3d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 --2- 172.21.15.42:0/3882274277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdf34076100 0x7fdf3419de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 -- 172.21.15.42:0/3882274277 >> 172.21.15.42:0/3882274277 conn(0x7fdf340fed50 msgr2=0x7fdf341003d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 -- 172.21.15.42:0/3882274277 shutdown_connections 2024-04-25T00:16:16.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.261+0000 7fdf3aa00640 1 -- 172.21.15.42:0/3882274277 wait complete. 2024-04-25T00:16:16.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:11:14.472869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:16:16.279 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:16:16.398 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:16 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] up:boot 2024-04-25T00:16:16.399 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:16 smithi106 ceph-mon[32510]: fsmap 12 up:standby 2024-04-25T00:16:16.399 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:16 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:16:16.399 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:16 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:16.399 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:16 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1422611372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:16.399 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:16 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:16.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.470+0000 7f3782a00640 1 -- 172.21.15.42:0/4260500314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c077340 msgr2=0x7f377c0777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.470+0000 7f3782a00640 1 --2- 172.21.15.42:0/4260500314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c077340 0x7f377c0777a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3764009900 tx=0x7f376402a410 comp rx=0 tx=0).stop 2024-04-25T00:16:16.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.470+0000 7f3782a00640 1 -- 172.21.15.42:0/4260500314 shutdown_connections 2024-04-25T00:16:16.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.470+0000 7f3782a00640 1 --2- 172.21.15.42:0/4260500314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f377c077ce0 0x7f377c078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.470+0000 7f3782a00640 1 --2- 172.21.15.42:0/4260500314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c077340 0x7f377c0777a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.470+0000 7f3782a00640 1 --2- 172.21.15.42:0/4260500314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c076100 0x7f377c076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.470+0000 7f3782a00640 1 -- 172.21.15.42:0/4260500314 >> 172.21.15.42:0/4260500314 conn(0x7f377c0fef30 msgr2=0x7f377c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.470+0000 7f3782a00640 1 -- 172.21.15.42:0/4260500314 shutdown_connections 2024-04-25T00:16:16.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.470+0000 7f3782a00640 1 -- 172.21.15.42:0/4260500314 wait complete. 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f3782a00640 1 Processor -- start 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f3782a00640 1 -- start start 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f3782a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c076100 0x7f377c19e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f3782a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f377c077340 0x7f377c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f3782a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c077ce0 0x7f377c1a5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f3782a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f377c113a70 con 0x7f377c077340 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f3782a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f377c1138f0 con 0x7f377c076100 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f3782a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f377c113bf0 con 0x7f377c077ce0 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f377be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c076100 0x7f377c19e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f377be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c076100 0x7f377c19e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42456/0 (socket says 172.21.15.42:42456) 2024-04-25T00:16:16.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.471+0000 7f377be00640 1 -- 172.21.15.42:0/392501995 learned_addr learned my addr 172.21.15.42:0/392501995 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f377b400640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f377c077340 0x7f377c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f3780a00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c077ce0 0x7f377c1a5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f377be00640 1 -- 172.21.15.42:0/392501995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c077ce0 msgr2=0x7f377c1a5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f377be00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c077ce0 0x7f377c1a5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f377be00640 1 -- 172.21.15.42:0/392501995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f377c077340 msgr2=0x7f377c19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f377be00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f377c077340 0x7f377c19e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f377be00640 1 -- 172.21.15.42:0/392501995 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377c1a5cf0 con 0x7f377c076100 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f377be00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c076100 0x7f377c19e0d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f376c00e9f0 tx=0x7f376c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f3778c00640 1 -- 172.21.15.42:0/392501995 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c00cd80 con 0x7f377c076100 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f3778c00640 1 -- 172.21.15.42:0/392501995 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f376c005500 con 0x7f377c076100 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.472+0000 7f3778c00640 1 -- 172.21.15.42:0/392501995 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376c010690 con 0x7f377c076100 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.473+0000 7f3782a00640 1 -- 172.21.15.42:0/392501995 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377c1a5fe0 con 0x7f377c076100 2024-04-25T00:16:16.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.473+0000 7f3782a00640 1 -- 172.21.15.42:0/392501995 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377c1a63f0 con 0x7f377c076100 2024-04-25T00:16:16.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.474+0000 7f3782a00640 1 -- 172.21.15.42:0/392501995 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377c110310 con 0x7f377c076100 2024-04-25T00:16:16.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.476+0000 7f3778c00640 1 -- 172.21.15.42:0/392501995 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f376c0046c0 con 0x7f377c076100 2024-04-25T00:16:16.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.477+0000 7f3778c00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37600a2c10 0x7f37600a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.477+0000 7f3778c00640 1 -- 172.21.15.42:0/392501995 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f376c0cefc0 con 0x7f377c076100 2024-04-25T00:16:16.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.477+0000 7f377b400640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37600a2c10 0x7f37600a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.478+0000 7f377b400640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37600a2c10 0x7f37600a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3764031440 tx=0x7f37640315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.481+0000 7f3778c00640 1 -- 172.21.15.42:0/392501995 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f376c09c320 con 0x7f377c076100 2024-04-25T00:16:16.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:16 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] up:boot 2024-04-25T00:16:16.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:16 smithi136 ceph-mon[32923]: fsmap 12 up:standby 2024-04-25T00:16:16.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:16 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:16:16.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:16 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:16.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:16 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1422611372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:16.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:16 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:16.614 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:16 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] up:boot 2024-04-25T00:16:16.614 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:16 smithi042 ceph-mon[28130]: fsmap 12 up:standby 2024-04-25T00:16:16.614 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:16 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:16:16.614 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:16 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:16.615 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:16 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1422611372' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:16.615 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:16 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:16.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.696+0000 7f3782a00640 1 -- 172.21.15.42:0/392501995 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f377c062450 con 0x7f377c076100 2024-04-25T00:16:16.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.697+0000 7f3778c00640 1 -- 172.21.15.42:0/392501995 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f376c086070 con 0x7f377c076100 2024-04-25T00:16:16.697 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-25T00:16:16.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 -- 172.21.15.42:0/392501995 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37600a2c10 msgr2=0x7f37600a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37600a2c10 0x7f37600a50d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f3764031440 tx=0x7f37640315b0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 -- 172.21.15.42:0/392501995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c076100 msgr2=0x7f377c19e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c076100 0x7f377c19e0d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f376c00e9f0 tx=0x7f376c00edf0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 -- 172.21.15.42:0/392501995 shutdown_connections 2024-04-25T00:16:16.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37600a2c10 0x7f37600a50d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f377c077ce0 0x7f377c1a5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f377c077340 0x7f377c19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 --2- 172.21.15.42:0/392501995 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c076100 0x7f377c19e0d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 -- 172.21.15.42:0/392501995 >> 172.21.15.42:0/392501995 conn(0x7f377c0fef30 msgr2=0x7f377c100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.700+0000 7f3782a00640 1 -- 172.21.15.42:0/392501995 shutdown_connections 2024-04-25T00:16:16.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.701+0000 7f3782a00640 1 -- 172.21.15.42:0/392501995 wait complete. 2024-04-25T00:16:16.730 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T00:16:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.928+0000 7fd27a400640 1 -- 172.21.15.42:0/206369051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd274107be0 msgr2=0x7fd274108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.928+0000 7fd27a400640 1 --2- 172.21.15.42:0/206369051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd274107be0 0x7fd274108000 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd264009900 tx=0x7fd26402a410 comp rx=0 tx=0).stop 2024-04-25T00:16:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.928+0000 7fd27a400640 1 -- 172.21.15.42:0/206369051 shutdown_connections 2024-04-25T00:16:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.928+0000 7fd27a400640 1 --2- 172.21.15.42:0/206369051 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274109790 0x7fd274110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.928+0000 7fd27a400640 1 --2- 172.21.15.42:0/206369051 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd274108dd0 0x7fd274109250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.928+0000 7fd27a400640 1 --2- 172.21.15.42:0/206369051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd274107be0 0x7fd274108000 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.929 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.928+0000 7fd27a400640 1 -- 172.21.15.42:0/206369051 >> 172.21.15.42:0/206369051 conn(0x7fd274076b80 msgr2=0x7fd274078fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:16.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.929+0000 7fd27a400640 1 -- 172.21.15.42:0/206369051 shutdown_connections 2024-04-25T00:16:16.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.929+0000 7fd27a400640 1 -- 172.21.15.42:0/206369051 wait complete. 2024-04-25T00:16:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.929+0000 7fd27a400640 1 Processor -- start 2024-04-25T00:16:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd27a400640 1 -- start start 2024-04-25T00:16:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd27a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd274107be0 0x7fd27419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd27a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274108dd0 0x7fd27419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd27a400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd274109790 0x7fd2741a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd27a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd2741138b0 con 0x7fd274107be0 2024-04-25T00:16:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd27a400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd274113730 con 0x7fd274109790 2024-04-25T00:16:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd27a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd274113a30 con 0x7fd274108dd0 2024-04-25T00:16:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd26be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274108dd0 0x7fd27419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd279000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd274107be0 0x7fd27419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd26be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274108dd0 0x7fd27419e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47586/0 (socket says 172.21.15.42:47586) 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd26be00640 1 -- 172.21.15.42:0/1585500730 learned_addr learned my addr 172.21.15.42:0/1585500730 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.930+0000 7fd279a00640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd274109790 0x7fd2741a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.931+0000 7fd26be00640 1 -- 172.21.15.42:0/1585500730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd274109790 msgr2=0x7fd2741a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.931+0000 7fd26be00640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd274109790 0x7fd2741a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.931+0000 7fd26be00640 1 -- 172.21.15.42:0/1585500730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd274107be0 msgr2=0x7fd27419df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.931+0000 7fd26be00640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd274107be0 0x7fd27419df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.931+0000 7fd26be00640 1 -- 172.21.15.42:0/1585500730 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2741a5b20 con 0x7fd274108dd0 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.931+0000 7fd26be00640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274108dd0 0x7fd27419e440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd26000e9f0 tx=0x7fd26000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.931+0000 7fd272000640 1 -- 172.21.15.42:0/1585500730 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd26000cd80 con 0x7fd274108dd0 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.931+0000 7fd272000640 1 -- 172.21.15.42:0/1585500730 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd260010910 con 0x7fd274108dd0 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.932+0000 7fd27a400640 1 -- 172.21.15.42:0/1585500730 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2741a5e10 con 0x7fd274108dd0 2024-04-25T00:16:16.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.932+0000 7fd272000640 1 -- 172.21.15.42:0/1585500730 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260018990 con 0x7fd274108dd0 2024-04-25T00:16:16.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.932+0000 7fd27a400640 1 -- 172.21.15.42:0/1585500730 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2741a62d0 con 0x7fd274108dd0 2024-04-25T00:16:16.934 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.933+0000 7fd27a400640 1 -- 172.21.15.42:0/1585500730 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27410ef00 con 0x7fd274108dd0 2024-04-25T00:16:16.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.935+0000 7fd272000640 1 -- 172.21.15.42:0/1585500730 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd260010ac0 con 0x7fd274108dd0 2024-04-25T00:16:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.937+0000 7fd272000640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2580a2b90 0x7fd2580a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.937+0000 7fd272000640 1 -- 172.21.15.42:0/1585500730 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fd2600cef60 con 0x7fd274108dd0 2024-04-25T00:16:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.937+0000 7fd279000640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2580a2b90 0x7fd2580a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:16.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.938+0000 7fd279000640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2580a2b90 0x7fd2580a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd26402aa00 tx=0x7fd2640315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:16.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:16.941+0000 7fd272000640 1 -- 172.21.15.42:0/1585500730 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd26009bc70 con 0x7fd274108dd0 2024-04-25T00:16:17.159 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.159+0000 7fd27a400640 1 -- 172.21.15.42:0/1585500730 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd2740623e0 con 0x7fd274108dd0 2024-04-25T00:16:17.160 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.160+0000 7fd272000640 1 -- 172.21.15.42:0/1585500730 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7fd26009ff70 con 0x7fd274108dd0 2024-04-25T00:16:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:17.160 INFO:teuthology.orchestra.run.smithi042.stdout:[ 2024-04-25T00:16:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:16:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: "poolnum": 1, 2024-04-25T00:16:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: "poolname": ".mgr" 2024-04-25T00:16:17.160 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:16:17.160 INFO:teuthology.orchestra.run.smithi042.stdout:] 2024-04-25T00:16:17.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 -- 172.21.15.42:0/1585500730 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2580a2b90 msgr2=0x7fd2580a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2580a2b90 0x7fd2580a5050 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd26402aa00 tx=0x7fd2640315b0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 -- 172.21.15.42:0/1585500730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274108dd0 msgr2=0x7fd27419e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274108dd0 0x7fd27419e440 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd26000e9f0 tx=0x7fd26000eeb0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 -- 172.21.15.42:0/1585500730 shutdown_connections 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2580a2b90 0x7fd2580a5050 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd274109790 0x7fd2741a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd274108dd0 0x7fd27419e440 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 --2- 172.21.15.42:0/1585500730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd274107be0 0x7fd27419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 -- 172.21.15.42:0/1585500730 >> 172.21.15.42:0/1585500730 conn(0x7fd274076b80 msgr2=0x7fd27410c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 -- 172.21.15.42:0/1585500730 shutdown_connections 2024-04-25T00:16:17.163 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.162+0000 7fd27a400640 1 -- 172.21.15.42:0/1585500730 wait complete. 2024-04-25T00:16:17.182 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T00:16:17.182 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T00:16:17.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.373+0000 7f30ebe00640 1 -- 172.21.15.42:0/2587075895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30e4102980 msgr2=0x7f30e4102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.373+0000 7f30ebe00640 1 --2- 172.21.15.42:0/2587075895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30e4102980 0x7f30e4102e00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009900 tx=0x7f30d002a410 comp rx=0 tx=0).stop 2024-04-25T00:16:17.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.373+0000 7f30ebe00640 1 -- 172.21.15.42:0/2587075895 shutdown_connections 2024-04-25T00:16:17.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.373+0000 7f30ebe00640 1 --2- 172.21.15.42:0/2587075895 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30e4103340 0x7f30e4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.373+0000 7f30ebe00640 1 --2- 172.21.15.42:0/2587075895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30e4102980 0x7f30e4102e00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.373+0000 7f30ebe00640 1 --2- 172.21.15.42:0/2587075895 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30e4101790 0x7f30e4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.373+0000 7f30ebe00640 1 -- 172.21.15.42:0/2587075895 >> 172.21.15.42:0/2587075895 conn(0x7f30e40fcd20 msgr2=0x7f30e40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:17.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.373+0000 7f30ebe00640 1 -- 172.21.15.42:0/2587075895 shutdown_connections 2024-04-25T00:16:17.374 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.373+0000 7f30ebe00640 1 -- 172.21.15.42:0/2587075895 wait complete. 2024-04-25T00:16:17.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.374+0000 7f30ebe00640 1 Processor -- start 2024-04-25T00:16:17.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.374+0000 7f30ebe00640 1 -- start start 2024-04-25T00:16:17.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.374+0000 7f30ebe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30e4101790 0x7f30e4199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.374+0000 7f30ebe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30e4102980 0x7f30e419a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.375 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.374+0000 7f30ebe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30e4103340 0x7f30e41a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.374+0000 7f30ebe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f30e410d410 con 0x7f30e4102980 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.374+0000 7f30ebe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f30e410d290 con 0x7f30e4103340 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.374+0000 7f30ebe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f30e410d590 con 0x7f30e4101790 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30e4101790 0x7f30e4199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30e4103340 0x7f30e41a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30e4101790 0x7f30e4199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47614/0 (socket says 172.21.15.42:47614) 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9400640 1 -- 172.21.15.42:0/1445832688 learned_addr learned my addr 172.21.15.42:0/1445832688 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e8a00640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30e4102980 0x7f30e419a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9e00640 1 -- 172.21.15.42:0/1445832688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30e4101790 msgr2=0x7f30e4199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9e00640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30e4101790 0x7f30e4199ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9e00640 1 -- 172.21.15.42:0/1445832688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30e4102980 msgr2=0x7f30e419a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9e00640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30e4102980 0x7f30e419a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9e00640 1 -- 172.21.15.42:0/1445832688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30e41a18e0 con 0x7f30e4103340 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9400640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30e4101790 0x7f30e4199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:17.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.375+0000 7f30e9e00640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30e4103340 0x7f30e41a1240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009fd0 tx=0x7f30dc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.376+0000 7f30e2000640 1 -- 172.21.15.42:0/1445832688 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00ce20 con 0x7f30e4103340 2024-04-25T00:16:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.376+0000 7f30e2000640 1 -- 172.21.15.42:0/1445832688 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30dc005020 con 0x7f30e4103340 2024-04-25T00:16:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.376+0000 7f30e2000640 1 -- 172.21.15.42:0/1445832688 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30dc010720 con 0x7f30e4103340 2024-04-25T00:16:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.376+0000 7f30ebe00640 1 -- 172.21.15.42:0/1445832688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30e41a1bd0 con 0x7f30e4103340 2024-04-25T00:16:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.376+0000 7f30ebe00640 1 -- 172.21.15.42:0/1445832688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30e41a20a0 con 0x7f30e4103340 2024-04-25T00:16:17.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.377+0000 7f30ebe00640 1 -- 172.21.15.42:0/1445832688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30e41a22d0 con 0x7f30e4103340 2024-04-25T00:16:17.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.381+0000 7f30e2000640 1 -- 172.21.15.42:0/1445832688 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f30dc005540 con 0x7f30e4103340 2024-04-25T00:16:17.382 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.382+0000 7f30e2000640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30cc0a2c10 0x7f30cc0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:17.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.382+0000 7f30e2000640 1 -- 172.21.15.42:0/1445832688 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f30dc0cf3b0 con 0x7f30e4103340 2024-04-25T00:16:17.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.383+0000 7f30e9400640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30cc0a2c10 0x7f30cc0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:17.384 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.383+0000 7f30e9400640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30cc0a2c10 0x7f30cc0a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30e419ab70 tx=0x7f30d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:17.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.386+0000 7f30e2000640 1 -- 172.21.15.42:0/1445832688 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f30dc09c710 con 0x7f30e4103340 2024-04-25T00:16:17.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:17 smithi136 ceph-mon[32923]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:17.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:17 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3882274277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:17.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:17 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/392501995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:17.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:17 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1585500730' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:16:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:17 smithi106 ceph-mon[32510]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:17 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3882274277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:17 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/392501995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:17 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1585500730' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:16:17.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:17.601+0000 7f30ebe00640 1 -- 172.21.15.42:0/1445832688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f30e41a26c0 con 0x7f30e4103340 2024-04-25T00:16:17.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:17 smithi042 ceph-mon[28130]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:17.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:17 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3882274277' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:17.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:17 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/392501995' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:16:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:17 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1585500730' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T00:16:18.242 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.242+0000 7f30e2000640 1 -- 172.21.15.42:0/1445832688 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f30dc09c010 con 0x7f30e4103340 2024-04-25T00:16:18.243 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' created 2024-04-25T00:16:18.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.247+0000 7f30ebe00640 1 -- 172.21.15.42:0/1445832688 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30cc0a2c10 msgr2=0x7f30cc0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.247+0000 7f30ebe00640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30cc0a2c10 0x7f30cc0a50d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f30e419ab70 tx=0x7f30d8005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:18.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.247+0000 7f30ebe00640 1 -- 172.21.15.42:0/1445832688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30e4103340 msgr2=0x7f30e41a1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.247+0000 7f30ebe00640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30e4103340 0x7f30e41a1240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f30dc009fd0 tx=0x7f30dc00ef10 comp rx=0 tx=0).stop 2024-04-25T00:16:18.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.248+0000 7f30ebe00640 1 -- 172.21.15.42:0/1445832688 shutdown_connections 2024-04-25T00:16:18.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.248+0000 7f30ebe00640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30cc0a2c10 0x7f30cc0a50d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.248+0000 7f30ebe00640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f30e4103340 0x7f30e41a1240 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.248+0000 7f30ebe00640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f30e4102980 0x7f30e419a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.248+0000 7f30ebe00640 1 --2- 172.21.15.42:0/1445832688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f30e4101790 0x7f30e4199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.248+0000 7f30ebe00640 1 -- 172.21.15.42:0/1445832688 >> 172.21.15.42:0/1445832688 conn(0x7f30e40fcd20 msgr2=0x7f30e4108150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.249+0000 7f30ebe00640 1 -- 172.21.15.42:0/1445832688 shutdown_connections 2024-04-25T00:16:18.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.249+0000 7f30ebe00640 1 -- 172.21.15.42:0/1445832688 wait complete. 2024-04-25T00:16:18.278 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T00:16:18.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.488+0000 7f20bc400640 1 -- 172.21.15.42:0/4085376660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b410dd60 msgr2=0x7f20b4110150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.488+0000 7f20bc400640 1 --2- 172.21.15.42:0/4085376660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b410dd60 0x7f20b4110150 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009be0 tx=0x7f20ac02a6c0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.488+0000 7f20bc400640 1 -- 172.21.15.42:0/4085376660 shutdown_connections 2024-04-25T00:16:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.488+0000 7f20bc400640 1 --2- 172.21.15.42:0/4085376660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b410dd60 0x7f20b4110150 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.488+0000 7f20bc400640 1 --2- 172.21.15.42:0/4085376660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b40741a0 0x7f20b4074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.488+0000 7f20bc400640 1 --2- 172.21.15.42:0/4085376660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b4075780 0x7f20b4073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.488+0000 7f20bc400640 1 -- 172.21.15.42:0/4085376660 >> 172.21.15.42:0/4085376660 conn(0x7f20b40fcd20 msgr2=0x7f20b40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.488+0000 7f20bc400640 1 -- 172.21.15.42:0/4085376660 shutdown_connections 2024-04-25T00:16:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.488+0000 7f20bc400640 1 -- 172.21.15.42:0/4085376660 wait complete. 2024-04-25T00:16:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.489+0000 7f20bc400640 1 Processor -- start 2024-04-25T00:16:18.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.489+0000 7f20bc400640 1 -- start start 2024-04-25T00:16:18.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20bc400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b40741a0 0x7f20b419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20bc400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b4075780 0x7f20b419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20bc400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b410dd60 0x7f20b41a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20bc400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4113990 con 0x7f20b4075780 2024-04-25T00:16:18.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20bc400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4113810 con 0x7f20b410dd60 2024-04-25T00:16:18.493 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20bc400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4113b10 con 0x7f20b40741a0 2024-04-25T00:16:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20b9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b4075780 0x7f20b419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20b9a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b40741a0 0x7f20b419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20b9000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b4075780 0x7f20b419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51182/0 (socket says 172.21.15.42:51182) 2024-04-25T00:16:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20b9a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b40741a0 0x7f20b419e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47632/0 (socket says 172.21.15.42:47632) 2024-04-25T00:16:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20ba400640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b410dd60 0x7f20b41a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20b9000640 1 -- 172.21.15.42:0/3032416855 learned_addr learned my addr 172.21.15.42:0/3032416855 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20ba400640 1 -- 172.21.15.42:0/3032416855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b40741a0 msgr2=0x7f20b419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20ba400640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b40741a0 0x7f20b419e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20ba400640 1 -- 172.21.15.42:0/3032416855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b4075780 msgr2=0x7f20b419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:18.494 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20ba400640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b4075780 0x7f20b419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:18.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.490+0000 7f20ba400640 1 -- 172.21.15.42:0/3032416855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b41a5c60 con 0x7f20b410dd60 2024-04-25T00:16:18.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.491+0000 7f20ba400640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b410dd60 0x7f20b41a55f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20ac02abe0 tx=0x7f20ac030d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.491+0000 7f20b2000640 1 -- 172.21.15.42:0/3032416855 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac041020 con 0x7f20b410dd60 2024-04-25T00:16:18.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.491+0000 7f20b2000640 1 -- 172.21.15.42:0/3032416855 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20ac037040 con 0x7f20b410dd60 2024-04-25T00:16:18.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.491+0000 7f20b2000640 1 -- 172.21.15.42:0/3032416855 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ac036070 con 0x7f20b410dd60 2024-04-25T00:16:18.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.491+0000 7f20bc400640 1 -- 172.21.15.42:0/3032416855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b41a5ef0 con 0x7f20b410dd60 2024-04-25T00:16:18.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.491+0000 7f20bc400640 1 -- 172.21.15.42:0/3032416855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b41a6300 con 0x7f20b410dd60 2024-04-25T00:16:18.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.492+0000 7f20bc400640 1 -- 172.21.15.42:0/3032416855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b4103f30 con 0x7f20b410dd60 2024-04-25T00:16:18.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.495+0000 7f20b2000640 1 -- 172.21.15.42:0/3032416855 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f20ac044020 con 0x7f20b410dd60 2024-04-25T00:16:18.496 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.496+0000 7f20b2000640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f209c0a2c10 0x7f209c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.496+0000 7f20b2000640 1 -- 172.21.15.42:0/3032416855 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7497+0+0 (secure 0 0 0) 0x7f20ac0ef750 con 0x7f20b410dd60 2024-04-25T00:16:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.496+0000 7f20b9a00640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f209c0a2c10 0x7f209c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:18.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.497+0000 7f20b9a00640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f209c0a2c10 0x7f209c0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009bd0 tx=0x7f20a8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:18.501 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.500+0000 7f20b2000640 1 -- 172.21.15.42:0/3032416855 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f20ac04b020 con 0x7f20b410dd60 2024-04-25T00:16:18.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:18 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1445832688' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:16:18.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:18 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:16:18.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:18 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:18.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:18 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1445832688' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:16:18.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:18 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:16:18.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:18 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:18.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:18 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1445832688' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:16:18.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:18 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T00:16:18.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:18 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:18.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:18.727+0000 7f20bc400640 1 -- 172.21.15.42:0/3032416855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f20b402d170 con 0x7f20b410dd60 2024-04-25T00:16:19.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.296+0000 7f20b2000640 1 -- 172.21.15.42:0/3032416855 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f20ac0bc940 con 0x7f20b410dd60 2024-04-25T00:16:19.298 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' created 2024-04-25T00:16:19.304 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.304+0000 7f20bc400640 1 -- 172.21.15.42:0/3032416855 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f209c0a2c10 msgr2=0x7f209c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.304+0000 7f20bc400640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f209c0a2c10 0x7f209c0a50d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009bd0 tx=0x7f20a8005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:19.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.304+0000 7f20bc400640 1 -- 172.21.15.42:0/3032416855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b410dd60 msgr2=0x7f20b41a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.304+0000 7f20bc400640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b410dd60 0x7f20b41a55f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f20ac02abe0 tx=0x7f20ac030d30 comp rx=0 tx=0).stop 2024-04-25T00:16:19.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.307+0000 7f20bc400640 1 -- 172.21.15.42:0/3032416855 shutdown_connections 2024-04-25T00:16:19.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.307+0000 7f20bc400640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f209c0a2c10 0x7f209c0a50d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.307+0000 7f20bc400640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b410dd60 0x7f20b41a55f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.307+0000 7f20bc400640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b4075780 0x7f20b419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.307+0000 7f20bc400640 1 --2- 172.21.15.42:0/3032416855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b40741a0 0x7f20b419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.307 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.307+0000 7f20bc400640 1 -- 172.21.15.42:0/3032416855 >> 172.21.15.42:0/3032416855 conn(0x7f20b40fcd20 msgr2=0x7f20b4101100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:19.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.307+0000 7f20bc400640 1 -- 172.21.15.42:0/3032416855 shutdown_connections 2024-04-25T00:16:19.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.308+0000 7f20bc400640 1 -- 172.21.15.42:0/3032416855 wait complete. 2024-04-25T00:16:19.337 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T00:16:19.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.515+0000 7ffb3be00640 1 -- 172.21.15.42:0/1867522380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb341039a0 msgr2=0x7ffb34103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.515+0000 7ffb3be00640 1 --2- 172.21.15.42:0/1867522380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb341039a0 0x7ffb34103dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009900 tx=0x7ffb2802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.515+0000 7ffb3be00640 1 -- 172.21.15.42:0/1867522380 shutdown_connections 2024-04-25T00:16:19.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.515+0000 7ffb3be00640 1 --2- 172.21.15.42:0/1867522380 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb34105550 0x7ffb3410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.515+0000 7ffb3be00640 1 --2- 172.21.15.42:0/1867522380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb34104b90 0x7ffb34105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.515+0000 7ffb3be00640 1 --2- 172.21.15.42:0/1867522380 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb341039a0 0x7ffb34103dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.515+0000 7ffb3be00640 1 -- 172.21.15.42:0/1867522380 >> 172.21.15.42:0/1867522380 conn(0x7ffb340feef0 msgr2=0x7ffb34101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:19.516 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.516+0000 7ffb3be00640 1 -- 172.21.15.42:0/1867522380 shutdown_connections 2024-04-25T00:16:19.517 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.516+0000 7ffb3be00640 1 -- 172.21.15.42:0/1867522380 wait complete. 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.516+0000 7ffb3be00640 1 Processor -- start 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb3be00640 1 -- start start 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb3be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb341039a0 0x7ffb34199d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb3be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb34104b90 0x7ffb3419a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb3be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb34105550 0x7ffb341a1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb3be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410f670 con 0x7ffb34105550 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb3be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410f4f0 con 0x7ffb341039a0 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb3be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3410f7f0 con 0x7ffb34104b90 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb38a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb34104b90 0x7ffb3419a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb39400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb341039a0 0x7ffb34199d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:19.518 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb38a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb34104b90 0x7ffb3419a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47638/0 (socket says 172.21.15.42:47638) 2024-04-25T00:16:19.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb38a00640 1 -- 172.21.15.42:0/1587544426 learned_addr learned my addr 172.21.15.42:0/1587544426 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:19.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb39400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb341039a0 0x7ffb34199d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38052/0 (socket says 172.21.15.42:38052) 2024-04-25T00:16:19.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.517+0000 7ffb39e00640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb34105550 0x7ffb341a1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:19.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb38a00640 1 -- 172.21.15.42:0/1587544426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb341039a0 msgr2=0x7ffb34199d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb38a00640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb341039a0 0x7ffb34199d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb38a00640 1 -- 172.21.15.42:0/1587544426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb34105550 msgr2=0x7ffb341a1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:19.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb38a00640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb34105550 0x7ffb341a1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:19.519 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb38a00640 1 -- 172.21.15.42:0/1587544426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb341a1910 con 0x7ffb34104b90 2024-04-25T00:16:19.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb38a00640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb34104b90 0x7ffb3419a240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000ea60 tx=0x7ffb2000ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:19.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb32000640 1 -- 172.21.15.42:0/1587544426 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2000cde0 con 0x7ffb34104b90 2024-04-25T00:16:19.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb3be00640 1 -- 172.21.15.42:0/1587544426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb341a1c00 con 0x7ffb34104b90 2024-04-25T00:16:19.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb32000640 1 -- 172.21.15.42:0/1587544426 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb200035f0 con 0x7ffb34104b90 2024-04-25T00:16:19.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb32000640 1 -- 172.21.15.42:0/1587544426 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb20010670 con 0x7ffb34104b90 2024-04-25T00:16:19.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.518+0000 7ffb3be00640 1 -- 172.21.15.42:0/1587544426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb341a2170 con 0x7ffb34104b90 2024-04-25T00:16:19.520 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.520+0000 7ffb3be00640 1 -- 172.21.15.42:0/1587544426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3410acc0 con 0x7ffb34104b90 2024-04-25T00:16:19.522 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.522+0000 7ffb32000640 1 -- 172.21.15.42:0/1587544426 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ffb200037a0 con 0x7ffb34104b90 2024-04-25T00:16:19.523 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.524+0000 7ffb32000640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb1c0a2b90 0x7ffb1c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:19.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.524+0000 7ffb32000640 1 -- 172.21.15.42:0/1587544426 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7879+0+0 (secure 0 0 0) 0x7ffb200cf270 con 0x7ffb34104b90 2024-04-25T00:16:19.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.524+0000 7ffb39400640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb1c0a2b90 0x7ffb1c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:19.524 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.524+0000 7ffb39400640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb1c0a2b90 0x7ffb1c0a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffb2802a9e0 tx=0x7ffb280315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:19.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.528+0000 7ffb32000640 1 -- 172.21.15.42:0/1587544426 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb2009c2e0 con 0x7ffb34104b90 2024-04-25T00:16:19.567 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:19 smithi106 ceph-mon[32510]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:19.568 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:19.568 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:19.568 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:19.568 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:19 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:16:19.568 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:19 smithi106 ceph-mon[32510]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:16:19.568 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:19.568 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:19.568 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:19 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3032416855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:16:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:19 smithi042 ceph-mon[28130]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:19 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:16:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:19 smithi042 ceph-mon[28130]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:16:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:19.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:19 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3032416855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:16:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:19 smithi136 ceph-mon[32923]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:19 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T00:16:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:19 smithi136 ceph-mon[32923]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T00:16:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:16:19.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:19 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3032416855' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:16:19.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:19.769+0000 7ffb3be00640 1 -- 172.21.15.42:0/1587544426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ffb340623e0 con 0x7ffb34104b90 2024-04-25T00:16:20.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.317+0000 7ffb32000640 1 -- 172.21.15.42:0/1587544426 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7ffb2009bbe0 con 0x7ffb34104b90 2024-04-25T00:16:20.318 INFO:teuthology.orchestra.run.smithi042.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T00:16:20.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.321+0000 7ffb3be00640 1 -- 172.21.15.42:0/1587544426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb1c0a2b90 msgr2=0x7ffb1c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:20.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.321+0000 7ffb3be00640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb1c0a2b90 0x7ffb1c0a5050 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffb2802a9e0 tx=0x7ffb280315b0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.321+0000 7ffb3be00640 1 -- 172.21.15.42:0/1587544426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb34104b90 msgr2=0x7ffb3419a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:20.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.321+0000 7ffb3be00640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb34104b90 0x7ffb3419a240 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000ea60 tx=0x7ffb2000ef20 comp rx=0 tx=0).stop 2024-04-25T00:16:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.321+0000 7ffb3be00640 1 -- 172.21.15.42:0/1587544426 shutdown_connections 2024-04-25T00:16:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.321+0000 7ffb3be00640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb1c0a2b90 0x7ffb1c0a5050 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.321+0000 7ffb3be00640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb34105550 0x7ffb341a1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.321+0000 7ffb3be00640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb34104b90 0x7ffb3419a240 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.321+0000 7ffb3be00640 1 --2- 172.21.15.42:0/1587544426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb341039a0 0x7ffb34199d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.321+0000 7ffb3be00640 1 -- 172.21.15.42:0/1587544426 >> 172.21.15.42:0/1587544426 conn(0x7ffb340feef0 msgr2=0x7ffb34107e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.322+0000 7ffb3be00640 1 -- 172.21.15.42:0/1587544426 shutdown_connections 2024-04-25T00:16:20.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.322+0000 7ffb3be00640 1 -- 172.21.15.42:0/1587544426 wait complete. 2024-04-25T00:16:20.350 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.529+0000 7f4933e00640 1 -- 172.21.15.42:0/2566577248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c102300 msgr2=0x7f492c1046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.529+0000 7f4933e00640 1 --2- 172.21.15.42:0/2566577248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c102300 0x7f492c1046f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4920009900 tx=0x7f492002a410 comp rx=0 tx=0).stop 2024-04-25T00:16:20.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.529+0000 7f4933e00640 1 -- 172.21.15.42:0/2566577248 shutdown_connections 2024-04-25T00:16:20.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.529+0000 7f4933e00640 1 --2- 172.21.15.42:0/2566577248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f492c107560 0x7f492c109950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.529+0000 7f4933e00640 1 --2- 172.21.15.42:0/2566577248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f492c104c30 0x7f492c107020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.529+0000 7f4933e00640 1 --2- 172.21.15.42:0/2566577248 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c102300 0x7f492c1046f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.529+0000 7f4933e00640 1 -- 172.21.15.42:0/2566577248 >> 172.21.15.42:0/2566577248 conn(0x7f492c0fbc30 msgr2=0x7f492c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:20.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.529+0000 7f4933e00640 1 -- 172.21.15.42:0/2566577248 shutdown_connections 2024-04-25T00:16:20.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.529+0000 7f4933e00640 1 -- 172.21.15.42:0/2566577248 wait complete. 2024-04-25T00:16:20.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.530+0000 7f4933e00640 1 Processor -- start 2024-04-25T00:16:20.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.530+0000 7f4933e00640 1 -- start start 2024-04-25T00:16:20.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.530+0000 7f4933e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f492c102300 0x7f492c199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:20.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.530+0000 7f4933e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f492c104c30 0x7f492c19a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:20.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.530+0000 7f4933e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c107560 0x7f492c1a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:20.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.530+0000 7f4933e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10d440 con 0x7f492c107560 2024-04-25T00:16:20.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.530+0000 7f4933e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10d2c0 con 0x7f492c104c30 2024-04-25T00:16:20.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.530+0000 7f4933e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f492c10d5c0 con 0x7f492c102300 2024-04-25T00:16:20.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4931e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c107560 0x7f492c1a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:20.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4931400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f492c102300 0x7f492c199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4930a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f492c104c30 0x7f492c19a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4930a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f492c104c30 0x7f492c19a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38066/0 (socket says 172.21.15.42:38066) 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4930a00640 1 -- 172.21.15.42:0/65239181 learned_addr learned my addr 172.21.15.42:0/65239181 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4930a00640 1 -- 172.21.15.42:0/65239181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f492c102300 msgr2=0x7f492c199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4930a00640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f492c102300 0x7f492c199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4930a00640 1 -- 172.21.15.42:0/65239181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c107560 msgr2=0x7f492c1a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4930a00640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c107560 0x7f492c1a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4930a00640 1 -- 172.21.15.42:0/65239181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f492c1a1940 con 0x7f492c104c30 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f4930a00640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f492c104c30 0x7f492c19a270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f491800e9c0 tx=0x7f491800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.531+0000 7f492a000640 1 -- 172.21.15.42:0/65239181 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491800cd80 con 0x7f492c104c30 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.532+0000 7f4933e00640 1 -- 172.21.15.42:0/65239181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f492c1a1c30 con 0x7f492c104c30 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.532+0000 7f492a000640 1 -- 172.21.15.42:0/65239181 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4918005500 con 0x7f492c104c30 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.532+0000 7f492a000640 1 -- 172.21.15.42:0/65239181 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918010690 con 0x7f492c104c30 2024-04-25T00:16:20.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.532+0000 7f4933e00640 1 -- 172.21.15.42:0/65239181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f492c1a2180 con 0x7f492c104c30 2024-04-25T00:16:20.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.533+0000 7f4933e00640 1 -- 172.21.15.42:0/65239181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492c109050 con 0x7f492c104c30 2024-04-25T00:16:20.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.535+0000 7f492a000640 1 -- 172.21.15.42:0/65239181 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f49180046c0 con 0x7f492c104c30 2024-04-25T00:16:20.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.538+0000 7f492a000640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49140a2b90 0x7f49140a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:20.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.538+0000 7f492a000640 1 -- 172.21.15.42:0/65239181 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f49180cf3c0 con 0x7f492c104c30 2024-04-25T00:16:20.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.538+0000 7f4931400640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49140a2b90 0x7f49140a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:20.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.538+0000 7f4931400640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49140a2b90 0x7f49140a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4920009230 tx=0x7f49200315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:20.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.540+0000 7f492a000640 1 -- 172.21.15.42:0/65239181 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f491809c3d0 con 0x7f492c104c30 2024-04-25T00:16:20.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:16:20.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:16:20.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:16:20.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1587544426' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:16:20.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:16:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:16:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:16:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:16:20.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:16:20.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:16:20.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1587544426' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:16:20.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:16:20.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:16:20.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:16:20.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.758+0000 7f4933e00640 1 -- 172.21.15.42:0/65239181 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f492c0623e0 con 0x7f492c104c30 2024-04-25T00:16:20.760 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.760+0000 7f492a000640 1 -- 172.21.15.42:0/65239181 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22230 (secure 0 0 0) 0x7f491809bcd0 con 0x7f492c104c30 2024-04-25T00:16:20.761 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:20.761 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":85,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:16:19.771094+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:20.763 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 -- 172.21.15.42:0/65239181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49140a2b90 msgr2=0x7f49140a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49140a2b90 0x7f49140a5050 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4920009230 tx=0x7f49200315b0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 -- 172.21.15.42:0/65239181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f492c104c30 msgr2=0x7f492c19a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f492c104c30 0x7f492c19a270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f491800e9c0 tx=0x7f491800ee80 comp rx=0 tx=0).stop 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 -- 172.21.15.42:0/65239181 shutdown_connections 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49140a2b90 0x7f49140a5050 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f492c107560 0x7f492c1a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f492c104c30 0x7f492c19a270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 --2- 172.21.15.42:0/65239181 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f492c102300 0x7f492c199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 -- 172.21.15.42:0/65239181 >> 172.21.15.42:0/65239181 conn(0x7f492c0fbc30 msgr2=0x7f492c101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 -- 172.21.15.42:0/65239181 shutdown_connections 2024-04-25T00:16:20.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.763+0000 7f4933e00640 1 -- 172.21.15.42:0/65239181 wait complete. 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1587544426' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:20.781 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:20.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.966+0000 7f118e400640 1 -- 172.21.15.42:0/3031999660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11881027a0 msgr2=0x7f1188102c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:20.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.966+0000 7f118e400640 1 --2- 172.21.15.42:0/3031999660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11881027a0 0x7f1188102c20 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1174009860 tx=0x7f117402a340 comp rx=0 tx=0).stop 2024-04-25T00:16:20.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.966+0000 7f118e400640 1 -- 172.21.15.42:0/3031999660 shutdown_connections 2024-04-25T00:16:20.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.966+0000 7f118e400640 1 --2- 172.21.15.42:0/3031999660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1188103160 0x7f11881099f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.966+0000 7f118e400640 1 --2- 172.21.15.42:0/3031999660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11881027a0 0x7f1188102c20 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.966+0000 7f118e400640 1 --2- 172.21.15.42:0/3031999660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11881015b0 0x7f11881019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.966+0000 7f118e400640 1 -- 172.21.15.42:0/3031999660 >> 172.21.15.42:0/3031999660 conn(0x7f11880fcb20 msgr2=0x7f11880fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:20.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.966+0000 7f118e400640 1 -- 172.21.15.42:0/3031999660 shutdown_connections 2024-04-25T00:16:20.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.966+0000 7f118e400640 1 -- 172.21.15.42:0/3031999660 wait complete. 2024-04-25T00:16:20.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.967+0000 7f118e400640 1 Processor -- start 2024-04-25T00:16:20.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.967+0000 7f118e400640 1 -- start start 2024-04-25T00:16:20.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f118e400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11881015b0 0x7f1188199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:20.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f118e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11881027a0 0x7f118819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:20.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f118e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1188103160 0x7f11881a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:20.970 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f118e400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f118810d230 con 0x7f11881027a0 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f118e400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f118810d0b0 con 0x7f1188103160 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f118e400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f118810d3b0 con 0x7f11881015b0 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f1187e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11881027a0 0x7f118819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f1187e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11881027a0 0x7f118819a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51230/0 (socket says 172.21.15.42:51230) 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f1187e00640 1 -- 172.21.15.42:0/3603204094 learned_addr learned my addr 172.21.15.42:0/3603204094 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f118d000640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11881015b0 0x7f1188199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f118da00640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1188103160 0x7f11881a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f1187e00640 1 -- 172.21.15.42:0/3603204094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11881015b0 msgr2=0x7f1188199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f1187e00640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11881015b0 0x7f1188199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f1187e00640 1 -- 172.21.15.42:0/3603204094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1188103160 msgr2=0x7f11881a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f1187e00640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1188103160 0x7f11881a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.968+0000 7f1187e00640 1 -- 172.21.15.42:0/3603204094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11881a1760 con 0x7f11881027a0 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.969+0000 7f1187e00640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11881027a0 0x7f118819a080 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1174005940 tx=0x7f1174033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.969+0000 7f1185600640 1 -- 172.21.15.42:0/3603204094 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117403f020 con 0x7f11881027a0 2024-04-25T00:16:20.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.969+0000 7f1185600640 1 -- 172.21.15.42:0/3603204094 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1174030850 con 0x7f11881027a0 2024-04-25T00:16:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.969+0000 7f1185600640 1 -- 172.21.15.42:0/3603204094 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1174037040 con 0x7f11881027a0 2024-04-25T00:16:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.969+0000 7f118e400640 1 -- 172.21.15.42:0/3603204094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11881a19f0 con 0x7f11881027a0 2024-04-25T00:16:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.969+0000 7f118e400640 1 -- 172.21.15.42:0/3603204094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11881a1ee0 con 0x7f11881027a0 2024-04-25T00:16:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.971+0000 7f118e400640 1 -- 172.21.15.42:0/3603204094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11881a20b0 con 0x7f11881027a0 2024-04-25T00:16:20.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.972+0000 7f1185600640 1 -- 172.21.15.42:0/3603204094 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f1174042020 con 0x7f11881027a0 2024-04-25T00:16:20.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.974+0000 7f1185600640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f11700a2b90 0x7f11700a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:20.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.974+0000 7f1185600640 1 -- 172.21.15.42:0/3603204094 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7979+0+0 (secure 0 0 0) 0x7f11740ef2b0 con 0x7f11881027a0 2024-04-25T00:16:20.974 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.974+0000 7f118d000640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f11700a2b90 0x7f11700a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:20.975 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.975+0000 7f118d000640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f11700a2b90 0x7f11700a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1188102600 tx=0x7f117c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:20.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:20.979+0000 7f1185600640 1 -- 172.21.15.42:0/3603204094 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1174049020 con 0x7f11881027a0 2024-04-25T00:16:21.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.201+0000 7f118e400640 1 -- 172.21.15.42:0/3603204094 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f11881a2810 con 0x7f11881027a0 2024-04-25T00:16:21.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.210+0000 7f1185600640 1 -- 172.21.15.42:0/3603204094 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+22230 (secure 0 0 0) 0x7f11740bc2c0 con 0x7f11881027a0 2024-04-25T00:16:21.210 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:21.210 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":85,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:16:19.771094+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:21.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 -- 172.21.15.42:0/3603204094 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f11700a2b90 msgr2=0x7f11700a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:21.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f11700a2b90 0x7f11700a5050 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1188102600 tx=0x7f117c005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 -- 172.21.15.42:0/3603204094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11881027a0 msgr2=0x7f118819a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11881027a0 0x7f118819a080 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1174005940 tx=0x7f1174033d90 comp rx=0 tx=0).stop 2024-04-25T00:16:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 -- 172.21.15.42:0/3603204094 shutdown_connections 2024-04-25T00:16:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f11700a2b90 0x7f11700a5050 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1188103160 0x7f11881a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f11881027a0 0x7f118819a080 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 --2- 172.21.15.42:0/3603204094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11881015b0 0x7f1188199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 -- 172.21.15.42:0/3603204094 >> 172.21.15.42:0/3603204094 conn(0x7f11880fcb20 msgr2=0x7f1188073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 -- 172.21.15.42:0/3603204094 shutdown_connections 2024-04-25T00:16:21.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.213+0000 7f118e400640 1 -- 172.21.15.42:0/3603204094 wait complete. 2024-04-25T00:16:21.237 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T00:16:21.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.426+0000 7f84dde00640 1 -- 172.21.15.42:0/778548682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d81020d0 msgr2=0x7f84d81044c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:21.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.426+0000 7f84dde00640 1 --2- 172.21.15.42:0/778548682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d81020d0 0x7f84d81044c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009900 tx=0x7f84cc02a410 comp rx=0 tx=0).stop 2024-04-25T00:16:21.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.426+0000 7f84dde00640 1 -- 172.21.15.42:0/778548682 shutdown_connections 2024-04-25T00:16:21.426 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.426+0000 7f84dde00640 1 --2- 172.21.15.42:0/778548682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84d8107330 0x7f84d8109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:21.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.426+0000 7f84dde00640 1 --2- 172.21.15.42:0/778548682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d8104a00 0x7f84d8106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:21.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.426+0000 7f84dde00640 1 --2- 172.21.15.42:0/778548682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d81020d0 0x7f84d81044c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:21.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.426+0000 7f84dde00640 1 -- 172.21.15.42:0/778548682 >> 172.21.15.42:0/778548682 conn(0x7f84d80fbae0 msgr2=0x7f84d80fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:21.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.426+0000 7f84dde00640 1 -- 172.21.15.42:0/778548682 shutdown_connections 2024-04-25T00:16:21.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.426+0000 7f84dde00640 1 -- 172.21.15.42:0/778548682 wait complete. 2024-04-25T00:16:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.427+0000 7f84dde00640 1 Processor -- start 2024-04-25T00:16:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.427+0000 7f84dde00640 1 -- start start 2024-04-25T00:16:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84dde00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d81020d0 0x7f84d8199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d8104a00 0x7f84d819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84dde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84d8107330 0x7f84d81a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:21.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84dde00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f84d810d340 con 0x7f84d8104a00 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84dde00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f84d810d1c0 con 0x7f84d81020d0 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84dde00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f84d810d4c0 con 0x7f84d8107330 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84d7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d8104a00 0x7f84d819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84d7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d8104a00 0x7f84d819a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51258/0 (socket says 172.21.15.42:51258) 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84d7e00640 1 -- 172.21.15.42:0/4142714699 learned_addr learned my addr 172.21.15.42:0/4142714699 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84dca00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d81020d0 0x7f84d8199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84dd400640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84d8107330 0x7f84d81a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84d7e00640 1 -- 172.21.15.42:0/4142714699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84d8107330 msgr2=0x7f84d81a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84d7e00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84d8107330 0x7f84d81a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84d7e00640 1 -- 172.21.15.42:0/4142714699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d81020d0 msgr2=0x7f84d8199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84d7e00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d81020d0 0x7f84d8199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:21.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.428+0000 7f84d7e00640 1 -- 172.21.15.42:0/4142714699 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d81a1790 con 0x7f84d8104a00 2024-04-25T00:16:21.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.429+0000 7f84d7e00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d8104a00 0x7f84d819a0b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e9a0 tx=0x7f84c400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:21.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.429+0000 7f84d5600640 1 -- 172.21.15.42:0/4142714699 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c400cd80 con 0x7f84d8104a00 2024-04-25T00:16:21.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.429+0000 7f84dde00640 1 -- 172.21.15.42:0/4142714699 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d81a1a80 con 0x7f84d8104a00 2024-04-25T00:16:21.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.429+0000 7f84d5600640 1 -- 172.21.15.42:0/4142714699 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84c40035f0 con 0x7f84d8104a00 2024-04-25T00:16:21.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.429+0000 7f84d5600640 1 -- 172.21.15.42:0/4142714699 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4010670 con 0x7f84d8104a00 2024-04-25T00:16:21.430 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.429+0000 7f84dde00640 1 -- 172.21.15.42:0/4142714699 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d81a1fd0 con 0x7f84d8104a00 2024-04-25T00:16:21.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.431+0000 7f84dde00640 1 -- 172.21.15.42:0/4142714699 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d8108e80 con 0x7f84d8104a00 2024-04-25T00:16:21.432 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.432+0000 7f84d5600640 1 -- 172.21.15.42:0/4142714699 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f84c40037a0 con 0x7f84d8104a00 2024-04-25T00:16:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.434+0000 7f84d5600640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f84c00a2b90 0x7f84c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.435+0000 7f84d5600640 1 -- 172.21.15.42:0/4142714699 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f84c40cf1d0 con 0x7f84d8104a00 2024-04-25T00:16:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.435+0000 7f84dca00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f84c00a2b90 0x7f84c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:21.435 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.435+0000 7f84dca00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f84c00a2b90 0x7f84c00a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f84cc031440 tx=0x7f84cc0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:21.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.438+0000 7f84d5600640 1 -- 172.21.15.42:0/4142714699 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f84c409c1e0 con 0x7f84d8104a00 2024-04-25T00:16:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: pgmap v311: 97 pgs: 13 creating+peering, 4 active+clean, 80 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:16:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:16:21.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:16:21.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:16:21.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/65239181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:21.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:21.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3603204094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:21.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:21.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:21.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:21.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:16:21.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: pgmap v311: 97 pgs: 13 creating+peering, 4 active+clean, 80 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:16:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:16:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:16:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:16:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/65239181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:21.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3603204094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:21.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:21.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:21.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:21.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:16:21.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:21.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:21.685+0000 7f84dde00640 1 -- 172.21.15.42:0/4142714699 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f84d8062340 con 0x7f84d8104a00 2024-04-25T00:16:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: pgmap v311: 97 pgs: 13 creating+peering, 4 active+clean, 80 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T00:16:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T00:16:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T00:16:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T00:16:21.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T00:16:21.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/65239181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:21.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:21.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3603204094' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:21.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:21.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:16:21.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:16:21.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T00:16:21.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:16:22.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.338+0000 7f84d5600640 1 -- 172.21.15.42:0/4142714699 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f84c409bae0 con 0x7f84d8104a00 2024-04-25T00:16:22.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 -- 172.21.15.42:0/4142714699 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f84c00a2b90 msgr2=0x7f84c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:22.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f84c00a2b90 0x7f84c00a5050 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f84cc031440 tx=0x7f84cc0315b0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 -- 172.21.15.42:0/4142714699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d8104a00 msgr2=0x7f84d819a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:22.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d8104a00 0x7f84d819a0b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e9a0 tx=0x7f84c400ee60 comp rx=0 tx=0).stop 2024-04-25T00:16:22.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 -- 172.21.15.42:0/4142714699 shutdown_connections 2024-04-25T00:16:22.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f84c00a2b90 0x7f84c00a5050 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f84d8107330 0x7f84d81a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f84d8104a00 0x7f84d819a0b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 --2- 172.21.15.42:0/4142714699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f84d81020d0 0x7f84d8199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 -- 172.21.15.42:0/4142714699 >> 172.21.15.42:0/4142714699 conn(0x7f84d80fbae0 msgr2=0x7f84d8101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:22.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 -- 172.21.15.42:0/4142714699 shutdown_connections 2024-04-25T00:16:22.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.342+0000 7f84dde00640 1 -- 172.21.15.42:0/4142714699 wait complete. 2024-04-25T00:16:22.365 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T00:16:22.365 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2024-04-25T00:16:22.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:22 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] up:active 2024-04-25T00:16:22.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:22 smithi136 ceph-mon[32923]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:22.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:22 smithi136 ceph-mon[32923]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:22.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:22 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/4142714699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:16:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.583+0000 7ff827000640 1 -- 172.21.15.42:0/926869052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff820076100 msgr2=0x7ff820076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.583+0000 7ff827000640 1 --2- 172.21.15.42:0/926869052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff820076100 0x7ff820076520 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff810009ca0 tx=0x7ff81002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.583+0000 7ff827000640 1 -- 172.21.15.42:0/926869052 shutdown_connections 2024-04-25T00:16:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.583+0000 7ff827000640 1 --2- 172.21.15.42:0/926869052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff820077ce0 0x7ff820078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.583+0000 7ff827000640 1 --2- 172.21.15.42:0/926869052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff820077340 0x7ff8200777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.583+0000 7ff827000640 1 --2- 172.21.15.42:0/926869052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff820076100 0x7ff820076520 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.583+0000 7ff827000640 1 -- 172.21.15.42:0/926869052 >> 172.21.15.42:0/926869052 conn(0x7ff8200fef30 msgr2=0x7ff820101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.583+0000 7ff827000640 1 -- 172.21.15.42:0/926869052 shutdown_connections 2024-04-25T00:16:22.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.584+0000 7ff827000640 1 -- 172.21.15.42:0/926869052 wait complete. 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.584+0000 7ff827000640 1 Processor -- start 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff827000640 1 -- start start 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff827000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff820076100 0x7ff82019e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff827000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff820077340 0x7ff82019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff827000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff820077ce0 0x7ff8201a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff827000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff820113a90 con 0x7ff820077ce0 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff827000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff820113910 con 0x7ff820076100 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff827000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff820113c10 con 0x7ff820077340 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff825000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff820077ce0 0x7ff8201a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff81fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff820076100 0x7ff82019e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff825000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff820077ce0 0x7ff8201a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51276/0 (socket says 172.21.15.42:51276) 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff825000640 1 -- 172.21.15.42:0/2486473319 learned_addr learned my addr 172.21.15.42:0/2486473319 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:22.587 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.585+0000 7ff817400640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff820077340 0x7ff82019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.586+0000 7ff825000640 1 -- 172.21.15.42:0/2486473319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff820077340 msgr2=0x7ff82019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.586+0000 7ff825000640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff820077340 0x7ff82019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.586+0000 7ff825000640 1 -- 172.21.15.42:0/2486473319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff820076100 msgr2=0x7ff82019e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.586+0000 7ff825000640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff820076100 0x7ff82019e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.586+0000 7ff825000640 1 -- 172.21.15.42:0/2486473319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8201a5d40 con 0x7ff820077ce0 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.586+0000 7ff81fe00640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff820076100 0x7ff82019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.586+0000 7ff817400640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff820077340 0x7ff82019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.586+0000 7ff825000640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff820077ce0 0x7ff8201a56d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff81800e9b0 tx=0x7ff81800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.586+0000 7ff81d600640 1 -- 172.21.15.42:0/2486473319 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81800cd80 con 0x7ff820077ce0 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.587+0000 7ff81d600640 1 -- 172.21.15.42:0/2486473319 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8180035f0 con 0x7ff820077ce0 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.587+0000 7ff827000640 1 -- 172.21.15.42:0/2486473319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8201a5fd0 con 0x7ff820077ce0 2024-04-25T00:16:22.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.587+0000 7ff81d600640 1 -- 172.21.15.42:0/2486473319 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818010670 con 0x7ff820077ce0 2024-04-25T00:16:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.587+0000 7ff827000640 1 -- 172.21.15.42:0/2486473319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8201a64a0 con 0x7ff820077ce0 2024-04-25T00:16:22.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:22 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] up:active 2024-04-25T00:16:22.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:22 smithi106 ceph-mon[32510]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:22.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:22 smithi106 ceph-mon[32510]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:22.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:22 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/4142714699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:16:22.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.588+0000 7ff827000640 1 -- 172.21.15.42:0/2486473319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff820110330 con 0x7ff820077ce0 2024-04-25T00:16:22.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.590+0000 7ff81d600640 1 -- 172.21.15.42:0/2486473319 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff8180037a0 con 0x7ff820077ce0 2024-04-25T00:16:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.591+0000 7ff81d600640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8040a2b90 0x7ff8040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.592+0000 7ff81d600640 1 -- 172.21.15.42:0/2486473319 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ff8180cf330 con 0x7ff820077ce0 2024-04-25T00:16:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.592+0000 7ff81fe00640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8040a2b90 0x7ff8040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:22.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.592+0000 7ff81fe00640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8040a2b90 0x7ff8040a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff810005e30 tx=0x7ff810004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:22.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.596+0000 7ff81d600640 1 -- 172.21.15.42:0/2486473319 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff81809c340 con 0x7ff820077ce0 2024-04-25T00:16:22.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:22 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] up:active 2024-04-25T00:16:22.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:22 smithi042 ceph-mon[28130]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:22.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:22 smithi042 ceph-mon[28130]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:22.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:22 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/4142714699' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T00:16:22.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:22.846+0000 7ff827000640 1 -- 172.21.15.42:0/2486473319 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7ff820062340 con 0x7ff820077ce0 2024-04-25T00:16:23.350 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.350+0000 7ff81d600640 1 -- 172.21.15.42:0/2486473319 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7ff81809bc40 con 0x7ff820077ce0 2024-04-25T00:16:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.354+0000 7ff827000640 1 -- 172.21.15.42:0/2486473319 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8040a2b90 msgr2=0x7ff8040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.354+0000 7ff827000640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8040a2b90 0x7ff8040a5050 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff810005e30 tx=0x7ff810004a40 comp rx=0 tx=0).stop 2024-04-25T00:16:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.354+0000 7ff827000640 1 -- 172.21.15.42:0/2486473319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff820077ce0 msgr2=0x7ff8201a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:23.354 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.354+0000 7ff827000640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff820077ce0 0x7ff8201a56d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff81800e9b0 tx=0x7ff81800ee70 comp rx=0 tx=0).stop 2024-04-25T00:16:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.354+0000 7ff827000640 1 -- 172.21.15.42:0/2486473319 shutdown_connections 2024-04-25T00:16:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.354+0000 7ff827000640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff8040a2b90 0x7ff8040a5050 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.354+0000 7ff827000640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff820077ce0 0x7ff8201a56d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.354+0000 7ff827000640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff820077340 0x7ff82019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.354+0000 7ff827000640 1 --2- 172.21.15.42:0/2486473319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff820076100 0x7ff82019e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.354+0000 7ff827000640 1 -- 172.21.15.42:0/2486473319 >> 172.21.15.42:0/2486473319 conn(0x7ff8200fef30 msgr2=0x7ff820100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:23.355 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.355+0000 7ff827000640 1 -- 172.21.15.42:0/2486473319 shutdown_connections 2024-04-25T00:16:23.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.355+0000 7ff827000640 1 -- 172.21.15.42:0/2486473319 wait complete. 2024-04-25T00:16:23.382 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T00:16:23.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.577+0000 7fe605e00640 1 -- 172.21.15.42:0/2334202781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe600076100 msgr2=0x7fe600076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:23.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.577+0000 7fe605e00640 1 --2- 172.21.15.42:0/2334202781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe600076100 0x7fe600076520 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009860 tx=0x7fe5f402a340 comp rx=0 tx=0).stop 2024-04-25T00:16:23.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.578+0000 7fe605e00640 1 -- 172.21.15.42:0/2334202781 shutdown_connections 2024-04-25T00:16:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.578+0000 7fe605e00640 1 --2- 172.21.15.42:0/2334202781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe600077ce0 0x7fe600078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.578+0000 7fe605e00640 1 --2- 172.21.15.42:0/2334202781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe600077340 0x7fe6000777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.578+0000 7fe605e00640 1 --2- 172.21.15.42:0/2334202781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe600076100 0x7fe600076520 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.578+0000 7fe605e00640 1 -- 172.21.15.42:0/2334202781 >> 172.21.15.42:0/2334202781 conn(0x7fe6000fedf0 msgr2=0x7fe600101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.578+0000 7fe605e00640 1 -- 172.21.15.42:0/2334202781 shutdown_connections 2024-04-25T00:16:23.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.578+0000 7fe605e00640 1 -- 172.21.15.42:0/2334202781 wait complete. 2024-04-25T00:16:23.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.579+0000 7fe605e00640 1 Processor -- start 2024-04-25T00:16:23.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.579+0000 7fe605e00640 1 -- start start 2024-04-25T00:16:23.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe605e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe600076100 0x7fe60019df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:23.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe605e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe600077340 0x7fe60019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe605e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe600077ce0 0x7fe6001a54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe605e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe600113910 con 0x7fe600077ce0 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe605e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe600113790 con 0x7fe600077340 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe605e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe600113a90 con 0x7fe600076100 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe605400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe600077ce0 0x7fe6001a54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe5ffe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe600077340 0x7fe60019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe605400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe600077ce0 0x7fe6001a54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51292/0 (socket says 172.21.15.42:51292) 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe605400640 1 -- 172.21.15.42:0/3972832645 learned_addr learned my addr 172.21.15.42:0/3972832645 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe604a00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe600076100 0x7fe60019df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe5ffe00640 1 -- 172.21.15.42:0/3972832645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe600076100 msgr2=0x7fe60019df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe5ffe00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe600076100 0x7fe60019df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe5ffe00640 1 -- 172.21.15.42:0/3972832645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe600077ce0 msgr2=0x7fe6001a54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:23.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe5ffe00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe600077ce0 0x7fe6001a54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:23.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.580+0000 7fe5ffe00640 1 -- 172.21.15.42:0/3972832645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6001a5b50 con 0x7fe600077340 2024-04-25T00:16:23.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.581+0000 7fe5ffe00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe600077340 0x7fe60019e470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0005fd0 tx=0x7fe5f000ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:23.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.581+0000 7fe5fd600640 1 -- 172.21.15.42:0/3972832645 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f000cd00 con 0x7fe600077340 2024-04-25T00:16:23.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.581+0000 7fe5fd600640 1 -- 172.21.15.42:0/3972832645 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5f00035f0 con 0x7fe600077340 2024-04-25T00:16:23.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.581+0000 7fe5fd600640 1 -- 172.21.15.42:0/3972832645 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0010670 con 0x7fe600077340 2024-04-25T00:16:23.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.581+0000 7fe605e00640 1 -- 172.21.15.42:0/3972832645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6001a5e40 con 0x7fe600077340 2024-04-25T00:16:23.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.581+0000 7fe605e00640 1 -- 172.21.15.42:0/3972832645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6001a62b0 con 0x7fe600077340 2024-04-25T00:16:23.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.583+0000 7fe605e00640 1 -- 172.21.15.42:0/3972832645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6001101f0 con 0x7fe600077340 2024-04-25T00:16:23.586 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.586+0000 7fe5fd600640 1 -- 172.21.15.42:0/3972832645 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe5f00037a0 con 0x7fe600077340 2024-04-25T00:16:23.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.588+0000 7fe5fd600640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5e80a2b90 0x7fe5e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:23.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.588+0000 7fe5fd600640 1 -- 172.21.15.42:0/3972832645 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe5f00cf160 con 0x7fe600077340 2024-04-25T00:16:23.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.588+0000 7fe604a00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5e80a2b90 0x7fe5e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:23.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:23 smithi106 ceph-mon[32510]: pgmap v314: 97 pgs: 23 creating+peering, 65 active+clean, 9 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-04-25T00:16:23.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:23 smithi106 ceph-mon[32510]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T00:16:23.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:23 smithi106 ceph-mon[32510]: Cluster is now healthy 2024-04-25T00:16:23.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:23 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/4142714699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:16:23.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:23 smithi106 ceph-mon[32510]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:23.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:23 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2486473319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T00:16:23.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.589+0000 7fe604a00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5e80a2b90 0x7fe5e80a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009830 tx=0x7fe5f4030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:23.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.592+0000 7fe5fd600640 1 -- 172.21.15.42:0/3972832645 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe5f0009510 con 0x7fe600077340 2024-04-25T00:16:23.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:23 smithi042 ceph-mon[28130]: pgmap v314: 97 pgs: 23 creating+peering, 65 active+clean, 9 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-04-25T00:16:23.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:23 smithi042 ceph-mon[28130]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T00:16:23.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:23 smithi042 ceph-mon[28130]: Cluster is now healthy 2024-04-25T00:16:23.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:23 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/4142714699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:16:23.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:23 smithi042 ceph-mon[28130]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:23.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:23 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2486473319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T00:16:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:23 smithi136 ceph-mon[32923]: pgmap v314: 97 pgs: 23 creating+peering, 65 active+clean, 9 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-04-25T00:16:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:23 smithi136 ceph-mon[32923]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T00:16:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:23 smithi136 ceph-mon[32923]: Cluster is now healthy 2024-04-25T00:16:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:23 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/4142714699' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T00:16:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:23 smithi136 ceph-mon[32923]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:23 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2486473319' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T00:16:23.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:23.839+0000 7fe605e00640 1 -- 172.21.15.42:0/3972832645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fe600062340 con 0x7fe600077340 2024-04-25T00:16:24.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.305+0000 7fe5fd600640 1 -- 172.21.15.42:0/3972832645 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) ==== 121+0+0 (secure 0 0 0) 0x7fe5f009c170 con 0x7fe600077340 2024-04-25T00:16:24.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.309+0000 7fe605e00640 1 -- 172.21.15.42:0/3972832645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5e80a2b90 msgr2=0x7fe5e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.309+0000 7fe605e00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5e80a2b90 0x7fe5e80a5050 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe5f4009830 tx=0x7fe5f4030550 comp rx=0 tx=0).stop 2024-04-25T00:16:24.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.309+0000 7fe605e00640 1 -- 172.21.15.42:0/3972832645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe600077340 msgr2=0x7fe60019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.309+0000 7fe605e00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe600077340 0x7fe60019e470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0005fd0 tx=0x7fe5f000ee50 comp rx=0 tx=0).stop 2024-04-25T00:16:24.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.309+0000 7fe605e00640 1 -- 172.21.15.42:0/3972832645 shutdown_connections 2024-04-25T00:16:24.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.309+0000 7fe605e00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5e80a2b90 0x7fe5e80a5050 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.309+0000 7fe605e00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe600077ce0 0x7fe6001a54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.309+0000 7fe605e00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe600077340 0x7fe60019e470 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.309+0000 7fe605e00640 1 --2- 172.21.15.42:0/3972832645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe600076100 0x7fe60019df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.309+0000 7fe605e00640 1 -- 172.21.15.42:0/3972832645 >> 172.21.15.42:0/3972832645 conn(0x7fe6000fedf0 msgr2=0x7fe6001008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:24.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.310+0000 7fe605e00640 1 -- 172.21.15.42:0/3972832645 shutdown_connections 2024-04-25T00:16:24.310 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.310+0000 7fe605e00640 1 -- 172.21.15.42:0/3972832645 wait complete. 2024-04-25T00:16:24.338 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-25T00:16:24.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.528+0000 7f2c13200640 1 -- 172.21.15.42:0/3004077633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c101670 msgr2=0x7f2c0c101a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.528+0000 7f2c13200640 1 --2- 172.21.15.42:0/3004077633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c101670 0x7f2c0c101a90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2c00009900 tx=0x7f2c0002a410 comp rx=0 tx=0).stop 2024-04-25T00:16:24.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.528+0000 7f2c13200640 1 -- 172.21.15.42:0/3004077633 shutdown_connections 2024-04-25T00:16:24.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.528+0000 7f2c13200640 1 --2- 172.21.15.42:0/3004077633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c103220 0x7f2c0c109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.528+0000 7f2c13200640 1 --2- 172.21.15.42:0/3004077633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c102860 0x7f2c0c102ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.528+0000 7f2c13200640 1 --2- 172.21.15.42:0/3004077633 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c101670 0x7f2c0c101a90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.528+0000 7f2c13200640 1 -- 172.21.15.42:0/3004077633 >> 172.21.15.42:0/3004077633 conn(0x7f2c0c0fcbe0 msgr2=0x7f2c0c0ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:24.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.528+0000 7f2c13200640 1 -- 172.21.15.42:0/3004077633 shutdown_connections 2024-04-25T00:16:24.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.528+0000 7f2c13200640 1 -- 172.21.15.42:0/3004077633 wait complete. 2024-04-25T00:16:24.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.529+0000 7f2c13200640 1 Processor -- start 2024-04-25T00:16:24.531 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.529+0000 7f2c13200640 1 -- start start 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c13200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c101670 0x7f2c0c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c13200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c102860 0x7f2c0c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c13200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c103220 0x7f2c0c074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c13200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10d2f0 con 0x7f2c0c101670 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c13200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10d170 con 0x7f2c0c103220 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c13200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10d470 con 0x7f2c0c102860 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c11e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c101670 0x7f2c0c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c12800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c103220 0x7f2c0c074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c12800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c103220 0x7f2c0c074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38152/0 (socket says 172.21.15.42:38152) 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c12800640 1 -- 172.21.15.42:0/3157228822 learned_addr learned my addr 172.21.15.42:0/3157228822 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c11e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c101670 0x7f2c0c0754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51306/0 (socket says 172.21.15.42:51306) 2024-04-25T00:16:24.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c11400640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c102860 0x7f2c0c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:24.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c12800640 1 -- 172.21.15.42:0/3157228822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c102860 msgr2=0x7f2c0c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c12800640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c102860 0x7f2c0c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c12800640 1 -- 172.21.15.42:0/3157228822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c101670 msgr2=0x7f2c0c0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:24.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c12800640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c101670 0x7f2c0c0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:24.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.530+0000 7f2c12800640 1 -- 172.21.15.42:0/3157228822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c074900 con 0x7f2c0c103220 2024-04-25T00:16:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.531+0000 7f2c12800640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c103220 0x7f2c0c074070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800eb10 tx=0x7f2c0800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.531+0000 7f2c06a00640 1 -- 172.21.15.42:0/3157228822 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08004b20 con 0x7f2c0c103220 2024-04-25T00:16:24.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.531+0000 7f2c06a00640 1 -- 172.21.15.42:0/3157228822 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c08004cc0 con 0x7f2c0c103220 2024-04-25T00:16:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.531+0000 7f2c13200640 1 -- 172.21.15.42:0/3157228822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c1a1ae0 con 0x7f2c0c103220 2024-04-25T00:16:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.531+0000 7f2c06a00640 1 -- 172.21.15.42:0/3157228822 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c08010760 con 0x7f2c0c103220 2024-04-25T00:16:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.531+0000 7f2c13200640 1 -- 172.21.15.42:0/3157228822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c1a2030 con 0x7f2c0c103220 2024-04-25T00:16:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.532+0000 7f2c13200640 1 -- 172.21.15.42:0/3157228822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0c108a10 con 0x7f2c0c103220 2024-04-25T00:16:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.534+0000 7f2c06a00640 1 -- 172.21.15.42:0/3157228822 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2c08005020 con 0x7f2c0c103220 2024-04-25T00:16:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.535+0000 7f2c06a00640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf40a2c10 0x7f2bf40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.535+0000 7f2c06a00640 1 -- 172.21.15.42:0/3157228822 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2c080cf500 con 0x7f2c0c103220 2024-04-25T00:16:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.535+0000 7f2c11e00640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf40a2c10 0x7f2bf40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.536+0000 7f2c11e00640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf40a2c10 0x7f2bf40a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c00005ab0 tx=0x7f2c00000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:24.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.539+0000 7f2c06a00640 1 -- 172.21.15.42:0/3157228822 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2c0809c510 con 0x7f2c0c103220 2024-04-25T00:16:24.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:16:24.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2486473319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T00:16:24.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:16:24.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:24.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-25T00:16:24.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-25T00:16:24.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3972832645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:16:24.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:16:24.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:16:24.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] up:active 2024-04-25T00:16:24.590 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:16:24.590 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:16:24.590 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: Cluster is now healthy 2024-04-25T00:16:24.590 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T00:16:24.590 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T00:16:24.590 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:24 smithi106 ceph-mon[32510]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T00:16:24.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:16:24.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2486473319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T00:16:24.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3972832645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] up:active 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: Cluster is now healthy 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T00:16:24.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:24 smithi042 ceph-mon[28130]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T00:16:24.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T00:16:24.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2486473319' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T00:16:24.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: daemon mds.a assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T00:16:24.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T00:16:24.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: fsmap cephfs:2 {0=b=up:active,1=a=up:creating} 10 up:standby 2024-04-25T00:16:24.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: daemon mds.a is now active in filesystem cephfs as rank 1 2024-04-25T00:16:24.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3972832645' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:16:24.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T00:16:24.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T00:16:24.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] up:active 2024-04-25T00:16:24.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: daemon mds.c assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T00:16:24.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T00:16:24.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: Cluster is now healthy 2024-04-25T00:16:24.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: fsmap cephfs:2 {0=b=up:active,1=a=up:active} 2 up:standby-replay 8 up:standby 2024-04-25T00:16:24.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:creating} 2 up:standby-replay 7 up:standby 2024-04-25T00:16:24.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:24 smithi136 ceph-mon[32923]: daemon mds.c is now active in filesystem cephfs as rank 2 2024-04-25T00:16:24.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:24.790+0000 7f2c13200640 1 -- 172.21.15.42:0/3157228822 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f2c0c062340 con 0x7f2c0c103220 2024-04-25T00:16:25.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.314+0000 7f2c06a00640 1 -- 172.21.15.42:0/3157228822 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v22) ==== 115+0+0 (secure 0 0 0) 0x7f2c0809be10 con 0x7f2c0c103220 2024-04-25T00:16:25.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.316+0000 7f2c13200640 1 -- 172.21.15.42:0/3157228822 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf40a2c10 msgr2=0x7f2bf40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.316+0000 7f2c13200640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf40a2c10 0x7f2bf40a50d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c00005ab0 tx=0x7f2c00000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.317+0000 7f2c13200640 1 -- 172.21.15.42:0/3157228822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c103220 msgr2=0x7f2c0c074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.317+0000 7f2c13200640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c103220 0x7f2c0c074070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800eb10 tx=0x7f2c0800c2d0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.317+0000 7f2c13200640 1 -- 172.21.15.42:0/3157228822 shutdown_connections 2024-04-25T00:16:25.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.317+0000 7f2c13200640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf40a2c10 0x7f2bf40a50d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.317+0000 7f2c13200640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c103220 0x7f2c0c074070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.317+0000 7f2c13200640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c102860 0x7f2c0c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.317+0000 7f2c13200640 1 --2- 172.21.15.42:0/3157228822 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c101670 0x7f2c0c0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.317+0000 7f2c13200640 1 -- 172.21.15.42:0/3157228822 >> 172.21.15.42:0/3157228822 conn(0x7f2c0c0fcbe0 msgr2=0x7f2c0c105be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:25.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.317+0000 7f2c13200640 1 -- 172.21.15.42:0/3157228822 shutdown_connections 2024-04-25T00:16:25.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.317+0000 7f2c13200640 1 -- 172.21.15.42:0/3157228822 wait complete. 2024-04-25T00:16:25.338 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T00:16:25.338 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-25T00:16:25.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.542+0000 7fb033200640 1 -- 172.21.15.42:0/1958399823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb034102830 msgr2=0x7fb034104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.542+0000 7fb033200640 1 --2- 172.21.15.42:0/1958399823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb034102830 0x7fb034104ca0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb020009900 tx=0x7fb02002a410 comp rx=0 tx=0).stop 2024-04-25T00:16:25.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.542+0000 7fb033200640 1 -- 172.21.15.42:0/1958399823 shutdown_connections 2024-04-25T00:16:25.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.542+0000 7fb033200640 1 --2- 172.21.15.42:0/1958399823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb0341051e0 0x7fb034105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.542+0000 7fb033200640 1 --2- 172.21.15.42:0/1958399823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb034102830 0x7fb034104ca0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.542+0000 7fb033200640 1 --2- 172.21.15.42:0/1958399823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0340ffec0 0x7fb0341022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.542+0000 7fb033200640 1 -- 172.21.15.42:0/1958399823 >> 172.21.15.42:0/1958399823 conn(0x7fb0340fbae0 msgr2=0x7fb0340fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:25.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.542+0000 7fb033200640 1 -- 172.21.15.42:0/1958399823 shutdown_connections 2024-04-25T00:16:25.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.542+0000 7fb033200640 1 -- 172.21.15.42:0/1958399823 wait complete. 2024-04-25T00:16:25.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.543+0000 7fb033200640 1 Processor -- start 2024-04-25T00:16:25.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb033200640 1 -- start start 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb033200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0340ffec0 0x7fb03419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb033200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb034102830 0x7fb03419e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb033200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0341051e0 0x7fb0341a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb033200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb034113910 con 0x7fb0340ffec0 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb033200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb034113790 con 0x7fb0341051e0 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb033200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb034113a90 con 0x7fb034102830 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb031e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0340ffec0 0x7fb03419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb032800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0341051e0 0x7fb0341a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb032800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0341051e0 0x7fb0341a54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38178/0 (socket says 172.21.15.42:38178) 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb032800640 1 -- 172.21.15.42:0/3626150320 learned_addr learned my addr 172.21.15.42:0/3626150320 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.544+0000 7fb031400640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb034102830 0x7fb03419e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb032800640 1 -- 172.21.15.42:0/3626150320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb034102830 msgr2=0x7fb03419e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb032800640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb034102830 0x7fb03419e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb032800640 1 -- 172.21.15.42:0/3626150320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0340ffec0 msgr2=0x7fb03419df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb032800640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0340ffec0 0x7fb03419df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb032800640 1 -- 172.21.15.42:0/3626150320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0341a5b20 con 0x7fb0341051e0 2024-04-25T00:16:25.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb032800640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0341051e0 0x7fb0341a54b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00eb10 tx=0x7fb02c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:25.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb02aa00640 1 -- 172.21.15.42:0/3626150320 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c004b20 con 0x7fb0341051e0 2024-04-25T00:16:25.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb02aa00640 1 -- 172.21.15.42:0/3626150320 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb02c004cc0 con 0x7fb0341051e0 2024-04-25T00:16:25.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb02aa00640 1 -- 172.21.15.42:0/3626150320 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c010760 con 0x7fb0341051e0 2024-04-25T00:16:25.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb033200640 1 -- 172.21.15.42:0/3626150320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0341a5e10 con 0x7fb0341051e0 2024-04-25T00:16:25.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.545+0000 7fb033200640 1 -- 172.21.15.42:0/3626150320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0341a6280 con 0x7fb0341051e0 2024-04-25T00:16:25.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.547+0000 7fb033200640 1 -- 172.21.15.42:0/3626150320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0340fd5d0 con 0x7fb0341051e0 2024-04-25T00:16:25.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.548+0000 7fb02aa00640 1 -- 172.21.15.42:0/3626150320 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb02c005020 con 0x7fb0341051e0 2024-04-25T00:16:25.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.549+0000 7fb02aa00640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0180a2c10 0x7fb0180a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:25.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.550+0000 7fb02aa00640 1 -- 172.21.15.42:0/3626150320 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fb02c0cf5b0 con 0x7fb0341051e0 2024-04-25T00:16:25.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.550+0000 7fb031e00640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0180a2c10 0x7fb0180a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:25.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.550+0000 7fb031e00640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0180a2c10 0x7fb0180a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb024009bd0 tx=0x7fb024005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:25.553 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.554+0000 7fb02aa00640 1 -- 172.21.15.42:0/3626150320 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fb02c09c5c0 con 0x7fb0341051e0 2024-04-25T00:16:25.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-mon[28130]: pgmap v315: 97 pgs: 23 creating+peering, 70 active+clean, 4 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 541 B/s wr, 5 op/s 2024-04-25T00:16:25.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3157228822' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:16:25.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:16:25.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:16:25.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] up:active 2024-04-25T00:16:25.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:16:25.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:16:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:25 smithi136 ceph-mon[32923]: pgmap v315: 97 pgs: 23 creating+peering, 70 active+clean, 4 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 541 B/s wr, 5 op/s 2024-04-25T00:16:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:25 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3157228822' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:16:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:25 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:16:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:25 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:16:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:25 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] up:active 2024-04-25T00:16:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:25 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:16:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:25 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:16:25.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.788+0000 7fb033200640 1 -- 172.21.15.42:0/3626150320 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb034061b80 con 0x7fb0180a2c10 2024-04-25T00:16:25.816 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.816+0000 7fb02aa00640 1 -- 172.21.15.42:0/3626150320 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb034061b80 con 0x7fb0180a2c10 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.818+0000 7fb033200640 1 -- 172.21.15.42:0/3626150320 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0180a2c10 msgr2=0x7fb0180a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.818+0000 7fb033200640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0180a2c10 0x7fb0180a50d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb024009bd0 tx=0x7fb024005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.818+0000 7fb033200640 1 -- 172.21.15.42:0/3626150320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0341051e0 msgr2=0x7fb0341a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.818+0000 7fb033200640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0341051e0 0x7fb0341a54b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00eb10 tx=0x7fb02c00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.819+0000 7fb033200640 1 -- 172.21.15.42:0/3626150320 shutdown_connections 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.819+0000 7fb033200640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb0180a2c10 0x7fb0180a50d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.819+0000 7fb033200640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0341051e0 0x7fb0341a54b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.819+0000 7fb033200640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb034102830 0x7fb03419e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.819+0000 7fb033200640 1 --2- 172.21.15.42:0/3626150320 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb0340ffec0 0x7fb03419df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.819+0000 7fb033200640 1 -- 172.21.15.42:0/3626150320 >> 172.21.15.42:0/3626150320 conn(0x7fb0340fbae0 msgr2=0x7fb034100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:25.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.819+0000 7fb033200640 1 -- 172.21.15.42:0/3626150320 shutdown_connections 2024-04-25T00:16:25.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:25.819+0000 7fb033200640 1 -- 172.21.15.42:0/3626150320 wait complete. 2024-04-25T00:16:25.837 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T00:16:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:25 smithi106 ceph-mon[32510]: pgmap v315: 97 pgs: 23 creating+peering, 70 active+clean, 4 unknown; 577 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 541 B/s wr, 5 op/s 2024-04-25T00:16:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:25 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3157228822' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:16:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:25 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T00:16:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:25 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T00:16:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:25 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] up:active 2024-04-25T00:16:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:25 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:16:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:25 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:16:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.027+0000 7f3089e00640 1 -- 172.21.15.42:0/1732959954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084108f90 msgr2=0x7f3084109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.027+0000 7f3089e00640 1 --2- 172.21.15.42:0/1732959954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084108f90 0x7f3084109410 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f306c009860 tx=0x7f306c02a340 comp rx=0 tx=0).stop 2024-04-25T00:16:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.027+0000 7f3089e00640 1 -- 172.21.15.42:0/1732959954 shutdown_connections 2024-04-25T00:16:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.027+0000 7f3089e00640 1 --2- 172.21.15.42:0/1732959954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084109950 0x7f30841101e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.027+0000 7f3089e00640 1 --2- 172.21.15.42:0/1732959954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084108f90 0x7f3084109410 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.027+0000 7f3089e00640 1 --2- 172.21.15.42:0/1732959954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3084107da0 0x7f30841081c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.027+0000 7f3089e00640 1 -- 172.21.15.42:0/1732959954 >> 172.21.15.42:0/1732959954 conn(0x7f3084076d40 msgr2=0x7f3084079180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.028+0000 7f3089e00640 1 -- 172.21.15.42:0/1732959954 shutdown_connections 2024-04-25T00:16:26.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.028+0000 7f3089e00640 1 -- 172.21.15.42:0/1732959954 wait complete. 2024-04-25T00:16:26.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.028+0000 7f3089e00640 1 Processor -- start 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3089e00640 1 -- start start 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3089e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3084107da0 0x7f308419e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3089e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084108f90 0x7f308419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3089e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084109950 0x7f30841a5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3089e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3084113a20 con 0x7f3084107da0 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3089e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f30841138a0 con 0x7f3084109950 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3089e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3084113ba0 con 0x7f3084108f90 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3082a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3084107da0 0x7f308419e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3083400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084109950 0x7f30841a5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3083400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084109950 0x7f30841a5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38230/0 (socket says 172.21.15.42:38230) 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3082a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3084107da0 0x7f308419e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51364/0 (socket says 172.21.15.42:51364) 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3083400640 1 -- 172.21.15.42:0/1093346624 learned_addr learned my addr 172.21.15.42:0/1093346624 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.029+0000 7f3082000640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084108f90 0x7f308419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:26.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3083400640 1 -- 172.21.15.42:0/1093346624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084108f90 msgr2=0x7f308419e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3083400640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084108f90 0x7f308419e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3083400640 1 -- 172.21.15.42:0/1093346624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3084107da0 msgr2=0x7f308419e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3083400640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3084107da0 0x7f308419e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3083400640 1 -- 172.21.15.42:0/1093346624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30841a5cb0 con 0x7f3084109950 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3083400640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084109950 0x7f30841a5640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f307c00e970 tx=0x7f307c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3088a00640 1 -- 172.21.15.42:0/1093346624 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c00ccc0 con 0x7f3084109950 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3088a00640 1 -- 172.21.15.42:0/1093346624 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f307c005500 con 0x7f3084109950 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3088a00640 1 -- 172.21.15.42:0/1093346624 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f307c010690 con 0x7f3084109950 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3089e00640 1 -- 172.21.15.42:0/1093346624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30841a5fa0 con 0x7f3084109950 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.030+0000 7f3089e00640 1 -- 172.21.15.42:0/1093346624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30841a6410 con 0x7f3084109950 2024-04-25T00:16:26.032 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.031+0000 7f3089e00640 1 -- 172.21.15.42:0/1093346624 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f308410f110 con 0x7f3084109950 2024-04-25T00:16:26.033 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.033+0000 7f3088a00640 1 -- 172.21.15.42:0/1093346624 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f307c004e50 con 0x7f3084109950 2024-04-25T00:16:26.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.034+0000 7f3088a00640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30680a2c10 0x7f30680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:26.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.035+0000 7f3088a00640 1 -- 172.21.15.42:0/1093346624 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f307c0cf320 con 0x7f3084109950 2024-04-25T00:16:26.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.035+0000 7f3082a00640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30680a2c10 0x7f30680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:26.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.035+0000 7f3082a00640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30680a2c10 0x7f30680a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3084108df0 tx=0x7f3074009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:26.039 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.039+0000 7f3088a00640 1 -- 172.21.15.42:0/1093346624 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f307c09c330 con 0x7f3084109950 2024-04-25T00:16:26.127 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:16:25.799+0000 7f81c0a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.127 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:16:25.799+0000 7f81c0a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.127 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:16:25.799+0000 7f81c0a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.127 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:16:25.799+0000 7f81c0a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.127 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:16:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:16:25.799+0000 7f81c0a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T00:16:26.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.271+0000 7f3089e00640 1 -- 172.21.15.42:0/1093346624 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3084061b80 con 0x7f30680a2c10 2024-04-25T00:16:26.305 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.305+0000 7f3088a00640 1 -- 172.21.15.42:0/1093346624 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3084061b80 con 0x7f30680a2c10 2024-04-25T00:16:26.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 -- 172.21.15.42:0/1093346624 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30680a2c10 msgr2=0x7f30680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:26.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30680a2c10 0x7f30680a50d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f3084108df0 tx=0x7f3074009500 comp rx=0 tx=0).stop 2024-04-25T00:16:26.308 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 -- 172.21.15.42:0/1093346624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084109950 msgr2=0x7f30841a5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084109950 0x7f30841a5640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f307c00e970 tx=0x7f307c00ee30 comp rx=0 tx=0).stop 2024-04-25T00:16:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 -- 172.21.15.42:0/1093346624 shutdown_connections 2024-04-25T00:16:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f30680a2c10 0x7f30680a50d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3084109950 0x7f30841a5640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3084108f90 0x7f308419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 --2- 172.21.15.42:0/1093346624 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3084107da0 0x7f308419e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 -- 172.21.15.42:0/1093346624 >> 172.21.15.42:0/1093346624 conn(0x7f3084076d40 msgr2=0x7f308410c2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 -- 172.21.15.42:0/1093346624 shutdown_connections 2024-04-25T00:16:26.309 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.308+0000 7f3089e00640 1 -- 172.21.15.42:0/1093346624 wait complete. 2024-04-25T00:16:26.327 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T00:16:26.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.546+0000 7fa761400640 1 -- 172.21.15.42:0/1908437568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa75c102970 msgr2=0x7fa75c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:26.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.546+0000 7fa761400640 1 --2- 172.21.15.42:0/1908437568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa75c102970 0x7fa75c102df0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009be0 tx=0x7fa74c02a6d0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.546+0000 7fa761400640 1 -- 172.21.15.42:0/1908437568 shutdown_connections 2024-04-25T00:16:26.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.546+0000 7fa761400640 1 --2- 172.21.15.42:0/1908437568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c103330 0x7fa75c109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.546+0000 7fa761400640 1 --2- 172.21.15.42:0/1908437568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa75c102970 0x7fa75c102df0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.546+0000 7fa761400640 1 --2- 172.21.15.42:0/1908437568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c101780 0x7fa75c101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.546+0000 7fa761400640 1 -- 172.21.15.42:0/1908437568 >> 172.21.15.42:0/1908437568 conn(0x7fa75c0fccf0 msgr2=0x7fa75c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:26.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.546+0000 7fa761400640 1 -- 172.21.15.42:0/1908437568 shutdown_connections 2024-04-25T00:16:26.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.546+0000 7fa761400640 1 -- 172.21.15.42:0/1908437568 wait complete. 2024-04-25T00:16:26.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.548+0000 7fa761400640 1 Processor -- start 2024-04-25T00:16:26.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.548+0000 7fa761400640 1 -- start start 2024-04-25T00:16:26.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.548+0000 7fa761400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa75c101780 0x7fa75c199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:26.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.548+0000 7fa761400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c102970 0x7fa75c19a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:26.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.548+0000 7fa761400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c103330 0x7fa75c1a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:26.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.548+0000 7fa761400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c10d3e0 con 0x7fa75c102970 2024-04-25T00:16:26.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.548+0000 7fa761400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c10d260 con 0x7fa75c103330 2024-04-25T00:16:26.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.548+0000 7fa761400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa75c10d560 con 0x7fa75c101780 2024-04-25T00:16:26.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa75c101780 0x7fa75c199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:26.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa75c101780 0x7fa75c199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47790/0 (socket says 172.21.15.42:47790) 2024-04-25T00:16:26.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75a800640 1 -- 172.21.15.42:0/1742520560 learned_addr learned my addr 172.21.15.42:0/1742520560 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa759e00640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c102970 0x7fa75c19a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75b200640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c103330 0x7fa75c1a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75a800640 1 -- 172.21.15.42:0/1742520560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c103330 msgr2=0x7fa75c1a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75a800640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c103330 0x7fa75c1a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75a800640 1 -- 172.21.15.42:0/1742520560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c102970 msgr2=0x7fa75c19a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75a800640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c102970 0x7fa75c19a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75a800640 1 -- 172.21.15.42:0/1742520560 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa75c1a18e0 con 0x7fa75c101780 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75b200640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c103330 0x7fa75c1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.549+0000 7fa75a800640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa75c101780 0x7fa75c199cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa74400e9f0 tx=0x7fa74400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.550+0000 7fa753400640 1 -- 172.21.15.42:0/1742520560 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74400ccc0 con 0x7fa75c101780 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.550+0000 7fa753400640 1 -- 172.21.15.42:0/1742520560 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7440035f0 con 0x7fa75c101780 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.550+0000 7fa753400640 1 -- 172.21.15.42:0/1742520560 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa74400ce60 con 0x7fa75c101780 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.550+0000 7fa761400640 1 -- 172.21.15.42:0/1742520560 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa75c1a1bd0 con 0x7fa75c101780 2024-04-25T00:16:26.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.550+0000 7fa761400640 1 -- 172.21.15.42:0/1742520560 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa75c1a20a0 con 0x7fa75c101780 2024-04-25T00:16:26.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.551+0000 7fa761400640 1 -- 172.21.15.42:0/1742520560 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa75c1a2270 con 0x7fa75c101780 2024-04-25T00:16:26.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.554+0000 7fa753400640 1 -- 172.21.15.42:0/1742520560 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa7440037a0 con 0x7fa75c101780 2024-04-25T00:16:26.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.556+0000 7fa753400640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa7400a2ca0 0x7fa7400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:26.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.556+0000 7fa753400640 1 -- 172.21.15.42:0/1742520560 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa7440cf370 con 0x7fa75c101780 2024-04-25T00:16:26.556 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.556+0000 7fa759e00640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa7400a2ca0 0x7fa7400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:26.557 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.557+0000 7fa759e00640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa7400a2ca0 0x7fa7400a5160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa74c02acc0 tx=0x7fa74c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:26.560 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.560+0000 7fa753400640 1 -- 172.21.15.42:0/1742520560 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa74409c220 con 0x7fa75c101780 2024-04-25T00:16:26.588 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b[63388]: 2024-04-25T00:16:26.224+0000 7f30a434c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T00:16:26.588 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b[63388]: 2024-04-25T00:16:26.224+0000 7f30a434c700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-04-25T00:16:26.588 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:16:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b[63388]: 2024-04-25T00:16:26.224+0000 7f30a434c700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-04-25T00:16:26.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.803+0000 7fa761400640 1 -- 172.21.15.42:0/1742520560 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fa75c075090 con 0x7fa7400a2ca0 2024-04-25T00:16:26.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.819+0000 7fa753400640 1 -- 172.21.15.42:0/1742520560 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa75c075090 con 0x7fa7400a2ca0 2024-04-25T00:16:26.820 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/qa/sv_0/c0c9b9cc-19dc-4ab7-a78b-05453e534b6c 2024-04-25T00:16:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.822+0000 7fa761400640 1 -- 172.21.15.42:0/1742520560 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa7400a2ca0 msgr2=0x7fa7400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.822+0000 7fa761400640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa7400a2ca0 0x7fa7400a5160 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fa74c02acc0 tx=0x7fa74c005fd0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.822+0000 7fa761400640 1 -- 172.21.15.42:0/1742520560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa75c101780 msgr2=0x7fa75c199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.822+0000 7fa761400640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa75c101780 0x7fa75c199cf0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa74400e9f0 tx=0x7fa74400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.823+0000 7fa761400640 1 -- 172.21.15.42:0/1742520560 shutdown_connections 2024-04-25T00:16:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.823+0000 7fa761400640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa7400a2ca0 0x7fa7400a5160 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.823+0000 7fa761400640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa75c103330 0x7fa75c1a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.823+0000 7fa761400640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa75c102970 0x7fa75c19a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.823+0000 7fa761400640 1 --2- 172.21.15.42:0/1742520560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa75c101780 0x7fa75c199cf0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:26.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.823+0000 7fa761400640 1 -- 172.21.15.42:0/1742520560 >> 172.21.15.42:0/1742520560 conn(0x7fa75c0fccf0 msgr2=0x7fa75c073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:26.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.823+0000 7fa761400640 1 -- 172.21.15.42:0/1742520560 shutdown_connections 2024-04-25T00:16:26.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:26.823+0000 7fa761400640 1 -- 172.21.15.42:0/1742520560 wait complete. 2024-04-25T00:16:26.843 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T00:16:27.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.065+0000 7f14d4c00640 1 -- 172.21.15.42:0/1428196148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d01016d0 msgr2=0x7f14d0101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.065+0000 7f14d4c00640 1 --2- 172.21.15.42:0/1428196148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d01016d0 0x7f14d0101af0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009ca0 tx=0x7f14c002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.065+0000 7f14d4c00640 1 -- 172.21.15.42:0/1428196148 shutdown_connections 2024-04-25T00:16:27.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.065+0000 7f14d4c00640 1 --2- 172.21.15.42:0/1428196148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d0103280 0x7f14d0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.065+0000 7f14d4c00640 1 --2- 172.21.15.42:0/1428196148 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d01028c0 0x7f14d0102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.065+0000 7f14d4c00640 1 --2- 172.21.15.42:0/1428196148 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d01016d0 0x7f14d0101af0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.065+0000 7f14d4c00640 1 -- 172.21.15.42:0/1428196148 >> 172.21.15.42:0/1428196148 conn(0x7f14d00fcc40 msgr2=0x7f14d00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:27.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.065+0000 7f14d4c00640 1 -- 172.21.15.42:0/1428196148 shutdown_connections 2024-04-25T00:16:27.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.065+0000 7f14d4c00640 1 -- 172.21.15.42:0/1428196148 wait complete. 2024-04-25T00:16:27.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.066+0000 7f14d4c00640 1 Processor -- start 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14d4c00640 1 -- start start 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14d4c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d01016d0 0x7f14d00754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14d4c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d01028c0 0x7f14d0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14d4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d0103280 0x7f14d0074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14d4c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010d330 con 0x7f14d01028c0 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14d4c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010d1b0 con 0x7f14d0103280 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14d4c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f14d010d4b0 con 0x7f14d01016d0 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14cea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d01028c0 0x7f14d0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14cea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d01028c0 0x7f14d0073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37154/0 (socket says 172.21.15.42:37154) 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14cea00640 1 -- 172.21.15.42:0/2171518590 learned_addr learned my addr 172.21.15.42:0/2171518590 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14cf400640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d01016d0 0x7f14d00754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.067+0000 7f14cfe00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d0103280 0x7f14d0074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.068+0000 7f14cea00640 1 -- 172.21.15.42:0/2171518590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d01016d0 msgr2=0x7f14d00754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.068+0000 7f14cea00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d01016d0 0x7f14d00754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.068+0000 7f14cea00640 1 -- 172.21.15.42:0/2171518590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d0103280 msgr2=0x7f14d0074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.068+0000 7f14cea00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d0103280 0x7f14d0074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.068+0000 7f14cea00640 1 -- 172.21.15.42:0/2171518590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d0074900 con 0x7f14d01028c0 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.068+0000 7f14cea00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d01028c0 0x7f14d0073b30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00e9f0 tx=0x7f14bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.068+0000 7f14c7e00640 1 -- 172.21.15.42:0/2171518590 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc00cd80 con 0x7f14d01028c0 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.068+0000 7f14d4c00640 1 -- 172.21.15.42:0/2171518590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d01a1a90 con 0x7f14d01028c0 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.068+0000 7f14c7e00640 1 -- 172.21.15.42:0/2171518590 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f14bc010910 con 0x7f14d01028c0 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.068+0000 7f14c7e00640 1 -- 172.21.15.42:0/2171518590 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc018950 con 0x7f14d01028c0 2024-04-25T00:16:27.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.069+0000 7f14d4c00640 1 -- 172.21.15.42:0/2171518590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d01a2030 con 0x7f14d01028c0 2024-04-25T00:16:27.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.070+0000 7f14d4c00640 1 -- 172.21.15.42:0/2171518590 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14d01089f0 con 0x7f14d01028c0 2024-04-25T00:16:27.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.072+0000 7f14c7e00640 1 -- 172.21.15.42:0/2171518590 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f14bc010ac0 con 0x7f14d01028c0 2024-04-25T00:16:27.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.074+0000 7f14c7e00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14b40a2ca0 0x7f14b40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.074+0000 7f14c7e00640 1 -- 172.21.15.42:0/2171518590 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f14bc0cf430 con 0x7f14d01028c0 2024-04-25T00:16:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.074+0000 7f14cf400640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14b40a2ca0 0x7f14b40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.075+0000 7f14cf400640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14b40a2ca0 0x7f14b40a5160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f14c0000f80 tx=0x7f14c0005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:27.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.078+0000 7f14c7e00640 1 -- 172.21.15.42:0/2171518590 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14bc09bc90 con 0x7f14d01028c0 2024-04-25T00:16:27.316 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.316+0000 7f14d4c00640 1 -- 172.21.15.42:0/2171518590 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f14d0061c10 con 0x7f14b40a2ca0 2024-04-25T00:16:27.356 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.356+0000 7f14c7e00640 1 -- 172.21.15.42:0/2171518590 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f14d0061c10 con 0x7f14b40a2ca0 2024-04-25T00:16:27.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 -- 172.21.15.42:0/2171518590 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14b40a2ca0 msgr2=0x7f14b40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14b40a2ca0 0x7f14b40a5160 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f14c0000f80 tx=0x7f14c0005f30 comp rx=0 tx=0).stop 2024-04-25T00:16:27.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 -- 172.21.15.42:0/2171518590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d01028c0 msgr2=0x7f14d0073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.359 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d01028c0 0x7f14d0073b30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00e9f0 tx=0x7f14bc00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 -- 172.21.15.42:0/2171518590 shutdown_connections 2024-04-25T00:16:27.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14b40a2ca0 0x7f14b40a5160 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f14d0103280 0x7f14d0074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f14d01028c0 0x7f14d0073b30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 --2- 172.21.15.42:0/2171518590 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14d01016d0 0x7f14d00754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 -- 172.21.15.42:0/2171518590 >> 172.21.15.42:0/2171518590 conn(0x7f14d00fcc40 msgr2=0x7f14d0105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:27.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 -- 172.21.15.42:0/2171518590 shutdown_connections 2024-04-25T00:16:27.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.359+0000 7f14d4c00640 1 -- 172.21.15.42:0/2171518590 wait complete. 2024-04-25T00:16:27.381 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T00:16:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.598+0000 7fc28ca00640 1 -- 172.21.15.42:0/964058869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc288104a70 msgr2=0x7fc288104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.598+0000 7fc28ca00640 1 --2- 172.21.15.42:0/964058869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc288104a70 0x7fc288104ef0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc274009860 tx=0x7fc27402a340 comp rx=0 tx=0).stop 2024-04-25T00:16:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.598+0000 7fc28ca00640 1 -- 172.21.15.42:0/964058869 shutdown_connections 2024-04-25T00:16:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.598+0000 7fc28ca00640 1 --2- 172.21.15.42:0/964058869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288105430 0x7fc28810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.598+0000 7fc28ca00640 1 --2- 172.21.15.42:0/964058869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc288104a70 0x7fc288104ef0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.598+0000 7fc28ca00640 1 --2- 172.21.15.42:0/964058869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc288103880 0x7fc288103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.598+0000 7fc28ca00640 1 -- 172.21.15.42:0/964058869 >> 172.21.15.42:0/964058869 conn(0x7fc2880fedf0 msgr2=0x7fc288101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.599+0000 7fc28ca00640 1 -- 172.21.15.42:0/964058869 shutdown_connections 2024-04-25T00:16:27.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.599+0000 7fc28ca00640 1 -- 172.21.15.42:0/964058869 wait complete. 2024-04-25T00:16:27.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.600+0000 7fc28ca00640 1 Processor -- start 2024-04-25T00:16:27.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.600+0000 7fc28ca00640 1 -- start start 2024-04-25T00:16:27.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.600+0000 7fc28ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc288103880 0x7fc288199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.600+0000 7fc28ca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc288104a70 0x7fc28819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.600+0000 7fc28ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288105430 0x7fc2881a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.600+0000 7fc28ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc28810f500 con 0x7fc288105430 2024-04-25T00:16:27.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.600+0000 7fc28ca00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc28810f380 con 0x7fc288104a70 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.600+0000 7fc28ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc28810f680 con 0x7fc288103880 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc287400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc288103880 0x7fc288199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc287e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288105430 0x7fc2881a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc287400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc288103880 0x7fc288199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59940/0 (socket says 172.21.15.42:59940) 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc287400640 1 -- 172.21.15.42:0/3752190426 learned_addr learned my addr 172.21.15.42:0/3752190426 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc286a00640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc288104a70 0x7fc28819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc287400640 1 -- 172.21.15.42:0/3752190426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc288104a70 msgr2=0x7fc28819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc287400640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc288104a70 0x7fc28819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc287400640 1 -- 172.21.15.42:0/3752190426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288105430 msgr2=0x7fc2881a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc287400640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288105430 0x7fc2881a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc287400640 1 -- 172.21.15.42:0/3752190426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2881a17c0 con 0x7fc288103880 2024-04-25T00:16:27.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.601+0000 7fc287400640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc288103880 0x7fc288199ba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc27800e9f0 tx=0x7fc27800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.602+0000 7fc27fe00640 1 -- 172.21.15.42:0/3752190426 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27800ccc0 con 0x7fc288103880 2024-04-25T00:16:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.602+0000 7fc27fe00640 1 -- 172.21.15.42:0/3752190426 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2780035f0 con 0x7fc288103880 2024-04-25T00:16:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.602+0000 7fc28ca00640 1 -- 172.21.15.42:0/3752190426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2881a1ab0 con 0x7fc288103880 2024-04-25T00:16:27.603 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.602+0000 7fc27fe00640 1 -- 172.21.15.42:0/3752190426 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27800ce60 con 0x7fc288103880 2024-04-25T00:16:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.602+0000 7fc28ca00640 1 -- 172.21.15.42:0/3752190426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2881a1fd0 con 0x7fc288103880 2024-04-25T00:16:27.604 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.603+0000 7fc28ca00640 1 -- 172.21.15.42:0/3752190426 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2880688d0 con 0x7fc288103880 2024-04-25T00:16:27.605 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.605+0000 7fc27fe00640 1 -- 172.21.15.42:0/3752190426 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc278003b10 con 0x7fc288103880 2024-04-25T00:16:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.606+0000 7fc27fe00640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc26c0a2ca0 0x7fc26c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.606+0000 7fc27fe00640 1 -- 172.21.15.42:0/3752190426 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fc2780cf410 con 0x7fc288103880 2024-04-25T00:16:27.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.606+0000 7fc286a00640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc26c0a2ca0 0x7fc26c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:27.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.607+0000 7fc286a00640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc26c0a2ca0 0x7fc26c0a5160 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc27402a930 tx=0x7fc274009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:27.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.611+0000 7fc27fe00640 1 -- 172.21.15.42:0/3752190426 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc27809c2c0 con 0x7fc288103880 2024-04-25T00:16:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:27 smithi042 ceph-mon[28130]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:27 smithi042 ceph-mon[28130]: pgmap v316: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.6 KiB/s wr, 15 op/s 2024-04-25T00:16:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:27 smithi042 ceph-mon[28130]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:27.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:27 smithi042 ceph-mon[28130]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-25T00:16:27.686 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:27 smithi106 ceph-mon[32510]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:27.686 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:27 smithi106 ceph-mon[32510]: pgmap v316: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.6 KiB/s wr, 15 op/s 2024-04-25T00:16:27.686 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:27 smithi106 ceph-mon[32510]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:27.686 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:27 smithi106 ceph-mon[32510]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-25T00:16:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:27 smithi136 ceph-mon[32923]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:27 smithi136 ceph-mon[32923]: pgmap v316: 97 pgs: 97 active+clean; 579 KiB data, 345 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.6 KiB/s wr, 15 op/s 2024-04-25T00:16:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:27 smithi136 ceph-mon[32923]: from='client.24665 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:27 smithi136 ceph-mon[32923]: mgrmap e17: x(active, since 8m), standbys: y, z 2024-04-25T00:16:27.852 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.852+0000 7fc28ca00640 1 -- 172.21.15.42:0/3752190426 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fc288061c10 con 0x7fc26c0a2ca0 2024-04-25T00:16:27.863 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.863+0000 7fc27fe00640 1 -- 172.21.15.42:0/3752190426 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fc288061c10 con 0x7fc26c0a2ca0 2024-04-25T00:16:27.863 INFO:teuthology.orchestra.run.smithi042.stdout:/volumes/qa/sv_1/66188292-0dcc-4827-bc9f-81c849b8381e 2024-04-25T00:16:27.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.865+0000 7fc28ca00640 1 -- 172.21.15.42:0/3752190426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc26c0a2ca0 msgr2=0x7fc26c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.865+0000 7fc28ca00640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc26c0a2ca0 0x7fc26c0a5160 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc27402a930 tx=0x7fc274009540 comp rx=0 tx=0).stop 2024-04-25T00:16:27.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.865+0000 7fc28ca00640 1 -- 172.21.15.42:0/3752190426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc288103880 msgr2=0x7fc288199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:27.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.865+0000 7fc28ca00640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc288103880 0x7fc288199ba0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc27800e9f0 tx=0x7fc27800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.866+0000 7fc28ca00640 1 -- 172.21.15.42:0/3752190426 shutdown_connections 2024-04-25T00:16:27.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.866+0000 7fc28ca00640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc26c0a2ca0 0x7fc26c0a5160 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.866+0000 7fc28ca00640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc288105430 0x7fc2881a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.866+0000 7fc28ca00640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc288104a70 0x7fc28819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.866+0000 7fc28ca00640 1 --2- 172.21.15.42:0/3752190426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc288103880 0x7fc288199ba0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:27.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.866+0000 7fc28ca00640 1 -- 172.21.15.42:0/3752190426 >> 172.21.15.42:0/3752190426 conn(0x7fc2880fedf0 msgr2=0x7fc288107e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:27.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.866+0000 7fc28ca00640 1 -- 172.21.15.42:0/3752190426 shutdown_connections 2024-04-25T00:16:27.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:27.866+0000 7fc28ca00640 1 -- 172.21.15.42:0/3752190426 wait complete. 2024-04-25T00:16:27.886 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:16:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.095+0000 7f6b1b200640 1 -- 172.21.15.42:0/3460288254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b14107460 msgr2=0x7f6b141098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.095+0000 7f6b1b200640 1 --2- 172.21.15.42:0/3460288254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b14107460 0x7f6b141098b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009860 tx=0x7f6b0c02a340 comp rx=0 tx=0).stop 2024-04-25T00:16:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.095+0000 7f6b1b200640 1 -- 172.21.15.42:0/3460288254 shutdown_connections 2024-04-25T00:16:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.095+0000 7f6b1b200640 1 --2- 172.21.15.42:0/3460288254 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b14107460 0x7f6b141098b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.095+0000 7f6b1b200640 1 --2- 172.21.15.42:0/3460288254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b14104b30 0x7f6b14106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.095+0000 7f6b1b200640 1 --2- 172.21.15.42:0/3460288254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b14102200 0x7f6b141045f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.095+0000 7f6b1b200640 1 -- 172.21.15.42:0/3460288254 >> 172.21.15.42:0/3460288254 conn(0x7f6b140fbbf0 msgr2=0x7f6b140fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.096+0000 7f6b1b200640 1 -- 172.21.15.42:0/3460288254 shutdown_connections 2024-04-25T00:16:28.096 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.096+0000 7f6b1b200640 1 -- 172.21.15.42:0/3460288254 wait complete. 2024-04-25T00:16:28.098 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.096+0000 7f6b1b200640 1 Processor -- start 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b1b200640 1 -- start start 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b1b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b14102200 0x7f6b14199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b1b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b14104b30 0x7f6b1419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b1b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b14107460 0x7f6b141a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b1b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410d420 con 0x7f6b14104b30 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b1b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410d2a0 con 0x7f6b14107460 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b1b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1410d5a0 con 0x7f6b14102200 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b13400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b14104b30 0x7f6b1419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b13400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b14104b30 0x7f6b1419a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37188/0 (socket says 172.21.15.42:37188) 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b13400640 1 -- 172.21.15.42:0/1540345226 learned_addr learned my addr 172.21.15.42:0/1540345226 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b13e00640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b14102200 0x7f6b14199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.097+0000 7f6b19200640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b14107460 0x7f6b141a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.098+0000 7f6b13400640 1 -- 172.21.15.42:0/1540345226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b14102200 msgr2=0x7f6b14199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.098+0000 7f6b13400640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b14102200 0x7f6b14199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.098+0000 7f6b13400640 1 -- 172.21.15.42:0/1540345226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b14107460 msgr2=0x7f6b141a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.098+0000 7f6b13400640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b14107460 0x7f6b141a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.098+0000 7f6b13400640 1 -- 172.21.15.42:0/1540345226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b141a18d0 con 0x7f6b14104b30 2024-04-25T00:16:28.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.098+0000 7f6b13400640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b14104b30 0x7f6b1419a1d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00e9f0 tx=0x7f6afc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:28.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.098+0000 7f6b10c00640 1 -- 172.21.15.42:0/1540345226 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc00ccc0 con 0x7f6b14104b30 2024-04-25T00:16:28.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.098+0000 7f6b10c00640 1 -- 172.21.15.42:0/1540345226 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6afc005500 con 0x7f6b14104b30 2024-04-25T00:16:28.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.098+0000 7f6b1b200640 1 -- 172.21.15.42:0/1540345226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b141a1bc0 con 0x7f6b14104b30 2024-04-25T00:16:28.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.098+0000 7f6b10c00640 1 -- 172.21.15.42:0/1540345226 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6afc010690 con 0x7f6b14104b30 2024-04-25T00:16:28.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.099+0000 7f6b1b200640 1 -- 172.21.15.42:0/1540345226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b141a20c0 con 0x7f6b14104b30 2024-04-25T00:16:28.100 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.100+0000 7f6b1b200640 1 -- 172.21.15.42:0/1540345226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b14108f50 con 0x7f6b14104b30 2024-04-25T00:16:28.102 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.103+0000 7f6b10c00640 1 -- 172.21.15.42:0/1540345226 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6afc0046c0 con 0x7f6b14104b30 2024-04-25T00:16:28.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.104+0000 7f6b10c00640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6af80a2c20 0x7f6af80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.104+0000 7f6b10c00640 1 -- 172.21.15.42:0/1540345226 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f6afc0cf640 con 0x7f6b14104b30 2024-04-25T00:16:28.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.104+0000 7f6b13e00640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6af80a2c20 0x7f6af80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.105+0000 7f6b13e00640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6af80a2c20 0x7f6af80a50e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b1419ab60 tx=0x7f6b04010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:28.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.105+0000 7f6b10c00640 1 -- 172.21.15.42:0/1540345226 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6afc053740 con 0x7f6b14104b30 2024-04-25T00:16:28.360 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.360+0000 7f6b1b200640 1 -- 172.21.15.42:0/1540345226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6b1402d3c0 con 0x7f6b14104b30 2024-04-25T00:16:28.361 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.361+0000 7f6b10c00640 1 -- 172.21.15.42:0/1540345226 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11868 (secure 0 0 0) 0x7f6afc09c4f0 con 0x7f6b14104b30 2024-04-25T00:16:28.362 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:28.362 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:16:19.771762+0000","modified":"2024-04-25T00:16:25.317872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24568,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24568":{"gid":24568,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6841/4068466248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4068466248},{"type":"v1","addr":"172.21.15.42:6841","nonce":4068466248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3349564106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3349564106},{"type":"v1","addr":"172.21.15.106:6839","nonce":3349564106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24517,24568,24569]},"id":1}]} 2024-04-25T00:16:28.362 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-04-25T00:16:28.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 -- 172.21.15.42:0/1540345226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6af80a2c20 msgr2=0x7f6af80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6af80a2c20 0x7f6af80a50e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b1419ab60 tx=0x7f6b04010040 comp rx=0 tx=0).stop 2024-04-25T00:16:28.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 -- 172.21.15.42:0/1540345226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b14104b30 msgr2=0x7f6b1419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b14104b30 0x7f6b1419a1d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00e9f0 tx=0x7f6afc00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 -- 172.21.15.42:0/1540345226 shutdown_connections 2024-04-25T00:16:28.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6af80a2c20 0x7f6af80a50e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b14107460 0x7f6b141a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b14104b30 0x7f6b1419a1d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 --2- 172.21.15.42:0/1540345226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b14102200 0x7f6b14199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 -- 172.21.15.42:0/1540345226 >> 172.21.15.42:0/1540345226 conn(0x7f6b140fbbf0 msgr2=0x7f6b141018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:28.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 -- 172.21.15.42:0/1540345226 shutdown_connections 2024-04-25T00:16:28.365 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.364+0000 7f6b1b200640 1 -- 172.21.15.42:0/1540345226 wait complete. 2024-04-25T00:16:28.387 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:16:28.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.590+0000 7fd690400640 1 -- 172.21.15.42:0/3730677337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6881017b0 msgr2=0x7fd688101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.590+0000 7fd690400640 1 --2- 172.21.15.42:0/3730677337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6881017b0 0x7fd688101bd0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd678009ca0 tx=0x7fd67802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.590+0000 7fd690400640 1 -- 172.21.15.42:0/3730677337 shutdown_connections 2024-04-25T00:16:28.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.590+0000 7fd690400640 1 --2- 172.21.15.42:0/3730677337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd688103360 0x7fd688109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.590+0000 7fd690400640 1 --2- 172.21.15.42:0/3730677337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd6881029a0 0x7fd688102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.590+0000 7fd690400640 1 --2- 172.21.15.42:0/3730677337 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6881017b0 0x7fd688101bd0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.590+0000 7fd690400640 1 -- 172.21.15.42:0/3730677337 >> 172.21.15.42:0/3730677337 conn(0x7fd6880fcd20 msgr2=0x7fd6880ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:28.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.590+0000 7fd690400640 1 -- 172.21.15.42:0/3730677337 shutdown_connections 2024-04-25T00:16:28.591 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.590+0000 7fd690400640 1 -- 172.21.15.42:0/3730677337 wait complete. 2024-04-25T00:16:28.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.591+0000 7fd690400640 1 Processor -- start 2024-04-25T00:16:28.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.591+0000 7fd690400640 1 -- start start 2024-04-25T00:16:28.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.591+0000 7fd690400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6881017b0 0x7fd688197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.591+0000 7fd690400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6881029a0 0x7fd688198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.591+0000 7fd690400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd688103360 0x7fd68819f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.591+0000 7fd690400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd68810d450 con 0x7fd6881017b0 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.591+0000 7fd690400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd68810d2d0 con 0x7fd6881029a0 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.591+0000 7fd690400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd68810d5d0 con 0x7fd688103360 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6881017b0 0x7fd688197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6881017b0 0x7fd688197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37206/0 (socket says 172.21.15.42:37206) 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68da00640 1 -- 172.21.15.42:0/1542715561 learned_addr learned my addr 172.21.15.42:0/1542715561 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68e400640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd688103360 0x7fd68819f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68d000640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6881029a0 0x7fd688198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68da00640 1 -- 172.21.15.42:0/1542715561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd688103360 msgr2=0x7fd68819f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68da00640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd688103360 0x7fd68819f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68da00640 1 -- 172.21.15.42:0/1542715561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6881029a0 msgr2=0x7fd688198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68da00640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6881029a0 0x7fd688198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68da00640 1 -- 172.21.15.42:0/1542715561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68819f720 con 0x7fd6881017b0 2024-04-25T00:16:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.592+0000 7fd68da00640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6881017b0 0x7fd688197b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd678000f80 tx=0x7fd6780319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.593+0000 7fd686000640 1 -- 172.21.15.42:0/1542715561 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678003bf0 con 0x7fd6881017b0 2024-04-25T00:16:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.593+0000 7fd686000640 1 -- 172.21.15.42:0/1542715561 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd678003d90 con 0x7fd6881017b0 2024-04-25T00:16:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.593+0000 7fd690400640 1 -- 172.21.15.42:0/1542715561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68819f9b0 con 0x7fd6881017b0 2024-04-25T00:16:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.593+0000 7fd686000640 1 -- 172.21.15.42:0/1542715561 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd67802fd30 con 0x7fd6881017b0 2024-04-25T00:16:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.593+0000 7fd690400640 1 -- 172.21.15.42:0/1542715561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd68819ff50 con 0x7fd6881017b0 2024-04-25T00:16:28.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.594+0000 7fd690400640 1 -- 172.21.15.42:0/1542715561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd688108bf0 con 0x7fd6881017b0 2024-04-25T00:16:28.596 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.596+0000 7fd686000640 1 -- 172.21.15.42:0/1542715561 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd678004040 con 0x7fd6881017b0 2024-04-25T00:16:28.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.598+0000 7fd686000640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd66c0a2ca0 0x7fd66c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:28.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.598+0000 7fd686000640 1 -- 172.21.15.42:0/1542715561 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd6780f02a0 con 0x7fd6881017b0 2024-04-25T00:16:28.598 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.598+0000 7fd68d000640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd66c0a2ca0 0x7fd66c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:28.599 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.599+0000 7fd68d000640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd66c0a2ca0 0x7fd66c0a5160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd688199010 tx=0x7fd67400b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:28.602 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.602+0000 7fd686000640 1 -- 172.21.15.42:0/1542715561 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd6780bd3f0 con 0x7fd6881017b0 2024-04-25T00:16:28.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:28 smithi136 ceph-mon[32923]: from='client.24679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:28.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:28 smithi136 ceph-mon[32923]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:28.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:28 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1540345226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:28.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.809+0000 7fd690400640 1 -- 172.21.15.42:0/1542715561 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd688062340 con 0x7fd6881017b0 2024-04-25T00:16:28.811 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.811+0000 7fd686000640 1 -- 172.21.15.42:0/1542715561 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22217 (secure 0 0 0) 0x7fd6780bccf0 con 0x7fd6881017b0 2024-04-25T00:16:28.811 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:28.812 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":86,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:16:21.219969+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:28.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 -- 172.21.15.42:0/1542715561 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd66c0a2ca0 msgr2=0x7fd66c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd66c0a2ca0 0x7fd66c0a5160 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd688199010 tx=0x7fd67400b390 comp rx=0 tx=0).stop 2024-04-25T00:16:28.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 -- 172.21.15.42:0/1542715561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6881017b0 msgr2=0x7fd688197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:28.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6881017b0 0x7fd688197b00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd678000f80 tx=0x7fd6780319f0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 -- 172.21.15.42:0/1542715561 shutdown_connections 2024-04-25T00:16:28.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd66c0a2ca0 0x7fd66c0a5160 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd688103360 0x7fd68819f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6881029a0 0x7fd688198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 --2- 172.21.15.42:0/1542715561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd6881017b0 0x7fd688197b00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:28.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 -- 172.21.15.42:0/1542715561 >> 172.21.15.42:0/1542715561 conn(0x7fd6880fcd20 msgr2=0x7fd688105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:28.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 -- 172.21.15.42:0/1542715561 shutdown_connections 2024-04-25T00:16:28.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:28.814+0000 7fd690400640 1 -- 172.21.15.42:0/1542715561 wait complete. 2024-04-25T00:16:28.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:28 smithi042 ceph-mon[28130]: from='client.24679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:28.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:28 smithi042 ceph-mon[28130]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:28.831 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:28 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1540345226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:28.834 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T00:16:28.834 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:16:28.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:28 smithi106 ceph-mon[32510]: from='client.24679 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:28.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:28 smithi106 ceph-mon[32510]: from='client.14577 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:28.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:28 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1540345226' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:16:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.047+0000 7f05f6a00640 1 -- 172.21.15.42:0/2220397417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05f0103160 msgr2=0x7f05f01099f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.047+0000 7f05f6a00640 1 --2- 172.21.15.42:0/2220397417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05f0103160 0x7f05f01099f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009be0 tx=0x7f05e802a6c0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.047+0000 7f05f6a00640 1 -- 172.21.15.42:0/2220397417 shutdown_connections 2024-04-25T00:16:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.047+0000 7f05f6a00640 1 --2- 172.21.15.42:0/2220397417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05f0103160 0x7f05f01099f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.047+0000 7f05f6a00640 1 --2- 172.21.15.42:0/2220397417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05f01027a0 0x7f05f0102c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.047+0000 7f05f6a00640 1 --2- 172.21.15.42:0/2220397417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f01015b0 0x7f05f01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.047+0000 7f05f6a00640 1 -- 172.21.15.42:0/2220397417 >> 172.21.15.42:0/2220397417 conn(0x7f05f00fcb20 msgr2=0x7f05f00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.047+0000 7f05f6a00640 1 -- 172.21.15.42:0/2220397417 shutdown_connections 2024-04-25T00:16:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.047+0000 7f05f6a00640 1 -- 172.21.15.42:0/2220397417 wait complete. 2024-04-25T00:16:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.048+0000 7f05f6a00640 1 Processor -- start 2024-04-25T00:16:29.050 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.048+0000 7f05f6a00640 1 -- start start 2024-04-25T00:16:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f01015b0 0x7f05f0199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05f01027a0 0x7f05f0199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05f0103160 0x7f05f01a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f6a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f05f010d210 con 0x7f05f01015b0 2024-04-25T00:16:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f6a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f05f010d090 con 0x7f05f0103160 2024-04-25T00:16:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f6a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05f010d390 con 0x7f05f01027a0 2024-04-25T00:16:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f01015b0 0x7f05f0199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f01015b0 0x7f05f0199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37236/0 (socket says 172.21.15.42:37236) 2024-04-25T00:16:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f5600640 1 -- 172.21.15.42:0/728479529 learned_addr learned my addr 172.21.15.42:0/728479529 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f6000640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05f0103160 0x7f05f01a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f4c00640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05f01027a0 0x7f05f0199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f5600640 1 -- 172.21.15.42:0/728479529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05f01027a0 msgr2=0x7f05f0199f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f5600640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05f01027a0 0x7f05f0199f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f5600640 1 -- 172.21.15.42:0/728479529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05f0103160 msgr2=0x7f05f01a0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f5600640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05f0103160 0x7f05f01a0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.049+0000 7f05f5600640 1 -- 172.21.15.42:0/728479529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f01a1650 con 0x7f05f01015b0 2024-04-25T00:16:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.050+0000 7f05f5600640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f01015b0 0x7f05f0199a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05e400e9f0 tx=0x7f05e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.050+0000 7f05ee000640 1 -- 172.21.15.42:0/728479529 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e400ccc0 con 0x7f05f01015b0 2024-04-25T00:16:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.050+0000 7f05ee000640 1 -- 172.21.15.42:0/728479529 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e40035f0 con 0x7f05f01015b0 2024-04-25T00:16:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.050+0000 7f05f6a00640 1 -- 172.21.15.42:0/728479529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f01a1940 con 0x7f05f01015b0 2024-04-25T00:16:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.050+0000 7f05ee000640 1 -- 172.21.15.42:0/728479529 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e400ce60 con 0x7f05f01015b0 2024-04-25T00:16:29.054 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.050+0000 7f05f6a00640 1 -- 172.21.15.42:0/728479529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01a1ee0 con 0x7f05f01015b0 2024-04-25T00:16:29.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.051+0000 7f05f6a00640 1 -- 172.21.15.42:0/728479529 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f01a2170 con 0x7f05f01015b0 2024-04-25T00:16:29.055 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.053+0000 7f05ee000640 1 -- 172.21.15.42:0/728479529 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f05e40037a0 con 0x7f05f01015b0 2024-04-25T00:16:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.055+0000 7f05ee000640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f05d80a2ca0 0x7f05d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.056+0000 7f05ee000640 1 -- 172.21.15.42:0/728479529 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f05e40cf2e0 con 0x7f05f01015b0 2024-04-25T00:16:29.056 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.056+0000 7f05f4c00640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f05d80a2ca0 0x7f05d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.057 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.056+0000 7f05f4c00640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f05d80a2ca0 0x7f05d80a5160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0095f0 tx=0x7f05dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:29.060 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.060+0000 7f05ee000640 1 -- 172.21.15.42:0/728479529 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f05e409c190 con 0x7f05f01015b0 2024-04-25T00:16:29.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.263+0000 7f05f6a00640 1 -- 172.21.15.42:0/728479529 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05f01a2340 con 0x7f05d80a2ca0 2024-04-25T00:16:29.282 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.282+0000 7f05ee000640 1 -- 172.21.15.42:0/728479529 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258739 (secure 0 0 0) 0x7f05f01a2340 con 0x7f05d80a2ca0 2024-04-25T00:16:29.287 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:29.289 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-25T00:16:29.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 -- 172.21.15.42:0/728479529 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f05d80a2ca0 msgr2=0x7f05d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f05d80a2ca0 0x7f05d80a5160 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f05dc0095f0 tx=0x7f05dc005f90 comp rx=0 tx=0).stop 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 -- 172.21.15.42:0/728479529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f01015b0 msgr2=0x7f05f0199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f01015b0 0x7f05f0199a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f05e400e9f0 tx=0x7f05e400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 -- 172.21.15.42:0/728479529 shutdown_connections 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f05d80a2ca0 0x7f05d80a5160 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05f0103160 0x7f05f01a0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05f01027a0 0x7f05f0199f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 --2- 172.21.15.42:0/728479529 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f05f01015b0 0x7f05f0199a30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 -- 172.21.15.42:0/728479529 >> 172.21.15.42:0/728479529 conn(0x7f05f00fcb20 msgr2=0x7f05f0073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.289+0000 7f05f6a00640 1 -- 172.21.15.42:0/728479529 shutdown_connections 2024-04-25T00:16:29.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.290+0000 7f05f6a00640 1 -- 172.21.15.42:0/728479529 wait complete. 2024-04-25T00:16:29.311 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-04-25T00:16:27.911057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":154,"num_read_kb":130,"num_write":298,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357836,"kb_used_data":12816,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583988,"statfs":{"total":1151940427776,"available":1151574003712,"internally_reserved":0,"allocated":13123584,"data_stored":7117891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4640,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":28,"num_read_kb":21,"num_write":117,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003607"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.377212+0000","last_change":"2024-04-25T00:16:20.250230+0000","last_active":"2024-04-25T00:16:23.377212+0000","last_peered":"2024-04-25T00:16:23.377212+0000","last_clean":"2024-04-25T00:16:23.377212+0000","last_became_active":"2024-04-25T00:16:20.249751+0000","last_became_peered":"2024-04-25T00:16:20.249751+0000","last_unstale":"2024-04-25T00:16:23.377212+0000","last_undegraded":"2024-04-25T00:16:23.377212+0000","last_fullsized":"2024-04-25T00:16:23.377212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:57.895905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700589+0000","last_change":"2024-04-25T00:16:20.237934+0000","last_active":"2024-04-25T00:16:21.700589+0000","last_peered":"2024-04-25T00:16:21.700589+0000","last_clean":"2024-04-25T00:16:21.700589+0000","last_became_active":"2024-04-25T00:16:20.236039+0000","last_became_peered":"2024-04-25T00:16:20.236039+0000","last_unstale":"2024-04-25T00:16:21.700589+0000","last_undegraded":"2024-04-25T00:16:21.700589+0000","last_fullsized":"2024-04-25T00:16:21.700589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:07.730977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700290+0000","last_change":"2024-04-25T00:16:20.248923+0000","last_active":"2024-04-25T00:16:21.700290+0000","last_peered":"2024-04-25T00:16:21.700290+0000","last_clean":"2024-04-25T00:16:21.700290+0000","last_became_active":"2024-04-25T00:16:20.248078+0000","last_became_peered":"2024-04-25T00:16:20.248078+0000","last_unstale":"2024-04-25T00:16:21.700290+0000","last_undegraded":"2024-04-25T00:16:21.700290+0000","last_fullsized":"2024-04-25T00:16:21.700290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:25.675449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.229280+0000","last_change":"2024-04-25T00:16:20.219294+0000","last_active":"2024-04-25T00:16:21.229280+0000","last_peered":"2024-04-25T00:16:21.229280+0000","last_clean":"2024-04-25T00:16:21.229280+0000","last_became_active":"2024-04-25T00:16:20.218990+0000","last_became_peered":"2024-04-25T00:16:20.218990+0000","last_unstale":"2024-04-25T00:16:21.229280+0000","last_undegraded":"2024-04-25T00:16:21.229280+0000","last_fullsized":"2024-04-25T00:16:21.229280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:07.201406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.477150+0000","last_change":"2024-04-25T00:16:20.238311+0000","last_active":"2024-04-25T00:16:21.477150+0000","last_peered":"2024-04-25T00:16:21.477150+0000","last_clean":"2024-04-25T00:16:21.477150+0000","last_became_active":"2024-04-25T00:16:20.238104+0000","last_became_peered":"2024-04-25T00:16:20.238104+0000","last_unstale":"2024-04-25T00:16:21.477150+0000","last_undegraded":"2024-04-25T00:16:21.477150+0000","last_fullsized":"2024-04-25T00:16:21.477150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:42.459941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:22.245080+0000","last_change":"2024-04-25T00:16:20.228307+0000","last_active":"2024-04-25T00:16:22.245080+0000","last_peered":"2024-04-25T00:16:22.245080+0000","last_clean":"2024-04-25T00:16:22.245080+0000","last_became_active":"2024-04-25T00:16:20.227999+0000","last_became_peered":"2024-04-25T00:16:20.227999+0000","last_unstale":"2024-04-25T00:16:22.245080+0000","last_undegraded":"2024-04-25T00:16:22.245080+0000","last_fullsized":"2024-04-25T00:16:22.245080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:57.910589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233060+0000","last_change":"2024-04-25T00:16:20.224382+0000","last_active":"2024-04-25T00:16:23.233060+0000","last_peered":"2024-04-25T00:16:23.233060+0000","last_clean":"2024-04-25T00:16:23.233060+0000","last_became_active":"2024-04-25T00:16:20.224122+0000","last_became_peered":"2024-04-25T00:16:20.224122+0000","last_unstale":"2024-04-25T00:16:23.233060+0000","last_undegraded":"2024-04-25T00:16:23.233060+0000","last_fullsized":"2024-04-25T00:16:23.233060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:19.651330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.225679+0000","last_change":"2024-04-25T00:16:20.227720+0000","last_active":"2024-04-25T00:16:21.225679+0000","last_peered":"2024-04-25T00:16:21.225679+0000","last_clean":"2024-04-25T00:16:21.225679+0000","last_became_active":"2024-04-25T00:16:20.227263+0000","last_became_peered":"2024-04-25T00:16:20.227263+0000","last_unstale":"2024-04-25T00:16:21.225679+0000","last_undegraded":"2024-04-25T00:16:21.225679+0000","last_fullsized":"2024-04-25T00:16:21.225679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:03.709172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375891+0000","last_change":"2024-04-25T00:16:20.217673+0000","last_active":"2024-04-25T00:16:23.375891+0000","last_peered":"2024-04-25T00:16:23.375891+0000","last_clean":"2024-04-25T00:16:23.375891+0000","last_became_active":"2024-04-25T00:16:20.216773+0000","last_became_peered":"2024-04-25T00:16:20.216773+0000","last_unstale":"2024-04-25T00:16:23.375891+0000","last_undegraded":"2024-04-25T00:16:23.375891+0000","last_fullsized":"2024-04-25T00:16:23.375891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.623501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:22.245468+0000","last_change":"2024-04-25T00:16:20.218460+0000","last_active":"2024-04-25T00:16:22.245468+0000","last_peered":"2024-04-25T00:16:22.245468+0000","last_clean":"2024-04-25T00:16:22.245468+0000","last_became_active":"2024-04-25T00:16:20.217198+0000","last_became_peered":"2024-04-25T00:16:20.217198+0000","last_unstale":"2024-04-25T00:16:22.245468+0000","last_undegraded":"2024-04-25T00:16:22.245468+0000","last_fullsized":"2024-04-25T00:16:22.245468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:06.511749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233369+0000","last_change":"2024-04-25T00:16:20.221990+0000","last_active":"2024-04-25T00:16:23.233369+0000","last_peered":"2024-04-25T00:16:23.233369+0000","last_clean":"2024-04-25T00:16:23.233369+0000","last_became_active":"2024-04-25T00:16:20.221635+0000","last_became_peered":"2024-04-25T00:16:20.221635+0000","last_unstale":"2024-04-25T00:16:23.233369+0000","last_undegraded":"2024-04-25T00:16:23.233369+0000","last_fullsized":"2024-04-25T00:16:23.233369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:31.205135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.229240+0000","last_change":"2024-04-25T00:16:20.219923+0000","last_active":"2024-04-25T00:16:21.229240+0000","last_peered":"2024-04-25T00:16:21.229240+0000","last_clean":"2024-04-25T00:16:21.229240+0000","last_became_active":"2024-04-25T00:16:20.219578+0000","last_became_peered":"2024-04-25T00:16:20.219578+0000","last_unstale":"2024-04-25T00:16:21.229240+0000","last_undegraded":"2024-04-25T00:16:21.229240+0000","last_fullsized":"2024-04-25T00:16:21.229240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:41.811382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383669+0000","last_change":"2024-04-25T00:16:20.240570+0000","last_active":"2024-04-25T00:16:23.383669+0000","last_peered":"2024-04-25T00:16:23.383669+0000","last_clean":"2024-04-25T00:16:23.383669+0000","last_became_active":"2024-04-25T00:16:20.238241+0000","last_became_peered":"2024-04-25T00:16:20.238241+0000","last_unstale":"2024-04-25T00:16:23.383669+0000","last_undegraded":"2024-04-25T00:16:23.383669+0000","last_fullsized":"2024-04-25T00:16:23.383669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:06.981816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700108+0000","last_change":"2024-04-25T00:16:20.234748+0000","last_active":"2024-04-25T00:16:21.700108+0000","last_peered":"2024-04-25T00:16:21.700108+0000","last_clean":"2024-04-25T00:16:21.700108+0000","last_became_active":"2024-04-25T00:16:20.234325+0000","last_became_peered":"2024-04-25T00:16:20.234325+0000","last_unstale":"2024-04-25T00:16:21.700108+0000","last_undegraded":"2024-04-25T00:16:21.700108+0000","last_fullsized":"2024-04-25T00:16:21.700108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:00.383533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.699878+0000","last_change":"2024-04-25T00:16:20.242987+0000","last_active":"2024-04-25T00:16:21.699878+0000","last_peered":"2024-04-25T00:16:21.699878+0000","last_clean":"2024-04-25T00:16:21.699878+0000","last_became_active":"2024-04-25T00:16:20.242625+0000","last_became_peered":"2024-04-25T00:16:20.242625+0000","last_unstale":"2024-04-25T00:16:21.699878+0000","last_undegraded":"2024-04-25T00:16:21.699878+0000","last_fullsized":"2024-04-25T00:16:21.699878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:16.618120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375122+0000","last_change":"2024-04-25T00:16:20.241985+0000","last_active":"2024-04-25T00:16:23.375122+0000","last_peered":"2024-04-25T00:16:23.375122+0000","last_clean":"2024-04-25T00:16:23.375122+0000","last_became_active":"2024-04-25T00:16:20.241646+0000","last_became_peered":"2024-04-25T00:16:20.241646+0000","last_unstale":"2024-04-25T00:16:23.375122+0000","last_undegraded":"2024-04-25T00:16:23.375122+0000","last_fullsized":"2024-04-25T00:16:23.375122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:44.094705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.377570+0000","last_change":"2024-04-25T00:16:20.250161+0000","last_active":"2024-04-25T00:16:23.377570+0000","last_peered":"2024-04-25T00:16:23.377570+0000","last_clean":"2024-04-25T00:16:23.377570+0000","last_became_active":"2024-04-25T00:16:20.249811+0000","last_became_peered":"2024-04-25T00:16:20.249811+0000","last_unstale":"2024-04-25T00:16:23.377570+0000","last_undegraded":"2024-04-25T00:16:23.377570+0000","last_fullsized":"2024-04-25T00:16:23.377570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:03.222602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.229066+0000","last_change":"2024-04-25T00:16:20.220859+0000","last_active":"2024-04-25T00:16:21.229066+0000","last_peered":"2024-04-25T00:16:21.229066+0000","last_clean":"2024-04-25T00:16:21.229066+0000","last_became_active":"2024-04-25T00:16:20.219864+0000","last_became_peered":"2024-04-25T00:16:20.219864+0000","last_unstale":"2024-04-25T00:16:21.229066+0000","last_undegraded":"2024-04-25T00:16:21.229066+0000","last_fullsized":"2024-04-25T00:16:21.229066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:23.830726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375929+0000","last_change":"2024-04-25T00:16:20.249293+0000","last_active":"2024-04-25T00:16:23.375929+0000","last_peered":"2024-04-25T00:16:23.375929+0000","last_clean":"2024-04-25T00:16:23.375929+0000","last_became_active":"2024-04-25T00:16:20.248952+0000","last_became_peered":"2024-04-25T00:16:20.248952+0000","last_unstale":"2024-04-25T00:16:23.375929+0000","last_undegraded":"2024-04-25T00:16:23.375929+0000","last_fullsized":"2024-04-25T00:16:23.375929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:02.124426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.325216+0000","last_change":"2024-04-25T00:16:19.310971+0000","last_active":"2024-04-25T00:16:24.325216+0000","last_peered":"2024-04-25T00:16:24.325216+0000","last_clean":"2024-04-25T00:16:24.325216+0000","last_became_active":"2024-04-25T00:16:19.309020+0000","last_became_peered":"2024-04-25T00:16:19.309020+0000","last_unstale":"2024-04-25T00:16:24.325216+0000","last_undegraded":"2024-04-25T00:16:24.325216+0000","last_fullsized":"2024-04-25T00:16:24.325216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:56.744138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.226036+0000","last_change":"2024-04-25T00:16:20.250665+0000","last_active":"2024-04-25T00:16:21.226036+0000","last_peered":"2024-04-25T00:16:21.226036+0000","last_clean":"2024-04-25T00:16:21.226036+0000","last_became_active":"2024-04-25T00:16:20.250338+0000","last_became_peered":"2024-04-25T00:16:20.250338+0000","last_unstale":"2024-04-25T00:16:21.226036+0000","last_undegraded":"2024-04-25T00:16:21.226036+0000","last_fullsized":"2024-04-25T00:16:21.226036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:17.018035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.496001+0000","last_change":"2024-04-25T00:16:19.327748+0000","last_active":"2024-04-25T00:16:21.496001+0000","last_peered":"2024-04-25T00:16:21.496001+0000","last_clean":"2024-04-25T00:16:21.496001+0000","last_became_active":"2024-04-25T00:16:19.327264+0000","last_became_peered":"2024-04-25T00:16:19.327264+0000","last_unstale":"2024-04-25T00:16:21.496001+0000","last_undegraded":"2024-04-25T00:16:21.496001+0000","last_fullsized":"2024-04-25T00:16:21.496001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:27.080423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700861+0000","last_change":"2024-04-25T00:16:20.243061+0000","last_active":"2024-04-25T00:16:21.700861+0000","last_peered":"2024-04-25T00:16:21.700861+0000","last_clean":"2024-04-25T00:16:21.700861+0000","last_became_active":"2024-04-25T00:16:20.242697+0000","last_became_peered":"2024-04-25T00:16:20.242697+0000","last_unstale":"2024-04-25T00:16:21.700861+0000","last_undegraded":"2024-04-25T00:16:21.700861+0000","last_fullsized":"2024-04-25T00:16:21.700861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:57.734902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.320426+0000","last_change":"2024-04-25T00:16:19.317122+0000","last_active":"2024-04-25T00:16:24.320426+0000","last_peered":"2024-04-25T00:16:24.320426+0000","last_clean":"2024-04-25T00:16:24.320426+0000","last_became_active":"2024-04-25T00:16:19.316754+0000","last_became_peered":"2024-04-25T00:16:19.316754+0000","last_unstale":"2024-04-25T00:16:24.320426+0000","last_undegraded":"2024-04-25T00:16:24.320426+0000","last_fullsized":"2024-04-25T00:16:24.320426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:28.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375611+0000","last_change":"2024-04-25T00:16:20.238611+0000","last_active":"2024-04-25T00:16:23.375611+0000","last_peered":"2024-04-25T00:16:23.375611+0000","last_clean":"2024-04-25T00:16:23.375611+0000","last_became_active":"2024-04-25T00:16:20.238257+0000","last_became_peered":"2024-04-25T00:16:20.238257+0000","last_unstale":"2024-04-25T00:16:23.375611+0000","last_undegraded":"2024-04-25T00:16:23.375611+0000","last_fullsized":"2024-04-25T00:16:23.375611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:13.490284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.477463+0000","last_change":"2024-04-25T00:16:19.308565+0000","last_active":"2024-04-25T00:16:21.477463+0000","last_peered":"2024-04-25T00:16:21.477463+0000","last_clean":"2024-04-25T00:16:21.477463+0000","last_became_active":"2024-04-25T00:16:19.308220+0000","last_became_peered":"2024-04-25T00:16:19.308220+0000","last_unstale":"2024-04-25T00:16:21.477463+0000","last_undegraded":"2024-04-25T00:16:21.477463+0000","last_fullsized":"2024-04-25T00:16:21.477463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:33.215491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376084+0000","last_change":"2024-04-25T00:16:20.220198+0000","last_active":"2024-04-25T00:16:23.376084+0000","last_peered":"2024-04-25T00:16:23.376084+0000","last_clean":"2024-04-25T00:16:23.376084+0000","last_became_active":"2024-04-25T00:16:20.219897+0000","last_became_peered":"2024-04-25T00:16:20.219897+0000","last_unstale":"2024-04-25T00:16:23.376084+0000","last_undegraded":"2024-04-25T00:16:23.376084+0000","last_fullsized":"2024-04-25T00:16:23.376084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:57.617707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:25.312196+0000","last_change":"2024-04-25T00:16:19.312725+0000","last_active":"2024-04-25T00:16:25.312196+0000","last_peered":"2024-04-25T00:16:25.312196+0000","last_clean":"2024-04-25T00:16:25.312196+0000","last_became_active":"2024-04-25T00:16:19.312115+0000","last_became_peered":"2024-04-25T00:16:19.312115+0000","last_unstale":"2024-04-25T00:16:25.312196+0000","last_undegraded":"2024-04-25T00:16:25.312196+0000","last_fullsized":"2024-04-25T00:16:25.312196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.661132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.477252+0000","last_change":"2024-04-25T00:16:20.233546+0000","last_active":"2024-04-25T00:16:21.477252+0000","last_peered":"2024-04-25T00:16:21.477252+0000","last_clean":"2024-04-25T00:16:21.477252+0000","last_became_active":"2024-04-25T00:16:20.233029+0000","last_became_peered":"2024-04-25T00:16:20.233029+0000","last_unstale":"2024-04-25T00:16:21.477252+0000","last_undegraded":"2024-04-25T00:16:21.477252+0000","last_fullsized":"2024-04-25T00:16:21.477252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:44.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.322639+0000","last_change":"2024-04-25T00:16:19.307512+0000","last_active":"2024-04-25T00:16:24.322639+0000","last_peered":"2024-04-25T00:16:24.322639+0000","last_clean":"2024-04-25T00:16:24.322639+0000","last_became_active":"2024-04-25T00:16:19.307164+0000","last_became_peered":"2024-04-25T00:16:19.307164+0000","last_unstale":"2024-04-25T00:16:24.322639+0000","last_undegraded":"2024-04-25T00:16:24.322639+0000","last_fullsized":"2024-04-25T00:16:24.322639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:37.560178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375187+0000","last_change":"2024-04-25T00:16:20.232758+0000","last_active":"2024-04-25T00:16:23.375187+0000","last_peered":"2024-04-25T00:16:23.375187+0000","last_clean":"2024-04-25T00:16:23.375187+0000","last_became_active":"2024-04-25T00:16:20.232443+0000","last_became_peered":"2024-04-25T00:16:20.232443+0000","last_unstale":"2024-04-25T00:16:23.375187+0000","last_undegraded":"2024-04-25T00:16:23.375187+0000","last_fullsized":"2024-04-25T00:16:23.375187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:31.029585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.325943+0000","last_change":"2024-04-25T00:16:19.313924+0000","last_active":"2024-04-25T00:16:24.325943+0000","last_peered":"2024-04-25T00:16:24.325943+0000","last_clean":"2024-04-25T00:16:24.325943+0000","last_became_active":"2024-04-25T00:16:19.313570+0000","last_became_peered":"2024-04-25T00:16:19.313570+0000","last_unstale":"2024-04-25T00:16:24.325943+0000","last_undegraded":"2024-04-25T00:16:24.325943+0000","last_fullsized":"2024-04-25T00:16:24.325943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:42.258910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.225559+0000","last_change":"2024-04-25T00:16:20.244150+0000","last_active":"2024-04-25T00:16:21.225559+0000","last_peered":"2024-04-25T00:16:21.225559+0000","last_clean":"2024-04-25T00:16:21.225559+0000","last_became_active":"2024-04-25T00:16:20.243802+0000","last_became_peered":"2024-04-25T00:16:20.243802+0000","last_unstale":"2024-04-25T00:16:21.225559+0000","last_undegraded":"2024-04-25T00:16:21.225559+0000","last_fullsized":"2024-04-25T00:16:21.225559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:10.192507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375112+0000","last_change":"2024-04-25T00:16:19.312361+0000","last_active":"2024-04-25T00:16:23.375112+0000","last_peered":"2024-04-25T00:16:23.375112+0000","last_clean":"2024-04-25T00:16:23.375112+0000","last_became_active":"2024-04-25T00:16:19.312031+0000","last_became_peered":"2024-04-25T00:16:19.312031+0000","last_unstale":"2024-04-25T00:16:23.375112+0000","last_undegraded":"2024-04-25T00:16:23.375112+0000","last_fullsized":"2024-04-25T00:16:23.375112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:10.062091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.228514+0000","last_change":"2024-04-25T00:16:20.231399+0000","last_active":"2024-04-25T00:16:21.228514+0000","last_peered":"2024-04-25T00:16:21.228514+0000","last_clean":"2024-04-25T00:16:21.228514+0000","last_became_active":"2024-04-25T00:16:20.230954+0000","last_became_peered":"2024-04-25T00:16:20.230954+0000","last_unstale":"2024-04-25T00:16:21.228514+0000","last_undegraded":"2024-04-25T00:16:21.228514+0000","last_fullsized":"2024-04-25T00:16:21.228514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:20.934927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.378905+0000","last_change":"2024-04-25T00:16:19.308591+0000","last_active":"2024-04-25T00:16:23.378905+0000","last_peered":"2024-04-25T00:16:23.378905+0000","last_clean":"2024-04-25T00:16:23.378905+0000","last_became_active":"2024-04-25T00:16:19.307987+0000","last_became_peered":"2024-04-25T00:16:19.307987+0000","last_unstale":"2024-04-25T00:16:23.378905+0000","last_undegraded":"2024-04-25T00:16:23.378905+0000","last_fullsized":"2024-04-25T00:16:23.378905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:32.739170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375812+0000","last_change":"2024-04-25T00:16:20.218545+0000","last_active":"2024-04-25T00:16:23.375812+0000","last_peered":"2024-04-25T00:16:23.375812+0000","last_clean":"2024-04-25T00:16:23.375812+0000","last_became_active":"2024-04-25T00:16:20.218267+0000","last_became_peered":"2024-04-25T00:16:20.218267+0000","last_unstale":"2024-04-25T00:16:23.375812+0000","last_undegraded":"2024-04-25T00:16:23.375812+0000","last_fullsized":"2024-04-25T00:16:23.375812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:47.725985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.379011+0000","last_change":"2024-04-25T00:16:19.311918+0000","last_active":"2024-04-25T00:16:23.379011+0000","last_peered":"2024-04-25T00:16:23.379011+0000","last_clean":"2024-04-25T00:16:23.379011+0000","last_became_active":"2024-04-25T00:16:19.311594+0000","last_became_peered":"2024-04-25T00:16:19.311594+0000","last_unstale":"2024-04-25T00:16:23.379011+0000","last_undegraded":"2024-04-25T00:16:23.379011+0000","last_fullsized":"2024-04-25T00:16:23.379011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:04.574190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700690+0000","last_change":"2024-04-25T00:16:20.248116+0000","last_active":"2024-04-25T00:16:21.700690+0000","last_peered":"2024-04-25T00:16:21.700690+0000","last_clean":"2024-04-25T00:16:21.700690+0000","last_became_active":"2024-04-25T00:16:20.247373+0000","last_became_peered":"2024-04-25T00:16:20.247373+0000","last_unstale":"2024-04-25T00:16:21.700690+0000","last_undegraded":"2024-04-25T00:16:21.700690+0000","last_fullsized":"2024-04-25T00:16:21.700690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:47.777779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.381082+0000","last_change":"2024-04-25T00:16:19.320640+0000","last_active":"2024-04-25T00:16:23.381082+0000","last_peered":"2024-04-25T00:16:23.381082+0000","last_clean":"2024-04-25T00:16:23.381082+0000","last_became_active":"2024-04-25T00:16:19.320273+0000","last_became_peered":"2024-04-25T00:16:19.320273+0000","last_unstale":"2024-04-25T00:16:23.381082+0000","last_undegraded":"2024-04-25T00:16:23.381082+0000","last_fullsized":"2024-04-25T00:16:23.381082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:10.843162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375646+0000","last_change":"2024-04-25T00:16:20.219378+0000","last_active":"2024-04-25T00:16:23.375646+0000","last_peered":"2024-04-25T00:16:23.375646+0000","last_clean":"2024-04-25T00:16:23.375646+0000","last_became_active":"2024-04-25T00:16:20.219069+0000","last_became_peered":"2024-04-25T00:16:20.219069+0000","last_unstale":"2024-04-25T00:16:23.375646+0000","last_undegraded":"2024-04-25T00:16:23.375646+0000","last_fullsized":"2024-04-25T00:16:23.375646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:10.818609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.310119+0000","last_change":"2024-04-25T00:16:19.315089+0000","last_active":"2024-04-25T00:16:24.310119+0000","last_peered":"2024-04-25T00:16:24.310119+0000","last_clean":"2024-04-25T00:16:24.310119+0000","last_became_active":"2024-04-25T00:16:19.314416+0000","last_became_peered":"2024-04-25T00:16:19.314416+0000","last_unstale":"2024-04-25T00:16:24.310119+0000","last_undegraded":"2024-04-25T00:16:24.310119+0000","last_fullsized":"2024-04-25T00:16:24.310119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:25.920010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.477370+0000","last_change":"2024-04-25T00:16:20.225678+0000","last_active":"2024-04-25T00:16:21.477370+0000","last_peered":"2024-04-25T00:16:21.477370+0000","last_clean":"2024-04-25T00:16:21.477370+0000","last_became_active":"2024-04-25T00:16:20.225146+0000","last_became_peered":"2024-04-25T00:16:20.225146+0000","last_unstale":"2024-04-25T00:16:21.477370+0000","last_undegraded":"2024-04-25T00:16:21.477370+0000","last_fullsized":"2024-04-25T00:16:21.477370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:56.003701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.382921+0000","last_change":"2024-04-25T00:16:19.327877+0000","last_active":"2024-04-25T00:16:23.382921+0000","last_peered":"2024-04-25T00:16:23.382921+0000","last_clean":"2024-04-25T00:16:23.382921+0000","last_became_active":"2024-04-25T00:16:19.327537+0000","last_became_peered":"2024-04-25T00:16:19.327537+0000","last_unstale":"2024-04-25T00:16:23.382921+0000","last_undegraded":"2024-04-25T00:16:23.382921+0000","last_fullsized":"2024-04-25T00:16:23.382921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:14.614232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383713+0000","last_change":"2024-04-25T00:16:20.253101+0000","last_active":"2024-04-25T00:16:23.383713+0000","last_peered":"2024-04-25T00:16:23.383713+0000","last_clean":"2024-04-25T00:16:23.383713+0000","last_became_active":"2024-04-25T00:16:20.252735+0000","last_became_peered":"2024-04-25T00:16:20.252735+0000","last_unstale":"2024-04-25T00:16:23.383713+0000","last_undegraded":"2024-04-25T00:16:23.383713+0000","last_fullsized":"2024-04-25T00:16:23.383713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:35.520309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.381386+0000","last_change":"2024-04-25T00:16:19.328428+0000","last_active":"2024-04-25T00:16:23.381386+0000","last_peered":"2024-04-25T00:16:23.381386+0000","last_clean":"2024-04-25T00:16:23.381386+0000","last_became_active":"2024-04-25T00:16:19.328112+0000","last_became_peered":"2024-04-25T00:16:19.328112+0000","last_unstale":"2024-04-25T00:16:23.381386+0000","last_undegraded":"2024-04-25T00:16:23.381386+0000","last_fullsized":"2024-04-25T00:16:23.381386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:09.130246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233454+0000","last_change":"2024-04-25T00:16:20.237922+0000","last_active":"2024-04-25T00:16:23.233454+0000","last_peered":"2024-04-25T00:16:23.233454+0000","last_clean":"2024-04-25T00:16:23.233454+0000","last_became_active":"2024-04-25T00:16:20.237594+0000","last_became_peered":"2024-04-25T00:16:20.237594+0000","last_unstale":"2024-04-25T00:16:23.233454+0000","last_undegraded":"2024-04-25T00:16:23.233454+0000","last_fullsized":"2024-04-25T00:16:23.233454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:26.517335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.331075+0000","last_change":"2024-04-25T00:16:19.312920+0000","last_active":"2024-04-25T00:16:26.331075+0000","last_peered":"2024-04-25T00:16:26.331075+0000","last_clean":"2024-04-25T00:16:26.331075+0000","last_became_active":"2024-04-25T00:16:19.312548+0000","last_became_peered":"2024-04-25T00:16:19.312548+0000","last_unstale":"2024-04-25T00:16:26.331075+0000","last_undegraded":"2024-04-25T00:16:26.331075+0000","last_fullsized":"2024-04-25T00:16:26.331075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:47.154402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233025+0000","last_change":"2024-04-25T00:16:20.221102+0000","last_active":"2024-04-25T00:16:23.233025+0000","last_peered":"2024-04-25T00:16:23.233025+0000","last_clean":"2024-04-25T00:16:23.233025+0000","last_became_active":"2024-04-25T00:16:20.220753+0000","last_became_peered":"2024-04-25T00:16:20.220753+0000","last_unstale":"2024-04-25T00:16:23.233025+0000","last_undegraded":"2024-04-25T00:16:23.233025+0000","last_fullsized":"2024-04-25T00:16:23.233025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:18.652865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.326937+0000","last_change":"2024-04-25T00:16:19.324181+0000","last_active":"2024-04-25T00:16:24.326937+0000","last_peered":"2024-04-25T00:16:24.326937+0000","last_clean":"2024-04-25T00:16:24.326937+0000","last_became_active":"2024-04-25T00:16:19.323821+0000","last_became_peered":"2024-04-25T00:16:19.323821+0000","last_unstale":"2024-04-25T00:16:24.326937+0000","last_undegraded":"2024-04-25T00:16:24.326937+0000","last_fullsized":"2024-04-25T00:16:24.326937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:34.323961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375521+0000","last_change":"2024-04-25T00:16:20.219629+0000","last_active":"2024-04-25T00:16:23.375521+0000","last_peered":"2024-04-25T00:16:23.375521+0000","last_clean":"2024-04-25T00:16:23.375521+0000","last_became_active":"2024-04-25T00:16:20.219230+0000","last_became_peered":"2024-04-25T00:16:20.219230+0000","last_unstale":"2024-04-25T00:16:23.375521+0000","last_undegraded":"2024-04-25T00:16:23.375521+0000","last_fullsized":"2024-04-25T00:16:23.375521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:52.410209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.307528+0000","last_change":"2024-04-25T00:16:19.306005+0000","last_active":"2024-04-25T00:16:24.307528+0000","last_peered":"2024-04-25T00:16:24.307528+0000","last_clean":"2024-04-25T00:16:24.307528+0000","last_became_active":"2024-04-25T00:16:19.305656+0000","last_became_peered":"2024-04-25T00:16:19.305656+0000","last_unstale":"2024-04-25T00:16:24.307528+0000","last_undegraded":"2024-04-25T00:16:24.307528+0000","last_fullsized":"2024-04-25T00:16:24.307528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:16.452352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376035+0000","last_change":"2024-04-25T00:16:20.217886+0000","last_active":"2024-04-25T00:16:23.376035+0000","last_peered":"2024-04-25T00:16:23.376035+0000","last_clean":"2024-04-25T00:16:23.376035+0000","last_became_active":"2024-04-25T00:16:20.217379+0000","last_became_peered":"2024-04-25T00:16:20.217379+0000","last_unstale":"2024-04-25T00:16:23.376035+0000","last_undegraded":"2024-04-25T00:16:23.376035+0000","last_fullsized":"2024-04-25T00:16:23.376035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:22.028791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.228302+0000","last_change":"2024-04-25T00:16:19.328466+0000","last_active":"2024-04-25T00:16:21.228302+0000","last_peered":"2024-04-25T00:16:21.228302+0000","last_clean":"2024-04-25T00:16:21.228302+0000","last_became_active":"2024-04-25T00:16:19.321077+0000","last_became_peered":"2024-04-25T00:16:19.321077+0000","last_unstale":"2024-04-25T00:16:21.228302+0000","last_undegraded":"2024-04-25T00:16:21.228302+0000","last_fullsized":"2024-04-25T00:16:21.228302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:55.296160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:22.245382+0000","last_change":"2024-04-25T00:16:20.249243+0000","last_active":"2024-04-25T00:16:22.245382+0000","last_peered":"2024-04-25T00:16:22.245382+0000","last_clean":"2024-04-25T00:16:22.245382+0000","last_became_active":"2024-04-25T00:16:20.248955+0000","last_became_peered":"2024-04-25T00:16:20.248955+0000","last_unstale":"2024-04-25T00:16:22.245382+0000","last_undegraded":"2024-04-25T00:16:22.245382+0000","last_fullsized":"2024-04-25T00:16:22.245382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:14.195688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233233+0000","last_change":"2024-04-25T00:16:19.316409+0000","last_active":"2024-04-25T00:16:23.233233+0000","last_peered":"2024-04-25T00:16:23.233233+0000","last_clean":"2024-04-25T00:16:23.233233+0000","last_became_active":"2024-04-25T00:16:19.315982+0000","last_became_peered":"2024-04-25T00:16:19.315982+0000","last_unstale":"2024-04-25T00:16:23.233233+0000","last_undegraded":"2024-04-25T00:16:23.233233+0000","last_fullsized":"2024-04-25T00:16:23.233233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.718600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:22.245225+0000","last_change":"2024-04-25T00:16:20.242119+0000","last_active":"2024-04-25T00:16:22.245225+0000","last_peered":"2024-04-25T00:16:22.245225+0000","last_clean":"2024-04-25T00:16:22.245225+0000","last_became_active":"2024-04-25T00:16:20.241702+0000","last_became_peered":"2024-04-25T00:16:20.241702+0000","last_unstale":"2024-04-25T00:16:22.245225+0000","last_undegraded":"2024-04-25T00:16:22.245225+0000","last_fullsized":"2024-04-25T00:16:22.245225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:26.652681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.377959+0000","last_change":"2024-04-25T00:16:20.220822+0000","last_active":"2024-04-25T00:16:23.377959+0000","last_peered":"2024-04-25T00:16:23.377959+0000","last_clean":"2024-04-25T00:16:23.377959+0000","last_became_active":"2024-04-25T00:16:20.220420+0000","last_became_peered":"2024-04-25T00:16:20.220420+0000","last_unstale":"2024-04-25T00:16:23.377959+0000","last_undegraded":"2024-04-25T00:16:23.377959+0000","last_fullsized":"2024-04-25T00:16:23.377959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:01.032770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.327022+0000","last_change":"2024-04-25T00:16:19.328872+0000","last_active":"2024-04-25T00:16:24.327022+0000","last_peered":"2024-04-25T00:16:24.327022+0000","last_clean":"2024-04-25T00:16:24.327022+0000","last_became_active":"2024-04-25T00:16:19.328439+0000","last_became_peered":"2024-04-25T00:16:19.328439+0000","last_unstale":"2024-04-25T00:16:24.327022+0000","last_undegraded":"2024-04-25T00:16:24.327022+0000","last_fullsized":"2024-04-25T00:16:24.327022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:05.832328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.226801+0000","last_change":"2024-04-25T00:16:20.242857+0000","last_active":"2024-04-25T00:16:21.226801+0000","last_peered":"2024-04-25T00:16:21.226801+0000","last_clean":"2024-04-25T00:16:21.226801+0000","last_became_active":"2024-04-25T00:16:20.240941+0000","last_became_peered":"2024-04-25T00:16:20.240941+0000","last_unstale":"2024-04-25T00:16:21.226801+0000","last_undegraded":"2024-04-25T00:16:21.226801+0000","last_fullsized":"2024-04-25T00:16:21.226801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:36.550571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.326012+0000","last_change":"2024-04-25T00:16:19.312914+0000","last_active":"2024-04-25T00:16:24.326012+0000","last_peered":"2024-04-25T00:16:24.326012+0000","last_clean":"2024-04-25T00:16:24.326012+0000","last_became_active":"2024-04-25T00:16:19.312578+0000","last_became_peered":"2024-04-25T00:16:19.312578+0000","last_unstale":"2024-04-25T00:16:24.326012+0000","last_undegraded":"2024-04-25T00:16:24.326012+0000","last_fullsized":"2024-04-25T00:16:24.326012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:37.035904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":339,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233053+0000","last_change":"2024-04-25T00:15:03.261320+0000","last_active":"2024-04-25T00:16:23.233053+0000","last_peered":"2024-04-25T00:16:23.233053+0000","last_clean":"2024-04-25T00:16:23.233053+0000","last_became_active":"2024-04-25T00:15:03.260804+0000","last_became_peered":"2024-04-25T00:15:03.260804+0000","last_unstale":"2024-04-25T00:16:23.233053+0000","last_undegraded":"2024-04-25T00:16:23.233053+0000","last_fullsized":"2024-04-25T00:16:23.233053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:14.473933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:14.473933+0000","last_clean_scrub_stamp":"2024-04-25T00:11:14.473933+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:35.876949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.382876+0000","last_change":"2024-04-25T00:16:20.238253+0000","last_active":"2024-04-25T00:16:23.382876+0000","last_peered":"2024-04-25T00:16:23.382876+0000","last_clean":"2024-04-25T00:16:23.382876+0000","last_became_active":"2024-04-25T00:16:20.237824+0000","last_became_peered":"2024-04-25T00:16:20.237824+0000","last_unstale":"2024-04-25T00:16:23.382876+0000","last_undegraded":"2024-04-25T00:16:23.382876+0000","last_fullsized":"2024-04-25T00:16:23.382876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:21.996513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.320490+0000","last_change":"2024-04-25T00:16:19.316194+0000","last_active":"2024-04-25T00:16:24.320490+0000","last_peered":"2024-04-25T00:16:24.320490+0000","last_clean":"2024-04-25T00:16:24.320490+0000","last_became_active":"2024-04-25T00:16:19.315625+0000","last_became_peered":"2024-04-25T00:16:19.315625+0000","last_unstale":"2024-04-25T00:16:24.320490+0000","last_undegraded":"2024-04-25T00:16:24.320490+0000","last_fullsized":"2024-04-25T00:16:24.320490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:23.426048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.379375+0000","last_change":"2024-04-25T00:16:19.324283+0000","last_active":"2024-04-25T00:16:23.379375+0000","last_peered":"2024-04-25T00:16:23.379375+0000","last_clean":"2024-04-25T00:16:23.379375+0000","last_became_active":"2024-04-25T00:16:19.323800+0000","last_became_peered":"2024-04-25T00:16:19.323800+0000","last_unstale":"2024-04-25T00:16:23.379375+0000","last_undegraded":"2024-04-25T00:16:23.379375+0000","last_fullsized":"2024-04-25T00:16:23.379375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:43.592775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383803+0000","last_change":"2024-04-25T00:16:20.239205+0000","last_active":"2024-04-25T00:16:23.383803+0000","last_peered":"2024-04-25T00:16:23.383803+0000","last_clean":"2024-04-25T00:16:23.383803+0000","last_became_active":"2024-04-25T00:16:20.237135+0000","last_became_peered":"2024-04-25T00:16:20.237135+0000","last_unstale":"2024-04-25T00:16:23.383803+0000","last_undegraded":"2024-04-25T00:16:23.383803+0000","last_fullsized":"2024-04-25T00:16:23.383803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:37.348227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.324981+0000","last_change":"2024-04-25T00:16:19.325868+0000","last_active":"2024-04-25T00:16:24.324981+0000","last_peered":"2024-04-25T00:16:24.324981+0000","last_clean":"2024-04-25T00:16:24.324981+0000","last_became_active":"2024-04-25T00:16:19.322820+0000","last_became_peered":"2024-04-25T00:16:19.322820+0000","last_unstale":"2024-04-25T00:16:24.324981+0000","last_undegraded":"2024-04-25T00:16:24.324981+0000","last_fullsized":"2024-04-25T00:16:24.324981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:50.982099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.226683+0000","last_change":"2024-04-25T00:16:20.248268+0000","last_active":"2024-04-25T00:16:21.226683+0000","last_peered":"2024-04-25T00:16:21.226683+0000","last_clean":"2024-04-25T00:16:21.226683+0000","last_became_active":"2024-04-25T00:16:20.247274+0000","last_became_peered":"2024-04-25T00:16:20.247274+0000","last_unstale":"2024-04-25T00:16:21.226683+0000","last_undegraded":"2024-04-25T00:16:21.226683+0000","last_fullsized":"2024-04-25T00:16:21.226683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:46.120277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.380422+0000","last_change":"2024-04-25T00:16:19.317256+0000","last_active":"2024-04-25T00:16:23.380422+0000","last_peered":"2024-04-25T00:16:23.380422+0000","last_clean":"2024-04-25T00:16:23.380422+0000","last_became_active":"2024-04-25T00:16:19.316931+0000","last_became_peered":"2024-04-25T00:16:19.316931+0000","last_unstale":"2024-04-25T00:16:23.380422+0000","last_undegraded":"2024-04-25T00:16:23.380422+0000","last_fullsized":"2024-04-25T00:16:23.380422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:49.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700421+0000","last_change":"2024-04-25T00:16:20.252271+0000","last_active":"2024-04-25T00:16:21.700421+0000","last_peered":"2024-04-25T00:16:21.700421+0000","last_clean":"2024-04-25T00:16:21.700421+0000","last_became_active":"2024-04-25T00:16:20.251930+0000","last_became_peered":"2024-04-25T00:16:20.251930+0000","last_unstale":"2024-04-25T00:16:21.700421+0000","last_undegraded":"2024-04-25T00:16:21.700421+0000","last_fullsized":"2024-04-25T00:16:21.700421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:07.313172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.381188+0000","last_change":"2024-04-25T00:16:19.309540+0000","last_active":"2024-04-25T00:16:23.381188+0000","last_peered":"2024-04-25T00:16:23.381188+0000","last_clean":"2024-04-25T00:16:23.381188+0000","last_became_active":"2024-04-25T00:16:19.309171+0000","last_became_peered":"2024-04-25T00:16:19.309171+0000","last_unstale":"2024-04-25T00:16:23.381188+0000","last_undegraded":"2024-04-25T00:16:23.381188+0000","last_fullsized":"2024-04-25T00:16:23.381188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:28.035966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.477222+0000","last_change":"2024-04-25T00:16:20.224710+0000","last_active":"2024-04-25T00:16:21.477222+0000","last_peered":"2024-04-25T00:16:21.477222+0000","last_clean":"2024-04-25T00:16:21.477222+0000","last_became_active":"2024-04-25T00:16:20.224385+0000","last_became_peered":"2024-04-25T00:16:20.224385+0000","last_unstale":"2024-04-25T00:16:21.477222+0000","last_undegraded":"2024-04-25T00:16:21.477222+0000","last_fullsized":"2024-04-25T00:16:21.477222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:29.161972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.382288+0000","last_change":"2024-04-25T00:16:19.327498+0000","last_active":"2024-04-25T00:16:23.382288+0000","last_peered":"2024-04-25T00:16:23.382288+0000","last_clean":"2024-04-25T00:16:23.382288+0000","last_became_active":"2024-04-25T00:16:19.326889+0000","last_became_peered":"2024-04-25T00:16:19.326889+0000","last_unstale":"2024-04-25T00:16:23.382288+0000","last_undegraded":"2024-04-25T00:16:23.382288+0000","last_fullsized":"2024-04-25T00:16:23.382288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:05.275999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376234+0000","last_change":"2024-04-25T00:16:20.242150+0000","last_active":"2024-04-25T00:16:23.376234+0000","last_peered":"2024-04-25T00:16:23.376234+0000","last_clean":"2024-04-25T00:16:23.376234+0000","last_became_active":"2024-04-25T00:16:20.241770+0000","last_became_peered":"2024-04-25T00:16:20.241770+0000","last_unstale":"2024-04-25T00:16:23.376234+0000","last_undegraded":"2024-04-25T00:16:23.376234+0000","last_fullsized":"2024-04-25T00:16:23.376234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:38.038980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375743+0000","last_change":"2024-04-25T00:16:19.328774+0000","last_active":"2024-04-25T00:16:23.375743+0000","last_peered":"2024-04-25T00:16:23.375743+0000","last_clean":"2024-04-25T00:16:23.375743+0000","last_became_active":"2024-04-25T00:16:19.328466+0000","last_became_peered":"2024-04-25T00:16:19.328466+0000","last_unstale":"2024-04-25T00:16:23.375743+0000","last_undegraded":"2024-04-25T00:16:23.375743+0000","last_fullsized":"2024-04-25T00:16:23.375743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:35.967205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.226962+0000","last_change":"2024-04-25T00:16:20.244330+0000","last_active":"2024-04-25T00:16:21.226962+0000","last_peered":"2024-04-25T00:16:21.226962+0000","last_clean":"2024-04-25T00:16:21.226962+0000","last_became_active":"2024-04-25T00:16:20.243946+0000","last_became_peered":"2024-04-25T00:16:20.243946+0000","last_unstale":"2024-04-25T00:16:21.226962+0000","last_undegraded":"2024-04-25T00:16:21.226962+0000","last_fullsized":"2024-04-25T00:16:21.226962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:37.790655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.378998+0000","last_change":"2024-04-25T00:16:19.313708+0000","last_active":"2024-04-25T00:16:23.378998+0000","last_peered":"2024-04-25T00:16:23.378998+0000","last_clean":"2024-04-25T00:16:23.378998+0000","last_became_active":"2024-04-25T00:16:19.311700+0000","last_became_peered":"2024-04-25T00:16:19.311700+0000","last_unstale":"2024-04-25T00:16:23.378998+0000","last_undegraded":"2024-04-25T00:16:23.378998+0000","last_fullsized":"2024-04-25T00:16:23.378998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:24.656229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233366+0000","last_change":"2024-04-25T00:16:20.240828+0000","last_active":"2024-04-25T00:16:23.233366+0000","last_peered":"2024-04-25T00:16:23.233366+0000","last_clean":"2024-04-25T00:16:23.233366+0000","last_became_active":"2024-04-25T00:16:20.240495+0000","last_became_peered":"2024-04-25T00:16:20.240495+0000","last_unstale":"2024-04-25T00:16:23.233366+0000","last_undegraded":"2024-04-25T00:16:23.233366+0000","last_fullsized":"2024-04-25T00:16:23.233366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:19.730968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376542+0000","last_change":"2024-04-25T00:16:19.324069+0000","last_active":"2024-04-25T00:16:23.376542+0000","last_peered":"2024-04-25T00:16:23.376542+0000","last_clean":"2024-04-25T00:16:23.376542+0000","last_became_active":"2024-04-25T00:16:19.323604+0000","last_became_peered":"2024-04-25T00:16:19.323604+0000","last_unstale":"2024-04-25T00:16:23.376542+0000","last_undegraded":"2024-04-25T00:16:23.376542+0000","last_fullsized":"2024-04-25T00:16:23.376542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:50.908834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233562+0000","last_change":"2024-04-25T00:16:20.220555+0000","last_active":"2024-04-25T00:16:23.233562+0000","last_peered":"2024-04-25T00:16:23.233562+0000","last_clean":"2024-04-25T00:16:23.233562+0000","last_became_active":"2024-04-25T00:16:20.219986+0000","last_became_peered":"2024-04-25T00:16:20.219986+0000","last_unstale":"2024-04-25T00:16:23.233562+0000","last_undegraded":"2024-04-25T00:16:23.233562+0000","last_fullsized":"2024-04-25T00:16:23.233562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:04.095555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375941+0000","last_change":"2024-04-25T00:16:19.328270+0000","last_active":"2024-04-25T00:16:23.375941+0000","last_peered":"2024-04-25T00:16:23.375941+0000","last_clean":"2024-04-25T00:16:23.375941+0000","last_became_active":"2024-04-25T00:16:19.327929+0000","last_became_peered":"2024-04-25T00:16:19.327929+0000","last_unstale":"2024-04-25T00:16:23.375941+0000","last_undegraded":"2024-04-25T00:16:23.375941+0000","last_fullsized":"2024-04-25T00:16:23.375941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:09.233345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233250+0000","last_change":"2024-04-25T00:16:20.237103+0000","last_active":"2024-04-25T00:16:23.233250+0000","last_peered":"2024-04-25T00:16:23.233250+0000","last_clean":"2024-04-25T00:16:23.233250+0000","last_became_active":"2024-04-25T00:16:20.236811+0000","last_became_peered":"2024-04-25T00:16:20.236811+0000","last_unstale":"2024-04-25T00:16:23.233250+0000","last_undegraded":"2024-04-25T00:16:23.233250+0000","last_fullsized":"2024-04-25T00:16:23.233250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:56.833659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383973+0000","last_change":"2024-04-25T00:16:19.323227+0000","last_active":"2024-04-25T00:16:23.383973+0000","last_peered":"2024-04-25T00:16:23.383973+0000","last_clean":"2024-04-25T00:16:23.383973+0000","last_became_active":"2024-04-25T00:16:19.322074+0000","last_became_peered":"2024-04-25T00:16:19.322074+0000","last_unstale":"2024-04-25T00:16:23.383973+0000","last_undegraded":"2024-04-25T00:16:23.383973+0000","last_fullsized":"2024-04-25T00:16:23.383973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:53.316171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233535+0000","last_change":"2024-04-25T00:16:20.222049+0000","last_active":"2024-04-25T00:16:23.233535+0000","last_peered":"2024-04-25T00:16:23.233535+0000","last_clean":"2024-04-25T00:16:23.233535+0000","last_became_active":"2024-04-25T00:16:20.221651+0000","last_became_peered":"2024-04-25T00:16:20.221651+0000","last_unstale":"2024-04-25T00:16:23.233535+0000","last_undegraded":"2024-04-25T00:16:23.233535+0000","last_fullsized":"2024-04-25T00:16:23.233535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:31.244463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376411+0000","last_change":"2024-04-25T00:16:20.223864+0000","last_active":"2024-04-25T00:16:23.376411+0000","last_peered":"2024-04-25T00:16:23.376411+0000","last_clean":"2024-04-25T00:16:23.376411+0000","last_became_active":"2024-04-25T00:16:20.223533+0000","last_became_peered":"2024-04-25T00:16:20.223533+0000","last_unstale":"2024-04-25T00:16:23.376411+0000","last_undegraded":"2024-04-25T00:16:23.376411+0000","last_fullsized":"2024-04-25T00:16:23.376411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:14.482038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376410+0000","last_change":"2024-04-25T00:16:20.214232+0000","last_active":"2024-04-25T00:16:23.376410+0000","last_peered":"2024-04-25T00:16:23.376410+0000","last_clean":"2024-04-25T00:16:23.376410+0000","last_became_active":"2024-04-25T00:16:20.213871+0000","last_became_peered":"2024-04-25T00:16:20.213871+0000","last_unstale":"2024-04-25T00:16:23.376410+0000","last_undegraded":"2024-04-25T00:16:23.376410+0000","last_fullsized":"2024-04-25T00:16:23.376410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:07.049327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375661+0000","last_change":"2024-04-25T00:16:20.232935+0000","last_active":"2024-04-25T00:16:23.375661+0000","last_peered":"2024-04-25T00:16:23.375661+0000","last_clean":"2024-04-25T00:16:23.375661+0000","last_became_active":"2024-04-25T00:16:20.232632+0000","last_became_peered":"2024-04-25T00:16:20.232632+0000","last_unstale":"2024-04-25T00:16:23.375661+0000","last_undegraded":"2024-04-25T00:16:23.375661+0000","last_fullsized":"2024-04-25T00:16:23.375661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:36.724694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700075+0000","last_change":"2024-04-25T00:16:20.244382+0000","last_active":"2024-04-25T00:16:21.700075+0000","last_peered":"2024-04-25T00:16:21.700075+0000","last_clean":"2024-04-25T00:16:21.700075+0000","last_became_active":"2024-04-25T00:16:20.241866+0000","last_became_peered":"2024-04-25T00:16:20.241866+0000","last_unstale":"2024-04-25T00:16:21.700075+0000","last_undegraded":"2024-04-25T00:16:21.700075+0000","last_fullsized":"2024-04-25T00:16:21.700075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:24.170878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383977+0000","last_change":"2024-04-25T00:16:20.241343+0000","last_active":"2024-04-25T00:16:23.383977+0000","last_peered":"2024-04-25T00:16:23.383977+0000","last_clean":"2024-04-25T00:16:23.383977+0000","last_became_active":"2024-04-25T00:16:20.238417+0000","last_became_peered":"2024-04-25T00:16:20.238417+0000","last_unstale":"2024-04-25T00:16:23.383977+0000","last_undegraded":"2024-04-25T00:16:23.383977+0000","last_fullsized":"2024-04-25T00:16:23.383977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:29.988773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376842+0000","last_change":"2024-04-25T00:16:20.219966+0000","last_active":"2024-04-25T00:16:23.376842+0000","last_peered":"2024-04-25T00:16:23.376842+0000","last_clean":"2024-04-25T00:16:23.376842+0000","last_became_active":"2024-04-25T00:16:20.219656+0000","last_became_peered":"2024-04-25T00:16:20.219656+0000","last_unstale":"2024-04-25T00:16:23.376842+0000","last_undegraded":"2024-04-25T00:16:23.376842+0000","last_fullsized":"2024-04-25T00:16:23.376842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:56.290610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376180+0000","last_change":"2024-04-25T00:16:20.219141+0000","last_active":"2024-04-25T00:16:23.376180+0000","last_peered":"2024-04-25T00:16:23.376180+0000","last_clean":"2024-04-25T00:16:23.376180+0000","last_became_active":"2024-04-25T00:16:20.218842+0000","last_became_peered":"2024-04-25T00:16:20.218842+0000","last_unstale":"2024-04-25T00:16:23.376180+0000","last_undegraded":"2024-04-25T00:16:23.376180+0000","last_fullsized":"2024-04-25T00:16:23.376180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:33.938226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.226186+0000","last_change":"2024-04-25T00:16:20.249183+0000","last_active":"2024-04-25T00:16:21.226186+0000","last_peered":"2024-04-25T00:16:21.226186+0000","last_clean":"2024-04-25T00:16:21.226186+0000","last_became_active":"2024-04-25T00:16:20.248355+0000","last_became_peered":"2024-04-25T00:16:20.248355+0000","last_unstale":"2024-04-25T00:16:21.226186+0000","last_undegraded":"2024-04-25T00:16:21.226186+0000","last_fullsized":"2024-04-25T00:16:21.226186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:57.738041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.377447+0000","last_change":"2024-04-25T00:16:20.227528+0000","last_active":"2024-04-25T00:16:23.377447+0000","last_peered":"2024-04-25T00:16:23.377447+0000","last_clean":"2024-04-25T00:16:23.377447+0000","last_became_active":"2024-04-25T00:16:20.227208+0000","last_became_peered":"2024-04-25T00:16:20.227208+0000","last_unstale":"2024-04-25T00:16:23.377447+0000","last_undegraded":"2024-04-25T00:16:23.377447+0000","last_fullsized":"2024-04-25T00:16:23.377447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:49.920850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.225735+0000","last_change":"2024-04-25T00:16:20.228157+0000","last_active":"2024-04-25T00:16:21.225735+0000","last_peered":"2024-04-25T00:16:21.225735+0000","last_clean":"2024-04-25T00:16:21.225735+0000","last_became_active":"2024-04-25T00:16:20.227659+0000","last_became_peered":"2024-04-25T00:16:20.227659+0000","last_unstale":"2024-04-25T00:16:21.225735+0000","last_undegraded":"2024-04-25T00:16:21.225735+0000","last_fullsized":"2024-04-25T00:16:21.225735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:30.579407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375515+0000","last_change":"2024-04-25T00:16:20.232237+0000","last_active":"2024-04-25T00:16:23.375515+0000","last_peered":"2024-04-25T00:16:23.375515+0000","last_clean":"2024-04-25T00:16:23.375515+0000","last_became_active":"2024-04-25T00:16:20.231897+0000","last_became_peered":"2024-04-25T00:16:20.231897+0000","last_unstale":"2024-04-25T00:16:23.375515+0000","last_undegraded":"2024-04-25T00:16:23.375515+0000","last_fullsized":"2024-04-25T00:16:23.375515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:23.919515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383588+0000","last_change":"2024-04-25T00:16:20.243454+0000","last_active":"2024-04-25T00:16:23.383588+0000","last_peered":"2024-04-25T00:16:23.383588+0000","last_clean":"2024-04-25T00:16:23.383588+0000","last_became_active":"2024-04-25T00:16:20.243112+0000","last_became_peered":"2024-04-25T00:16:20.243112+0000","last_unstale":"2024-04-25T00:16:23.383588+0000","last_undegraded":"2024-04-25T00:16:23.383588+0000","last_fullsized":"2024-04-25T00:16:23.383588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:58.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.377159+0000","last_change":"2024-04-25T00:16:20.220947+0000","last_active":"2024-04-25T00:16:23.377159+0000","last_peered":"2024-04-25T00:16:23.377159+0000","last_clean":"2024-04-25T00:16:23.377159+0000","last_became_active":"2024-04-25T00:16:20.220662+0000","last_became_peered":"2024-04-25T00:16:20.220662+0000","last_unstale":"2024-04-25T00:16:23.377159+0000","last_undegraded":"2024-04-25T00:16:23.377159+0000","last_fullsized":"2024-04-25T00:16:23.377159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:44.840549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":28,"num_read_kb":21,"num_write":117,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":14972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":444893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1037869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":445095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037791,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":445049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1536000,"data_stored":1036471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":444401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":445923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":445789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":446579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":444712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":444955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36848,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708304,"statfs":{"total":95995035648,"available":95957303296,"internally_reserved":0,"allocated":1548288,"data_stored":1036155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:16:29.316 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T00:16:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.572+0000 7f5cd9c00640 1 -- 172.21.15.42:0/249822507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4104b80 msgr2=0x7f5cd4105000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.572+0000 7f5cd9c00640 1 --2- 172.21.15.42:0/249822507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4104b80 0x7f5cd4105000 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009900 tx=0x7f5cbc02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.573+0000 7f5cd9c00640 1 -- 172.21.15.42:0/249822507 shutdown_connections 2024-04-25T00:16:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.573+0000 7f5cd9c00640 1 --2- 172.21.15.42:0/249822507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cd4105540 0x7f5cd410bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.573+0000 7f5cd9c00640 1 --2- 172.21.15.42:0/249822507 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4104b80 0x7f5cd4105000 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.573+0000 7f5cd9c00640 1 --2- 172.21.15.42:0/249822507 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd4103990 0x7f5cd4103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.573+0000 7f5cd9c00640 1 -- 172.21.15.42:0/249822507 >> 172.21.15.42:0/249822507 conn(0x7f5cd40fef00 msgr2=0x7f5cd4101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:29.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.573+0000 7f5cd9c00640 1 -- 172.21.15.42:0/249822507 shutdown_connections 2024-04-25T00:16:29.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.573+0000 7f5cd9c00640 1 -- 172.21.15.42:0/249822507 wait complete. 2024-04-25T00:16:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd9c00640 1 Processor -- start 2024-04-25T00:16:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd9c00640 1 -- start start 2024-04-25T00:16:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4103990 0x7f5cd4199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd9c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd4104b80 0x7f5cd419a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cd4105540 0x7f5cd41a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd9c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410f610 con 0x7f5cd4103990 2024-04-25T00:16:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd9c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410f490 con 0x7f5cd4104b80 2024-04-25T00:16:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd9c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd410f790 con 0x7f5cd4105540 2024-04-25T00:16:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4103990 0x7f5cd4199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4103990 0x7f5cd4199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37250/0 (socket says 172.21.15.42:37250) 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.574+0000 7f5cd2a00640 1 -- 172.21.15.42:0/2298745435 learned_addr learned my addr 172.21.15.42:0/2298745435 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5cd3400640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cd4105540 0x7f5cd41a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5cd2000640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd4104b80 0x7f5cd419a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5cd2a00640 1 -- 172.21.15.42:0/2298745435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cd4105540 msgr2=0x7f5cd41a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5cd2a00640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cd4105540 0x7f5cd41a1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5cd2a00640 1 -- 172.21.15.42:0/2298745435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd4104b80 msgr2=0x7f5cd419a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5cd2a00640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd4104b80 0x7f5cd419a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5cd2a00640 1 -- 172.21.15.42:0/2298745435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd41a18c0 con 0x7f5cd4103990 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5cd2a00640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4103990 0x7f5cd4199c80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400ebc0 tx=0x7f5cc400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5ccb400640 1 -- 172.21.15.42:0/2298745435 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4003b70 con 0x7f5cd4103990 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5ccb400640 1 -- 172.21.15.42:0/2298745435 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cc4003d10 con 0x7f5cd4103990 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.575+0000 7f5cd9c00640 1 -- 172.21.15.42:0/2298745435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd41a1bb0 con 0x7f5cd4103990 2024-04-25T00:16:29.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.576+0000 7f5ccb400640 1 -- 172.21.15.42:0/2298745435 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc4019980 con 0x7f5cd4103990 2024-04-25T00:16:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.576+0000 7f5cd9c00640 1 -- 172.21.15.42:0/2298745435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd41a20d0 con 0x7f5cd4103990 2024-04-25T00:16:29.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.577+0000 7f5cd9c00640 1 -- 172.21.15.42:0/2298745435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd410acb0 con 0x7f5cd4103990 2024-04-25T00:16:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.579+0000 7f5ccb400640 1 -- 172.21.15.42:0/2298745435 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5cc4019b20 con 0x7f5cd4103990 2024-04-25T00:16:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.580+0000 7f5ccb400640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5cb80a2c20 0x7f5cb80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.581+0000 7f5ccb400640 1 -- 172.21.15.42:0/2298745435 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5cc40cf830 con 0x7f5cd4103990 2024-04-25T00:16:29.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.581+0000 7f5cd2000640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5cb80a2c20 0x7f5cb80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:29.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.581+0000 7f5cd2000640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5cb80a2c20 0x7f5cb80a50e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0314e0 tx=0x7f5cbc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:29.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.585+0000 7f5ccb400640 1 -- 172.21.15.42:0/2298745435 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5cc409c7e0 con 0x7f5cd4103990 2024-04-25T00:16:29.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:29 smithi042 ceph-mon[28130]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:29.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:29 smithi042 ceph-mon[28130]: pgmap v317: 97 pgs: 97 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.6 KiB/s wr, 18 op/s 2024-04-25T00:16:29.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:29 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1542715561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:29 smithi136 ceph-mon[32923]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:29 smithi136 ceph-mon[32923]: pgmap v317: 97 pgs: 97 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.6 KiB/s wr, 18 op/s 2024-04-25T00:16:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:29 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1542715561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:29.786 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.786+0000 7f5cd9c00640 1 -- 172.21.15.42:0/2298745435 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5cd4061b80 con 0x7f5cb80a2c20 2024-04-25T00:16:29.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.798+0000 7f5ccb400640 1 -- 172.21.15.42:0/2298745435 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258739 (secure 0 0 0) 0x7f5cd4061b80 con 0x7f5cb80a2c20 2024-04-25T00:16:29.799 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:29.803 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-25T00:16:29.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 -- 172.21.15.42:0/2298745435 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5cb80a2c20 msgr2=0x7f5cb80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5cb80a2c20 0x7f5cb80a50e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc0314e0 tx=0x7f5cbc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 -- 172.21.15.42:0/2298745435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4103990 msgr2=0x7f5cd4199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:29.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4103990 0x7f5cd4199c80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5cc400ebc0 tx=0x7f5cc400c6a0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 -- 172.21.15.42:0/2298745435 shutdown_connections 2024-04-25T00:16:29.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5cb80a2c20 0x7f5cb80a50e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5cd4105540 0x7f5cd41a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cd4104b80 0x7f5cd419a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 --2- 172.21.15.42:0/2298745435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5cd4103990 0x7f5cd4199c80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:29.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 -- 172.21.15.42:0/2298745435 >> 172.21.15.42:0/2298745435 conn(0x7f5cd40fef00 msgr2=0x7f5cd4107e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:29.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 -- 172.21.15.42:0/2298745435 shutdown_connections 2024-04-25T00:16:29.806 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:29.805+0000 7f5cd9c00640 1 -- 172.21.15.42:0/2298745435 wait complete. 2024-04-25T00:16:29.825 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-04-25T00:16:27.911057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":58,"num_object_clones":0,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":0,"num_read":154,"num_read_kb":130,"num_write":298,"num_write_kb":2975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":240,"ondisk_log_size":240,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":357836,"kb_used_data":12816,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583988,"statfs":{"total":1151940427776,"available":1151574003712,"internally_reserved":0,"allocated":13123584,"data_stored":7117891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4640,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":28,"num_read_kb":21,"num_write":117,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003607"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.377212+0000","last_change":"2024-04-25T00:16:20.250230+0000","last_active":"2024-04-25T00:16:23.377212+0000","last_peered":"2024-04-25T00:16:23.377212+0000","last_clean":"2024-04-25T00:16:23.377212+0000","last_became_active":"2024-04-25T00:16:20.249751+0000","last_became_peered":"2024-04-25T00:16:20.249751+0000","last_unstale":"2024-04-25T00:16:23.377212+0000","last_undegraded":"2024-04-25T00:16:23.377212+0000","last_fullsized":"2024-04-25T00:16:23.377212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:57.895905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700589+0000","last_change":"2024-04-25T00:16:20.237934+0000","last_active":"2024-04-25T00:16:21.700589+0000","last_peered":"2024-04-25T00:16:21.700589+0000","last_clean":"2024-04-25T00:16:21.700589+0000","last_became_active":"2024-04-25T00:16:20.236039+0000","last_became_peered":"2024-04-25T00:16:20.236039+0000","last_unstale":"2024-04-25T00:16:21.700589+0000","last_undegraded":"2024-04-25T00:16:21.700589+0000","last_fullsized":"2024-04-25T00:16:21.700589+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:07.730977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700290+0000","last_change":"2024-04-25T00:16:20.248923+0000","last_active":"2024-04-25T00:16:21.700290+0000","last_peered":"2024-04-25T00:16:21.700290+0000","last_clean":"2024-04-25T00:16:21.700290+0000","last_became_active":"2024-04-25T00:16:20.248078+0000","last_became_peered":"2024-04-25T00:16:20.248078+0000","last_unstale":"2024-04-25T00:16:21.700290+0000","last_undegraded":"2024-04-25T00:16:21.700290+0000","last_fullsized":"2024-04-25T00:16:21.700290+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:25.675449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.229280+0000","last_change":"2024-04-25T00:16:20.219294+0000","last_active":"2024-04-25T00:16:21.229280+0000","last_peered":"2024-04-25T00:16:21.229280+0000","last_clean":"2024-04-25T00:16:21.229280+0000","last_became_active":"2024-04-25T00:16:20.218990+0000","last_became_peered":"2024-04-25T00:16:20.218990+0000","last_unstale":"2024-04-25T00:16:21.229280+0000","last_undegraded":"2024-04-25T00:16:21.229280+0000","last_fullsized":"2024-04-25T00:16:21.229280+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:30:07.201406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.477150+0000","last_change":"2024-04-25T00:16:20.238311+0000","last_active":"2024-04-25T00:16:21.477150+0000","last_peered":"2024-04-25T00:16:21.477150+0000","last_clean":"2024-04-25T00:16:21.477150+0000","last_became_active":"2024-04-25T00:16:20.238104+0000","last_became_peered":"2024-04-25T00:16:20.238104+0000","last_unstale":"2024-04-25T00:16:21.477150+0000","last_undegraded":"2024-04-25T00:16:21.477150+0000","last_fullsized":"2024-04-25T00:16:21.477150+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:42.459941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:22.245080+0000","last_change":"2024-04-25T00:16:20.228307+0000","last_active":"2024-04-25T00:16:22.245080+0000","last_peered":"2024-04-25T00:16:22.245080+0000","last_clean":"2024-04-25T00:16:22.245080+0000","last_became_active":"2024-04-25T00:16:20.227999+0000","last_became_peered":"2024-04-25T00:16:20.227999+0000","last_unstale":"2024-04-25T00:16:22.245080+0000","last_undegraded":"2024-04-25T00:16:22.245080+0000","last_fullsized":"2024-04-25T00:16:22.245080+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:57.910589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233060+0000","last_change":"2024-04-25T00:16:20.224382+0000","last_active":"2024-04-25T00:16:23.233060+0000","last_peered":"2024-04-25T00:16:23.233060+0000","last_clean":"2024-04-25T00:16:23.233060+0000","last_became_active":"2024-04-25T00:16:20.224122+0000","last_became_peered":"2024-04-25T00:16:20.224122+0000","last_unstale":"2024-04-25T00:16:23.233060+0000","last_undegraded":"2024-04-25T00:16:23.233060+0000","last_fullsized":"2024-04-25T00:16:23.233060+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:19.651330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.225679+0000","last_change":"2024-04-25T00:16:20.227720+0000","last_active":"2024-04-25T00:16:21.225679+0000","last_peered":"2024-04-25T00:16:21.225679+0000","last_clean":"2024-04-25T00:16:21.225679+0000","last_became_active":"2024-04-25T00:16:20.227263+0000","last_became_peered":"2024-04-25T00:16:20.227263+0000","last_unstale":"2024-04-25T00:16:21.225679+0000","last_undegraded":"2024-04-25T00:16:21.225679+0000","last_fullsized":"2024-04-25T00:16:21.225679+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:03.709172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375891+0000","last_change":"2024-04-25T00:16:20.217673+0000","last_active":"2024-04-25T00:16:23.375891+0000","last_peered":"2024-04-25T00:16:23.375891+0000","last_clean":"2024-04-25T00:16:23.375891+0000","last_became_active":"2024-04-25T00:16:20.216773+0000","last_became_peered":"2024-04-25T00:16:20.216773+0000","last_unstale":"2024-04-25T00:16:23.375891+0000","last_undegraded":"2024-04-25T00:16:23.375891+0000","last_fullsized":"2024-04-25T00:16:23.375891+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.623501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:22.245468+0000","last_change":"2024-04-25T00:16:20.218460+0000","last_active":"2024-04-25T00:16:22.245468+0000","last_peered":"2024-04-25T00:16:22.245468+0000","last_clean":"2024-04-25T00:16:22.245468+0000","last_became_active":"2024-04-25T00:16:20.217198+0000","last_became_peered":"2024-04-25T00:16:20.217198+0000","last_unstale":"2024-04-25T00:16:22.245468+0000","last_undegraded":"2024-04-25T00:16:22.245468+0000","last_fullsized":"2024-04-25T00:16:22.245468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:06.511749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233369+0000","last_change":"2024-04-25T00:16:20.221990+0000","last_active":"2024-04-25T00:16:23.233369+0000","last_peered":"2024-04-25T00:16:23.233369+0000","last_clean":"2024-04-25T00:16:23.233369+0000","last_became_active":"2024-04-25T00:16:20.221635+0000","last_became_peered":"2024-04-25T00:16:20.221635+0000","last_unstale":"2024-04-25T00:16:23.233369+0000","last_undegraded":"2024-04-25T00:16:23.233369+0000","last_fullsized":"2024-04-25T00:16:23.233369+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:31.205135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.229240+0000","last_change":"2024-04-25T00:16:20.219923+0000","last_active":"2024-04-25T00:16:21.229240+0000","last_peered":"2024-04-25T00:16:21.229240+0000","last_clean":"2024-04-25T00:16:21.229240+0000","last_became_active":"2024-04-25T00:16:20.219578+0000","last_became_peered":"2024-04-25T00:16:20.219578+0000","last_unstale":"2024-04-25T00:16:21.229240+0000","last_undegraded":"2024-04-25T00:16:21.229240+0000","last_fullsized":"2024-04-25T00:16:21.229240+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:41.811382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383669+0000","last_change":"2024-04-25T00:16:20.240570+0000","last_active":"2024-04-25T00:16:23.383669+0000","last_peered":"2024-04-25T00:16:23.383669+0000","last_clean":"2024-04-25T00:16:23.383669+0000","last_became_active":"2024-04-25T00:16:20.238241+0000","last_became_peered":"2024-04-25T00:16:20.238241+0000","last_unstale":"2024-04-25T00:16:23.383669+0000","last_undegraded":"2024-04-25T00:16:23.383669+0000","last_fullsized":"2024-04-25T00:16:23.383669+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:06.981816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700108+0000","last_change":"2024-04-25T00:16:20.234748+0000","last_active":"2024-04-25T00:16:21.700108+0000","last_peered":"2024-04-25T00:16:21.700108+0000","last_clean":"2024-04-25T00:16:21.700108+0000","last_became_active":"2024-04-25T00:16:20.234325+0000","last_became_peered":"2024-04-25T00:16:20.234325+0000","last_unstale":"2024-04-25T00:16:21.700108+0000","last_undegraded":"2024-04-25T00:16:21.700108+0000","last_fullsized":"2024-04-25T00:16:21.700108+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:00.383533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.699878+0000","last_change":"2024-04-25T00:16:20.242987+0000","last_active":"2024-04-25T00:16:21.699878+0000","last_peered":"2024-04-25T00:16:21.699878+0000","last_clean":"2024-04-25T00:16:21.699878+0000","last_became_active":"2024-04-25T00:16:20.242625+0000","last_became_peered":"2024-04-25T00:16:20.242625+0000","last_unstale":"2024-04-25T00:16:21.699878+0000","last_undegraded":"2024-04-25T00:16:21.699878+0000","last_fullsized":"2024-04-25T00:16:21.699878+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:16.618120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375122+0000","last_change":"2024-04-25T00:16:20.241985+0000","last_active":"2024-04-25T00:16:23.375122+0000","last_peered":"2024-04-25T00:16:23.375122+0000","last_clean":"2024-04-25T00:16:23.375122+0000","last_became_active":"2024-04-25T00:16:20.241646+0000","last_became_peered":"2024-04-25T00:16:20.241646+0000","last_unstale":"2024-04-25T00:16:23.375122+0000","last_undegraded":"2024-04-25T00:16:23.375122+0000","last_fullsized":"2024-04-25T00:16:23.375122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:44.094705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.377570+0000","last_change":"2024-04-25T00:16:20.250161+0000","last_active":"2024-04-25T00:16:23.377570+0000","last_peered":"2024-04-25T00:16:23.377570+0000","last_clean":"2024-04-25T00:16:23.377570+0000","last_became_active":"2024-04-25T00:16:20.249811+0000","last_became_peered":"2024-04-25T00:16:20.249811+0000","last_unstale":"2024-04-25T00:16:23.377570+0000","last_undegraded":"2024-04-25T00:16:23.377570+0000","last_fullsized":"2024-04-25T00:16:23.377570+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:03.222602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.229066+0000","last_change":"2024-04-25T00:16:20.220859+0000","last_active":"2024-04-25T00:16:21.229066+0000","last_peered":"2024-04-25T00:16:21.229066+0000","last_clean":"2024-04-25T00:16:21.229066+0000","last_became_active":"2024-04-25T00:16:20.219864+0000","last_became_peered":"2024-04-25T00:16:20.219864+0000","last_unstale":"2024-04-25T00:16:21.229066+0000","last_undegraded":"2024-04-25T00:16:21.229066+0000","last_fullsized":"2024-04-25T00:16:21.229066+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:23.830726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375929+0000","last_change":"2024-04-25T00:16:20.249293+0000","last_active":"2024-04-25T00:16:23.375929+0000","last_peered":"2024-04-25T00:16:23.375929+0000","last_clean":"2024-04-25T00:16:23.375929+0000","last_became_active":"2024-04-25T00:16:20.248952+0000","last_became_peered":"2024-04-25T00:16:20.248952+0000","last_unstale":"2024-04-25T00:16:23.375929+0000","last_undegraded":"2024-04-25T00:16:23.375929+0000","last_fullsized":"2024-04-25T00:16:23.375929+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:02.124426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.325216+0000","last_change":"2024-04-25T00:16:19.310971+0000","last_active":"2024-04-25T00:16:24.325216+0000","last_peered":"2024-04-25T00:16:24.325216+0000","last_clean":"2024-04-25T00:16:24.325216+0000","last_became_active":"2024-04-25T00:16:19.309020+0000","last_became_peered":"2024-04-25T00:16:19.309020+0000","last_unstale":"2024-04-25T00:16:24.325216+0000","last_undegraded":"2024-04-25T00:16:24.325216+0000","last_fullsized":"2024-04-25T00:16:24.325216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:56.744138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.226036+0000","last_change":"2024-04-25T00:16:20.250665+0000","last_active":"2024-04-25T00:16:21.226036+0000","last_peered":"2024-04-25T00:16:21.226036+0000","last_clean":"2024-04-25T00:16:21.226036+0000","last_became_active":"2024-04-25T00:16:20.250338+0000","last_became_peered":"2024-04-25T00:16:20.250338+0000","last_unstale":"2024-04-25T00:16:21.226036+0000","last_undegraded":"2024-04-25T00:16:21.226036+0000","last_fullsized":"2024-04-25T00:16:21.226036+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:17.018035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.496001+0000","last_change":"2024-04-25T00:16:19.327748+0000","last_active":"2024-04-25T00:16:21.496001+0000","last_peered":"2024-04-25T00:16:21.496001+0000","last_clean":"2024-04-25T00:16:21.496001+0000","last_became_active":"2024-04-25T00:16:19.327264+0000","last_became_peered":"2024-04-25T00:16:19.327264+0000","last_unstale":"2024-04-25T00:16:21.496001+0000","last_undegraded":"2024-04-25T00:16:21.496001+0000","last_fullsized":"2024-04-25T00:16:21.496001+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:27.080423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700861+0000","last_change":"2024-04-25T00:16:20.243061+0000","last_active":"2024-04-25T00:16:21.700861+0000","last_peered":"2024-04-25T00:16:21.700861+0000","last_clean":"2024-04-25T00:16:21.700861+0000","last_became_active":"2024-04-25T00:16:20.242697+0000","last_became_peered":"2024-04-25T00:16:20.242697+0000","last_unstale":"2024-04-25T00:16:21.700861+0000","last_undegraded":"2024-04-25T00:16:21.700861+0000","last_fullsized":"2024-04-25T00:16:21.700861+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:57.734902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.320426+0000","last_change":"2024-04-25T00:16:19.317122+0000","last_active":"2024-04-25T00:16:24.320426+0000","last_peered":"2024-04-25T00:16:24.320426+0000","last_clean":"2024-04-25T00:16:24.320426+0000","last_became_active":"2024-04-25T00:16:19.316754+0000","last_became_peered":"2024-04-25T00:16:19.316754+0000","last_unstale":"2024-04-25T00:16:24.320426+0000","last_undegraded":"2024-04-25T00:16:24.320426+0000","last_fullsized":"2024-04-25T00:16:24.320426+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:28.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375611+0000","last_change":"2024-04-25T00:16:20.238611+0000","last_active":"2024-04-25T00:16:23.375611+0000","last_peered":"2024-04-25T00:16:23.375611+0000","last_clean":"2024-04-25T00:16:23.375611+0000","last_became_active":"2024-04-25T00:16:20.238257+0000","last_became_peered":"2024-04-25T00:16:20.238257+0000","last_unstale":"2024-04-25T00:16:23.375611+0000","last_undegraded":"2024-04-25T00:16:23.375611+0000","last_fullsized":"2024-04-25T00:16:23.375611+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:13.490284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.477463+0000","last_change":"2024-04-25T00:16:19.308565+0000","last_active":"2024-04-25T00:16:21.477463+0000","last_peered":"2024-04-25T00:16:21.477463+0000","last_clean":"2024-04-25T00:16:21.477463+0000","last_became_active":"2024-04-25T00:16:19.308220+0000","last_became_peered":"2024-04-25T00:16:19.308220+0000","last_unstale":"2024-04-25T00:16:21.477463+0000","last_undegraded":"2024-04-25T00:16:21.477463+0000","last_fullsized":"2024-04-25T00:16:21.477463+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:33.215491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376084+0000","last_change":"2024-04-25T00:16:20.220198+0000","last_active":"2024-04-25T00:16:23.376084+0000","last_peered":"2024-04-25T00:16:23.376084+0000","last_clean":"2024-04-25T00:16:23.376084+0000","last_became_active":"2024-04-25T00:16:20.219897+0000","last_became_peered":"2024-04-25T00:16:20.219897+0000","last_unstale":"2024-04-25T00:16:23.376084+0000","last_undegraded":"2024-04-25T00:16:23.376084+0000","last_fullsized":"2024-04-25T00:16:23.376084+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:57.617707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:25.312196+0000","last_change":"2024-04-25T00:16:19.312725+0000","last_active":"2024-04-25T00:16:25.312196+0000","last_peered":"2024-04-25T00:16:25.312196+0000","last_clean":"2024-04-25T00:16:25.312196+0000","last_became_active":"2024-04-25T00:16:19.312115+0000","last_became_peered":"2024-04-25T00:16:19.312115+0000","last_unstale":"2024-04-25T00:16:25.312196+0000","last_undegraded":"2024-04-25T00:16:25.312196+0000","last_fullsized":"2024-04-25T00:16:25.312196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.661132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":5,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.477252+0000","last_change":"2024-04-25T00:16:20.233546+0000","last_active":"2024-04-25T00:16:21.477252+0000","last_peered":"2024-04-25T00:16:21.477252+0000","last_clean":"2024-04-25T00:16:21.477252+0000","last_became_active":"2024-04-25T00:16:20.233029+0000","last_became_peered":"2024-04-25T00:16:20.233029+0000","last_unstale":"2024-04-25T00:16:21.477252+0000","last_undegraded":"2024-04-25T00:16:21.477252+0000","last_fullsized":"2024-04-25T00:16:21.477252+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:44.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.322639+0000","last_change":"2024-04-25T00:16:19.307512+0000","last_active":"2024-04-25T00:16:24.322639+0000","last_peered":"2024-04-25T00:16:24.322639+0000","last_clean":"2024-04-25T00:16:24.322639+0000","last_became_active":"2024-04-25T00:16:19.307164+0000","last_became_peered":"2024-04-25T00:16:19.307164+0000","last_unstale":"2024-04-25T00:16:24.322639+0000","last_undegraded":"2024-04-25T00:16:24.322639+0000","last_fullsized":"2024-04-25T00:16:24.322639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:37.560178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375187+0000","last_change":"2024-04-25T00:16:20.232758+0000","last_active":"2024-04-25T00:16:23.375187+0000","last_peered":"2024-04-25T00:16:23.375187+0000","last_clean":"2024-04-25T00:16:23.375187+0000","last_became_active":"2024-04-25T00:16:20.232443+0000","last_became_peered":"2024-04-25T00:16:20.232443+0000","last_unstale":"2024-04-25T00:16:23.375187+0000","last_undegraded":"2024-04-25T00:16:23.375187+0000","last_fullsized":"2024-04-25T00:16:23.375187+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:31.029585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.325943+0000","last_change":"2024-04-25T00:16:19.313924+0000","last_active":"2024-04-25T00:16:24.325943+0000","last_peered":"2024-04-25T00:16:24.325943+0000","last_clean":"2024-04-25T00:16:24.325943+0000","last_became_active":"2024-04-25T00:16:19.313570+0000","last_became_peered":"2024-04-25T00:16:19.313570+0000","last_unstale":"2024-04-25T00:16:24.325943+0000","last_undegraded":"2024-04-25T00:16:24.325943+0000","last_fullsized":"2024-04-25T00:16:24.325943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:42.258910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.225559+0000","last_change":"2024-04-25T00:16:20.244150+0000","last_active":"2024-04-25T00:16:21.225559+0000","last_peered":"2024-04-25T00:16:21.225559+0000","last_clean":"2024-04-25T00:16:21.225559+0000","last_became_active":"2024-04-25T00:16:20.243802+0000","last_became_peered":"2024-04-25T00:16:20.243802+0000","last_unstale":"2024-04-25T00:16:21.225559+0000","last_undegraded":"2024-04-25T00:16:21.225559+0000","last_fullsized":"2024-04-25T00:16:21.225559+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:10.192507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375112+0000","last_change":"2024-04-25T00:16:19.312361+0000","last_active":"2024-04-25T00:16:23.375112+0000","last_peered":"2024-04-25T00:16:23.375112+0000","last_clean":"2024-04-25T00:16:23.375112+0000","last_became_active":"2024-04-25T00:16:19.312031+0000","last_became_peered":"2024-04-25T00:16:19.312031+0000","last_unstale":"2024-04-25T00:16:23.375112+0000","last_undegraded":"2024-04-25T00:16:23.375112+0000","last_fullsized":"2024-04-25T00:16:23.375112+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:10.062091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.228514+0000","last_change":"2024-04-25T00:16:20.231399+0000","last_active":"2024-04-25T00:16:21.228514+0000","last_peered":"2024-04-25T00:16:21.228514+0000","last_clean":"2024-04-25T00:16:21.228514+0000","last_became_active":"2024-04-25T00:16:20.230954+0000","last_became_peered":"2024-04-25T00:16:20.230954+0000","last_unstale":"2024-04-25T00:16:21.228514+0000","last_undegraded":"2024-04-25T00:16:21.228514+0000","last_fullsized":"2024-04-25T00:16:21.228514+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:20.934927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.378905+0000","last_change":"2024-04-25T00:16:19.308591+0000","last_active":"2024-04-25T00:16:23.378905+0000","last_peered":"2024-04-25T00:16:23.378905+0000","last_clean":"2024-04-25T00:16:23.378905+0000","last_became_active":"2024-04-25T00:16:19.307987+0000","last_became_peered":"2024-04-25T00:16:19.307987+0000","last_unstale":"2024-04-25T00:16:23.378905+0000","last_undegraded":"2024-04-25T00:16:23.378905+0000","last_fullsized":"2024-04-25T00:16:23.378905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:32.739170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375812+0000","last_change":"2024-04-25T00:16:20.218545+0000","last_active":"2024-04-25T00:16:23.375812+0000","last_peered":"2024-04-25T00:16:23.375812+0000","last_clean":"2024-04-25T00:16:23.375812+0000","last_became_active":"2024-04-25T00:16:20.218267+0000","last_became_peered":"2024-04-25T00:16:20.218267+0000","last_unstale":"2024-04-25T00:16:23.375812+0000","last_undegraded":"2024-04-25T00:16:23.375812+0000","last_fullsized":"2024-04-25T00:16:23.375812+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:47.725985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.379011+0000","last_change":"2024-04-25T00:16:19.311918+0000","last_active":"2024-04-25T00:16:23.379011+0000","last_peered":"2024-04-25T00:16:23.379011+0000","last_clean":"2024-04-25T00:16:23.379011+0000","last_became_active":"2024-04-25T00:16:19.311594+0000","last_became_peered":"2024-04-25T00:16:19.311594+0000","last_unstale":"2024-04-25T00:16:23.379011+0000","last_undegraded":"2024-04-25T00:16:23.379011+0000","last_fullsized":"2024-04-25T00:16:23.379011+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:04.574190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700690+0000","last_change":"2024-04-25T00:16:20.248116+0000","last_active":"2024-04-25T00:16:21.700690+0000","last_peered":"2024-04-25T00:16:21.700690+0000","last_clean":"2024-04-25T00:16:21.700690+0000","last_became_active":"2024-04-25T00:16:20.247373+0000","last_became_peered":"2024-04-25T00:16:20.247373+0000","last_unstale":"2024-04-25T00:16:21.700690+0000","last_undegraded":"2024-04-25T00:16:21.700690+0000","last_fullsized":"2024-04-25T00:16:21.700690+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:47.777779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.381082+0000","last_change":"2024-04-25T00:16:19.320640+0000","last_active":"2024-04-25T00:16:23.381082+0000","last_peered":"2024-04-25T00:16:23.381082+0000","last_clean":"2024-04-25T00:16:23.381082+0000","last_became_active":"2024-04-25T00:16:19.320273+0000","last_became_peered":"2024-04-25T00:16:19.320273+0000","last_unstale":"2024-04-25T00:16:23.381082+0000","last_undegraded":"2024-04-25T00:16:23.381082+0000","last_fullsized":"2024-04-25T00:16:23.381082+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:10.843162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375646+0000","last_change":"2024-04-25T00:16:20.219378+0000","last_active":"2024-04-25T00:16:23.375646+0000","last_peered":"2024-04-25T00:16:23.375646+0000","last_clean":"2024-04-25T00:16:23.375646+0000","last_became_active":"2024-04-25T00:16:20.219069+0000","last_became_peered":"2024-04-25T00:16:20.219069+0000","last_unstale":"2024-04-25T00:16:23.375646+0000","last_undegraded":"2024-04-25T00:16:23.375646+0000","last_fullsized":"2024-04-25T00:16:23.375646+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:10.818609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.310119+0000","last_change":"2024-04-25T00:16:19.315089+0000","last_active":"2024-04-25T00:16:24.310119+0000","last_peered":"2024-04-25T00:16:24.310119+0000","last_clean":"2024-04-25T00:16:24.310119+0000","last_became_active":"2024-04-25T00:16:19.314416+0000","last_became_peered":"2024-04-25T00:16:19.314416+0000","last_unstale":"2024-04-25T00:16:24.310119+0000","last_undegraded":"2024-04-25T00:16:24.310119+0000","last_fullsized":"2024-04-25T00:16:24.310119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:25.920010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.477370+0000","last_change":"2024-04-25T00:16:20.225678+0000","last_active":"2024-04-25T00:16:21.477370+0000","last_peered":"2024-04-25T00:16:21.477370+0000","last_clean":"2024-04-25T00:16:21.477370+0000","last_became_active":"2024-04-25T00:16:20.225146+0000","last_became_peered":"2024-04-25T00:16:20.225146+0000","last_unstale":"2024-04-25T00:16:21.477370+0000","last_undegraded":"2024-04-25T00:16:21.477370+0000","last_fullsized":"2024-04-25T00:16:21.477370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:56.003701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.382921+0000","last_change":"2024-04-25T00:16:19.327877+0000","last_active":"2024-04-25T00:16:23.382921+0000","last_peered":"2024-04-25T00:16:23.382921+0000","last_clean":"2024-04-25T00:16:23.382921+0000","last_became_active":"2024-04-25T00:16:19.327537+0000","last_became_peered":"2024-04-25T00:16:19.327537+0000","last_unstale":"2024-04-25T00:16:23.382921+0000","last_undegraded":"2024-04-25T00:16:23.382921+0000","last_fullsized":"2024-04-25T00:16:23.382921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:14.614232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383713+0000","last_change":"2024-04-25T00:16:20.253101+0000","last_active":"2024-04-25T00:16:23.383713+0000","last_peered":"2024-04-25T00:16:23.383713+0000","last_clean":"2024-04-25T00:16:23.383713+0000","last_became_active":"2024-04-25T00:16:20.252735+0000","last_became_peered":"2024-04-25T00:16:20.252735+0000","last_unstale":"2024-04-25T00:16:23.383713+0000","last_undegraded":"2024-04-25T00:16:23.383713+0000","last_fullsized":"2024-04-25T00:16:23.383713+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:15:35.520309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.381386+0000","last_change":"2024-04-25T00:16:19.328428+0000","last_active":"2024-04-25T00:16:23.381386+0000","last_peered":"2024-04-25T00:16:23.381386+0000","last_clean":"2024-04-25T00:16:23.381386+0000","last_became_active":"2024-04-25T00:16:19.328112+0000","last_became_peered":"2024-04-25T00:16:19.328112+0000","last_unstale":"2024-04-25T00:16:23.381386+0000","last_undegraded":"2024-04-25T00:16:23.381386+0000","last_fullsized":"2024-04-25T00:16:23.381386+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:09.130246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233454+0000","last_change":"2024-04-25T00:16:20.237922+0000","last_active":"2024-04-25T00:16:23.233454+0000","last_peered":"2024-04-25T00:16:23.233454+0000","last_clean":"2024-04-25T00:16:23.233454+0000","last_became_active":"2024-04-25T00:16:20.237594+0000","last_became_peered":"2024-04-25T00:16:20.237594+0000","last_unstale":"2024-04-25T00:16:23.233454+0000","last_undegraded":"2024-04-25T00:16:23.233454+0000","last_fullsized":"2024-04-25T00:16:23.233454+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:26.517335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:26.331075+0000","last_change":"2024-04-25T00:16:19.312920+0000","last_active":"2024-04-25T00:16:26.331075+0000","last_peered":"2024-04-25T00:16:26.331075+0000","last_clean":"2024-04-25T00:16:26.331075+0000","last_became_active":"2024-04-25T00:16:19.312548+0000","last_became_peered":"2024-04-25T00:16:19.312548+0000","last_unstale":"2024-04-25T00:16:26.331075+0000","last_undegraded":"2024-04-25T00:16:26.331075+0000","last_fullsized":"2024-04-25T00:16:26.331075+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:47.154402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233025+0000","last_change":"2024-04-25T00:16:20.221102+0000","last_active":"2024-04-25T00:16:23.233025+0000","last_peered":"2024-04-25T00:16:23.233025+0000","last_clean":"2024-04-25T00:16:23.233025+0000","last_became_active":"2024-04-25T00:16:20.220753+0000","last_became_peered":"2024-04-25T00:16:20.220753+0000","last_unstale":"2024-04-25T00:16:23.233025+0000","last_undegraded":"2024-04-25T00:16:23.233025+0000","last_fullsized":"2024-04-25T00:16:23.233025+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:18.652865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.326937+0000","last_change":"2024-04-25T00:16:19.324181+0000","last_active":"2024-04-25T00:16:24.326937+0000","last_peered":"2024-04-25T00:16:24.326937+0000","last_clean":"2024-04-25T00:16:24.326937+0000","last_became_active":"2024-04-25T00:16:19.323821+0000","last_became_peered":"2024-04-25T00:16:19.323821+0000","last_unstale":"2024-04-25T00:16:24.326937+0000","last_undegraded":"2024-04-25T00:16:24.326937+0000","last_fullsized":"2024-04-25T00:16:24.326937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:34.323961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375521+0000","last_change":"2024-04-25T00:16:20.219629+0000","last_active":"2024-04-25T00:16:23.375521+0000","last_peered":"2024-04-25T00:16:23.375521+0000","last_clean":"2024-04-25T00:16:23.375521+0000","last_became_active":"2024-04-25T00:16:20.219230+0000","last_became_peered":"2024-04-25T00:16:20.219230+0000","last_unstale":"2024-04-25T00:16:23.375521+0000","last_undegraded":"2024-04-25T00:16:23.375521+0000","last_fullsized":"2024-04-25T00:16:23.375521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:44:52.410209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.307528+0000","last_change":"2024-04-25T00:16:19.306005+0000","last_active":"2024-04-25T00:16:24.307528+0000","last_peered":"2024-04-25T00:16:24.307528+0000","last_clean":"2024-04-25T00:16:24.307528+0000","last_became_active":"2024-04-25T00:16:19.305656+0000","last_became_peered":"2024-04-25T00:16:19.305656+0000","last_unstale":"2024-04-25T00:16:24.307528+0000","last_undegraded":"2024-04-25T00:16:24.307528+0000","last_fullsized":"2024-04-25T00:16:24.307528+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:16.452352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376035+0000","last_change":"2024-04-25T00:16:20.217886+0000","last_active":"2024-04-25T00:16:23.376035+0000","last_peered":"2024-04-25T00:16:23.376035+0000","last_clean":"2024-04-25T00:16:23.376035+0000","last_became_active":"2024-04-25T00:16:20.217379+0000","last_became_peered":"2024-04-25T00:16:20.217379+0000","last_unstale":"2024-04-25T00:16:23.376035+0000","last_undegraded":"2024-04-25T00:16:23.376035+0000","last_fullsized":"2024-04-25T00:16:23.376035+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:22.028791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.228302+0000","last_change":"2024-04-25T00:16:19.328466+0000","last_active":"2024-04-25T00:16:21.228302+0000","last_peered":"2024-04-25T00:16:21.228302+0000","last_clean":"2024-04-25T00:16:21.228302+0000","last_became_active":"2024-04-25T00:16:19.321077+0000","last_became_peered":"2024-04-25T00:16:19.321077+0000","last_unstale":"2024-04-25T00:16:21.228302+0000","last_undegraded":"2024-04-25T00:16:21.228302+0000","last_fullsized":"2024-04-25T00:16:21.228302+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:55.296160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:22.245382+0000","last_change":"2024-04-25T00:16:20.249243+0000","last_active":"2024-04-25T00:16:22.245382+0000","last_peered":"2024-04-25T00:16:22.245382+0000","last_clean":"2024-04-25T00:16:22.245382+0000","last_became_active":"2024-04-25T00:16:20.248955+0000","last_became_peered":"2024-04-25T00:16:20.248955+0000","last_unstale":"2024-04-25T00:16:22.245382+0000","last_undegraded":"2024-04-25T00:16:22.245382+0000","last_fullsized":"2024-04-25T00:16:22.245382+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:14.195688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233233+0000","last_change":"2024-04-25T00:16:19.316409+0000","last_active":"2024-04-25T00:16:23.233233+0000","last_peered":"2024-04-25T00:16:23.233233+0000","last_clean":"2024-04-25T00:16:23.233233+0000","last_became_active":"2024-04-25T00:16:19.315982+0000","last_became_peered":"2024-04-25T00:16:19.315982+0000","last_unstale":"2024-04-25T00:16:23.233233+0000","last_undegraded":"2024-04-25T00:16:23.233233+0000","last_fullsized":"2024-04-25T00:16:23.233233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.718600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:22.245225+0000","last_change":"2024-04-25T00:16:20.242119+0000","last_active":"2024-04-25T00:16:22.245225+0000","last_peered":"2024-04-25T00:16:22.245225+0000","last_clean":"2024-04-25T00:16:22.245225+0000","last_became_active":"2024-04-25T00:16:20.241702+0000","last_became_peered":"2024-04-25T00:16:20.241702+0000","last_unstale":"2024-04-25T00:16:22.245225+0000","last_undegraded":"2024-04-25T00:16:22.245225+0000","last_fullsized":"2024-04-25T00:16:22.245225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:26.652681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.377959+0000","last_change":"2024-04-25T00:16:20.220822+0000","last_active":"2024-04-25T00:16:23.377959+0000","last_peered":"2024-04-25T00:16:23.377959+0000","last_clean":"2024-04-25T00:16:23.377959+0000","last_became_active":"2024-04-25T00:16:20.220420+0000","last_became_peered":"2024-04-25T00:16:20.220420+0000","last_unstale":"2024-04-25T00:16:23.377959+0000","last_undegraded":"2024-04-25T00:16:23.377959+0000","last_fullsized":"2024-04-25T00:16:23.377959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:07:01.032770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.327022+0000","last_change":"2024-04-25T00:16:19.328872+0000","last_active":"2024-04-25T00:16:24.327022+0000","last_peered":"2024-04-25T00:16:24.327022+0000","last_clean":"2024-04-25T00:16:24.327022+0000","last_became_active":"2024-04-25T00:16:19.328439+0000","last_became_peered":"2024-04-25T00:16:19.328439+0000","last_unstale":"2024-04-25T00:16:24.327022+0000","last_undegraded":"2024-04-25T00:16:24.327022+0000","last_fullsized":"2024-04-25T00:16:24.327022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:05.832328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.226801+0000","last_change":"2024-04-25T00:16:20.242857+0000","last_active":"2024-04-25T00:16:21.226801+0000","last_peered":"2024-04-25T00:16:21.226801+0000","last_clean":"2024-04-25T00:16:21.226801+0000","last_became_active":"2024-04-25T00:16:20.240941+0000","last_became_peered":"2024-04-25T00:16:20.240941+0000","last_unstale":"2024-04-25T00:16:21.226801+0000","last_undegraded":"2024-04-25T00:16:21.226801+0000","last_fullsized":"2024-04-25T00:16:21.226801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:36.550571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.326012+0000","last_change":"2024-04-25T00:16:19.312914+0000","last_active":"2024-04-25T00:16:24.326012+0000","last_peered":"2024-04-25T00:16:24.326012+0000","last_clean":"2024-04-25T00:16:24.326012+0000","last_became_active":"2024-04-25T00:16:19.312578+0000","last_became_peered":"2024-04-25T00:16:19.312578+0000","last_unstale":"2024-04-25T00:16:24.326012+0000","last_undegraded":"2024-04-25T00:16:24.326012+0000","last_fullsized":"2024-04-25T00:16:24.326012+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:37.035904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":339,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233053+0000","last_change":"2024-04-25T00:15:03.261320+0000","last_active":"2024-04-25T00:16:23.233053+0000","last_peered":"2024-04-25T00:16:23.233053+0000","last_clean":"2024-04-25T00:16:23.233053+0000","last_became_active":"2024-04-25T00:15:03.260804+0000","last_became_peered":"2024-04-25T00:15:03.260804+0000","last_unstale":"2024-04-25T00:16:23.233053+0000","last_undegraded":"2024-04-25T00:16:23.233053+0000","last_fullsized":"2024-04-25T00:16:23.233053+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:14.473933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:14.473933+0000","last_clean_scrub_stamp":"2024-04-25T00:11:14.473933+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:35.876949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.382876+0000","last_change":"2024-04-25T00:16:20.238253+0000","last_active":"2024-04-25T00:16:23.382876+0000","last_peered":"2024-04-25T00:16:23.382876+0000","last_clean":"2024-04-25T00:16:23.382876+0000","last_became_active":"2024-04-25T00:16:20.237824+0000","last_became_peered":"2024-04-25T00:16:20.237824+0000","last_unstale":"2024-04-25T00:16:23.382876+0000","last_undegraded":"2024-04-25T00:16:23.382876+0000","last_fullsized":"2024-04-25T00:16:23.382876+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:21.996513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.320490+0000","last_change":"2024-04-25T00:16:19.316194+0000","last_active":"2024-04-25T00:16:24.320490+0000","last_peered":"2024-04-25T00:16:24.320490+0000","last_clean":"2024-04-25T00:16:24.320490+0000","last_became_active":"2024-04-25T00:16:19.315625+0000","last_became_peered":"2024-04-25T00:16:19.315625+0000","last_unstale":"2024-04-25T00:16:24.320490+0000","last_undegraded":"2024-04-25T00:16:24.320490+0000","last_fullsized":"2024-04-25T00:16:24.320490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:23.426048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.379375+0000","last_change":"2024-04-25T00:16:19.324283+0000","last_active":"2024-04-25T00:16:23.379375+0000","last_peered":"2024-04-25T00:16:23.379375+0000","last_clean":"2024-04-25T00:16:23.379375+0000","last_became_active":"2024-04-25T00:16:19.323800+0000","last_became_peered":"2024-04-25T00:16:19.323800+0000","last_unstale":"2024-04-25T00:16:23.379375+0000","last_undegraded":"2024-04-25T00:16:23.379375+0000","last_fullsized":"2024-04-25T00:16:23.379375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:43.592775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383803+0000","last_change":"2024-04-25T00:16:20.239205+0000","last_active":"2024-04-25T00:16:23.383803+0000","last_peered":"2024-04-25T00:16:23.383803+0000","last_clean":"2024-04-25T00:16:23.383803+0000","last_became_active":"2024-04-25T00:16:20.237135+0000","last_became_peered":"2024-04-25T00:16:20.237135+0000","last_unstale":"2024-04-25T00:16:23.383803+0000","last_undegraded":"2024-04-25T00:16:23.383803+0000","last_fullsized":"2024-04-25T00:16:23.383803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:37.348227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:24.324981+0000","last_change":"2024-04-25T00:16:19.325868+0000","last_active":"2024-04-25T00:16:24.324981+0000","last_peered":"2024-04-25T00:16:24.324981+0000","last_clean":"2024-04-25T00:16:24.324981+0000","last_became_active":"2024-04-25T00:16:19.322820+0000","last_became_peered":"2024-04-25T00:16:19.322820+0000","last_unstale":"2024-04-25T00:16:24.324981+0000","last_undegraded":"2024-04-25T00:16:24.324981+0000","last_fullsized":"2024-04-25T00:16:24.324981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:50.982099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.226683+0000","last_change":"2024-04-25T00:16:20.248268+0000","last_active":"2024-04-25T00:16:21.226683+0000","last_peered":"2024-04-25T00:16:21.226683+0000","last_clean":"2024-04-25T00:16:21.226683+0000","last_became_active":"2024-04-25T00:16:20.247274+0000","last_became_peered":"2024-04-25T00:16:20.247274+0000","last_unstale":"2024-04-25T00:16:21.226683+0000","last_undegraded":"2024-04-25T00:16:21.226683+0000","last_fullsized":"2024-04-25T00:16:21.226683+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:46.120277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.380422+0000","last_change":"2024-04-25T00:16:19.317256+0000","last_active":"2024-04-25T00:16:23.380422+0000","last_peered":"2024-04-25T00:16:23.380422+0000","last_clean":"2024-04-25T00:16:23.380422+0000","last_became_active":"2024-04-25T00:16:19.316931+0000","last_became_peered":"2024-04-25T00:16:19.316931+0000","last_unstale":"2024-04-25T00:16:23.380422+0000","last_undegraded":"2024-04-25T00:16:23.380422+0000","last_fullsized":"2024-04-25T00:16:23.380422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:49.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700421+0000","last_change":"2024-04-25T00:16:20.252271+0000","last_active":"2024-04-25T00:16:21.700421+0000","last_peered":"2024-04-25T00:16:21.700421+0000","last_clean":"2024-04-25T00:16:21.700421+0000","last_became_active":"2024-04-25T00:16:20.251930+0000","last_became_peered":"2024-04-25T00:16:20.251930+0000","last_unstale":"2024-04-25T00:16:21.700421+0000","last_undegraded":"2024-04-25T00:16:21.700421+0000","last_fullsized":"2024-04-25T00:16:21.700421+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:07.313172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.381188+0000","last_change":"2024-04-25T00:16:19.309540+0000","last_active":"2024-04-25T00:16:23.381188+0000","last_peered":"2024-04-25T00:16:23.381188+0000","last_clean":"2024-04-25T00:16:23.381188+0000","last_became_active":"2024-04-25T00:16:19.309171+0000","last_became_peered":"2024-04-25T00:16:19.309171+0000","last_unstale":"2024-04-25T00:16:23.381188+0000","last_undegraded":"2024-04-25T00:16:23.381188+0000","last_fullsized":"2024-04-25T00:16:23.381188+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:08:28.035966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.477222+0000","last_change":"2024-04-25T00:16:20.224710+0000","last_active":"2024-04-25T00:16:21.477222+0000","last_peered":"2024-04-25T00:16:21.477222+0000","last_clean":"2024-04-25T00:16:21.477222+0000","last_became_active":"2024-04-25T00:16:20.224385+0000","last_became_peered":"2024-04-25T00:16:20.224385+0000","last_unstale":"2024-04-25T00:16:21.477222+0000","last_undegraded":"2024-04-25T00:16:21.477222+0000","last_fullsized":"2024-04-25T00:16:21.477222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:29.161972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.382288+0000","last_change":"2024-04-25T00:16:19.327498+0000","last_active":"2024-04-25T00:16:23.382288+0000","last_peered":"2024-04-25T00:16:23.382288+0000","last_clean":"2024-04-25T00:16:23.382288+0000","last_became_active":"2024-04-25T00:16:19.326889+0000","last_became_peered":"2024-04-25T00:16:19.326889+0000","last_unstale":"2024-04-25T00:16:23.382288+0000","last_undegraded":"2024-04-25T00:16:23.382288+0000","last_fullsized":"2024-04-25T00:16:23.382288+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:05.275999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376234+0000","last_change":"2024-04-25T00:16:20.242150+0000","last_active":"2024-04-25T00:16:23.376234+0000","last_peered":"2024-04-25T00:16:23.376234+0000","last_clean":"2024-04-25T00:16:23.376234+0000","last_became_active":"2024-04-25T00:16:20.241770+0000","last_became_peered":"2024-04-25T00:16:20.241770+0000","last_unstale":"2024-04-25T00:16:23.376234+0000","last_undegraded":"2024-04-25T00:16:23.376234+0000","last_fullsized":"2024-04-25T00:16:23.376234+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:38.038980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375743+0000","last_change":"2024-04-25T00:16:19.328774+0000","last_active":"2024-04-25T00:16:23.375743+0000","last_peered":"2024-04-25T00:16:23.375743+0000","last_clean":"2024-04-25T00:16:23.375743+0000","last_became_active":"2024-04-25T00:16:19.328466+0000","last_became_peered":"2024-04-25T00:16:19.328466+0000","last_unstale":"2024-04-25T00:16:23.375743+0000","last_undegraded":"2024-04-25T00:16:23.375743+0000","last_fullsized":"2024-04-25T00:16:23.375743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:35.967205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.226962+0000","last_change":"2024-04-25T00:16:20.244330+0000","last_active":"2024-04-25T00:16:21.226962+0000","last_peered":"2024-04-25T00:16:21.226962+0000","last_clean":"2024-04-25T00:16:21.226962+0000","last_became_active":"2024-04-25T00:16:20.243946+0000","last_became_peered":"2024-04-25T00:16:20.243946+0000","last_unstale":"2024-04-25T00:16:21.226962+0000","last_undegraded":"2024-04-25T00:16:21.226962+0000","last_fullsized":"2024-04-25T00:16:21.226962+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:37.790655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.378998+0000","last_change":"2024-04-25T00:16:19.313708+0000","last_active":"2024-04-25T00:16:23.378998+0000","last_peered":"2024-04-25T00:16:23.378998+0000","last_clean":"2024-04-25T00:16:23.378998+0000","last_became_active":"2024-04-25T00:16:19.311700+0000","last_became_peered":"2024-04-25T00:16:19.311700+0000","last_unstale":"2024-04-25T00:16:23.378998+0000","last_undegraded":"2024-04-25T00:16:23.378998+0000","last_fullsized":"2024-04-25T00:16:23.378998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:24.656229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233366+0000","last_change":"2024-04-25T00:16:20.240828+0000","last_active":"2024-04-25T00:16:23.233366+0000","last_peered":"2024-04-25T00:16:23.233366+0000","last_clean":"2024-04-25T00:16:23.233366+0000","last_became_active":"2024-04-25T00:16:20.240495+0000","last_became_peered":"2024-04-25T00:16:20.240495+0000","last_unstale":"2024-04-25T00:16:23.233366+0000","last_undegraded":"2024-04-25T00:16:23.233366+0000","last_fullsized":"2024-04-25T00:16:23.233366+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:19.730968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376542+0000","last_change":"2024-04-25T00:16:19.324069+0000","last_active":"2024-04-25T00:16:23.376542+0000","last_peered":"2024-04-25T00:16:23.376542+0000","last_clean":"2024-04-25T00:16:23.376542+0000","last_became_active":"2024-04-25T00:16:19.323604+0000","last_became_peered":"2024-04-25T00:16:19.323604+0000","last_unstale":"2024-04-25T00:16:23.376542+0000","last_undegraded":"2024-04-25T00:16:23.376542+0000","last_fullsized":"2024-04-25T00:16:23.376542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:50.908834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233562+0000","last_change":"2024-04-25T00:16:20.220555+0000","last_active":"2024-04-25T00:16:23.233562+0000","last_peered":"2024-04-25T00:16:23.233562+0000","last_clean":"2024-04-25T00:16:23.233562+0000","last_became_active":"2024-04-25T00:16:20.219986+0000","last_became_peered":"2024-04-25T00:16:20.219986+0000","last_unstale":"2024-04-25T00:16:23.233562+0000","last_undegraded":"2024-04-25T00:16:23.233562+0000","last_fullsized":"2024-04-25T00:16:23.233562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:04.095555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375941+0000","last_change":"2024-04-25T00:16:19.328270+0000","last_active":"2024-04-25T00:16:23.375941+0000","last_peered":"2024-04-25T00:16:23.375941+0000","last_clean":"2024-04-25T00:16:23.375941+0000","last_became_active":"2024-04-25T00:16:19.327929+0000","last_became_peered":"2024-04-25T00:16:19.327929+0000","last_unstale":"2024-04-25T00:16:23.375941+0000","last_undegraded":"2024-04-25T00:16:23.375941+0000","last_fullsized":"2024-04-25T00:16:23.375941+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:09.233345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233250+0000","last_change":"2024-04-25T00:16:20.237103+0000","last_active":"2024-04-25T00:16:23.233250+0000","last_peered":"2024-04-25T00:16:23.233250+0000","last_clean":"2024-04-25T00:16:23.233250+0000","last_became_active":"2024-04-25T00:16:20.236811+0000","last_became_peered":"2024-04-25T00:16:20.236811+0000","last_unstale":"2024-04-25T00:16:23.233250+0000","last_undegraded":"2024-04-25T00:16:23.233250+0000","last_fullsized":"2024-04-25T00:16:23.233250+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:56.833659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383973+0000","last_change":"2024-04-25T00:16:19.323227+0000","last_active":"2024-04-25T00:16:23.383973+0000","last_peered":"2024-04-25T00:16:23.383973+0000","last_clean":"2024-04-25T00:16:23.383973+0000","last_became_active":"2024-04-25T00:16:19.322074+0000","last_became_peered":"2024-04-25T00:16:19.322074+0000","last_unstale":"2024-04-25T00:16:23.383973+0000","last_undegraded":"2024-04-25T00:16:23.383973+0000","last_fullsized":"2024-04-25T00:16:23.383973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:53.316171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.233535+0000","last_change":"2024-04-25T00:16:20.222049+0000","last_active":"2024-04-25T00:16:23.233535+0000","last_peered":"2024-04-25T00:16:23.233535+0000","last_clean":"2024-04-25T00:16:23.233535+0000","last_became_active":"2024-04-25T00:16:20.221651+0000","last_became_peered":"2024-04-25T00:16:20.221651+0000","last_unstale":"2024-04-25T00:16:23.233535+0000","last_undegraded":"2024-04-25T00:16:23.233535+0000","last_fullsized":"2024-04-25T00:16:23.233535+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:31.244463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376411+0000","last_change":"2024-04-25T00:16:20.223864+0000","last_active":"2024-04-25T00:16:23.376411+0000","last_peered":"2024-04-25T00:16:23.376411+0000","last_clean":"2024-04-25T00:16:23.376411+0000","last_became_active":"2024-04-25T00:16:20.223533+0000","last_became_peered":"2024-04-25T00:16:20.223533+0000","last_unstale":"2024-04-25T00:16:23.376411+0000","last_undegraded":"2024-04-25T00:16:23.376411+0000","last_fullsized":"2024-04-25T00:16:23.376411+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:14.482038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376410+0000","last_change":"2024-04-25T00:16:20.214232+0000","last_active":"2024-04-25T00:16:23.376410+0000","last_peered":"2024-04-25T00:16:23.376410+0000","last_clean":"2024-04-25T00:16:23.376410+0000","last_became_active":"2024-04-25T00:16:20.213871+0000","last_became_peered":"2024-04-25T00:16:20.213871+0000","last_unstale":"2024-04-25T00:16:23.376410+0000","last_undegraded":"2024-04-25T00:16:23.376410+0000","last_fullsized":"2024-04-25T00:16:23.376410+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:07.049327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375661+0000","last_change":"2024-04-25T00:16:20.232935+0000","last_active":"2024-04-25T00:16:23.375661+0000","last_peered":"2024-04-25T00:16:23.375661+0000","last_clean":"2024-04-25T00:16:23.375661+0000","last_became_active":"2024-04-25T00:16:20.232632+0000","last_became_peered":"2024-04-25T00:16:20.232632+0000","last_unstale":"2024-04-25T00:16:23.375661+0000","last_undegraded":"2024-04-25T00:16:23.375661+0000","last_fullsized":"2024-04-25T00:16:23.375661+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:36.724694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.700075+0000","last_change":"2024-04-25T00:16:20.244382+0000","last_active":"2024-04-25T00:16:21.700075+0000","last_peered":"2024-04-25T00:16:21.700075+0000","last_clean":"2024-04-25T00:16:21.700075+0000","last_became_active":"2024-04-25T00:16:20.241866+0000","last_became_peered":"2024-04-25T00:16:20.241866+0000","last_unstale":"2024-04-25T00:16:21.700075+0000","last_undegraded":"2024-04-25T00:16:21.700075+0000","last_fullsized":"2024-04-25T00:16:21.700075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:24.170878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383977+0000","last_change":"2024-04-25T00:16:20.241343+0000","last_active":"2024-04-25T00:16:23.383977+0000","last_peered":"2024-04-25T00:16:23.383977+0000","last_clean":"2024-04-25T00:16:23.383977+0000","last_became_active":"2024-04-25T00:16:20.238417+0000","last_became_peered":"2024-04-25T00:16:20.238417+0000","last_unstale":"2024-04-25T00:16:23.383977+0000","last_undegraded":"2024-04-25T00:16:23.383977+0000","last_fullsized":"2024-04-25T00:16:23.383977+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:29.988773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376842+0000","last_change":"2024-04-25T00:16:20.219966+0000","last_active":"2024-04-25T00:16:23.376842+0000","last_peered":"2024-04-25T00:16:23.376842+0000","last_clean":"2024-04-25T00:16:23.376842+0000","last_became_active":"2024-04-25T00:16:20.219656+0000","last_became_peered":"2024-04-25T00:16:20.219656+0000","last_unstale":"2024-04-25T00:16:23.376842+0000","last_undegraded":"2024-04-25T00:16:23.376842+0000","last_fullsized":"2024-04-25T00:16:23.376842+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:56.290610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.376180+0000","last_change":"2024-04-25T00:16:20.219141+0000","last_active":"2024-04-25T00:16:23.376180+0000","last_peered":"2024-04-25T00:16:23.376180+0000","last_clean":"2024-04-25T00:16:23.376180+0000","last_became_active":"2024-04-25T00:16:20.218842+0000","last_became_peered":"2024-04-25T00:16:20.218842+0000","last_unstale":"2024-04-25T00:16:23.376180+0000","last_undegraded":"2024-04-25T00:16:23.376180+0000","last_fullsized":"2024-04-25T00:16:23.376180+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:33.938226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.226186+0000","last_change":"2024-04-25T00:16:20.249183+0000","last_active":"2024-04-25T00:16:21.226186+0000","last_peered":"2024-04-25T00:16:21.226186+0000","last_clean":"2024-04-25T00:16:21.226186+0000","last_became_active":"2024-04-25T00:16:20.248355+0000","last_became_peered":"2024-04-25T00:16:20.248355+0000","last_unstale":"2024-04-25T00:16:21.226186+0000","last_undegraded":"2024-04-25T00:16:21.226186+0000","last_fullsized":"2024-04-25T00:16:21.226186+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:57.738041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.377447+0000","last_change":"2024-04-25T00:16:20.227528+0000","last_active":"2024-04-25T00:16:23.377447+0000","last_peered":"2024-04-25T00:16:23.377447+0000","last_clean":"2024-04-25T00:16:23.377447+0000","last_became_active":"2024-04-25T00:16:20.227208+0000","last_became_peered":"2024-04-25T00:16:20.227208+0000","last_unstale":"2024-04-25T00:16:23.377447+0000","last_undegraded":"2024-04-25T00:16:23.377447+0000","last_fullsized":"2024-04-25T00:16:23.377447+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:49.920850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:21.225735+0000","last_change":"2024-04-25T00:16:20.228157+0000","last_active":"2024-04-25T00:16:21.225735+0000","last_peered":"2024-04-25T00:16:21.225735+0000","last_clean":"2024-04-25T00:16:21.225735+0000","last_became_active":"2024-04-25T00:16:20.227659+0000","last_became_peered":"2024-04-25T00:16:20.227659+0000","last_unstale":"2024-04-25T00:16:21.225735+0000","last_undegraded":"2024-04-25T00:16:21.225735+0000","last_fullsized":"2024-04-25T00:16:21.225735+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:30.579407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.375515+0000","last_change":"2024-04-25T00:16:20.232237+0000","last_active":"2024-04-25T00:16:23.375515+0000","last_peered":"2024-04-25T00:16:23.375515+0000","last_clean":"2024-04-25T00:16:23.375515+0000","last_became_active":"2024-04-25T00:16:20.231897+0000","last_became_peered":"2024-04-25T00:16:20.231897+0000","last_unstale":"2024-04-25T00:16:23.375515+0000","last_undegraded":"2024-04-25T00:16:23.375515+0000","last_fullsized":"2024-04-25T00:16:23.375515+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:23.919515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.383588+0000","last_change":"2024-04-25T00:16:20.243454+0000","last_active":"2024-04-25T00:16:23.383588+0000","last_peered":"2024-04-25T00:16:23.383588+0000","last_clean":"2024-04-25T00:16:23.383588+0000","last_became_active":"2024-04-25T00:16:20.243112+0000","last_became_peered":"2024-04-25T00:16:20.243112+0000","last_unstale":"2024-04-25T00:16:23.383588+0000","last_undegraded":"2024-04-25T00:16:23.383588+0000","last_fullsized":"2024-04-25T00:16:23.383588+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:58.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-04-25T00:16:23.377159+0000","last_change":"2024-04-25T00:16:20.220947+0000","last_active":"2024-04-25T00:16:23.377159+0000","last_peered":"2024-04-25T00:16:23.377159+0000","last_clean":"2024-04-25T00:16:23.377159+0000","last_became_active":"2024-04-25T00:16:20.220662+0000","last_became_peered":"2024-04-25T00:16:20.220662+0000","last_unstale":"2024-04-25T00:16:23.377159+0000","last_undegraded":"2024-04-25T00:16:23.377159+0000","last_fullsized":"2024-04-25T00:16:23.377159+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:44.840549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":56,"num_object_clones":0,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":0,"num_read":28,"num_read_kb":21,"num_write":117,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":37,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":233472,"data_stored":14972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103,"ondisk_log_size":103,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350990,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":444893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1037869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708896,"statfs":{"total":95995035648,"available":95957909504,"internally_reserved":0,"allocated":942080,"data_stored":445095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037791,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":445049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28520,"kb_used_data":1500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716632,"statfs":{"total":95995035648,"available":95965831168,"internally_reserved":0,"allocated":1536000,"data_stored":1036471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495658,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":929792,"data_stored":444401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":445923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":954368,"data_stored":445789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":950272,"data_stored":446579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":444712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":937984,"data_stored":444955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36848,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708304,"statfs":{"total":95995035648,"available":95957303296,"internally_reserved":0,"allocated":1548288,"data_stored":1036155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:16:29.830 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T00:16:29.831 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T00:16:29.831 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T00:16:29.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:29 smithi106 ceph-mon[32510]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:16:29.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:29 smithi106 ceph-mon[32510]: pgmap v317: 97 pgs: 97 active+clean; 581 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 4.6 KiB/s wr, 18 op/s 2024-04-25T00:16:29.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:29 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1542715561' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:30.337 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:30.674 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:30 smithi042 ceph-mon[28130]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:30.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:30 smithi136 ceph-mon[32923]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:30.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:30 smithi106 ceph-mon[32510]: from='client.14601 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:31.529 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:31 smithi042 ceph-mon[28130]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:31.530 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:31 smithi042 ceph-mon[28130]: pgmap v318: 97 pgs: 97 active+clean; 582 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.1 KiB/s wr, 16 op/s 2024-04-25T00:16:31.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:31 smithi136 ceph-mon[32923]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:31.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:31 smithi136 ceph-mon[32923]: pgmap v318: 97 pgs: 97 active+clean; 582 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.1 KiB/s wr, 16 op/s 2024-04-25T00:16:31.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:31 smithi106 ceph-mon[32510]: from='client.14607 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:16:31.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:31 smithi106 ceph-mon[32510]: pgmap v318: 97 pgs: 97 active+clean; 582 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4.1 KiB/s wr, 16 op/s 2024-04-25T00:16:32.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.766+0000 7f3f42600700 1 -- 172.21.15.42:0/2278880891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3c074c90 msgr2=0x7f3f3c075110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.766+0000 7f3f42600700 1 --2- 172.21.15.42:0/2278880891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3c074c90 0x7f3f3c075110 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3f24009990 tx=0x7f3f24009ca0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.767+0000 7f3f42600700 1 -- 172.21.15.42:0/2278880891 shutdown_connections 2024-04-25T00:16:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.767+0000 7f3f42600700 1 --2- 172.21.15.42:0/2278880891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c10e890 0x7f3f3c110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.767+0000 7f3f42600700 1 --2- 172.21.15.42:0/2278880891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f3c076260 0x7f3f3c0746c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.767+0000 7f3f42600700 1 --2- 172.21.15.42:0/2278880891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3c074c90 0x7f3f3c075110 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.767+0000 7f3f42600700 1 -- 172.21.15.42:0/2278880891 >> 172.21.15.42:0/2278880891 conn(0x7f3f3c0fd850 msgr2=0x7f3f3c0ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.767+0000 7f3f42600700 1 -- 172.21.15.42:0/2278880891 shutdown_connections 2024-04-25T00:16:32.768 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.767+0000 7f3f42600700 1 -- 172.21.15.42:0/2278880891 wait complete. 2024-04-25T00:16:32.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.768+0000 7f3f42600700 1 Processor -- start 2024-04-25T00:16:32.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.768+0000 7f3f42600700 1 -- start start 2024-04-25T00:16:32.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.768+0000 7f3f42600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3c074c90 0x7f3f3c19ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f42600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f3c076260 0x7f3f3c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.769 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f42600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c10e890 0x7f3f3c1a6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f42600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c1144e0 con 0x7f3f3c074c90 2024-04-25T00:16:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f42600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c114360 con 0x7f3f3c10e890 2024-04-25T00:16:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f42600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c114660 con 0x7f3f3c076260 2024-04-25T00:16:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f3b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3c074c90 0x7f3f3c19ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f3b400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3c074c90 0x7f3f3c19ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37268/0 (socket says 172.21.15.42:37268) 2024-04-25T00:16:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f3b400700 1 -- 172.21.15.42:0/1286175515 learned_addr learned my addr 172.21.15.42:0/1286175515 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f3be00700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c10e890 0x7f3f3c1a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f33e00700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f3c076260 0x7f3f3c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f33e00700 1 -- 172.21.15.42:0/1286175515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c10e890 msgr2=0x7f3f3c1a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f33e00700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c10e890 0x7f3f3c1a6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f33e00700 1 -- 172.21.15.42:0/1286175515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3c074c90 msgr2=0x7f3f3c19ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f33e00700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3c074c90 0x7f3f3c19ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f33e00700 1 -- 172.21.15.42:0/1286175515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c1a66a0 con 0x7f3f3c076260 2024-04-25T00:16:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.769+0000 7f3f3b400700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3c074c90 0x7f3f3c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.770+0000 7f3f33e00700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f3c076260 0x7f3f3c19ef80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f240098e0 tx=0x7f3f2400bbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:32.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.770+0000 7f3f38c00700 1 -- 172.21.15.42:0/1286175515 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24026070 con 0x7f3f3c076260 2024-04-25T00:16:32.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.770+0000 7f3f42600700 1 -- 172.21.15.42:0/1286175515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c1a6930 con 0x7f3f3c076260 2024-04-25T00:16:32.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.770+0000 7f3f38c00700 1 -- 172.21.15.42:0/1286175515 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f24003d70 con 0x7f3f3c076260 2024-04-25T00:16:32.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.770+0000 7f3f38c00700 1 -- 172.21.15.42:0/1286175515 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f24021410 con 0x7f3f3c076260 2024-04-25T00:16:32.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.770+0000 7f3f42600700 1 -- 172.21.15.42:0/1286175515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c1a6de0 con 0x7f3f3c076260 2024-04-25T00:16:32.772 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.772+0000 7f3f42600700 1 -- 172.21.15.42:0/1286175515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f3c0507a0 con 0x7f3f3c076260 2024-04-25T00:16:32.773 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.773+0000 7f3f38c00700 1 -- 172.21.15.42:0/1286175515 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3f24029070 con 0x7f3f3c076260 2024-04-25T00:16:32.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.775+0000 7f3f38c00700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3f200a2d60 0x7f3f200a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:32.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.775+0000 7f3f38c00700 1 -- 172.21.15.42:0/1286175515 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3f240cff00 con 0x7f3f3c076260 2024-04-25T00:16:32.775 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.775+0000 7f3f3b400700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3f200a2d60 0x7f3f200a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:32.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.776+0000 7f3f3b400700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3f200a2d60 0x7f3f200a5220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c005950 tx=0x7f3f2c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:32.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:32.779+0000 7f3f38c00700 1 -- 172.21.15.42:0/1286175515 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3f2405d740 con 0x7f3f3c076260 2024-04-25T00:16:33.099 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.099+0000 7f3f42600700 1 -- 172.21.15.42:0/1286175515 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f3f3c062e40 con 0x7f3f3c076260 2024-04-25T00:16:33.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.107+0000 7f3f38c00700 1 -- 172.21.15.42:0/1286175515 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f3f24013070 con 0x7f3f3c076260 2024-04-25T00:16:33.108 INFO:teuthology.orchestra.run.smithi042.stdout:[client.0] 2024-04-25T00:16:33.108 INFO:teuthology.orchestra.run.smithi042.stdout: key = AQDhoClmg/gHBhAAhAS1w17qoKj0SpQ++/qAgg== 2024-04-25T00:16:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.110+0000 7f3f42600700 1 -- 172.21.15.42:0/1286175515 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3f200a2d60 msgr2=0x7f3f200a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.110+0000 7f3f42600700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3f200a2d60 0x7f3f200a5220 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c005950 tx=0x7f3f2c009260 comp rx=0 tx=0).stop 2024-04-25T00:16:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.110+0000 7f3f42600700 1 -- 172.21.15.42:0/1286175515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f3c076260 msgr2=0x7f3f3c19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.110+0000 7f3f42600700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f3c076260 0x7f3f3c19ef80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f240098e0 tx=0x7f3f2400bbb0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.111+0000 7f3f42600700 1 -- 172.21.15.42:0/1286175515 shutdown_connections 2024-04-25T00:16:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.111+0000 7f3f42600700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3f200a2d60 0x7f3f200a5220 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.111+0000 7f3f42600700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f3c10e890 0x7f3f3c1a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.111+0000 7f3f42600700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3f3c074c90 0x7f3f3c19ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.111+0000 7f3f42600700 1 --2- 172.21.15.42:0/1286175515 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3f3c076260 0x7f3f3c19ef80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:33.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.111+0000 7f3f42600700 1 -- 172.21.15.42:0/1286175515 >> 172.21.15.42:0/1286175515 conn(0x7f3f3c0fd850 msgr2=0x7f3f3c101c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:33.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.111+0000 7f3f42600700 1 -- 172.21.15.42:0/1286175515 shutdown_connections 2024-04-25T00:16:33.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:33.111+0000 7f3f42600700 1 -- 172.21.15.42:0/1286175515 wait complete. 2024-04-25T00:16:33.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:33 smithi136 ceph-mon[32923]: pgmap v319: 97 pgs: 97 active+clean; 623 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-04-25T00:16:33.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:33 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1286175515' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:16:33.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:33 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:16:33.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:33 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:16:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:33 smithi106 ceph-mon[32510]: pgmap v319: 97 pgs: 97 active+clean; 623 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-04-25T00:16:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:33 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1286175515' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:16:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:33 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:16:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:33 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:16:33.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:33 smithi042 ceph-mon[28130]: pgmap v319: 97 pgs: 97 active+clean; 623 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.0 KiB/s wr, 16 op/s 2024-04-25T00:16:33.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:33 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1286175515' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:16:33.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:33 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T00:16:33.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:33 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T00:16:34.114 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:16:34.114 DEBUG:teuthology.orchestra.run.smithi042:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T00:16:34.114 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T00:16:34.257 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T00:16:34.257 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T00:16:34.257 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T00:16:34.748 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:35.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:35 smithi042 ceph-mon[28130]: pgmap v320: 97 pgs: 97 active+clean; 623 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-04-25T00:16:35.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:35 smithi136 ceph-mon[32923]: pgmap v320: 97 pgs: 97 active+clean; 623 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-04-25T00:16:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:35 smithi106 ceph-mon[32510]: pgmap v320: 97 pgs: 97 active+clean; 623 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 7.0 KiB/s wr, 14 op/s 2024-04-25T00:16:37.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.179+0000 7fd07f400700 1 -- 172.21.15.42:0/3683491446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd078104450 msgr2=0x7fd0781048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:37.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.179+0000 7fd07f400700 1 --2- 172.21.15.42:0/3683491446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd078104450 0x7fd0781048d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd068009990 tx=0x7fd068009ca0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.180+0000 7fd07f400700 1 -- 172.21.15.42:0/3683491446 shutdown_connections 2024-04-25T00:16:37.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.180+0000 7fd07f400700 1 --2- 172.21.15.42:0/3683491446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd078104e10 0x7fd078111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.180+0000 7fd07f400700 1 --2- 172.21.15.42:0/3683491446 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd07810a4e0 0x7fd07810a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.180+0000 7fd07f400700 1 --2- 172.21.15.42:0/3683491446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd078104450 0x7fd0781048d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.180+0000 7fd07f400700 1 -- 172.21.15.42:0/3683491446 >> 172.21.15.42:0/3683491446 conn(0x7fd0780fff40 msgr2=0x7fd078102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:37.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.180+0000 7fd07f400700 1 -- 172.21.15.42:0/3683491446 shutdown_connections 2024-04-25T00:16:37.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.180+0000 7fd07f400700 1 -- 172.21.15.42:0/3683491446 wait complete. 2024-04-25T00:16:37.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.181+0000 7fd07f400700 1 Processor -- start 2024-04-25T00:16:37.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.181+0000 7fd07f400700 1 -- start start 2024-04-25T00:16:37.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd07f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078104450 0x7fd0781a0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd07f400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd078104e10 0x7fd0781a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd07f400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd07810a4e0 0x7fd07819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd07f400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd0781147c0 con 0x7fd078104450 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd07f400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd078114640 con 0x7fd078104e10 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd07f400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd078114940 con 0x7fd07810a4e0 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd07ca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078104450 0x7fd0781a0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd077e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd078104e10 0x7fd0781a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd07ca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078104450 0x7fd0781a0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47830/0 (socket says 172.21.15.42:47830) 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd07ca00700 1 -- 172.21.15.42:0/3546015251 learned_addr learned my addr 172.21.15.42:0/3546015251 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.182+0000 7fd07d400700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd07810a4e0 0x7fd07819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.183+0000 7fd07ca00700 1 -- 172.21.15.42:0/3546015251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd07810a4e0 msgr2=0x7fd07819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:37.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.183+0000 7fd07ca00700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd07810a4e0 0x7fd07819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.183+0000 7fd07ca00700 1 -- 172.21.15.42:0/3546015251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd078104e10 msgr2=0x7fd0781a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:37.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.183+0000 7fd07ca00700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd078104e10 0x7fd0781a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.183+0000 7fd07ca00700 1 -- 172.21.15.42:0/3546015251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd07819b1a0 con 0x7fd078104450 2024-04-25T00:16:37.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.183+0000 7fd077e00700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd078104e10 0x7fd0781a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:37.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.183+0000 7fd07ca00700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078104450 0x7fd0781a0880 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00e9a0 tx=0x7fd06c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:37.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.183+0000 7fd075600700 1 -- 172.21.15.42:0/3546015251 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c00ca10 con 0x7fd078104450 2024-04-25T00:16:37.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.184+0000 7fd075600700 1 -- 172.21.15.42:0/3546015251 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd06c00cbb0 con 0x7fd078104450 2024-04-25T00:16:37.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.184+0000 7fd07f400700 1 -- 172.21.15.42:0/3546015251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd07819b490 con 0x7fd078104450 2024-04-25T00:16:37.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.184+0000 7fd075600700 1 -- 172.21.15.42:0/3546015251 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c018680 con 0x7fd078104450 2024-04-25T00:16:37.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.184+0000 7fd07f400700 1 -- 172.21.15.42:0/3546015251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0781a7750 con 0x7fd078104450 2024-04-25T00:16:37.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.185+0000 7fd07f400700 1 -- 172.21.15.42:0/3546015251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd078068f20 con 0x7fd078104450 2024-04-25T00:16:37.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.187+0000 7fd075600700 1 -- 172.21.15.42:0/3546015251 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd06c010430 con 0x7fd078104450 2024-04-25T00:16:37.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.188+0000 7fd075600700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd0600a2cb0 0x7fd0600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:37.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.188+0000 7fd075600700 1 -- 172.21.15.42:0/3546015251 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fd06c0cf2d0 con 0x7fd078104450 2024-04-25T00:16:37.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.189+0000 7fd077e00700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd0600a2cb0 0x7fd0600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:37.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.189+0000 7fd077e00700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd0600a2cb0 0x7fd0600a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd078109aa0 tx=0x7fd068005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:37.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.193+0000 7fd075600700 1 -- 172.21.15.42:0/3546015251 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd06c09c180 con 0x7fd078104450 2024-04-25T00:16:37.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.487+0000 7fd07f400700 1 -- 172.21.15.42:0/3546015251 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd07802db30 con 0x7fd078104450 2024-04-25T00:16:37.497 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.497+0000 7fd075600700 1 -- 172.21.15.42:0/3546015251 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263197 (secure 0 0 0) 0x7fd06c09ba80 con 0x7fd078104450 2024-04-25T00:16:37.502 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:37.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.502+0000 7fd07f400700 1 -- 172.21.15.42:0/3546015251 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd0600a2cb0 msgr2=0x7fd0600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:37.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.502+0000 7fd07f400700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd0600a2cb0 0x7fd0600a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd078109aa0 tx=0x7fd068005fd0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.503+0000 7fd07f400700 1 -- 172.21.15.42:0/3546015251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078104450 msgr2=0x7fd0781a0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:37.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.503+0000 7fd07f400700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078104450 0x7fd0781a0880 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd06c00e9a0 tx=0x7fd06c00ecb0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.503+0000 7fd07f400700 1 -- 172.21.15.42:0/3546015251 shutdown_connections 2024-04-25T00:16:37.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.503+0000 7fd07f400700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd0600a2cb0 0x7fd0600a5170 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.503+0000 7fd07f400700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd07810a4e0 0x7fd07819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.503+0000 7fd07f400700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd078104450 0x7fd0781a0880 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.503+0000 7fd07f400700 1 --2- 172.21.15.42:0/3546015251 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd078104e10 0x7fd0781a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:37.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.503+0000 7fd07f400700 1 -- 172.21.15.42:0/3546015251 >> 172.21.15.42:0/3546015251 conn(0x7fd0780fff40 msgr2=0x7fd078100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:37.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.503+0000 7fd07f400700 1 -- 172.21.15.42:0/3546015251 shutdown_connections 2024-04-25T00:16:37.505 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:37.503+0000 7fd07f400700 1 -- 172.21.15.42:0/3546015251 wait complete. 2024-04-25T00:16:37.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:37 smithi136 ceph-mon[32923]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.3 KiB/s wr, 16 op/s 2024-04-25T00:16:37.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:37 smithi106 ceph-mon[32510]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.3 KiB/s wr, 16 op/s 2024-04-25T00:16:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:37 smithi042 ceph-mon[28130]: pgmap v321: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 8.3 KiB/s wr, 16 op/s 2024-04-25T00:16:38.445 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6800","nonce":384891702},{"type":"v1","addr":"172.21.15.42:6801","nonce":384891702}]},"active_addr":"172.21.15.42:6801/384891702","active_change":"2024-04-25T00:08:25.764082+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.42:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1157984729}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":3660948450}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":639384425}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":4114878478}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.42:0","nonce":1605599300}]}]} 2024-04-25T00:16:38.452 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T00:16:38.452 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T00:16:38.452 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:16:38.522 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:38 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3546015251' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:16:38.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:38 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3546015251' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:16:38.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:38 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3546015251' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T00:16:38.891 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:39.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:39 smithi042 ceph-mon[28130]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-04-25T00:16:39.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:39 smithi136 ceph-mon[32923]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-04-25T00:16:39.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:39 smithi106 ceph-mon[32510]: pgmap v322: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 6.6 KiB/s wr, 12 op/s 2024-04-25T00:16:41.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.256+0000 7f0056000700 1 -- 172.21.15.42:0/99356919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0050069f40 msgr2=0x7f005006a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:41.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.257+0000 7f0056000700 1 --2- 172.21.15.42:0/99356919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0050069f40 0x7f005006a3a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0038009990 tx=0x7f0038009ca0 comp rx=0 tx=0).stop 2024-04-25T00:16:41.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.257+0000 7f0056000700 1 -- 172.21.15.42:0/99356919 shutdown_connections 2024-04-25T00:16:41.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.257+0000 7f0056000700 1 --2- 172.21.15.42:0/99356919 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f005006a8e0 0x7f005010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:41.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.257+0000 7f0056000700 1 --2- 172.21.15.42:0/99356919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00501082b0 0x7f0050108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:41.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.257+0000 7f0056000700 1 --2- 172.21.15.42:0/99356919 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0050069f40 0x7f005006a3a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:41.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.257+0000 7f0056000700 1 -- 172.21.15.42:0/99356919 >> 172.21.15.42:0/99356919 conn(0x7f00500780e0 msgr2=0x7f00500784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:41.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.257+0000 7f0056000700 1 -- 172.21.15.42:0/99356919 shutdown_connections 2024-04-25T00:16:41.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.257+0000 7f0056000700 1 -- 172.21.15.42:0/99356919 wait complete. 2024-04-25T00:16:41.260 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.258+0000 7f0056000700 1 Processor -- start 2024-04-25T00:16:41.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f0056000700 1 -- start start 2024-04-25T00:16:41.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f0056000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0050069f40 0x7f0050101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:41.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f0056000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005006a8e0 0x7f0050101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:41.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f0056000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00501082b0 0x7f0050101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:41.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f0056000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0050112510 con 0x7f00501082b0 2024-04-25T00:16:41.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f0056000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0050112390 con 0x7f0050069f40 2024-04-25T00:16:41.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f0056000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0050112690 con 0x7f005006a8e0 2024-04-25T00:16:41.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f004fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00501082b0 0x7f0050101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:41.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f004fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00501082b0 0x7f0050101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47858/0 (socket says 172.21.15.42:47858) 2024-04-25T00:16:41.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f004fe00700 1 -- 172.21.15.42:0/2608819645 learned_addr learned my addr 172.21.15.42:0/2608819645 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:41.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f0046a00700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005006a8e0 0x7f0050101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:41.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.259+0000 7f004f400700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0050069f40 0x7f0050101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:41.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.260+0000 7f004fe00700 1 -- 172.21.15.42:0/2608819645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005006a8e0 msgr2=0x7f0050101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:41.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.260+0000 7f004fe00700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005006a8e0 0x7f0050101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:41.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.260+0000 7f004fe00700 1 -- 172.21.15.42:0/2608819645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0050069f40 msgr2=0x7f0050101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:41.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.260+0000 7f004fe00700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0050069f40 0x7f0050101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:41.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.260+0000 7f004fe00700 1 -- 172.21.15.42:0/2608819645 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00501a6f40 con 0x7f00501082b0 2024-04-25T00:16:41.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.260+0000 7f0046a00700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005006a8e0 0x7f0050101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:16:41.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.260+0000 7f004fe00700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00501082b0 0x7f0050101c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f004800d980 tx=0x7f004800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:41.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.260+0000 7f004cc00700 1 -- 172.21.15.42:0/2608819645 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048016070 con 0x7f00501082b0 2024-04-25T00:16:41.263 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.261+0000 7f004cc00700 1 -- 172.21.15.42:0/2608819645 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0048004d10 con 0x7f00501082b0 2024-04-25T00:16:41.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.261+0000 7f0056000700 1 -- 172.21.15.42:0/2608819645 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00501a7230 con 0x7f00501082b0 2024-04-25T00:16:41.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.261+0000 7f004cc00700 1 -- 172.21.15.42:0/2608819645 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00480056c0 con 0x7f00501082b0 2024-04-25T00:16:41.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.261+0000 7f0056000700 1 -- 172.21.15.42:0/2608819645 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00501a76f0 con 0x7f00501082b0 2024-04-25T00:16:41.264 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.262+0000 7f0056000700 1 -- 172.21.15.42:0/2608819645 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0050068f20 con 0x7f00501082b0 2024-04-25T00:16:41.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.264+0000 7f004cc00700 1 -- 172.21.15.42:0/2608819645 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0048004750 con 0x7f00501082b0 2024-04-25T00:16:41.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.265+0000 7f004cc00700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00340a2cb0 0x7f00340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:41.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.265+0000 7f004cc00700 1 -- 172.21.15.42:0/2608819645 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f00480cede0 con 0x7f00501082b0 2024-04-25T00:16:41.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.265+0000 7f004f400700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00340a2cb0 0x7f00340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:41.267 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.266+0000 7f004f400700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00340a2cb0 0x7f00340a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f004000f540 tx=0x7f004000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:41.271 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.270+0000 7f004cc00700 1 -- 172.21.15.42:0/2608819645 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f004801f020 con 0x7f00501082b0 2024-04-25T00:16:41.504 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.502+0000 7f0056000700 1 -- 172.21.15.42:0/2608819645 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f005002dcf0 con 0x7f00501082b0 2024-04-25T00:16:41.506 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.505+0000 7f004cc00700 1 -- 172.21.15.42:0/2608819645 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22217 (secure 0 0 0) 0x7f004809bc90 con 0x7f00501082b0 2024-04-25T00:16:41.507 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:41.507 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":86,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:16:21.219969+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:41.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.508+0000 7f0056000700 1 -- 172.21.15.42:0/2608819645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00340a2cb0 msgr2=0x7f00340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:41.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.508+0000 7f0056000700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00340a2cb0 0x7f00340a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f004000f540 tx=0x7f004000b040 comp rx=0 tx=0).stop 2024-04-25T00:16:41.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.508+0000 7f0056000700 1 -- 172.21.15.42:0/2608819645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00501082b0 msgr2=0x7f0050101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:41.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.508+0000 7f0056000700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00501082b0 0x7f0050101c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f004800d980 tx=0x7f004800dc90 comp rx=0 tx=0).stop 2024-04-25T00:16:41.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.509+0000 7f0056000700 1 -- 172.21.15.42:0/2608819645 shutdown_connections 2024-04-25T00:16:41.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.509+0000 7f0056000700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00340a2cb0 0x7f00340a5170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:41.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.509+0000 7f0056000700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f00501082b0 0x7f0050101c80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:41.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.509+0000 7f0056000700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0050069f40 0x7f0050101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:41.510 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.509+0000 7f0056000700 1 --2- 172.21.15.42:0/2608819645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f005006a8e0 0x7f0050101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:41.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.509+0000 7f0056000700 1 -- 172.21.15.42:0/2608819645 >> 172.21.15.42:0/2608819645 conn(0x7f00500780e0 msgr2=0x7f00500ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:41.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.509+0000 7f0056000700 1 -- 172.21.15.42:0/2608819645 shutdown_connections 2024-04-25T00:16:41.511 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:41.509+0000 7f0056000700 1 -- 172.21.15.42:0/2608819645 wait complete. 2024-04-25T00:16:41.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:41 smithi136 ceph-mon[32923]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-04-25T00:16:41.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:41 smithi106 ceph-mon[32510]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-04-25T00:16:41.839 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:41 smithi042 ceph-mon[28130]: pgmap v323: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 8 op/s 2024-04-25T00:16:42.361 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T00:16:42.361 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T00:16:42.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:42 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2608819645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:42.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:42 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2608819645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:42.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:42 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2608819645' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:42.846 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:43.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:43 smithi042 ceph-mon[28130]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-25T00:16:43.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:43 smithi136 ceph-mon[32923]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-25T00:16:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:43 smithi106 ceph-mon[32510]: pgmap v324: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 5.2 KiB/s wr, 9 op/s 2024-04-25T00:16:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.317+0000 7f4780a00700 1 -- 172.21.15.42:0/542525137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f477c06a8e0 msgr2=0x7f477c10f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.317+0000 7f4780a00700 1 --2- 172.21.15.42:0/542525137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f477c06a8e0 0x7f477c10f0f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4774009f90 tx=0x7f477400b330 comp rx=0 tx=0).stop 2024-04-25T00:16:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.317+0000 7f4780a00700 1 -- 172.21.15.42:0/542525137 shutdown_connections 2024-04-25T00:16:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.317+0000 7f4780a00700 1 --2- 172.21.15.42:0/542525137 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f477c06a8e0 0x7f477c10f0f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:45.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.317+0000 7f4780a00700 1 --2- 172.21.15.42:0/542525137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f477c1082b0 0x7f477c108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:45.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.317+0000 7f4780a00700 1 --2- 172.21.15.42:0/542525137 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f477c069f40 0x7f477c06a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:45.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.317+0000 7f4780a00700 1 -- 172.21.15.42:0/542525137 >> 172.21.15.42:0/542525137 conn(0x7f477c0780e0 msgr2=0x7f477c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:45.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.317+0000 7f4780a00700 1 -- 172.21.15.42:0/542525137 shutdown_connections 2024-04-25T00:16:45.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.317+0000 7f4780a00700 1 -- 172.21.15.42:0/542525137 wait complete. 2024-04-25T00:16:45.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.318+0000 7f4780a00700 1 Processor -- start 2024-04-25T00:16:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.318+0000 7f4780a00700 1 -- start start 2024-04-25T00:16:45.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f4780a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f477c069f40 0x7f477c1a0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f4780a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f477c06a8e0 0x7f477c1a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f4780a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f477c1082b0 0x7f477c19aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f4780a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f477c112550 con 0x7f477c1082b0 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f4780a00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1123d0 con 0x7f477c069f40 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f4780a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f477c1126d0 con 0x7f477c06a8e0 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f4779e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f477c069f40 0x7f477c1a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f477a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f477c1082b0 0x7f477c19aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f4779e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f477c069f40 0x7f477c1a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58726/0 (socket says 172.21.15.42:58726) 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f477a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f477c1082b0 0x7f477c19aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47876/0 (socket says 172.21.15.42:47876) 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f4779e00700 1 -- 172.21.15.42:0/3452530314 learned_addr learned my addr 172.21.15.42:0/3452530314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.319+0000 7f4779400700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f477c06a8e0 0x7f477c1a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:45.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.320+0000 7f4779e00700 1 -- 172.21.15.42:0/3452530314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f477c06a8e0 msgr2=0x7f477c1a0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.320+0000 7f4779e00700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f477c06a8e0 0x7f477c1a0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.320+0000 7f4779e00700 1 -- 172.21.15.42:0/3452530314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f477c1082b0 msgr2=0x7f477c19aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.320+0000 7f4779e00700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f477c1082b0 0x7f477c19aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.320+0000 7f4779e00700 1 -- 172.21.15.42:0/3452530314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c19b180 con 0x7f477c069f40 2024-04-25T00:16:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.320+0000 7f4779e00700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f477c069f40 0x7f477c1a0860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e9a0 tx=0x7f476c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.320+0000 7f4772a00700 1 -- 172.21.15.42:0/3452530314 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c00c9c0 con 0x7f477c069f40 2024-04-25T00:16:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.320+0000 7f4772a00700 1 -- 172.21.15.42:0/3452530314 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f476c00cb60 con 0x7f477c069f40 2024-04-25T00:16:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.320+0000 7f4780a00700 1 -- 172.21.15.42:0/3452530314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477c19b470 con 0x7f477c069f40 2024-04-25T00:16:45.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.320+0000 7f4772a00700 1 -- 172.21.15.42:0/3452530314 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c018660 con 0x7f477c069f40 2024-04-25T00:16:45.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.321+0000 7f4780a00700 1 -- 172.21.15.42:0/3452530314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477c1a7730 con 0x7f477c069f40 2024-04-25T00:16:45.324 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.322+0000 7f4780a00700 1 -- 172.21.15.42:0/3452530314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f477c068f20 con 0x7f477c069f40 2024-04-25T00:16:45.325 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.324+0000 7f4772a00700 1 -- 172.21.15.42:0/3452530314 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f476c00cd10 con 0x7f477c069f40 2024-04-25T00:16:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.325+0000 7f4772a00700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f47600a2cb0 0x7f47600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.325+0000 7f4772a00700 1 -- 172.21.15.42:0/3452530314 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f476c0cf420 con 0x7f477c069f40 2024-04-25T00:16:45.326 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.325+0000 7f4779400700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f47600a2cb0 0x7f47600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:45.327 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.326+0000 7f4779400700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f47600a2cb0 0x7f47600a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f477c103190 tx=0x7f4768009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:45.328 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.327+0000 7f4772a00700 1 -- 172.21.15.42:0/3452530314 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f476c09c2d0 con 0x7f477c069f40 2024-04-25T00:16:45.561 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.560+0000 7f4780a00700 1 -- 172.21.15.42:0/3452530314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f477c1a7960 con 0x7f477c069f40 2024-04-25T00:16:45.563 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.562+0000 7f4772a00700 1 -- 172.21.15.42:0/3452530314 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+22217 (secure 0 0 0) 0x7f476c09bbd0 con 0x7f477c069f40 2024-04-25T00:16:45.564 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:16:45.564 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":86,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:16:21.219969+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:16:45.566 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 -- 172.21.15.42:0/3452530314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f47600a2cb0 msgr2=0x7f47600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f47600a2cb0 0x7f47600a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f477c103190 tx=0x7f4768009310 comp rx=0 tx=0).stop 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 -- 172.21.15.42:0/3452530314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f477c069f40 msgr2=0x7f477c1a0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f477c069f40 0x7f477c1a0860 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e9a0 tx=0x7f476c00ecb0 comp rx=0 tx=0).stop 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 -- 172.21.15.42:0/3452530314 shutdown_connections 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f47600a2cb0 0x7f47600a5170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f477c1082b0 0x7f477c19aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f477c069f40 0x7f477c1a0860 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 --2- 172.21.15.42:0/3452530314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f477c06a8e0 0x7f477c1a0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 -- 172.21.15.42:0/3452530314 >> 172.21.15.42:0/3452530314 conn(0x7f477c0780e0 msgr2=0x7f477c10d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 -- 172.21.15.42:0/3452530314 shutdown_connections 2024-04-25T00:16:45.567 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:45.565+0000 7f4780a00700 1 -- 172.21.15.42:0/3452530314 wait complete. 2024-04-25T00:16:45.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:45 smithi136 ceph-mon[32923]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T00:16:45.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:45 smithi106 ceph-mon[32510]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T00:16:45.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:45 smithi042 ceph-mon[28130]: pgmap v325: 97 pgs: 97 active+clean; 636 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T00:16:46.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T00:16:46.533 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T00:16:46.534 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T00:16:46.534 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T00:16:46.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T00:16:46.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T00:16:46.535 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T00:16:46.536 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T00:16:46.536 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T00:16:46.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T00:16:46.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T00:16:46.537 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T00:16:46.697 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:46 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3452530314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:46.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:46 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3452530314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:46.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:46 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3452530314' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:16:47.739 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:47 smithi106 ceph-mon[32510]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T00:16:47.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:47 smithi136 ceph-mon[32923]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T00:16:47.780 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.782 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.786 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.788 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.790 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.793 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.794 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.802 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.814 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.815 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.817 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.835 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:16:47.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:47 smithi042 ceph-mon[28130]: pgmap v326: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T00:16:49.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:49 smithi042 ceph-mon[28130]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:16:49.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:49 smithi136 ceph-mon[32923]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:16:49.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:49 smithi106 ceph-mon[32510]: pgmap v327: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T00:16:51.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:51 smithi042 ceph-mon[28130]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:16:51.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:51 smithi136 ceph-mon[32923]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:16:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:51 smithi106 ceph-mon[32510]: pgmap v328: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:16:53.703 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:53 smithi042 ceph-mon[28130]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T00:16:53.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:53 smithi136 ceph-mon[32923]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T00:16:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:53 smithi106 ceph-mon[32510]: pgmap v329: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T00:16:55.727 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:55 smithi042 ceph-mon[28130]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:16:55.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:55 smithi136 ceph-mon[32923]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:16:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:55 smithi106 ceph-mon[32510]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T00:16:57.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.269+0000 7f49d4400700 1 -- 172.21.15.42:0/2464679536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49cc104c70 msgr2=0x7f49cc105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:57.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.269+0000 7f49d4400700 1 --2- 172.21.15.42:0/2464679536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49cc104c70 0x7f49cc105100 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f49c400b0a0 tx=0x7f49c400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:16:57.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.270+0000 7f49d4400700 1 -- 172.21.15.42:0/2464679536 shutdown_connections 2024-04-25T00:16:57.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.270+0000 7f49d4400700 1 --2- 172.21.15.42:0/2464679536 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49cc104c70 0x7f49cc105100 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:57.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.270+0000 7f49d4400700 1 --2- 172.21.15.42:0/2464679536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc107160 0x7f49cc069f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:57.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.270+0000 7f49d4400700 1 --2- 172.21.15.42:0/2464679536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc06a480 0x7f49cc06a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:57.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.270+0000 7f49d4400700 1 -- 172.21.15.42:0/2464679536 >> 172.21.15.42:0/2464679536 conn(0x7f49cc0780e0 msgr2=0x7f49cc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:57.318 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.270+0000 7f49d4400700 1 -- 172.21.15.42:0/2464679536 shutdown_connections 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.270+0000 7f49d4400700 1 -- 172.21.15.42:0/2464679536 wait complete. 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.271+0000 7f49d4400700 1 Processor -- start 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.271+0000 7f49d4400700 1 -- start start 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.271+0000 7f49d4400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06a480 0x7f49cc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d4400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc104c70 0x7f49cc19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d4400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49cc107160 0x7f49cc1a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d4400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc118b40 con 0x7f49cc107160 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d4400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc1189c0 con 0x7f49cc06a480 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d4400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49cc118cc0 con 0x7f49cc104c70 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d1a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06a480 0x7f49cc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d1000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc104c70 0x7f49cc19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:57.319 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d1a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06a480 0x7f49cc19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45074/0 (socket says 172.21.15.42:45074) 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d1a00700 1 -- 172.21.15.42:0/1229874978 learned_addr learned my addr 172.21.15.42:0/1229874978 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d2400700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49cc107160 0x7f49cc1a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d1a00700 1 -- 172.21.15.42:0/1229874978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc104c70 msgr2=0x7f49cc19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d1a00700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f49cc104c70 0x7f49cc19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d1a00700 1 -- 172.21.15.42:0/1229874978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49cc107160 msgr2=0x7f49cc1a3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d1a00700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f49cc107160 0x7f49cc1a3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.272+0000 7f49d1a00700 1 -- 172.21.15.42:0/1229874978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49cc1a3fb0 con 0x7f49cc06a480 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.273+0000 7f49d1a00700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06a480 0x7f49cc19eff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e9a0 tx=0x7f49c000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.273+0000 7f49ca000700 1 -- 172.21.15.42:0/1229874978 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c000ca10 con 0x7f49cc06a480 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.273+0000 7f49ca000700 1 -- 172.21.15.42:0/1229874978 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49c000cbb0 con 0x7f49cc06a480 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.273+0000 7f49ca000700 1 -- 172.21.15.42:0/1229874978 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0018680 con 0x7f49cc06a480 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.273+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49cc1a42a0 con 0x7f49cc06a480 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.273+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49cc1abb20 con 0x7f49cc06a480 2024-04-25T00:16:57.320 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.274+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f49cc1046b0 con 0x7f49cc06a480 2024-04-25T00:16:57.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.276+0000 7f49ca000700 1 -- 172.21.15.42:0/1229874978 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f49c0010430 con 0x7f49cc06a480 2024-04-25T00:16:57.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.277+0000 7f49ca000700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49b40a2d30 0x7f49b40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:57.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.278+0000 7f49ca000700 1 -- 172.21.15.42:0/1229874978 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f49c00cfa50 con 0x7f49cc06a480 2024-04-25T00:16:57.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.278+0000 7f49d1000700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49b40a2d30 0x7f49b40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:57.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.278+0000 7f49ca000700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] conn(0x7f49b40ac510 0x7f49b40ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:16:57.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.278+0000 7f49ca000700 1 -- 172.21.15.42:0/1229874978 --> [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49c001bec0 con 0x7f49b40ac510 2024-04-25T00:16:57.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.278+0000 7f49ca000700 1 -- 172.21.15.42:0/1229874978 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f49c00cfe00 con 0x7f49cc06a480 2024-04-25T00:16:57.321 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.278+0000 7f49d2400700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] conn(0x7f49b40ac510 0x7f49b40ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.278+0000 7f49d1000700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49b40a2d30 0x7f49b40a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f49cc06a210 tx=0x7f49b80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.279+0000 7f49d2400700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] conn(0x7f49b40ac510 0x7f49b40ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.280+0000 7f49ca000700 1 -- 172.21.15.42:0/1229874978 <== osd.8 v2:172.21.15.136:6816/3281651567 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49c001bec0 con 0x7f49b40ac510 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.299+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 --> [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f49cc101750 con 0x7f49b40ac510 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.300+0000 7f49ca000700 1 -- 172.21.15.42:0/1229874978 <== osd.8 v2:172.21.15.136:6816/3281651567 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f49cc101750 con 0x7f49b40ac510 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.300+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 >> [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] conn(0x7f49b40ac510 msgr2=0x7f49b40ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.300+0000 7f49d4400700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] conn(0x7f49b40ac510 0x7f49b40ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.300+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49b40a2d30 msgr2=0x7f49b40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.300+0000 7f49d4400700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49b40a2d30 0x7f49b40a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f49cc06a210 tx=0x7f49b80073d0 comp rx=0 tx=0).stop 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.300+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06a480 msgr2=0x7f49cc19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.300+0000 7f49d4400700 1 --2- 172.21.15.42:0/1229874978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06a480 0x7f49cc19eff0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f49c000e9a0 tx=0x7f49c000ecb0 comp rx=0 tx=0).stop 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.300+0000 7f49d2400700 1 -- 172.21.15.42:0/1229874978 reap_dead start 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.300+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 shutdown_connections 2024-04-25T00:16:57.322 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.300+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 >> 172.21.15.42:0/1229874978 conn(0x7f49cc0780e0 msgr2=0x7f49cc103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:16:57.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.301+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 shutdown_connections 2024-04-25T00:16:57.323 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:16:57.301+0000 7f49d4400700 1 -- 172.21.15.42:0/1229874978 wait complete. 2024-04-25T00:16:57.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:57 smithi042 ceph-mon[28130]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T00:16:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:57 smithi136 ceph-mon[32923]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T00:16:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:57 smithi106 ceph-mon[32510]: pgmap v331: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-04-25T00:16:59.618 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:16:59 smithi042 ceph-mon[28130]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T00:16:59.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:16:59 smithi136 ceph-mon[32923]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T00:16:59.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:16:59 smithi106 ceph-mon[32510]: pgmap v332: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.138+0000 7f78a5400700 1 -- 172.21.15.42:0/1396278908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78a006a8e0 msgr2=0x7f78a010f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.138+0000 7f78a5400700 1 --2- 172.21.15.42:0/1396278908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78a006a8e0 0x7f78a010f0f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f789800b0a0 tx=0x7f789800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.139+0000 7f78a5400700 1 -- 172.21.15.42:0/1396278908 shutdown_connections 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.139+0000 7f78a5400700 1 --2- 172.21.15.42:0/1396278908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78a006a8e0 0x7f78a010f0f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.139+0000 7f78a5400700 1 --2- 172.21.15.42:0/1396278908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a01082b0 0x7f78a0108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.139+0000 7f78a5400700 1 --2- 172.21.15.42:0/1396278908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0069f40 0x7f78a006a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.139+0000 7f78a5400700 1 -- 172.21.15.42:0/1396278908 >> 172.21.15.42:0/1396278908 conn(0x7f78a00780e0 msgr2=0x7f78a00784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.139+0000 7f78a5400700 1 -- 172.21.15.42:0/1396278908 shutdown_connections 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.139+0000 7f78a5400700 1 -- 172.21.15.42:0/1396278908 wait complete. 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.140+0000 7f78a5400700 1 Processor -- start 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.141+0000 7f78a5400700 1 -- start start 2024-04-25T00:17:00.196 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.141+0000 7f78a5400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0069f40 0x7f78a01a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.141+0000 7f78a5400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a006a8e0 0x7f78a01a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.141+0000 7f78a5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78a01082b0 0x7f78a019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.141+0000 7f78a5400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0112580 con 0x7f78a01082b0 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.141+0000 7f78a5400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0112400 con 0x7f78a0069f40 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.141+0000 7f78a5400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f78a0112700 con 0x7f78a006a8e0 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789f200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78a01082b0 0x7f78a019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a006a8e0 0x7f78a01a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789e800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0069f40 0x7f78a01a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789de00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a006a8e0 0x7f78a01a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:36458/0 (socket says 172.21.15.42:36458) 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789de00700 1 -- 172.21.15.42:0/1331090308 learned_addr learned my addr 172.21.15.42:0/1331090308 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789e800700 1 -- 172.21.15.42:0/1331090308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a006a8e0 msgr2=0x7f78a01a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789e800700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f78a006a8e0 0x7f78a01a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:00.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789e800700 1 -- 172.21.15.42:0/1331090308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78a01082b0 msgr2=0x7f78a019aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:00.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789e800700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f78a01082b0 0x7f78a019aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:00.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789e800700 1 -- 172.21.15.42:0/1331090308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78a019b170 con 0x7f78a0069f40 2024-04-25T00:17:00.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.142+0000 7f789e800700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0069f40 0x7f78a01a0850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f789000e9f0 tx=0x7f789000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:00.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.143+0000 7f7897400700 1 -- 172.21.15.42:0/1331090308 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890004220 con 0x7f78a0069f40 2024-04-25T00:17:00.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.143+0000 7f7897400700 1 -- 172.21.15.42:0/1331090308 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7890004d10 con 0x7f78a0069f40 2024-04-25T00:17:00.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.143+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78a019b460 con 0x7f78a0069f40 2024-04-25T00:17:00.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.143+0000 7f7897400700 1 -- 172.21.15.42:0/1331090308 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890005020 con 0x7f78a0069f40 2024-04-25T00:17:00.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.143+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78a01a7720 con 0x7f78a0069f40 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.144+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78a010ded0 con 0x7f78a0069f40 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.146+0000 7f7897400700 1 -- 172.21.15.42:0/1331090308 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7890004750 con 0x7f78a0069f40 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.147+0000 7f7897400700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f78840a2cb0 0x7f78840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.148+0000 7f7897400700 1 -- 172.21.15.42:0/1331090308 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f78900cf1f0 con 0x7f78a0069f40 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.148+0000 7f789de00700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f78840a2cb0 0x7f78840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.148+0000 7f7897400700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] conn(0x7f78840ac490 0x7f78840ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.148+0000 7f7897400700 1 -- 172.21.15.42:0/1331090308 --> [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78840aef60 con 0x7f78840ac490 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.148+0000 7f7897400700 1 -- 172.21.15.42:0/1331090308 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f78900cf5a0 con 0x7f78a0069f40 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.148+0000 7f789de00700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f78840a2cb0 0x7f78840a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f788c009680 tx=0x7f788c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.148+0000 7f789f200700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] conn(0x7f78840ac490 0x7f78840ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.149+0000 7f789f200700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] conn(0x7f78840ac490 0x7f78840ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.150+0000 7f7897400700 1 -- 172.21.15.42:0/1331090308 <== osd.5 v2:172.21.15.136:6808/3211315252 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78840aef60 con 0x7f78840ac490 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.168+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 --> [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f78a0103960 con 0x7f78840ac490 2024-04-25T00:17:00.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.169+0000 7f7897400700 1 -- 172.21.15.42:0/1331090308 <== osd.5 v2:172.21.15.136:6808/3211315252 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f78a0103960 con 0x7f78840ac490 2024-04-25T00:17:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.169+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 >> [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] conn(0x7f78840ac490 msgr2=0x7f78840ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.169+0000 7f78a5400700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] conn(0x7f78840ac490 0x7f78840ae8b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.170+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f78840a2cb0 msgr2=0x7f78840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.170+0000 7f78a5400700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f78840a2cb0 0x7f78840a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f788c009680 tx=0x7f788c009260 comp rx=0 tx=0).stop 2024-04-25T00:17:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.170+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0069f40 msgr2=0x7f78a01a0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.170+0000 7f78a5400700 1 --2- 172.21.15.42:0/1331090308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78a0069f40 0x7f78a01a0850 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f789000e9f0 tx=0x7f789000ed00 comp rx=0 tx=0).stop 2024-04-25T00:17:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.170+0000 7f789f200700 1 -- 172.21.15.42:0/1331090308 reap_dead start 2024-04-25T00:17:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.170+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 shutdown_connections 2024-04-25T00:17:00.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.170+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 >> 172.21.15.42:0/1331090308 conn(0x7f78a00780e0 msgr2=0x7f78a010d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:00.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.170+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 shutdown_connections 2024-04-25T00:17:00.201 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:00.170+0000 7f78a5400700 1 -- 172.21.15.42:0/1331090308 wait complete. 2024-04-25T00:17:01.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:01 smithi042 ceph-mon[28130]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:17:01.635 INFO:teuthology.orchestra.run.smithi042.stdout:257698037798 2024-04-25T00:17:01.635 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T00:17:01.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:01 smithi136 ceph-mon[32923]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:17:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:01 smithi106 ceph-mon[32510]: pgmap v333: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:17:03.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.040+0000 7effa9600700 1 -- 172.21.15.42:0/4068420045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effa40745d0 msgr2=0x7effa40749b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:03.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.040+0000 7effa9600700 1 --2- 172.21.15.42:0/4068420045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effa40745d0 0x7effa40749b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7eff940098f0 tx=0x7eff94009c00 comp rx=0 tx=0).stop 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.040+0000 7effa9600700 1 -- 172.21.15.42:0/4068420045 shutdown_connections 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.040+0000 7effa9600700 1 --2- 172.21.15.42:0/4068420045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa4113170 0x7effa4115560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.040+0000 7effa9600700 1 --2- 172.21.15.42:0/4068420045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effa40745d0 0x7effa40749b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.040+0000 7effa9600700 1 --2- 172.21.15.42:0/4068420045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4074ef0 0x7effa4112c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.040+0000 7effa9600700 1 -- 172.21.15.42:0/4068420045 >> 172.21.15.42:0/4068420045 conn(0x7effa40fdcb0 msgr2=0x7effa41000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.040+0000 7effa9600700 1 -- 172.21.15.42:0/4068420045 shutdown_connections 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.040+0000 7effa9600700 1 -- 172.21.15.42:0/4068420045 wait complete. 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.041+0000 7effa9600700 1 Processor -- start 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.041+0000 7effa9600700 1 -- start start 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa9600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa40745d0 0x7effa419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:03.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa9600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4074ef0 0x7effa419f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa9600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effa4113170 0x7effa41a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa9600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7effa4118ad0 con 0x7effa40745d0 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa9600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7effa4118950 con 0x7effa4113170 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa9600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7effa4118c50 con 0x7effa4074ef0 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa40745d0 0x7effa419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa40745d0 0x7effa419efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33214/0 (socket says 172.21.15.42:33214) 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa3e00700 1 -- 172.21.15.42:0/3516975473 learned_addr learned my addr 172.21.15.42:0/3516975473 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa8c00700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effa4113170 0x7effa41a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7eff9be00700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4074ef0 0x7effa419f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.042+0000 7effa3e00700 1 -- 172.21.15.42:0/3516975473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4074ef0 msgr2=0x7effa419f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:03.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa3e00700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4074ef0 0x7effa419f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa3e00700 1 -- 172.21.15.42:0/3516975473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effa4113170 msgr2=0x7effa41a3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa3e00700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effa4113170 0x7effa41a3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa3e00700 1 -- 172.21.15.42:0/3516975473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa41a3fb0 con 0x7effa40745d0 2024-04-25T00:17:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa8c00700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effa4113170 0x7effa41a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa3e00700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa40745d0 0x7effa419efa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff94005950 tx=0x7eff9400fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa1600700 1 -- 172.21.15.42:0/3516975473 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94005140 con 0x7effa40745d0 2024-04-25T00:17:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa1600700 1 -- 172.21.15.42:0/3516975473 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff9400b560 con 0x7effa40745d0 2024-04-25T00:17:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa1600700 1 -- 172.21.15.42:0/3516975473 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff94003ae0 con 0x7effa40745d0 2024-04-25T00:17:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa41a4240 con 0x7effa40745d0 2024-04-25T00:17:03.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.043+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa41abb20 con 0x7effa40745d0 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.044+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7effa4062940 con 0x7effa40745d0 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.046+0000 7effa1600700 1 -- 172.21.15.42:0/3516975473 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7eff9400ba80 con 0x7effa40745d0 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.048+0000 7effa1600700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff880a2d30 0x7eff880a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.048+0000 7effa1600700 1 -- 172.21.15.42:0/3516975473 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7eff940d0440 con 0x7effa40745d0 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.048+0000 7eff9be00700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff880a2d30 0x7eff880a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.048+0000 7effa1600700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] conn(0x7eff880ac3e0 0x7eff880ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.048+0000 7effa1600700 1 -- 172.21.15.42:0/3516975473 --> [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7eff94001d30 con 0x7eff880ac3e0 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.048+0000 7effa1600700 1 -- 172.21.15.42:0/3516975473 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7eff940d07f0 con 0x7effa40745d0 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.048+0000 7effa8c00700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] conn(0x7eff880ac3e0 0x7eff880ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.049+0000 7eff9be00700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff880a2d30 0x7eff880a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009750 tx=0x7eff8c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.049+0000 7effa8c00700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] conn(0x7eff880ac3e0 0x7eff880ae800 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:03.139 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.050+0000 7effa1600700 1 -- 172.21.15.42:0/3516975473 <== osd.6 v2:172.21.15.42:6818/2564442522 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7eff94001d30 con 0x7eff880ac3e0 2024-04-25T00:17:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.068+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 --> [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7effa4062d40 con 0x7eff880ac3e0 2024-04-25T00:17:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.069+0000 7effa1600700 1 -- 172.21.15.42:0/3516975473 <== osd.6 v2:172.21.15.42:6818/2564442522 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7effa4062d40 con 0x7eff880ac3e0 2024-04-25T00:17:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] conn(0x7eff880ac3e0 msgr2=0x7eff880ae800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa9600700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] conn(0x7eff880ac3e0 0x7eff880ae800 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff880a2d30 msgr2=0x7eff880a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:03.140 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa9600700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff880a2d30 0x7eff880a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009750 tx=0x7eff8c009310 comp rx=0 tx=0).stop 2024-04-25T00:17:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa40745d0 msgr2=0x7effa419efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa9600700 1 --2- 172.21.15.42:0/3516975473 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effa40745d0 0x7effa419efa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7eff94005950 tx=0x7eff9400fb20 comp rx=0 tx=0).stop 2024-04-25T00:17:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa8c00700 1 -- 172.21.15.42:0/3516975473 reap_dead start 2024-04-25T00:17:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 shutdown_connections 2024-04-25T00:17:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 >> 172.21.15.42:0/3516975473 conn(0x7effa40fdcb0 msgr2=0x7effa40fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 shutdown_connections 2024-04-25T00:17:03.141 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:03.070+0000 7effa9600700 1 -- 172.21.15.42:0/3516975473 wait complete. 2024-04-25T00:17:03.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:03 smithi136 ceph-mon[32923]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:17:03.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:03 smithi106 ceph-mon[32510]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:17:03.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:03 smithi042 ceph-mon[28130]: pgmap v334: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:17:04.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.731+0000 7f35b7400700 1 -- 172.21.15.42:0/899944689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35b0069f40 msgr2=0x7f35b006a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.731+0000 7f35b7400700 1 --2- 172.21.15.42:0/899944689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35b0069f40 0x7f35b006a3a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f359c009990 tx=0x7f359c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.731+0000 7f35b7400700 1 -- 172.21.15.42:0/899944689 shutdown_connections 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.731+0000 7f35b7400700 1 --2- 172.21.15.42:0/899944689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b006a8e0 0x7f35b010f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.731+0000 7f35b7400700 1 --2- 172.21.15.42:0/899944689 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b01082b0 0x7f35b0108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.731+0000 7f35b7400700 1 --2- 172.21.15.42:0/899944689 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35b0069f40 0x7f35b006a3a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.731+0000 7f35b7400700 1 -- 172.21.15.42:0/899944689 >> 172.21.15.42:0/899944689 conn(0x7f35b00780e0 msgr2=0x7f35b00784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.731+0000 7f35b7400700 1 -- 172.21.15.42:0/899944689 shutdown_connections 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.731+0000 7f35b7400700 1 -- 172.21.15.42:0/899944689 wait complete. 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.733+0000 7f35b7400700 1 Processor -- start 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.733+0000 7f35b7400700 1 -- start start 2024-04-25T00:17:04.386 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.733+0000 7f35b7400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b0069f40 0x7f35b019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.733+0000 7f35b7400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b006a8e0 0x7f35b019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.733+0000 7f35b7400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35b01082b0 0x7f35b0198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.733+0000 7f35b7400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f35b0112560 con 0x7f35b01082b0 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.733+0000 7f35b7400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01123e0 con 0x7f35b006a8e0 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.733+0000 7f35b7400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f35b01126e0 con 0x7f35b0069f40 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.734+0000 7f35b4a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b0069f40 0x7f35b019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.734+0000 7f35b5400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35b01082b0 0x7f35b0198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.734+0000 7f35b4a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b0069f40 0x7f35b019e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:36484/0 (socket says 172.21.15.42:36484) 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.734+0000 7f35b4a00700 1 -- 172.21.15.42:0/2530473044 learned_addr learned my addr 172.21.15.42:0/2530473044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.734+0000 7f35afe00700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b006a8e0 0x7f35b019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.734+0000 7f35b4a00700 1 -- 172.21.15.42:0/2530473044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b006a8e0 msgr2=0x7f35b019ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.734+0000 7f35b4a00700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35b006a8e0 0x7f35b019ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.734+0000 7f35b4a00700 1 -- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35b01082b0 msgr2=0x7f35b0198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.734+0000 7f35b4a00700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f35b01082b0 0x7f35b0198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.734+0000 7f35b4a00700 1 -- 172.21.15.42:0/2530473044 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b0198fe0 con 0x7f35b0069f40 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.735+0000 7f35b4a00700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b0069f40 0x7f35b019e6c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f35a400e9a0 tx=0x7f35a400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.735+0000 7f35ad600700 1 -- 172.21.15.42:0/2530473044 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a400cbd0 con 0x7f35b0069f40 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.735+0000 7f35ad600700 1 -- 172.21.15.42:0/2530473044 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35a400cd70 con 0x7f35b0069f40 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.735+0000 7f35ad600700 1 -- 172.21.15.42:0/2530473044 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a4018710 con 0x7f35b0069f40 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.736+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b01992d0 con 0x7f35b0069f40 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.736+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b01a5590 con 0x7f35b0069f40 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.737+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35b0062980 con 0x7f35b0069f40 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.739+0000 7f35ad600700 1 -- 172.21.15.42:0/2530473044 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f35a4010430 con 0x7f35b0069f40 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.740+0000 7f35ad600700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35980a2d30 0x7f35980a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.741+0000 7f35ad600700 1 -- 172.21.15.42:0/2530473044 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f35a40cfac0 con 0x7f35b0069f40 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.741+0000 7f35ad600700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] conn(0x7f35980ac510 0x7f35980ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.388 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.741+0000 7f35afe00700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35980a2d30 0x7f35980a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.741+0000 7f35ad600700 1 -- 172.21.15.42:0/2530473044 --> [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35980aefe0 con 0x7f35980ac510 2024-04-25T00:17:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.741+0000 7f35ad600700 1 -- 172.21.15.42:0/2530473044 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f35a40cfe70 con 0x7f35b0069f40 2024-04-25T00:17:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.741+0000 7f35b5400700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] conn(0x7f35980ac510 0x7f35980ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.742+0000 7f35afe00700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35980a2d30 0x7f35980a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f359c00b2e0 tx=0x7f359c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.742+0000 7f35b5400700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] conn(0x7f35980ac510 0x7f35980ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.743+0000 7f35ad600700 1 -- 172.21.15.42:0/2530473044 <== osd.3 v2:172.21.15.42:6810/2791262023 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35980aefe0 con 0x7f35980ac510 2024-04-25T00:17:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.762+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 --> [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35b0062e50 con 0x7f35980ac510 2024-04-25T00:17:04.389 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.762+0000 7f35ad600700 1 -- 172.21.15.42:0/2530473044 <== osd.3 v2:172.21.15.42:6810/2791262023 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f35b0062e50 con 0x7f35980ac510 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.763+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] conn(0x7f35980ac510 msgr2=0x7f35980ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.763+0000 7f35b7400700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] conn(0x7f35980ac510 0x7f35980ae930 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.763+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35980a2d30 msgr2=0x7f35980a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.763+0000 7f35b7400700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35980a2d30 0x7f35980a51f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f359c00b2e0 tx=0x7f359c01a040 comp rx=0 tx=0).stop 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.763+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b0069f40 msgr2=0x7f35b019e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.763+0000 7f35b7400700 1 --2- 172.21.15.42:0/2530473044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f35b0069f40 0x7f35b019e6c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f35a400e9a0 tx=0x7f35a400ed60 comp rx=0 tx=0).stop 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.763+0000 7f35b5400700 1 -- 172.21.15.42:0/2530473044 reap_dead start 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.763+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 shutdown_connections 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.763+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 >> 172.21.15.42:0/2530473044 conn(0x7f35b00780e0 msgr2=0x7f35b00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.764+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 shutdown_connections 2024-04-25T00:17:04.390 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:02.764+0000 7f35b7400700 1 -- 172.21.15.42:0/2530473044 wait complete. 2024-04-25T00:17:04.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.462+0000 7f9f93a00700 1 -- 172.21.15.42:0/3346171118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c10ece0 msgr2=0x7f9f8c1110d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.462+0000 7f9f93a00700 1 --2- 172.21.15.42:0/3346171118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c10ece0 0x7f9f8c1110d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400b0a0 tx=0x7f9f8400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.462+0000 7f9f93a00700 1 -- 172.21.15.42:0/3346171118 shutdown_connections 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.462+0000 7f9f93a00700 1 --2- 172.21.15.42:0/3346171118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c10ece0 0x7f9f8c1110d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.462+0000 7f9f93a00700 1 --2- 172.21.15.42:0/3346171118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8c069f40 0x7f9f8c06a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.462+0000 7f9f93a00700 1 --2- 172.21.15.42:0/3346171118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f8c06a860 0x7f9f8c10e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.462+0000 7f9f93a00700 1 -- 172.21.15.42:0/3346171118 >> 172.21.15.42:0/3346171118 conn(0x7f9f8c0780e0 msgr2=0x7f9f8c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.462+0000 7f9f93a00700 1 -- 172.21.15.42:0/3346171118 shutdown_connections 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.462+0000 7f9f93a00700 1 -- 172.21.15.42:0/3346171118 wait complete. 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.463+0000 7f9f93a00700 1 Processor -- start 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.464+0000 7f9f93a00700 1 -- start start 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.464+0000 7f9f93a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c069f40 0x7f9f8c1a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.464+0000 7f9f93a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8c06a860 0x7f9f8c1a0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.464+0000 7f9f93a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f8c10ece0 0x7f9f8c19a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.464+0000 7f9f93a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c114730 con 0x7f9f8c06a860 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.464+0000 7f9f93a00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c1145b0 con 0x7f9f8c069f40 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.464+0000 7f9f93a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9f8c1148b0 con 0x7f9f8c10ece0 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f8be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8c06a860 0x7f9f8c1a0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f8be00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8c06a860 0x7f9f8c1a0d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33226/0 (socket says 172.21.15.42:33226) 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f8be00700 1 -- 172.21.15.42:0/1532623142 learned_addr learned my addr 172.21.15.42:0/1532623142 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f91000700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c069f40 0x7f9f8c1a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f91a00700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f8c10ece0 0x7f9f8c19a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f8be00700 1 -- 172.21.15.42:0/1532623142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f8c10ece0 msgr2=0x7f9f8c19a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f8be00700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f8c10ece0 0x7f9f8c19a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f8be00700 1 -- 172.21.15.42:0/1532623142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c069f40 msgr2=0x7f9f8c1a0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f8be00700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f8c069f40 0x7f9f8c1a0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f8be00700 1 -- 172.21.15.42:0/1532623142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f8c19b130 con 0x7f9f8c06a860 2024-04-25T00:17:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.465+0000 7f9f8be00700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8c06a860 0x7f9f8c1a0d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800e9c0 tx=0x7f9f7800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.466+0000 7f9f89600700 1 -- 172.21.15.42:0/1532623142 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7800cbd0 con 0x7f9f8c06a860 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.466+0000 7f9f89600700 1 -- 172.21.15.42:0/1532623142 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f7800cd70 con 0x7f9f8c06a860 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.466+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8c19b420 con 0x7f9f8c06a860 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.466+0000 7f9f89600700 1 -- 172.21.15.42:0/1532623142 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78010430 con 0x7f9f8c06a860 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.466+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f8c1a76e0 con 0x7f9f8c06a860 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.467+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f8c0629c0 con 0x7f9f8c06a860 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.469+0000 7f9f89600700 1 -- 172.21.15.42:0/1532623142 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9f78004750 con 0x7f9f8c06a860 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.470+0000 7f9f89600700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f740a2d30 0x7f9f740a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.470+0000 7f9f89600700 1 -- 172.21.15.42:0/1532623142 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f9f780cf6f0 con 0x7f9f8c06a860 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.471+0000 7f9f91000700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f740a2d30 0x7f9f740a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.471+0000 7f9f89600700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] conn(0x7f9f740ac510 0x7f9f740ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.471+0000 7f9f89600700 1 -- 172.21.15.42:0/1532623142 --> [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f740aefe0 con 0x7f9f740ac510 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.471+0000 7f9f89600700 1 -- 172.21.15.42:0/1532623142 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f9f780cfaa0 con 0x7f9f8c06a860 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.471+0000 7f9f91a00700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] conn(0x7f9f740ac510 0x7f9f740ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.471+0000 7f9f91000700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f740a2d30 0x7f9f740a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c19bb10 tx=0x7f9f80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.472+0000 7f9f91a00700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] conn(0x7f9f740ac510 0x7f9f740ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:04.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.473+0000 7f9f89600700 1 -- 172.21.15.42:0/1532623142 <== osd.2 v2:172.21.15.136:6800/930826162 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f740aefe0 con 0x7f9f740ac510 2024-04-25T00:17:04.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.494+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 --> [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f8c062e40 con 0x7f9f740ac510 2024-04-25T00:17:04.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.495+0000 7f9f89600700 1 -- 172.21.15.42:0/1532623142 <== osd.2 v2:172.21.15.136:6800/930826162 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9f8c062e40 con 0x7f9f740ac510 2024-04-25T00:17:04.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.495+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 >> [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] conn(0x7f9f740ac510 msgr2=0x7f9f740ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.495+0000 7f9f93a00700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] conn(0x7f9f740ac510 0x7f9f740ae930 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.496+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f740a2d30 msgr2=0x7f9f740a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.496+0000 7f9f93a00700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f740a2d30 0x7f9f740a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c19bb10 tx=0x7f9f80009260 comp rx=0 tx=0).stop 2024-04-25T00:17:04.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.496+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8c06a860 msgr2=0x7f9f8c1a0d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:04.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.496+0000 7f9f93a00700 1 --2- 172.21.15.42:0/1532623142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f8c06a860 0x7f9f8c1a0d50 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800e9c0 tx=0x7f9f7800ecd0 comp rx=0 tx=0).stop 2024-04-25T00:17:04.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.496+0000 7f9f91a00700 1 -- 172.21.15.42:0/1532623142 reap_dead start 2024-04-25T00:17:04.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.496+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 shutdown_connections 2024-04-25T00:17:04.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.496+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 >> 172.21.15.42:0/1532623142 conn(0x7f9f8c0780e0 msgr2=0x7f9f8c1100f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:04.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.496+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 shutdown_connections 2024-04-25T00:17:04.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:04.496+0000 7f9f93a00700 1 -- 172.21.15.42:0/1532623142 wait complete. 2024-04-25T00:17:05.759 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:05 smithi042 ceph-mon[28130]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:17:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:05 smithi136 ceph-mon[32923]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:17:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:05 smithi106 ceph-mon[32510]: pgmap v335: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 170 B/s wr, 5 op/s 2024-04-25T00:17:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.078+0000 7f2f22200700 1 -- 172.21.15.42:0/3761521662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f1c1130e0 msgr2=0x7f2f1c1154d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.078+0000 7f2f22200700 1 --2- 172.21.15.42:0/3761521662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f1c1130e0 0x7f2f1c1154d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400b0a0 tx=0x7f2f1400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.078+0000 7f2f22200700 1 -- 172.21.15.42:0/3761521662 shutdown_connections 2024-04-25T00:17:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.078+0000 7f2f22200700 1 --2- 172.21.15.42:0/3761521662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f1c1130e0 0x7f2f1c1154d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.078+0000 7f2f22200700 1 --2- 172.21.15.42:0/3761521662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f1c076760 0x7f2f1c076b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.078+0000 7f2f22200700 1 --2- 172.21.15.42:0/3761521662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f1c077080 0x7f2f1c112ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.103 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.078+0000 7f2f22200700 1 -- 172.21.15.42:0/3761521662 >> 172.21.15.42:0/3761521662 conn(0x7f2f1c0fff70 msgr2=0x7f2f1c102390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.078+0000 7f2f22200700 1 -- 172.21.15.42:0/3761521662 shutdown_connections 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.078+0000 7f2f22200700 1 -- 172.21.15.42:0/3761521662 wait complete. 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.079+0000 7f2f22200700 1 Processor -- start 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.079+0000 7f2f22200700 1 -- start start 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f22200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f1c076760 0x7f2f1c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f22200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f1c077080 0x7f2f1c19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f22200700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f1c1130e0 0x7f2f1c1a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f22200700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c118bd0 con 0x7f2f1c1130e0 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f22200700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c118a50 con 0x7f2f1c077080 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f22200700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1c118d50 con 0x7f2f1c076760 2024-04-25T00:17:07.104 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f1c077080 0x7f2f1c19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f1aa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f1c077080 0x7f2f1c19f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53122/0 (socket says 172.21.15.42:53122) 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f1aa00700 1 -- 172.21.15.42:0/723026787 learned_addr learned my addr 172.21.15.42:0/723026787 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f1be00700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f1c1130e0 0x7f2f1c1a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.080+0000 7f2f1b400700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f1c076760 0x7f2f1c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.081+0000 7f2f1aa00700 1 -- 172.21.15.42:0/723026787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f1c076760 msgr2=0x7f2f1c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.081+0000 7f2f1aa00700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2f1c076760 0x7f2f1c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.081+0000 7f2f1aa00700 1 -- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f1c1130e0 msgr2=0x7f2f1c1a38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.081+0000 7f2f1aa00700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2f1c1130e0 0x7f2f1c1a38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.081+0000 7f2f1aa00700 1 -- 172.21.15.42:0/723026787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f1c1a4030 con 0x7f2f1c077080 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.081+0000 7f2f1aa00700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f1c077080 0x7f2f1c19f560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800eb30 tx=0x7f2f0800ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.081+0000 7f2f13e00700 1 -- 172.21.15.42:0/723026787 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800cc90 con 0x7f2f1c077080 2024-04-25T00:17:07.105 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.081+0000 7f2f13e00700 1 -- 172.21.15.42:0/723026787 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f08004500 con 0x7f2f1c077080 2024-04-25T00:17:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.081+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f1c1a4320 con 0x7f2f1c077080 2024-04-25T00:17:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.081+0000 7f2f13e00700 1 -- 172.21.15.42:0/723026787 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08010430 con 0x7f2f1c077080 2024-04-25T00:17:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.082+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1c1abba0 con 0x7f2f1c077080 2024-04-25T00:17:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.082+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f1c0629c0 con 0x7f2f1c077080 2024-04-25T00:17:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.087+0000 7f2f13e00700 1 -- 172.21.15.42:0/723026787 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2f08043b10 con 0x7f2f1c077080 2024-04-25T00:17:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.088+0000 7f2f13e00700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f000a2d30 0x7f2f000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.089+0000 7f2f13e00700 1 -- 172.21.15.42:0/723026787 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f2f080cf500 con 0x7f2f1c077080 2024-04-25T00:17:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.089+0000 7f2f1b400700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f000a2d30 0x7f2f000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.089+0000 7f2f13e00700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] conn(0x7f2f000ac510 0x7f2f000ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:07.106 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.089+0000 7f2f13e00700 1 -- 172.21.15.42:0/723026787 --> [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f08044e50 con 0x7f2f000ac510 2024-04-25T00:17:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.089+0000 7f2f13e00700 1 -- 172.21.15.42:0/723026787 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f2f080cf8b0 con 0x7f2f1c077080 2024-04-25T00:17:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.089+0000 7f2f1be00700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] conn(0x7f2f000ac510 0x7f2f000ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.090+0000 7f2f1b400700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f000a2d30 0x7f2f000a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009bf0 tx=0x7f2f0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.090+0000 7f2f1be00700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] conn(0x7f2f000ac510 0x7f2f000ae930 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:07.107 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.091+0000 7f2f13e00700 1 -- 172.21.15.42:0/723026787 <== osd.0 v2:172.21.15.42:6802/2959872296 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2f08044e50 con 0x7f2f000ac510 2024-04-25T00:17:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.110+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 --> [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f1c062df0 con 0x7f2f000ac510 2024-04-25T00:17:07.111 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.111+0000 7f2f13e00700 1 -- 172.21.15.42:0/723026787 <== osd.0 v2:172.21.15.42:6802/2959872296 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2f1c062df0 con 0x7f2f000ac510 2024-04-25T00:17:07.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.111+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] conn(0x7f2f000ac510 msgr2=0x7f2f000ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.111+0000 7f2f22200700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] conn(0x7f2f000ac510 0x7f2f000ae930 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.111+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f000a2d30 msgr2=0x7f2f000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.111+0000 7f2f22200700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f000a2d30 0x7f2f000a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c009bf0 tx=0x7f2f0c009260 comp rx=0 tx=0).stop 2024-04-25T00:17:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.111+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f1c077080 msgr2=0x7f2f1c19f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.111+0000 7f2f22200700 1 --2- 172.21.15.42:0/723026787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f1c077080 0x7f2f1c19f560 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800eb30 tx=0x7f2f0800ee40 comp rx=0 tx=0).stop 2024-04-25T00:17:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.111+0000 7f2f1be00700 1 -- 172.21.15.42:0/723026787 reap_dead start 2024-04-25T00:17:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.111+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 shutdown_connections 2024-04-25T00:17:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.111+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 >> 172.21.15.42:0/723026787 conn(0x7f2f1c0fff70 msgr2=0x7f2f1c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.112+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 shutdown_connections 2024-04-25T00:17:07.113 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.112+0000 7f2f22200700 1 -- 172.21.15.42:0/723026787 wait complete. 2024-04-25T00:17:07.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:07 smithi042 ceph-mon[28130]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T00:17:07.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:07 smithi136 ceph-mon[32923]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T00:17:07.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:07 smithi106 ceph-mon[32510]: pgmap v336: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-04-25T00:17:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.891+0000 7ffa28a00700 1 -- 172.21.15.42:0/3244697281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa2406a480 msgr2=0x7ffa2406a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.891+0000 7ffa28a00700 1 --2- 172.21.15.42:0/3244697281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa2406a480 0x7ffa2406a8e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa100098f0 tx=0x7ffa10009c00 comp rx=0 tx=0).stop 2024-04-25T00:17:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.891+0000 7ffa28a00700 1 -- 172.21.15.42:0/3244697281 shutdown_connections 2024-04-25T00:17:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.891+0000 7ffa28a00700 1 --2- 172.21.15.42:0/3244697281 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa24104c70 0x7ffa24105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.891+0000 7ffa28a00700 1 --2- 172.21.15.42:0/3244697281 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa24107160 0x7ffa24069f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.891+0000 7ffa28a00700 1 --2- 172.21.15.42:0/3244697281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa2406a480 0x7ffa2406a8e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.891+0000 7ffa28a00700 1 -- 172.21.15.42:0/3244697281 >> 172.21.15.42:0/3244697281 conn(0x7ffa240780e0 msgr2=0x7ffa240784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.891+0000 7ffa28a00700 1 -- 172.21.15.42:0/3244697281 shutdown_connections 2024-04-25T00:17:07.893 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.891+0000 7ffa28a00700 1 -- 172.21.15.42:0/3244697281 wait complete. 2024-04-25T00:17:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.892+0000 7ffa28a00700 1 Processor -- start 2024-04-25T00:17:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.892+0000 7ffa28a00700 1 -- start start 2024-04-25T00:17:07.894 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa28a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa2406a480 0x7ffa2419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa28a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa24104c70 0x7ffa2419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa28a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa24107160 0x7ffa241a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa28a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa24118b60 con 0x7ffa24104c70 2024-04-25T00:17:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa28a00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa241189e0 con 0x7ffa24107160 2024-04-25T00:17:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa28a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa24118ce0 con 0x7ffa2406a480 2024-04-25T00:17:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa21400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa24104c70 0x7ffa2419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa21400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa24104c70 0x7ffa2419f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42376/0 (socket says 172.21.15.42:42376) 2024-04-25T00:17:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa21e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa2406a480 0x7ffa2419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa21400700 1 -- 172.21.15.42:0/2364966690 learned_addr learned my addr 172.21.15.42:0/2364966690 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:07.895 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.893+0000 7ffa22800700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa24107160 0x7ffa241a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa21400700 1 -- 172.21.15.42:0/2364966690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa2406a480 msgr2=0x7ffa2419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa21400700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa2406a480 0x7ffa2419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa21400700 1 -- 172.21.15.42:0/2364966690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa24107160 msgr2=0x7ffa241a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa21400700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa24107160 0x7ffa241a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa21400700 1 -- 172.21.15.42:0/2364966690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa241a4080 con 0x7ffa24104c70 2024-04-25T00:17:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa21e00700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa2406a480 0x7ffa2419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:07.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa21400700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa24104c70 0x7ffa2419f5b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005950 tx=0x7ffa1000fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:07.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa1aa00700 1 -- 172.21.15.42:0/2364966690 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa100051f0 con 0x7ffa24104c70 2024-04-25T00:17:07.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa1aa00700 1 -- 172.21.15.42:0/2364966690 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa1000b560 con 0x7ffa24104c70 2024-04-25T00:17:07.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa241a4310 con 0x7ffa24104c70 2024-04-25T00:17:07.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.894+0000 7ffa1aa00700 1 -- 172.21.15.42:0/2364966690 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10003a40 con 0x7ffa24104c70 2024-04-25T00:17:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.895+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa241abbf0 con 0x7ffa24104c70 2024-04-25T00:17:07.898 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.895+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffa241046b0 con 0x7ffa24104c70 2024-04-25T00:17:07.899 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.898+0000 7ffa1aa00700 1 -- 172.21.15.42:0/2364966690 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffa1000ba80 con 0x7ffa24104c70 2024-04-25T00:17:07.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.899+0000 7ffa1aa00700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa080a2cb0 0x7ffa080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:07.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.899+0000 7ffa1aa00700 1 -- 172.21.15.42:0/2364966690 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7ffa100d04c0 con 0x7ffa24104c70 2024-04-25T00:17:07.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.899+0000 7ffa21e00700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa080a2cb0 0x7ffa080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:07.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.900+0000 7ffa1aa00700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] conn(0x7ffa080ac360 0x7ffa080ae780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:07.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.900+0000 7ffa1aa00700 1 -- 172.21.15.42:0/2364966690 --> [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa1000b0b0 con 0x7ffa080ac360 2024-04-25T00:17:07.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.900+0000 7ffa1aa00700 1 -- 172.21.15.42:0/2364966690 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7ffa100d0870 con 0x7ffa24104c70 2024-04-25T00:17:07.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.900+0000 7ffa22800700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] conn(0x7ffa080ac360 0x7ffa080ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:07.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.900+0000 7ffa21e00700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa080a2cb0 0x7ffa080a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa14005fd0 tx=0x7ffa14009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:07.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.900+0000 7ffa22800700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] conn(0x7ffa080ac360 0x7ffa080ae780 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:07.902 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.901+0000 7ffa1aa00700 1 -- 172.21.15.42:0/2364966690 <== osd.7 v2:172.21.15.106:6816/178560796 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa1000b0b0 con 0x7ffa080ac360 2024-04-25T00:17:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.921+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 --> [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffa24101750 con 0x7ffa080ac360 2024-04-25T00:17:07.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.922+0000 7ffa1aa00700 1 -- 172.21.15.42:0/2364966690 <== osd.7 v2:172.21.15.106:6816/178560796 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffa24101750 con 0x7ffa080ac360 2024-04-25T00:17:07.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.923+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 >> [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] conn(0x7ffa080ac360 msgr2=0x7ffa080ae780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.923+0000 7ffa28a00700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] conn(0x7ffa080ac360 0x7ffa080ae780 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:07.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.923+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa080a2cb0 msgr2=0x7ffa080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.923+0000 7ffa28a00700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa080a2cb0 0x7ffa080a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffa14005fd0 tx=0x7ffa14009310 comp rx=0 tx=0).stop 2024-04-25T00:17:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.923+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa24104c70 msgr2=0x7ffa2419f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.923+0000 7ffa28a00700 1 --2- 172.21.15.42:0/2364966690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa24104c70 0x7ffa2419f5b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffa10005950 tx=0x7ffa1000fb20 comp rx=0 tx=0).stop 2024-04-25T00:17:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.923+0000 7ffa22800700 1 -- 172.21.15.42:0/2364966690 reap_dead start 2024-04-25T00:17:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.923+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 shutdown_connections 2024-04-25T00:17:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.923+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 >> 172.21.15.42:0/2364966690 conn(0x7ffa240780e0 msgr2=0x7ffa24103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.924+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 shutdown_connections 2024-04-25T00:17:07.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:07.924+0000 7ffa28a00700 1 -- 172.21.15.42:0/2364966690 wait complete. 2024-04-25T00:17:08.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.668+0000 7f3a3ee00700 1 -- 172.21.15.42:0/2530492506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a38102d60 msgr2=0x7f3a38106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:08.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.668+0000 7f3a3ee00700 1 --2- 172.21.15.42:0/2530492506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a38102d60 0x7f3a38106db0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3a240099c0 tx=0x7f3a24009cd0 comp rx=0 tx=0).stop 2024-04-25T00:17:08.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.669+0000 7f3a3ee00700 1 -- 172.21.15.42:0/2530492506 shutdown_connections 2024-04-25T00:17:08.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.669+0000 7f3a3ee00700 1 --2- 172.21.15.42:0/2530492506 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a38107490 0x7f3a3810f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:08.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.669+0000 7f3a3ee00700 1 --2- 172.21.15.42:0/2530492506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a381023b0 0x7f3a38102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:08.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.669+0000 7f3a3ee00700 1 --2- 172.21.15.42:0/2530492506 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a38102d60 0x7f3a38106db0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:08.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.669+0000 7f3a3ee00700 1 -- 172.21.15.42:0/2530492506 >> 172.21.15.42:0/2530492506 conn(0x7f3a380fdc80 msgr2=0x7f3a381000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:08.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.669+0000 7f3a3ee00700 1 -- 172.21.15.42:0/2530492506 shutdown_connections 2024-04-25T00:17:08.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.669+0000 7f3a3ee00700 1 -- 172.21.15.42:0/2530492506 wait complete. 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.670+0000 7f3a3ee00700 1 Processor -- start 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.670+0000 7f3a3ee00700 1 -- start start 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.670+0000 7f3a3ee00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a381023b0 0x7f3a381a07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3ee00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a38102d60 0x7f3a381a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3ee00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a38107490 0x7f3a3819a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3ee00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3a381124c0 con 0x7f3a38107490 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3ee00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38112340 con 0x7f3a381023b0 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3ee00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a38112640 con 0x7f3a38102d60 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3e400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a38107490 0x7f3a3819a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3da00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a381023b0 0x7f3a381a07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3e400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a38107490 0x7f3a3819a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42392/0 (socket says 172.21.15.42:42392) 2024-04-25T00:17:08.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3e400700 1 -- 172.21.15.42:0/3539659874 learned_addr learned my addr 172.21.15.42:0/3539659874 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:08.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3d000700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a38102d60 0x7f3a381a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:08.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3da00700 1 -- 172.21.15.42:0/3539659874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a38102d60 msgr2=0x7f3a381a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:08.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3da00700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a38102d60 0x7f3a381a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:08.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3da00700 1 -- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a38107490 msgr2=0x7f3a3819a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:08.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3da00700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a38107490 0x7f3a3819a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:08.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3da00700 1 -- 172.21.15.42:0/3539659874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3819b100 con 0x7f3a381023b0 2024-04-25T00:17:08.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.671+0000 7f3a3e400700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a38107490 0x7f3a3819a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:08.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.672+0000 7f3a3da00700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a381023b0 0x7f3a381a07e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00e9f0 tx=0x7f3a2c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:08.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.672+0000 7f3a36000700 1 -- 172.21.15.42:0/3539659874 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c004220 con 0x7f3a381023b0 2024-04-25T00:17:08.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.672+0000 7f3a36000700 1 -- 172.21.15.42:0/3539659874 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a2c004d10 con 0x7f3a381023b0 2024-04-25T00:17:08.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.672+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a3819b3f0 con 0x7f3a381023b0 2024-04-25T00:17:08.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.672+0000 7f3a36000700 1 -- 172.21.15.42:0/3539659874 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a2c005020 con 0x7f3a381023b0 2024-04-25T00:17:08.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.672+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a381a76b0 con 0x7f3a381023b0 2024-04-25T00:17:08.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.673+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3a380ff920 con 0x7f3a381023b0 2024-04-25T00:17:08.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.675+0000 7f3a36000700 1 -- 172.21.15.42:0/3539659874 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3a2c004750 con 0x7f3a381023b0 2024-04-25T00:17:08.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.677+0000 7f3a36000700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a200a2cb0 0x7f3a200a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:08.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.677+0000 7f3a36000700 1 -- 172.21.15.42:0/3539659874 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f3a2c0cefd0 con 0x7f3a381023b0 2024-04-25T00:17:08.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.677+0000 7f3a3d000700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a200a2cb0 0x7f3a200a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:08.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.677+0000 7f3a36000700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] conn(0x7f3a200ac490 0x7f3a200ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:08.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.677+0000 7f3a36000700 1 -- 172.21.15.42:0/3539659874 --> [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3a200aef60 con 0x7f3a200ac490 2024-04-25T00:17:08.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.677+0000 7f3a36000700 1 -- 172.21.15.42:0/3539659874 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f3a2c0d15f0 con 0x7f3a381023b0 2024-04-25T00:17:08.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.677+0000 7f3a3e400700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] conn(0x7f3a200ac490 0x7f3a200ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:08.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.678+0000 7f3a3d000700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a200a2cb0 0x7f3a200a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3a24000c00 tx=0x7f3a24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:08.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.678+0000 7f3a3e400700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] conn(0x7f3a200ac490 0x7f3a200ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:08.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.679+0000 7f3a36000700 1 -- 172.21.15.42:0/3539659874 <== osd.9 v2:172.21.15.42:6826/876648842 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3a200aef60 con 0x7f3a200ac490 2024-04-25T00:17:08.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.697+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 --> [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3a38103890 con 0x7f3a200ac490 2024-04-25T00:17:08.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.698+0000 7f3a36000700 1 -- 172.21.15.42:0/3539659874 <== osd.9 v2:172.21.15.42:6826/876648842 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3a38103890 con 0x7f3a200ac490 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.698+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] conn(0x7f3a200ac490 msgr2=0x7f3a200ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.698+0000 7f3a3ee00700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] conn(0x7f3a200ac490 0x7f3a200ae8b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.698+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a200a2cb0 msgr2=0x7f3a200a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.698+0000 7f3a3ee00700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a200a2cb0 0x7f3a200a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3a24000c00 tx=0x7f3a24005fb0 comp rx=0 tx=0).stop 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.698+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a381023b0 msgr2=0x7f3a381a07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.698+0000 7f3a3ee00700 1 --2- 172.21.15.42:0/3539659874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a381023b0 0x7f3a381a07e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c00e9f0 tx=0x7f3a2c00ed00 comp rx=0 tx=0).stop 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.698+0000 7f3a3e400700 1 -- 172.21.15.42:0/3539659874 reap_dead start 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.698+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 shutdown_connections 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.698+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 >> 172.21.15.42:0/3539659874 conn(0x7f3a380fdc80 msgr2=0x7f3a3810ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.699+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 shutdown_connections 2024-04-25T00:17:08.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:08.699+0000 7f3a3ee00700 1 -- 172.21.15.42:0/3539659874 wait complete. 2024-04-25T00:17:09.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.332+0000 7f1fd8e00700 1 -- 172.21.15.42:0/2001006435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fd4069f40 msgr2=0x7f1fd406a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:09.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.332+0000 7f1fd8e00700 1 --2- 172.21.15.42:0/2001006435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fd4069f40 0x7f1fd406a3a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0009990 tx=0x7f1fc0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:09.436 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.332+0000 7f1fd8e00700 1 -- 172.21.15.42:0/2001006435 shutdown_connections 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.332+0000 7f1fd8e00700 1 --2- 172.21.15.42:0/2001006435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fd406a8e0 0x7f1fd410f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.332+0000 7f1fd8e00700 1 --2- 172.21.15.42:0/2001006435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd41082b0 0x7f1fd4108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.332+0000 7f1fd8e00700 1 --2- 172.21.15.42:0/2001006435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fd4069f40 0x7f1fd406a3a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.332+0000 7f1fd8e00700 1 -- 172.21.15.42:0/2001006435 >> 172.21.15.42:0/2001006435 conn(0x7f1fd40780e0 msgr2=0x7f1fd40784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.332+0000 7f1fd8e00700 1 -- 172.21.15.42:0/2001006435 shutdown_connections 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.332+0000 7f1fd8e00700 1 -- 172.21.15.42:0/2001006435 wait complete. 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.334+0000 7f1fd8e00700 1 Processor -- start 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.334+0000 7f1fd8e00700 1 -- start start 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.335+0000 7f1fd8e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fd4069f40 0x7f1fd419c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.335+0000 7f1fd8e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd406a8e0 0x7f1fd419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.335+0000 7f1fd8e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fd41082b0 0x7f1fd4196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.335+0000 7f1fd8e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd41103a0 con 0x7f1fd41082b0 2024-04-25T00:17:09.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.335+0000 7f1fd8e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4110220 con 0x7f1fd4069f40 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.335+0000 7f1fd8e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd4110520 con 0x7f1fd406a8e0 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.335+0000 7f1fd1e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fd4069f40 0x7f1fd419c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.335+0000 7f1fd1e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fd4069f40 0x7f1fd419c410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53204/0 (socket says 172.21.15.42:53204) 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.335+0000 7f1fd1e00700 1 -- 172.21.15.42:0/3791798672 learned_addr learned my addr 172.21.15.42:0/3791798672 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.335+0000 7f1fd2800700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fd41082b0 0x7f1fd4196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.336+0000 7f1fd1400700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd406a8e0 0x7f1fd419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.336+0000 7f1fd1e00700 1 -- 172.21.15.42:0/3791798672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd406a8e0 msgr2=0x7f1fd419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.336+0000 7f1fd1e00700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fd406a8e0 0x7f1fd419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.336+0000 7f1fd1e00700 1 -- 172.21.15.42:0/3791798672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fd41082b0 msgr2=0x7f1fd4196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.336+0000 7f1fd1e00700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fd41082b0 0x7f1fd4196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.336+0000 7f1fd1e00700 1 -- 172.21.15.42:0/3791798672 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd4196d80 con 0x7f1fd4069f40 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.336+0000 7f1fd2800700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1fd41082b0 0x7f1fd4196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.337+0000 7f1fd1e00700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fd4069f40 0x7f1fd419c410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4012fc0 tx=0x7f1fc40103a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:09.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.337+0000 7f1fcaa00700 1 -- 172.21.15.42:0/3791798672 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc40109b0 con 0x7f1fd4069f40 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.337+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd4197010 con 0x7f1fd4069f40 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.337+0000 7f1fcaa00700 1 -- 172.21.15.42:0/3791798672 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1fc401d070 con 0x7f1fd4069f40 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.337+0000 7f1fcaa00700 1 -- 172.21.15.42:0/3791798672 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4018480 con 0x7f1fd4069f40 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.337+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd41a32e0 con 0x7f1fd4069f40 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.338+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1fd40629c0 con 0x7f1fd4069f40 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.340+0000 7f1fcaa00700 1 -- 172.21.15.42:0/3791798672 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1fc4016050 con 0x7f1fd4069f40 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.342+0000 7f1fcaa00700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1fb80a2cb0 0x7f1fb80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.342+0000 7f1fcaa00700 1 -- 172.21.15.42:0/3791798672 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f1fc40d39d0 con 0x7f1fd4069f40 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.342+0000 7f1fd1400700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1fb80a2cb0 0x7f1fb80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.342+0000 7f1fcaa00700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] conn(0x7f1fb80ac490 0x7f1fb80ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:09.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.342+0000 7f1fcaa00700 1 -- 172.21.15.42:0/3791798672 --> [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1fc400fe50 con 0x7f1fb80ac490 2024-04-25T00:17:09.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.342+0000 7f1fcaa00700 1 -- 172.21.15.42:0/3791798672 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f1fc40d3d80 con 0x7f1fd4069f40 2024-04-25T00:17:09.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.343+0000 7f1fd2800700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] conn(0x7f1fb80ac490 0x7f1fb80ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:09.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.343+0000 7f1fd1400700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1fb80a2cb0 0x7f1fb80a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0006010 tx=0x7f1fc001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:09.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.343+0000 7f1fd2800700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] conn(0x7f1fb80ac490 0x7f1fb80ae8b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:09.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.344+0000 7f1fcaa00700 1 -- 172.21.15.42:0/3791798672 <== osd.11 v2:172.21.15.136:6824/3705953158 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1fc400fe50 con 0x7f1fb80ac490 2024-04-25T00:17:09.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.363+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 --> [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1fd4062dc0 con 0x7f1fb80ac490 2024-04-25T00:17:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.363+0000 7f1fcaa00700 1 -- 172.21.15.42:0/3791798672 <== osd.11 v2:172.21.15.136:6824/3705953158 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1fd4062dc0 con 0x7f1fb80ac490 2024-04-25T00:17:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 >> [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] conn(0x7f1fb80ac490 msgr2=0x7f1fb80ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd8e00700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] conn(0x7f1fb80ac490 0x7f1fb80ae8b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1fb80a2cb0 msgr2=0x7f1fb80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd8e00700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1fb80a2cb0 0x7f1fb80a5170 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0006010 tx=0x7f1fc001a040 comp rx=0 tx=0).stop 2024-04-25T00:17:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fd4069f40 msgr2=0x7f1fd419c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd8e00700 1 --2- 172.21.15.42:0/3791798672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1fd4069f40 0x7f1fd419c410 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1fc4012fc0 tx=0x7f1fc40103a0 comp rx=0 tx=0).stop 2024-04-25T00:17:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd2800700 1 -- 172.21.15.42:0/3791798672 reap_dead start 2024-04-25T00:17:09.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 shutdown_connections 2024-04-25T00:17:09.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 >> 172.21.15.42:0/3791798672 conn(0x7f1fd40780e0 msgr2=0x7f1fd40ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:09.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 shutdown_connections 2024-04-25T00:17:09.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.364+0000 7f1fd8e00700 1 -- 172.21.15.42:0/3791798672 wait complete. 2024-04-25T00:17:09.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:09 smithi042 ceph-mon[28130]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T00:17:09.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:09 smithi136 ceph-mon[32923]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T00:17:09.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:09 smithi106 ceph-mon[32510]: pgmap v337: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-04-25T00:17:10.739 INFO:teuthology.orchestra.run.smithi042.stdout:107374182466 2024-04-25T00:17:10.740 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T00:17:10.994 INFO:teuthology.orchestra.run.smithi042.stdout:163208757302 2024-04-25T00:17:10.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T00:17:11.298 INFO:teuthology.orchestra.run.smithi042.stdout:77309411400 2024-04-25T00:17:11.298 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T00:17:11.752 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:11 smithi042 ceph-mon[28130]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:11.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:11 smithi136 ceph-mon[32923]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:11.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:11 smithi106 ceph-mon[32510]: pgmap v338: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:12.435 INFO:teuthology.orchestra.run.smithi042.stdout:34359738451 2024-04-25T00:17:12.435 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T00:17:12.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.242+0000 7f5eefe00700 1 -- 172.21.15.42:0/2234083743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ef010e9a0 msgr2=0x7f5ef0110e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:12.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.242+0000 7f5eefe00700 1 --2- 172.21.15.42:0/2234083743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ef010e9a0 0x7f5ef0110e10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8009f90 tx=0x7f5ee800b330 comp rx=0 tx=0).stop 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.242+0000 7f5eefe00700 1 -- 172.21.15.42:0/2234083743 shutdown_connections 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.242+0000 7f5eefe00700 1 --2- 172.21.15.42:0/2234083743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ef010e9a0 0x7f5ef0110e10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.242+0000 7f5eefe00700 1 --2- 172.21.15.42:0/2234083743 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef0074660 0x7f5ef0074a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.242+0000 7f5eefe00700 1 --2- 172.21.15.42:0/2234083743 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ef0074f80 0x7f5ef010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.242+0000 7f5eefe00700 1 -- 172.21.15.42:0/2234083743 >> 172.21.15.42:0/2234083743 conn(0x7f5ef00fdcb0 msgr2=0x7f5ef01000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.242+0000 7f5eefe00700 1 -- 172.21.15.42:0/2234083743 shutdown_connections 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.242+0000 7f5eefe00700 1 -- 172.21.15.42:0/2234083743 wait complete. 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.243+0000 7f5eefe00700 1 Processor -- start 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.243+0000 7f5eefe00700 1 -- start start 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eefe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef0074660 0x7f5ef01a0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eefe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ef0074f80 0x7f5ef01a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eefe00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ef010e9a0 0x7f5ef019a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eefe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0114690 con 0x7f5ef0074f80 2024-04-25T00:17:12.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eefe00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0114510 con 0x7f5ef010e9a0 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eefe00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0114810 con 0x7f5ef0074660 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eee000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ef0074f80 0x7f5ef01a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eeea00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef0074660 0x7f5ef01a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eee000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ef0074f80 0x7f5ef01a0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42410/0 (socket says 172.21.15.42:42410) 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eee000700 1 -- 172.21.15.42:0/3998026782 learned_addr learned my addr 172.21.15.42:0/3998026782 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.244+0000 7f5eef400700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ef010e9a0 0x7f5ef019a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.245+0000 7f5eeea00700 1 -- 172.21.15.42:0/3998026782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ef010e9a0 msgr2=0x7f5ef019a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.245+0000 7f5eeea00700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5ef010e9a0 0x7f5ef019a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.245+0000 7f5eeea00700 1 -- 172.21.15.42:0/3998026782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ef0074f80 msgr2=0x7f5ef01a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.245+0000 7f5eeea00700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5ef0074f80 0x7f5ef01a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.245+0000 7f5eeea00700 1 -- 172.21.15.42:0/3998026782 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef019b0b0 con 0x7f5ef0074660 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.245+0000 7f5eeea00700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef0074660 0x7f5ef01a0790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000e9c0 tx=0x7f5ee000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.245+0000 7f5ee7400700 1 -- 172.21.15.42:0/3998026782 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0004220 con 0x7f5ef0074660 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.245+0000 7f5ee7400700 1 -- 172.21.15.42:0/3998026782 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ee0004d10 con 0x7f5ef0074660 2024-04-25T00:17:12.737 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.245+0000 7f5ee7400700 1 -- 172.21.15.42:0/3998026782 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000ce60 con 0x7f5ef0074660 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.245+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef019b3a0 con 0x7f5ef0074660 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.246+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef01a7660 con 0x7f5ef0074660 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.246+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ef0062940 con 0x7f5ef0074660 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.249+0000 7f5ee7400700 1 -- 172.21.15.42:0/3998026782 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5ee0004750 con 0x7f5ef0074660 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.250+0000 7f5ee7400700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ed40a2d30 0x7f5ed40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.250+0000 7f5ee7400700 1 -- 172.21.15.42:0/3998026782 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7f5ee00cf0c0 con 0x7f5ef0074660 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.250+0000 7f5eee000700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ed40a2d30 0x7f5ed40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.250+0000 7f5ee7400700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] conn(0x7f5ed40ac510 0x7f5ed40ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.251+0000 7f5ee7400700 1 -- 172.21.15.42:0/3998026782 --> [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ee0019e50 con 0x7f5ed40ac510 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.251+0000 7f5ee7400700 1 -- 172.21.15.42:0/3998026782 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7f5ee00cf470 con 0x7f5ef0074660 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.251+0000 7f5eef400700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] conn(0x7f5ed40ac510 0x7f5ed40ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.251+0000 7f5eee000700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ed40a2d30 0x7f5ed40a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0111320 tx=0x7f5edc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.252+0000 7f5eef400700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] conn(0x7f5ed40ac510 0x7f5ed40ae930 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.252+0000 7f5ee7400700 1 -- 172.21.15.42:0/3998026782 <== osd.4 v2:172.21.15.106:6808/2825344351 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ee0019e50 con 0x7f5ed40ac510 2024-04-25T00:17:12.738 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.271+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 --> [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ef0062dc0 con 0x7f5ed40ac510 2024-04-25T00:17:12.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.272+0000 7f5ee7400700 1 -- 172.21.15.42:0/3998026782 <== osd.4 v2:172.21.15.106:6808/2825344351 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5ef0062dc0 con 0x7f5ed40ac510 2024-04-25T00:17:12.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 >> [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] conn(0x7f5ed40ac510 msgr2=0x7f5ed40ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:12.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eefe00700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] conn(0x7f5ed40ac510 0x7f5ed40ae930 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:12.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ed40a2d30 msgr2=0x7f5ed40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:12.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eefe00700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5ed40a2d30 0x7f5ed40a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0111320 tx=0x7f5edc009260 comp rx=0 tx=0).stop 2024-04-25T00:17:12.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef0074660 msgr2=0x7f5ef01a0790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:12.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eefe00700 1 --2- 172.21.15.42:0/3998026782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5ef0074660 0x7f5ef01a0790 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000e9c0 tx=0x7f5ee000ecd0 comp rx=0 tx=0).stop 2024-04-25T00:17:12.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eef400700 1 -- 172.21.15.42:0/3998026782 reap_dead start 2024-04-25T00:17:12.739 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 shutdown_connections 2024-04-25T00:17:12.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 >> 172.21.15.42:0/3998026782 conn(0x7f5ef00fdcb0 msgr2=0x7f5ef00fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:12.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 shutdown_connections 2024-04-25T00:17:12.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:09.273+0000 7f5eefe00700 1 -- 172.21.15.42:0/3998026782 wait complete. 2024-04-25T00:17:13.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.717+0000 7fe39c600700 1 -- 172.21.15.42:0/1410388330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe39410e860 msgr2=0x7fe39410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:13.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.717+0000 7fe39c600700 1 --2- 172.21.15.42:0/1410388330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe39410e860 0x7fe39410ec40 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe388009990 tx=0x7fe388009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:13.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.717+0000 7fe39c600700 1 -- 172.21.15.42:0/1410388330 shutdown_connections 2024-04-25T00:17:13.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.717+0000 7fe39c600700 1 --2- 172.21.15.42:0/1410388330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe394109220 0x7fe394115730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:13.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.717+0000 7fe39c600700 1 --2- 172.21.15.42:0/1410388330 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe39410e860 0x7fe39410ec40 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:13.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.717+0000 7fe39c600700 1 --2- 172.21.15.42:0/1410388330 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe394108860 0x7fe394108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:13.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.717+0000 7fe39c600700 1 -- 172.21.15.42:0/1410388330 >> 172.21.15.42:0/1410388330 conn(0x7fe394075a00 msgr2=0x7fe394107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:13.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.717+0000 7fe39c600700 1 -- 172.21.15.42:0/1410388330 shutdown_connections 2024-04-25T00:17:13.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.717+0000 7fe39c600700 1 -- 172.21.15.42:0/1410388330 wait complete. 2024-04-25T00:17:13.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.718+0000 7fe39c600700 1 Processor -- start 2024-04-25T00:17:13.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.719+0000 7fe39c600700 1 -- start start 2024-04-25T00:17:13.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.719+0000 7fe39c600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe394108860 0x7fe39419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:13.721 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.719+0000 7fe39c600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe394109220 0x7fe39419f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.719+0000 7fe39c600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe39410e860 0x7fe3941a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe39c600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe394118be0 con 0x7fe39410e860 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe39c600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe394118a60 con 0x7fe394108860 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe39c600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe394118d60 con 0x7fe394109220 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe399c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe394108860 0x7fe39419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe399200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe394109220 0x7fe39419f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe39a600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe39410e860 0x7fe3941a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe399200700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe394109220 0x7fe39419f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:37376/0 (socket says 172.21.15.42:37376) 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe399200700 1 -- 172.21.15.42:0/521728762 learned_addr learned my addr 172.21.15.42:0/521728762 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe399c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe394108860 0x7fe39419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53220/0 (socket says 172.21.15.42:53220) 2024-04-25T00:17:13.722 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe399c00700 1 -- 172.21.15.42:0/521728762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe394109220 msgr2=0x7fe39419f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe399c00700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe394109220 0x7fe39419f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe399c00700 1 -- 172.21.15.42:0/521728762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe39410e860 msgr2=0x7fe3941a38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe399c00700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe39410e860 0x7fe3941a38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.720+0000 7fe399c00700 1 -- 172.21.15.42:0/521728762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3941a4000 con 0x7fe394108860 2024-04-25T00:17:13.723 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.721+0000 7fe39a600700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe39410e860 0x7fe3941a38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.721+0000 7fe399c00700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe394108860 0x7fe39419eff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe388000c00 tx=0x7fe38800bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.721+0000 7fe392000700 1 -- 172.21.15.42:0/521728762 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe388026070 con 0x7fe394108860 2024-04-25T00:17:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.721+0000 7fe392000700 1 -- 172.21.15.42:0/521728762 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe388003d70 con 0x7fe394108860 2024-04-25T00:17:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.721+0000 7fe392000700 1 -- 172.21.15.42:0/521728762 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38800fd90 con 0x7fe394108860 2024-04-25T00:17:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.721+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3941a4290 con 0x7fe394108860 2024-04-25T00:17:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.722+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3941abb70 con 0x7fe394108860 2024-04-25T00:17:13.724 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.722+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe3941a4720 con 0x7fe394108860 2024-04-25T00:17:13.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.724+0000 7fe392000700 1 -- 172.21.15.42:0/521728762 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe388029070 con 0x7fe394108860 2024-04-25T00:17:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.726+0000 7fe392000700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe37c0a2d30 0x7fe37c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.726+0000 7fe392000700 1 -- 172.21.15.42:0/521728762 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe3880d0490 con 0x7fe394108860 2024-04-25T00:17:13.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.726+0000 7fe399200700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe37c0a2d30 0x7fe37c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.726+0000 7fe392000700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] conn(0x7fe37c0ac510 0x7fe37c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.726+0000 7fe392000700 1 -- 172.21.15.42:0/521728762 --> [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe37c0aefe0 con 0x7fe37c0ac510 2024-04-25T00:17:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.726+0000 7fe392000700 1 -- 172.21.15.42:0/521728762 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fe3880d0840 con 0x7fe394108860 2024-04-25T00:17:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.727+0000 7fe39a600700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] conn(0x7fe37c0ac510 0x7fe37c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:13.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.727+0000 7fe399200700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe37c0a2d30 0x7fe37c0a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe380009680 tx=0x7fe380009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:13.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.727+0000 7fe39a600700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] conn(0x7fe37c0ac510 0x7fe37c0ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:13.729 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.728+0000 7fe392000700 1 -- 172.21.15.42:0/521728762 <== osd.1 v2:172.21.15.106:6800/2488433173 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe37c0aefe0 con 0x7fe37c0ac510 2024-04-25T00:17:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.747+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 --> [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe394062ce0 con 0x7fe37c0ac510 2024-04-25T00:17:13.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.747+0000 7fe392000700 1 -- 172.21.15.42:0/521728762 <== osd.1 v2:172.21.15.106:6800/2488433173 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe394062ce0 con 0x7fe37c0ac510 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.748+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 >> [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] conn(0x7fe37c0ac510 msgr2=0x7fe37c0ae930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.748+0000 7fe39c600700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] conn(0x7fe37c0ac510 0x7fe37c0ae930 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.748+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe37c0a2d30 msgr2=0x7fe37c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.748+0000 7fe39c600700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe37c0a2d30 0x7fe37c0a51f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe380009680 tx=0x7fe380009260 comp rx=0 tx=0).stop 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.748+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe394108860 msgr2=0x7fe39419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.748+0000 7fe39c600700 1 --2- 172.21.15.42:0/521728762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe394108860 0x7fe39419eff0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe388000c00 tx=0x7fe38800bab0 comp rx=0 tx=0).stop 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.748+0000 7fe39a600700 1 -- 172.21.15.42:0/521728762 reap_dead start 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.748+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 shutdown_connections 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.748+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 >> 172.21.15.42:0/521728762 conn(0x7fe394075a00 msgr2=0x7fe394075f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.748+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 shutdown_connections 2024-04-25T00:17:13.750 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:13.749+0000 7fe39c600700 1 -- 172.21.15.42:0/521728762 wait complete. 2024-04-25T00:17:13.816 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:13 smithi042 ceph-mon[28130]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:17:13.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:13 smithi106 ceph-mon[32510]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:17:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:13 smithi136 ceph-mon[32923]: pgmap v339: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-04-25T00:17:14.096 INFO:teuthology.orchestra.run.smithi042.stdout:347892350999 2024-04-25T00:17:14.096 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T00:17:14.414 INFO:teuthology.orchestra.run.smithi042.stdout:197568495666 2024-04-25T00:17:14.415 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T00:17:14.762 INFO:teuthology.orchestra.run.smithi042.stdout:219043332140 2024-04-25T00:17:14.762 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T00:17:15.519 INFO:teuthology.orchestra.run.smithi042.stdout:141733920830 2024-04-25T00:17:15.519 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T00:17:15.601 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:15 smithi042 ceph-mon[28130]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:15 smithi106 ceph-mon[32510]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:15 smithi136 ceph-mon[32923]: pgmap v340: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:16.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.075+0000 7fce4d000700 1 -- 172.21.15.42:0/3953914929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48076760 msgr2=0x7fce48076b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:16.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.075+0000 7fce4d000700 1 --2- 172.21.15.42:0/3953914929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48076760 0x7fce48076b40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fce38009990 tx=0x7fce38009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:16.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.075+0000 7fce4d000700 1 -- 172.21.15.42:0/3953914929 shutdown_connections 2024-04-25T00:17:16.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.075+0000 7fce4d000700 1 --2- 172.21.15.42:0/3953914929 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce481130c0 0x7fce481154b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:16.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.075+0000 7fce4d000700 1 --2- 172.21.15.42:0/3953914929 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48076760 0x7fce48076b40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:16.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.075+0000 7fce4d000700 1 --2- 172.21.15.42:0/3953914929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce48077080 0x7fce48112b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:16.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.075+0000 7fce4d000700 1 -- 172.21.15.42:0/3953914929 >> 172.21.15.42:0/3953914929 conn(0x7fce480fff10 msgr2=0x7fce48102330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:16.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.075+0000 7fce4d000700 1 -- 172.21.15.42:0/3953914929 shutdown_connections 2024-04-25T00:17:16.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.075+0000 7fce4d000700 1 -- 172.21.15.42:0/3953914929 wait complete. 2024-04-25T00:17:16.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.076+0000 7fce4d000700 1 Processor -- start 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce4d000700 1 -- start start 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce4d000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48076760 0x7fce4819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce4d000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce48077080 0x7fce4819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce4d000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce481130c0 0x7fce481a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce4d000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fce48118be0 con 0x7fce48076760 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce4d000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fce48118a60 con 0x7fce48077080 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce4d000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce48118d60 con 0x7fce481130c0 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce45e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48076760 0x7fce4819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce45e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48076760 0x7fce4819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42458/0 (socket says 172.21.15.42:42458) 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce45e00700 1 -- 172.21.15.42:0/1773013185 learned_addr learned my addr 172.21.15.42:0/1773013185 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:16.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.077+0000 7fce45400700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce48077080 0x7fce4819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:16.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.078+0000 7fce46800700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce481130c0 0x7fce481a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:16.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.078+0000 7fce45400700 1 -- 172.21.15.42:0/1773013185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce481130c0 msgr2=0x7fce481a38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:16.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.078+0000 7fce45400700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce481130c0 0x7fce481a38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:16.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.078+0000 7fce45400700 1 -- 172.21.15.42:0/1773013185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48076760 msgr2=0x7fce4819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:16.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.078+0000 7fce45400700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48076760 0x7fce4819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:16.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.078+0000 7fce45400700 1 -- 172.21.15.42:0/1773013185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce481a4020 con 0x7fce48077080 2024-04-25T00:17:16.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.078+0000 7fce45e00700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fce48076760 0x7fce4819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:16.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.079+0000 7fce45400700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce48077080 0x7fce4819f550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce3400e9f0 tx=0x7fce3400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:16.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.079+0000 7fce3ea00700 1 -- 172.21.15.42:0/1773013185 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce3400cbd0 con 0x7fce48077080 2024-04-25T00:17:16.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.079+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce481a4310 con 0x7fce48077080 2024-04-25T00:17:16.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.079+0000 7fce3ea00700 1 -- 172.21.15.42:0/1773013185 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce3400cd70 con 0x7fce48077080 2024-04-25T00:17:16.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.079+0000 7fce3ea00700 1 -- 172.21.15.42:0/1773013185 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce34018710 con 0x7fce48077080 2024-04-25T00:17:16.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.079+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce481abb90 con 0x7fce48077080 2024-04-25T00:17:16.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.080+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fce48115570 con 0x7fce48077080 2024-04-25T00:17:16.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.083+0000 7fce3ea00700 1 -- 172.21.15.42:0/1773013185 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fce34010430 con 0x7fce48077080 2024-04-25T00:17:16.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.085+0000 7fce3ea00700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fce2c0a2cb0 0x7fce2c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:16.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.085+0000 7fce3ea00700 1 -- 172.21.15.42:0/1773013185 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fce340cf340 con 0x7fce48077080 2024-04-25T00:17:16.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.085+0000 7fce45e00700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fce2c0a2cb0 0x7fce2c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:16.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.085+0000 7fce3ea00700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] conn(0x7fce2c0ac490 0x7fce2c0ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:16.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.085+0000 7fce3ea00700 1 -- 172.21.15.42:0/1773013185 --> [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce3401ee50 con 0x7fce2c0ac490 2024-04-25T00:17:16.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.085+0000 7fce3ea00700 1 -- 172.21.15.42:0/1773013185 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) ==== 24+0+0 (secure 0 0 0) 0x7fce340d6050 con 0x7fce48077080 2024-04-25T00:17:16.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.086+0000 7fce45e00700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fce2c0a2cb0 0x7fce2c0a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fce38009750 tx=0x7fce3800b280 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:16.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.086+0000 7fce46800700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] conn(0x7fce2c0ac490 0x7fce2c0ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:16.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.086+0000 7fce46800700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] conn(0x7fce2c0ac490 0x7fce2c0ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:16.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.087+0000 7fce3ea00700 1 -- 172.21.15.42:0/1773013185 <== osd.10 v2:172.21.15.106:6824/3446161379 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce3401ee50 con 0x7fce2c0ac490 2024-04-25T00:17:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.106+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 --> [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fce48077ea0 con 0x7fce2c0ac490 2024-04-25T00:17:16.108 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.107+0000 7fce3ea00700 1 -- 172.21.15.42:0/1773013185 <== osd.10 v2:172.21.15.106:6824/3446161379 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fce48077ea0 con 0x7fce2c0ac490 2024-04-25T00:17:16.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 >> [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] conn(0x7fce2c0ac490 msgr2=0x7fce2c0ae8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:16.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce4d000700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] conn(0x7fce2c0ac490 0x7fce2c0ae8b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:16.109 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fce2c0a2cb0 msgr2=0x7fce2c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce4d000700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fce2c0a2cb0 0x7fce2c0a5170 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fce38009750 tx=0x7fce3800b280 comp rx=0 tx=0).stop 2024-04-25T00:17:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce48077080 msgr2=0x7fce4819f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce4d000700 1 --2- 172.21.15.42:0/1773013185 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce48077080 0x7fce4819f550 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce3400e9f0 tx=0x7fce3400ed00 comp rx=0 tx=0).stop 2024-04-25T00:17:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce46800700 1 -- 172.21.15.42:0/1773013185 reap_dead start 2024-04-25T00:17:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 shutdown_connections 2024-04-25T00:17:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 >> 172.21.15.42:0/1773013185 conn(0x7fce480fff10 msgr2=0x7fce48102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 shutdown_connections 2024-04-25T00:17:16.110 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:16.108+0000 7fce4d000700 1 -- 172.21.15.42:0/1773013185 wait complete. 2024-04-25T00:17:17.058 INFO:teuthology.orchestra.run.smithi042.stdout:55834574926 2024-04-25T00:17:17.059 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T00:17:17.118 INFO:teuthology.orchestra.run.smithi042.stdout:292057776161 2024-04-25T00:17:17.118 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T00:17:17.684 INFO:teuthology.orchestra.run.smithi042.stdout:313532612637 2024-04-25T00:17:17.685 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T00:17:17.733 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:17.733 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:17.735 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:17.741 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:17.750 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:17.759 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:17.779 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:17.789 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:17 smithi042 ceph-mon[28130]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T00:17:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:17 smithi106 ceph-mon[32510]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T00:17:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:17 smithi136 ceph-mon[32923]: pgmap v341: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T00:17:19.830 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:19 smithi042 ceph-mon[28130]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:17:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:19 smithi106 ceph-mon[32510]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:17:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:19 smithi136 ceph-mon[32923]: pgmap v342: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:17:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:21 smithi106 ceph-mon[32510]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:21.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:21 smithi042 ceph-mon[28130]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:21.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:21.963 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:21 smithi136 ceph-mon[32923]: pgmap v343: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-04-25T00:17:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:17:22.274 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:23.206 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:23.208 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:23 smithi106 ceph-mon[32510]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:17:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:23.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:23 smithi042 ceph-mon[28130]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:17:23.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:23.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:23.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:23.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:23 smithi136 ceph-mon[32923]: pgmap v344: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T00:17:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.451+0000 7fa921e00700 1 -- 172.21.15.42:0/3176036144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa91c107160 msgr2=0x7fa91c069f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.452+0000 7fa921e00700 1 --2- 172.21.15.42:0/3176036144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa91c107160 0x7fa91c069f40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009990 tx=0x7fa90c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.452+0000 7fa921e00700 1 -- 172.21.15.42:0/3176036144 shutdown_connections 2024-04-25T00:17:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.452+0000 7fa921e00700 1 --2- 172.21.15.42:0/3176036144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa91c104c70 0x7fa91c105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.452+0000 7fa921e00700 1 --2- 172.21.15.42:0/3176036144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa91c107160 0x7fa91c069f40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.452+0000 7fa921e00700 1 --2- 172.21.15.42:0/3176036144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa91c06a480 0x7fa91c06a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.452+0000 7fa921e00700 1 -- 172.21.15.42:0/3176036144 >> 172.21.15.42:0/3176036144 conn(0x7fa91c0780e0 msgr2=0x7fa91c0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.452+0000 7fa921e00700 1 -- 172.21.15.42:0/3176036144 shutdown_connections 2024-04-25T00:17:24.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.452+0000 7fa921e00700 1 -- 172.21.15.42:0/3176036144 wait complete. 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.454+0000 7fa921e00700 1 Processor -- start 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.454+0000 7fa921e00700 1 -- start start 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.454+0000 7fa921e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa91c06a480 0x7fa91c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.454+0000 7fa921e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa91c104c70 0x7fa91c19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.454+0000 7fa921e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa91c107160 0x7fa91c1a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.454+0000 7fa921e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c118b20 con 0x7fa91c06a480 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.454+0000 7fa921e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c1189a0 con 0x7fa91c104c70 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.454+0000 7fa921e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c118ca0 con 0x7fa91c107160 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.455+0000 7fa91b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa91c107160 0x7fa91c1a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.455+0000 7fa91b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa91c107160 0x7fa91c1a3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:40740/0 (socket says 172.21.15.42:40740) 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.455+0000 7fa91b400700 1 -- 172.21.15.42:0/2938981505 learned_addr learned my addr 172.21.15.42:0/2938981505 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.455+0000 7fa91a000700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa91c104c70 0x7fa91c19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:24.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.455+0000 7fa91b400700 1 -- 172.21.15.42:0/2938981505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa91c104c70 msgr2=0x7fa91c19f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.455+0000 7fa91b400700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa91c104c70 0x7fa91c19f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.455+0000 7fa91b400700 1 -- 172.21.15.42:0/2938981505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa91c06a480 msgr2=0x7fa91c19efb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.456+0000 7fa91b400700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa91c06a480 0x7fa91c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.456+0000 7fa91b400700 1 -- 172.21.15.42:0/2938981505 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c1a3fc0 con 0x7fa91c107160 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.457+0000 7fa91b400700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa91c107160 0x7fa91c1a3880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa91400ef30 tx=0x7fa91400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.458+0000 7fa920a00700 1 -- 172.21.15.42:0/2938981505 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9140092e0 con 0x7fa91c107160 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.458+0000 7fa920a00700 1 -- 172.21.15.42:0/2938981505 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa914004bb0 con 0x7fa91c107160 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.458+0000 7fa920a00700 1 -- 172.21.15.42:0/2938981505 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa914004280 con 0x7fa91c107160 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.458+0000 7fa921e00700 1 -- 172.21.15.42:0/2938981505 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c1a4250 con 0x7fa91c107160 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.458+0000 7fa921e00700 1 -- 172.21.15.42:0/2938981505 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c1abb30 con 0x7fa91c107160 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.459+0000 7fa921e00700 1 -- 172.21.15.42:0/2938981505 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa91c068f20 con 0x7fa91c107160 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.466+0000 7fa920a00700 1 -- 172.21.15.42:0/2938981505 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa91401b030 con 0x7fa91c107160 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.468+0000 7fa920a00700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa9000a2d30 0x7fa9000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:24.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.468+0000 7fa91aa00700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa9000a2d30 0x7fa9000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:24.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.468+0000 7fa920a00700 1 -- 172.21.15.42:0/2938981505 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fa9140cefb0 con 0x7fa91c107160 2024-04-25T00:17:24.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.468+0000 7fa920a00700 1 -- 172.21.15.42:0/2938981505 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa914022020 con 0x7fa91c107160 2024-04-25T00:17:24.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.469+0000 7fa91aa00700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa9000a2d30 0x7fa9000a51f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00b2e0 tx=0x7fa90c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:24.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.712+0000 7fa921e00700 1 -- 172.21.15.42:0/2938981505 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fa91c1abd00 con 0x7fa91c107160 2024-04-25T00:17:24.718 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.713+0000 7fa920a00700 1 -- 172.21.15.42:0/2938981505 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa914010040 con 0x7fa91c107160 2024-04-25T00:17:24.718 INFO:teuthology.orchestra.run.smithi042.stdout:77309411403 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.716+0000 7fa921e00700 1 -- 172.21.15.42:0/2938981505 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa9000a2d30 msgr2=0x7fa9000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.716+0000 7fa921e00700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa9000a2d30 0x7fa9000a51f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00b2e0 tx=0x7fa90c005fb0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.716+0000 7fa921e00700 1 -- 172.21.15.42:0/2938981505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa91c107160 msgr2=0x7fa91c1a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.716+0000 7fa921e00700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa91c107160 0x7fa91c1a3880 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa91400ef30 tx=0x7fa91400c3a0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.716+0000 7fa921e00700 1 -- 172.21.15.42:0/2938981505 shutdown_connections 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.716+0000 7fa921e00700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa9000a2d30 0x7fa9000a51f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.716+0000 7fa921e00700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa91c107160 0x7fa91c1a3880 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.716+0000 7fa921e00700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa91c06a480 0x7fa91c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.716+0000 7fa921e00700 1 --2- 172.21.15.42:0/2938981505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa91c104c70 0x7fa91c19f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.716+0000 7fa921e00700 1 -- 172.21.15.42:0/2938981505 >> 172.21.15.42:0/2938981505 conn(0x7fa91c0780e0 msgr2=0x7fa91c100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:24.719 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.717+0000 7fa921e00700 1 -- 172.21.15.42:0/2938981505 shutdown_connections 2024-04-25T00:17:24.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:24.717+0000 7fa921e00700 1 -- 172.21.15.42:0/2938981505 wait complete. 2024-04-25T00:17:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:25 smithi106 ceph-mon[32510]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:25 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2938981505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:17:25.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:25 smithi042 ceph-mon[28130]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:25.843 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:25 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2938981505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:17:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:25 smithi136 ceph-mon[32923]: pgmap v345: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:25 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2938981505' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T00:17:26.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T00:17:26.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T00:17:26.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T00:17:26.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-25T00:17:26.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-25T00:17:26.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-25T00:17:26.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:17:26.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:26 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3890617199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:17:26.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T00:17:26.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T00:17:26.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T00:17:26.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-25T00:17:26.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-25T00:17:26.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-25T00:17:26.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:17:26.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:26 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3890617199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:17:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-25T00:17:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T00:17:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-04-25T00:17:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-04-25T00:17:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]: dispatch 2024-04-25T00:17:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]: dispatch 2024-04-25T00:17:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]: dispatch 2024-04-25T00:17:27.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:26 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3890617199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T00:17:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.409+0000 7fe47fc00700 1 -- 172.21.15.42:0/2558887135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4781082b0 msgr2=0x7fe478108690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.409+0000 7fe47fc00700 1 --2- 172.21.15.42:0/2558887135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4781082b0 0x7fe478108690 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe46c009990 tx=0x7fe46c009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.409+0000 7fe47fc00700 1 -- 172.21.15.42:0/2558887135 shutdown_connections 2024-04-25T00:17:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.409+0000 7fe47fc00700 1 --2- 172.21.15.42:0/2558887135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe47806a8e0 0x7fe47810f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.409+0000 7fe47fc00700 1 --2- 172.21.15.42:0/2558887135 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4781082b0 0x7fe478108690 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.409+0000 7fe47fc00700 1 --2- 172.21.15.42:0/2558887135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe478069f40 0x7fe47806a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.183 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.409+0000 7fe47fc00700 1 -- 172.21.15.42:0/2558887135 >> 172.21.15.42:0/2558887135 conn(0x7fe4780780e0 msgr2=0x7fe4780784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.409+0000 7fe47fc00700 1 -- 172.21.15.42:0/2558887135 shutdown_connections 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.409+0000 7fe47fc00700 1 -- 172.21.15.42:0/2558887135 wait complete. 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.410+0000 7fe47fc00700 1 Processor -- start 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.410+0000 7fe47fc00700 1 -- start start 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47fc00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe478069f40 0x7fe47819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47fc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe47806a8e0 0x7fe47819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47fc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4781082b0 0x7fe4781988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47fc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe478112540 con 0x7fe4781082b0 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47fc00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781123c0 con 0x7fe478069f40 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47fc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe4781126c0 con 0x7fe47806a8e0 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47d200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe478069f40 0x7fe47819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47d200700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe478069f40 0x7fe47819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:54678/0 (socket says 172.21.15.42:54678) 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47d200700 1 -- 172.21.15.42:0/1173271828 learned_addr learned my addr 172.21.15.42:0/1173271828 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:27.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47dc00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4781082b0 0x7fe4781988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe477e00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe47806a8e0 0x7fe47819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47d200700 1 -- 172.21.15.42:0/1173271828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe47806a8e0 msgr2=0x7fe47819ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47d200700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe47806a8e0 0x7fe47819ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47d200700 1 -- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4781082b0 msgr2=0x7fe4781988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47d200700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4781082b0 0x7fe4781988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47d200700 1 -- 172.21.15.42:0/1173271828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe478199020 con 0x7fe478069f40 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.411+0000 7fe47dc00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4781082b0 0x7fe4781988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.412+0000 7fe47d200700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe478069f40 0x7fe47819e700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe46c000c00 tx=0x7fe46c00bab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.412+0000 7fe475600700 1 -- 172.21.15.42:0/1173271828 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c027070 con 0x7fe478069f40 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.412+0000 7fe475600700 1 -- 172.21.15.42:0/1173271828 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe46c003d70 con 0x7fe478069f40 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.412+0000 7fe475600700 1 -- 172.21.15.42:0/1173271828 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00f460 con 0x7fe478069f40 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.412+0000 7fe47fc00700 1 -- 172.21.15.42:0/1173271828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4781992b0 con 0x7fe478069f40 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.412+0000 7fe47fc00700 1 -- 172.21.15.42:0/1173271828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4781a55d0 con 0x7fe478069f40 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.413+0000 7fe47fc00700 1 -- 172.21.15.42:0/1173271828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe478068f20 con 0x7fe478069f40 2024-04-25T00:17:27.185 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.416+0000 7fe475600700 1 -- 172.21.15.42:0/1173271828 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe46c02a070 con 0x7fe478069f40 2024-04-25T00:17:27.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.417+0000 7fe475600700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4600a2d30 0x7fe4600a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:27.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.417+0000 7fe475600700 1 -- 172.21.15.42:0/1173271828 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe46c0d03a0 con 0x7fe478069f40 2024-04-25T00:17:27.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.417+0000 7fe477e00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4600a2d30 0x7fe4600a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:27.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.418+0000 7fe477e00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4600a2d30 0x7fe4600a51f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe478078800 tx=0x7fe464009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:27.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.419+0000 7fe475600700 1 -- 172.21.15.42:0/1173271828 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe46c013070 con 0x7fe478069f40 2024-04-25T00:17:27.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.656+0000 7fe47fc00700 1 -- 172.21.15.42:0/1173271828 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe47802db30 con 0x7fe478069f40 2024-04-25T00:17:27.186 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.657+0000 7fe475600700 1 -- 172.21.15.42:0/1173271828 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe46c09d2e0 con 0x7fe478069f40 2024-04-25T00:17:27.186 INFO:teuthology.orchestra.run.smithi042.stdout:34359738455 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 -- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4600a2d30 msgr2=0x7fe4600a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4600a2d30 0x7fe4600a51f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe478078800 tx=0x7fe464009260 comp rx=0 tx=0).stop 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 -- 172.21.15.42:0/1173271828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe478069f40 msgr2=0x7fe47819e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe478069f40 0x7fe47819e700 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe46c000c00 tx=0x7fe46c00bab0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 -- 172.21.15.42:0/1173271828 shutdown_connections 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe4600a2d30 0x7fe4600a51f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe4781082b0 0x7fe4781988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe478069f40 0x7fe47819e700 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 --2- 172.21.15.42:0/1173271828 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe47806a8e0 0x7fe47819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 -- 172.21.15.42:0/1173271828 >> 172.21.15.42:0/1173271828 conn(0x7fe4780780e0 msgr2=0x7fe4780fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 -- 172.21.15.42:0/1173271828 shutdown_connections 2024-04-25T00:17:27.187 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.660+0000 7fe47fc00700 1 -- 172.21.15.42:0/1173271828 wait complete. 2024-04-25T00:17:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:27 smithi106 ceph-mon[32510]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:17:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T00:17:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T00:17:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T00:17:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-25T00:17:27.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-25T00:17:27.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-25T00:17:27.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:17:27.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:27 smithi106 ceph-mon[32510]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:17:27.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:27 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1173271828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:17:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:27 smithi042 ceph-mon[28130]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:17:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T00:17:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T00:17:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T00:17:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-25T00:17:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-25T00:17:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-25T00:17:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:17:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:27 smithi042 ceph-mon[28130]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:17:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:27 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1173271828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:17:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:27 smithi136 ceph-mon[32923]: pgmap v346: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:17:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-25T00:17:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T00:17:28.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-04-25T00:17:28.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-04-25T00:17:28.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.12", "id": [3, 9]}]': finished 2024-04-25T00:17:28.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 6]}]': finished 2024-04-25T00:17:28.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3c", "id": [4, 9]}]': finished 2024-04-25T00:17:28.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:27 smithi136 ceph-mon[32923]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T00:17:28.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:27 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1173271828' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T00:17:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.080+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3981214303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a0104e50 msgr2=0x7fe9a0111310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.080+0000 7fe9a7a00700 1 --2- 172.21.15.42:0/3981214303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a0104e50 0x7fe9a0111310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe99800b0a0 tx=0x7fe99800b3b0 comp rx=0 tx=0).stop 2024-04-25T00:17:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.080+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3981214303 shutdown_connections 2024-04-25T00:17:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.080+0000 7fe9a7a00700 1 --2- 172.21.15.42:0/3981214303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a0104e50 0x7fe9a0111310 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.080+0000 7fe9a7a00700 1 --2- 172.21.15.42:0/3981214303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a010a540 0x7fe9a010a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.080+0000 7fe9a7a00700 1 --2- 172.21.15.42:0/3981214303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9a0104490 0x7fe9a0104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.080+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3981214303 >> 172.21.15.42:0/3981214303 conn(0x7fe9a00fff60 msgr2=0x7fe9a0102380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.080+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3981214303 shutdown_connections 2024-04-25T00:17:28.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.080+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3981214303 wait complete. 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.081+0000 7fe9a7a00700 1 Processor -- start 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.081+0000 7fe9a7a00700 1 -- start start 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe9a7a00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a0104490 0x7fe9a01a0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe9a7a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9a0104e50 0x7fe9a01a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe9a7a00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a010a540 0x7fe9a019a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe9a7a00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a01147c0 con 0x7fe9a0104490 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe9a7a00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a0114640 con 0x7fe9a0104e50 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe9a7a00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a0114940 con 0x7fe9a010a540 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe9a5000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a0104490 0x7fe9a01a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe9a5000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a0104490 0x7fe9a01a0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55840/0 (socket says 172.21.15.42:55840) 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe9a5000700 1 -- 172.21.15.42:0/3890617199 learned_addr learned my addr 172.21.15.42:0/3890617199 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe99fe00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9a0104e50 0x7fe9a01a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:28.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.082+0000 7fe9a5a00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a010a540 0x7fe9a019a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe99fe00700 1 -- 172.21.15.42:0/3890617199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a010a540 msgr2=0x7fe9a019a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe99fe00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a010a540 0x7fe9a019a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe99fe00700 1 -- 172.21.15.42:0/3890617199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a0104490 msgr2=0x7fe9a01a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe99fe00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a0104490 0x7fe9a01a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe99fe00700 1 -- 172.21.15.42:0/3890617199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a019b080 con 0x7fe9a0104e50 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe9a5000700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a0104490 0x7fe9a01a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe9a5a00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a010a540 0x7fe9a019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe99fe00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9a0104e50 0x7fe9a01a0ca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe99000e9f0 tx=0x7fe99000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe99d600700 1 -- 172.21.15.42:0/3890617199 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99000c9c0 con 0x7fe9a0104e50 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe99d600700 1 -- 172.21.15.42:0/3890617199 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe99000cb60 con 0x7fe9a0104e50 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3890617199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a019b370 con 0x7fe9a0104e50 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe99d600700 1 -- 172.21.15.42:0/3890617199 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe990018680 con 0x7fe9a0104e50 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.083+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3890617199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a01a7630 con 0x7fe9a0104e50 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.084+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3890617199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a0068f20 con 0x7fe9a0104e50 2024-04-25T00:17:28.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.086+0000 7fe99d600700 1 -- 172.21.15.42:0/3890617199 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe990010430 con 0x7fe9a0104e50 2024-04-25T00:17:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.088+0000 7fe99d600700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9880a2cb0 0x7fe9880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.088+0000 7fe99d600700 1 -- 172.21.15.42:0/3890617199 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 7979+0+0 (secure 0 0 0) 0x7fe9900cf230 con 0x7fe9a0104e50 2024-04-25T00:17:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.088+0000 7fe9a5000700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9880a2cb0 0x7fe9880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.089+0000 7fe9a5000700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9880a2cb0 0x7fe9880a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9a019ba60 tx=0x7fe994009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:28.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.092+0000 7fe99d600700 1 -- 172.21.15.42:0/3890617199 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe99009c0e0 con 0x7fe9a0104e50 2024-04-25T00:17:28.744 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:28 smithi042 ceph-mon[28130]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:17:28.744 INFO:teuthology.orchestra.run.smithi042.stdout:257698037803 2024-04-25T00:17:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.334+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3890617199 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe9a01a7860 con 0x7fe9a0104e50 2024-04-25T00:17:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.334+0000 7fe99d600700 1 -- 172.21.15.42:0/3890617199 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe99009b9e0 con 0x7fe9a0104e50 2024-04-25T00:17:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3890617199 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9880a2cb0 msgr2=0x7fe9880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9880a2cb0 0x7fe9880a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fe9a019ba60 tx=0x7fe994009260 comp rx=0 tx=0).stop 2024-04-25T00:17:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3890617199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9a0104e50 msgr2=0x7fe9a01a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9a0104e50 0x7fe9a01a0ca0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe99000e9f0 tx=0x7fe99000ed00 comp rx=0 tx=0).stop 2024-04-25T00:17:28.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3890617199 shutdown_connections 2024-04-25T00:17:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe9880a2cb0 0x7fe9880a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe9a010a540 0x7fe9a019a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe9a0104490 0x7fe9a01a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 --2- 172.21.15.42:0/3890617199 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe9a0104e50 0x7fe9a01a0ca0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3890617199 >> 172.21.15.42:0/3890617199 conn(0x7fe9a00fff60 msgr2=0x7fe9a0101460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3890617199 shutdown_connections 2024-04-25T00:17:28.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:26.337+0000 7fe9a7a00700 1 -- 172.21.15.42:0/3890617199 wait complete. 2024-04-25T00:17:28.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:28 smithi106 ceph-mon[32510]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:17:29.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:28 smithi136 ceph-mon[32923]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T00:17:29.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:29 smithi106 ceph-mon[32510]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-25T00:17:29.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:29 smithi106 ceph-mon[32510]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T00:17:29.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:29 smithi042 ceph-mon[28130]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-25T00:17:29.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:29 smithi042 ceph-mon[28130]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T00:17:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:29 smithi136 ceph-mon[32923]: pgmap v349: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 7 op/s 2024-04-25T00:17:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:29 smithi136 ceph-mon[32923]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-25T00:17:30.781 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411400 got 77309411403 for osd.2 2024-04-25T00:17:30.781 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:31.788 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:31 smithi042 ceph-mon[28130]: pgmap v350: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.534+0000 7f852c800700 1 -- 172.21.15.42:0/303801236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8524101130 msgr2=0x7f85241015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.534+0000 7f852c800700 1 --2- 172.21.15.42:0/303801236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8524101130 0x7f85241015b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8514009990 tx=0x7f8514009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.534+0000 7f852c800700 1 -- 172.21.15.42:0/303801236 shutdown_connections 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.534+0000 7f852c800700 1 --2- 172.21.15.42:0/303801236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524101af0 0x7f852410dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.534+0000 7f852c800700 1 --2- 172.21.15.42:0/303801236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85241071a0 0x7f8524107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.534+0000 7f852c800700 1 --2- 172.21.15.42:0/303801236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8524101130 0x7f85241015b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.534+0000 7f852c800700 1 -- 172.21.15.42:0/303801236 >> 172.21.15.42:0/303801236 conn(0x7f85240780e0 msgr2=0x7f85240784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.534+0000 7f852c800700 1 -- 172.21.15.42:0/303801236 shutdown_connections 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.534+0000 7f852c800700 1 -- 172.21.15.42:0/303801236 wait complete. 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.535+0000 7f852c800700 1 Processor -- start 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.536+0000 7f852c800700 1 -- start start 2024-04-25T00:17:31.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.536+0000 7f852c800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524101130 0x7f852419e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.536+0000 7f852c800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8524101af0 0x7f852419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.536+0000 7f852c800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85241071a0 0x7f8524198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.536+0000 7f852c800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8524111460 con 0x7f85241071a0 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.536+0000 7f852c800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f85241112e0 con 0x7f8524101af0 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.536+0000 7f852c800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85241115e0 con 0x7f8524101130 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f8529e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524101130 0x7f852419e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f852a800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85241071a0 0x7f8524198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f8529e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524101130 0x7f852419e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:43984/0 (socket says 172.21.15.42:43984) 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f8529e00700 1 -- 172.21.15.42:0/2695040671 learned_addr learned my addr 172.21.15.42:0/2695040671 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f8529400700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8524101af0 0x7f852419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f8529e00700 1 -- 172.21.15.42:0/2695040671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8524101af0 msgr2=0x7f852419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f8529e00700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8524101af0 0x7f852419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f8529e00700 1 -- 172.21.15.42:0/2695040671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85241071a0 msgr2=0x7f8524198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f8529e00700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85241071a0 0x7f8524198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f8529e00700 1 -- 172.21.15.42:0/2695040671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8524199040 con 0x7f8524101130 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f852a800700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85241071a0 0x7f8524198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.537+0000 7f8529e00700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524101130 0x7f852419e690 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f851800e9a0 tx=0x7f851800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.538+0000 7f851ea00700 1 -- 172.21.15.42:0/2695040671 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851800ca10 con 0x7f8524101130 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.538+0000 7f852c800700 1 -- 172.21.15.42:0/2695040671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8524199330 con 0x7f8524101130 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.538+0000 7f851ea00700 1 -- 172.21.15.42:0/2695040671 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f851800cbb0 con 0x7f8524101130 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.538+0000 7f851ea00700 1 -- 172.21.15.42:0/2695040671 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8518018680 con 0x7f8524101130 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.538+0000 7f852c800700 1 -- 172.21.15.42:0/2695040671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85241a54f0 con 0x7f8524101130 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.539+0000 7f852c800700 1 -- 172.21.15.42:0/2695040671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8524102850 con 0x7f8524101130 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.541+0000 7f851ea00700 1 -- 172.21.15.42:0/2695040671 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8518010430 con 0x7f8524101130 2024-04-25T00:17:31.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.542+0000 7f851ea00700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f850c0a2cb0 0x7f850c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:31.804 INFO:teuthology.orchestra.run.smithi042.stdout:163208757308 2024-04-25T00:17:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.543+0000 7f851ea00700 1 -- 172.21.15.42:0/2695040671 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f85180cf810 con 0x7f8524101130 2024-04-25T00:17:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.543+0000 7f8529400700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f850c0a2cb0 0x7f850c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.543+0000 7f8529400700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f850c0a2cb0 0x7f850c0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85240786e0 tx=0x7f851400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.545+0000 7f851ea00700 1 -- 172.21.15.42:0/2695040671 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f851809c5e0 con 0x7f8524101130 2024-04-25T00:17:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.787+0000 7f852c800700 1 -- 172.21.15.42:0/2695040671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f85241a5750 con 0x7f8524101130 2024-04-25T00:17:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.788+0000 7f851ea00700 1 -- 172.21.15.42:0/2695040671 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f851809bee0 con 0x7f8524101130 2024-04-25T00:17:31.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.791+0000 7f852c800700 1 -- 172.21.15.42:0/2695040671 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f850c0a2cb0 msgr2=0x7f850c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.791+0000 7f852c800700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f850c0a2cb0 0x7f850c0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f85240786e0 tx=0x7f851400b260 comp rx=0 tx=0).stop 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.792+0000 7f852c800700 1 -- 172.21.15.42:0/2695040671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524101130 msgr2=0x7f852419e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.792+0000 7f852c800700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524101130 0x7f852419e690 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f851800e9a0 tx=0x7f851800ecb0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.792+0000 7f852c800700 1 -- 172.21.15.42:0/2695040671 shutdown_connections 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.792+0000 7f852c800700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f850c0a2cb0 0x7f850c0a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.792+0000 7f852c800700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85241071a0 0x7f8524198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.792+0000 7f852c800700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8524101130 0x7f852419e690 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.792+0000 7f852c800700 1 --2- 172.21.15.42:0/2695040671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8524101af0 0x7f852419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.792+0000 7f852c800700 1 -- 172.21.15.42:0/2695040671 >> 172.21.15.42:0/2695040671 conn(0x7f85240780e0 msgr2=0x7f8524100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.792+0000 7f852c800700 1 -- 172.21.15.42:0/2695040671 shutdown_connections 2024-04-25T00:17:31.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:31.792+0000 7f852c800700 1 -- 172.21.15.42:0/2695040671 wait complete. 2024-04-25T00:17:31.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:31 smithi106 ceph-mon[32510]: pgmap v350: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-04-25T00:17:32.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:31 smithi136 ceph-mon[32923]: pgmap v350: 97 pgs: 2 peering, 95 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s 2024-04-25T00:17:32.790 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:32 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2695040671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:17:33.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:32 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2695040671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:17:33.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:32 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2695040671' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T00:17:33.847 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:33 smithi042 ceph-mon[28130]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:33.847 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:33 smithi042 ceph-mon[28130]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-25T00:17:33.847 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:33 smithi042 ceph-mon[28130]: Cluster is now healthy 2024-04-25T00:17:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:33 smithi136 ceph-mon[32923]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:33 smithi136 ceph-mon[32923]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-25T00:17:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:33 smithi136 ceph-mon[32923]: Cluster is now healthy 2024-04-25T00:17:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:33 smithi106 ceph-mon[32510]: pgmap v351: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:33 smithi106 ceph-mon[32510]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-25T00:17:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:33 smithi106 ceph-mon[32510]: Cluster is now healthy 2024-04-25T00:17:34.436 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738451 got 34359738455 for osd.0 2024-04-25T00:17:34.436 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:34.943 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.943+0000 7f1d32e00700 1 -- 172.21.15.42:0/2915782890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d2c102d60 msgr2=0x7f1d2c106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:34.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.943+0000 7f1d32e00700 1 --2- 172.21.15.42:0/2915782890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d2c102d60 0x7f1d2c106db0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1d180099c0 tx=0x7f1d18009cd0 comp rx=0 tx=0).stop 2024-04-25T00:17:34.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.943+0000 7f1d32e00700 1 -- 172.21.15.42:0/2915782890 shutdown_connections 2024-04-25T00:17:34.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.943+0000 7f1d32e00700 1 --2- 172.21.15.42:0/2915782890 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2c107490 0x7f1d2c10f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:34.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.943+0000 7f1d32e00700 1 --2- 172.21.15.42:0/2915782890 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2c1023b0 0x7f1d2c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:34.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.943+0000 7f1d32e00700 1 --2- 172.21.15.42:0/2915782890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d2c102d60 0x7f1d2c106db0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:34.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.943+0000 7f1d32e00700 1 -- 172.21.15.42:0/2915782890 >> 172.21.15.42:0/2915782890 conn(0x7f1d2c0fdc40 msgr2=0x7f1d2c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:34.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.943+0000 7f1d32e00700 1 -- 172.21.15.42:0/2915782890 shutdown_connections 2024-04-25T00:17:34.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.943+0000 7f1d32e00700 1 -- 172.21.15.42:0/2915782890 wait complete. 2024-04-25T00:17:34.944 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.944+0000 7f1d32e00700 1 Processor -- start 2024-04-25T00:17:34.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.944+0000 7f1d32e00700 1 -- start start 2024-04-25T00:17:34.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d32e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2c1023b0 0x7f1d2c074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:34.945 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d32e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2c102d60 0x7f1d2c074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:34.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d32e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d2c107490 0x7f1d2c074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:34.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d32e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c112470 con 0x7f1d2c102d60 2024-04-25T00:17:34.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d32e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1122f0 con 0x7f1d2c1023b0 2024-04-25T00:17:34.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d32e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2c1125f0 con 0x7f1d2c107490 2024-04-25T00:17:34.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d31000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2c102d60 0x7f1d2c074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:34.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d31a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2c1023b0 0x7f1d2c074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:34.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d31a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2c1023b0 0x7f1d2c074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58158/0 (socket says 172.21.15.42:58158) 2024-04-25T00:17:34.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d31a00700 1 -- 172.21.15.42:0/3731374359 learned_addr learned my addr 172.21.15.42:0/3731374359 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:34.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.945+0000 7f1d32400700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d2c107490 0x7f1d2c074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:34.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.946+0000 7f1d31a00700 1 -- 172.21.15.42:0/3731374359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d2c107490 msgr2=0x7f1d2c074fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:34.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.946+0000 7f1d31a00700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d2c107490 0x7f1d2c074fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:34.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.946+0000 7f1d31a00700 1 -- 172.21.15.42:0/3731374359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2c102d60 msgr2=0x7f1d2c074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:34.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.946+0000 7f1d31a00700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2c102d60 0x7f1d2c074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:34.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.946+0000 7f1d31a00700 1 -- 172.21.15.42:0/3731374359 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2c1a6e40 con 0x7f1d2c1023b0 2024-04-25T00:17:34.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.946+0000 7f1d32400700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d2c107490 0x7f1d2c074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:34.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.946+0000 7f1d31a00700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2c1023b0 0x7f1d2c074530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000ef30 tx=0x7f1d2000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:34.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.946+0000 7f1d2a000700 1 -- 172.21.15.42:0/3731374359 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d200092e0 con 0x7f1d2c1023b0 2024-04-25T00:17:34.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.946+0000 7f1d2a000700 1 -- 172.21.15.42:0/3731374359 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d20004bb0 con 0x7f1d2c1023b0 2024-04-25T00:17:34.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.947+0000 7f1d32e00700 1 -- 172.21.15.42:0/3731374359 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2c1a70d0 con 0x7f1d2c1023b0 2024-04-25T00:17:34.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.947+0000 7f1d2a000700 1 -- 172.21.15.42:0/3731374359 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d20004280 con 0x7f1d2c1023b0 2024-04-25T00:17:34.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.947+0000 7f1d32e00700 1 -- 172.21.15.42:0/3731374359 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d2c1a7590 con 0x7f1d2c1023b0 2024-04-25T00:17:34.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.948+0000 7f1d32e00700 1 -- 172.21.15.42:0/3731374359 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d2c068f20 con 0x7f1d2c1023b0 2024-04-25T00:17:34.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.950+0000 7f1d2a000700 1 -- 172.21.15.42:0/3731374359 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1d2001b030 con 0x7f1d2c1023b0 2024-04-25T00:17:34.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.951+0000 7f1d2a000700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d140a2cb0 0x7f1d140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:34.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.951+0000 7f1d2a000700 1 -- 172.21.15.42:0/3731374359 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1d200cf150 con 0x7f1d2c1023b0 2024-04-25T00:17:34.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.951+0000 7f1d31000700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d140a2cb0 0x7f1d140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:34.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.952+0000 7f1d31000700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d140a2cb0 0x7f1d140a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d180098e0 tx=0x7f1d18005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:34.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:34.954+0000 7f1d2a000700 1 -- 172.21.15.42:0/3731374359 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1d20022020 con 0x7f1d2c1023b0 2024-04-25T00:17:35.192 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.192+0000 7f1d32e00700 1 -- 172.21.15.42:0/3731374359 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1d2c02db30 con 0x7f1d2c1023b0 2024-04-25T00:17:35.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.193+0000 7f1d2a000700 1 -- 172.21.15.42:0/3731374359 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1d20010040 con 0x7f1d2c1023b0 2024-04-25T00:17:35.193 INFO:teuthology.orchestra.run.smithi042.stdout:141733920835 2024-04-25T00:17:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.195+0000 7f1d32e00700 1 -- 172.21.15.42:0/3731374359 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d140a2cb0 msgr2=0x7f1d140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.195+0000 7f1d32e00700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d140a2cb0 0x7f1d140a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1d180098e0 tx=0x7f1d18005960 comp rx=0 tx=0).stop 2024-04-25T00:17:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.195+0000 7f1d32e00700 1 -- 172.21.15.42:0/3731374359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2c1023b0 msgr2=0x7f1d2c074530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:35.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.195+0000 7f1d32e00700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2c1023b0 0x7f1d2c074530 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1d2000ef30 tx=0x7f1d2000c3a0 comp rx=0 tx=0).stop 2024-04-25T00:17:35.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.196+0000 7f1d32e00700 1 -- 172.21.15.42:0/3731374359 shutdown_connections 2024-04-25T00:17:35.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.196+0000 7f1d32e00700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1d140a2cb0 0x7f1d140a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:35.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.196+0000 7f1d32e00700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d2c107490 0x7f1d2c074fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:35.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.196+0000 7f1d32e00700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2c1023b0 0x7f1d2c074530 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:35.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.196+0000 7f1d32e00700 1 --2- 172.21.15.42:0/3731374359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1d2c102d60 0x7f1d2c074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:35.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.196+0000 7f1d32e00700 1 -- 172.21.15.42:0/3731374359 >> 172.21.15.42:0/3731374359 conn(0x7f1d2c0fdc40 msgr2=0x7f1d2c0fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:35.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.196+0000 7f1d32e00700 1 -- 172.21.15.42:0/3731374359 shutdown_connections 2024-04-25T00:17:35.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:35.196+0000 7f1d32e00700 1 -- 172.21.15.42:0/3731374359 wait complete. 2024-04-25T00:17:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:35 smithi042 ceph-mon[28130]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:35 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3731374359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:17:36.011 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037798 got 257698037803 for osd.8 2024-04-25T00:17:36.011 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:35 smithi136 ceph-mon[32923]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:35 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3731374359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:17:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:35 smithi106 ceph-mon[32510]: pgmap v352: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:35 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3731374359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T00:17:36.666 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.665+0000 7f5733e00700 1 -- 172.21.15.42:0/3143748745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734103320 msgr2=0x7f5734103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.666+0000 7f5733e00700 1 --2- 172.21.15.42:0/3143748745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734103320 0x7f5734103700 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f572400b0a0 tx=0x7f572400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.666+0000 7f5733e00700 1 -- 172.21.15.42:0/3143748745 shutdown_connections 2024-04-25T00:17:36.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.666+0000 7f5733e00700 1 --2- 172.21.15.42:0/3143748745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f573410c750 0x7f573410ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.666+0000 7f5733e00700 1 --2- 172.21.15.42:0/3143748745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734103320 0x7f5734103700 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.666+0000 7f5733e00700 1 --2- 172.21.15.42:0/3143748745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5734103c40 0x7f573410c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.666+0000 7f5733e00700 1 -- 172.21.15.42:0/3143748745 >> 172.21.15.42:0/3143748745 conn(0x7f57340fcb50 msgr2=0x7f57340fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:36.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.666+0000 7f5733e00700 1 -- 172.21.15.42:0/3143748745 shutdown_connections 2024-04-25T00:17:36.667 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.666+0000 7f5733e00700 1 -- 172.21.15.42:0/3143748745 wait complete. 2024-04-25T00:17:36.668 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.667+0000 7f5733e00700 1 Processor -- start 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.667+0000 7f5733e00700 1 -- start start 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5733e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5734103320 0x7f57341a07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5733e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734103c40 0x7f57341a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5733e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f573410c750 0x7f573419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5733e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f57341124e0 con 0x7f5734103c40 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5733e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5734112360 con 0x7f5734103320 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5733e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5734112660 con 0x7f573410c750 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5732a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5734103320 0x7f57341a07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5732a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5734103320 0x7f57341a07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58174/0 (socket says 172.21.15.42:58174) 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5732a00700 1 -- 172.21.15.42:0/2312236661 learned_addr learned my addr 172.21.15.42:0/2312236661 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f572a000700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734103c40 0x7f57341a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5733400700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f573410c750 0x7f573419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5732a00700 1 -- 172.21.15.42:0/2312236661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f573410c750 msgr2=0x7f573419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5732a00700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f573410c750 0x7f573419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5732a00700 1 -- 172.21.15.42:0/2312236661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734103c40 msgr2=0x7f57341a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5732a00700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734103c40 0x7f57341a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.668+0000 7f5732a00700 1 -- 172.21.15.42:0/2312236661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f573419b0d0 con 0x7f5734103320 2024-04-25T00:17:36.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.669+0000 7f5732a00700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5734103320 0x7f57341a07b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f572400b070 tx=0x7f5724004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:36.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.669+0000 7f572be00700 1 -- 172.21.15.42:0/2312236661 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5724021070 con 0x7f5734103320 2024-04-25T00:17:36.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.669+0000 7f572be00700 1 -- 172.21.15.42:0/2312236661 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5724003c00 con 0x7f5734103320 2024-04-25T00:17:36.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.669+0000 7f5733e00700 1 -- 172.21.15.42:0/2312236661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f573419b360 con 0x7f5734103320 2024-04-25T00:17:36.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.669+0000 7f572be00700 1 -- 172.21.15.42:0/2312236661 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57240048a0 con 0x7f5734103320 2024-04-25T00:17:36.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.669+0000 7f5733e00700 1 -- 172.21.15.42:0/2312236661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57341a7680 con 0x7f5734103320 2024-04-25T00:17:36.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.670+0000 7f5733e00700 1 -- 172.21.15.42:0/2312236661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5734068f20 con 0x7f5734103320 2024-04-25T00:17:36.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.672+0000 7f572be00700 1 -- 172.21.15.42:0/2312236661 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f5724004020 con 0x7f5734103320 2024-04-25T00:17:36.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.673+0000 7f572be00700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f57180a2e50 0x7f57180a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:36.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.674+0000 7f572be00700 1 -- 172.21.15.42:0/2312236661 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f572400f020 con 0x7f5734103320 2024-04-25T00:17:36.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.674+0000 7f572a000700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f57180a2e50 0x7f57180a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:36.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.674+0000 7f572a000700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f57180a2e50 0x7f57180a5310 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f573410e030 tx=0x7f571c00b360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:36.678 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.678+0000 7f572be00700 1 -- 172.21.15.42:0/2312236661 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f572402a020 con 0x7f5734103320 2024-04-25T00:17:36.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.916+0000 7f5733e00700 1 -- 172.21.15.42:0/2312236661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f57341a7850 con 0x7f5734103320 2024-04-25T00:17:36.918 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.917+0000 7f572be00700 1 -- 172.21.15.42:0/2312236661 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f572409d550 con 0x7f5734103320 2024-04-25T00:17:36.918 INFO:teuthology.orchestra.run.smithi042.stdout:107374182472 2024-04-25T00:17:36.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.920+0000 7f5733e00700 1 -- 172.21.15.42:0/2312236661 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f57180a2e50 msgr2=0x7f57180a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.920+0000 7f5733e00700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f57180a2e50 0x7f57180a5310 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f573410e030 tx=0x7f571c00b360 comp rx=0 tx=0).stop 2024-04-25T00:17:36.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.920+0000 7f5733e00700 1 -- 172.21.15.42:0/2312236661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5734103320 msgr2=0x7f57341a07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:36.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.920+0000 7f5733e00700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5734103320 0x7f57341a07b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f572400b070 tx=0x7f5724004cf0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.920+0000 7f5733e00700 1 -- 172.21.15.42:0/2312236661 shutdown_connections 2024-04-25T00:17:36.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.920+0000 7f5733e00700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f57180a2e50 0x7f57180a5310 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.920+0000 7f5733e00700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f573410c750 0x7f573419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.920+0000 7f5733e00700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5734103320 0x7f57341a07b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.920+0000 7f5733e00700 1 --2- 172.21.15.42:0/2312236661 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5734103c40 0x7f57341a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:36.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.920+0000 7f5733e00700 1 -- 172.21.15.42:0/2312236661 >> 172.21.15.42:0/2312236661 conn(0x7f57340fcb50 msgr2=0x7f573410e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:36.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.921+0000 7f5733e00700 1 -- 172.21.15.42:0/2312236661 shutdown_connections 2024-04-25T00:17:36.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:36.921+0000 7f5733e00700 1 -- 172.21.15.42:0/2312236661 wait complete. 2024-04-25T00:17:37.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:37 smithi042 ceph-mon[28130]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:37.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:37 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2312236661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:17:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:37 smithi136 ceph-mon[32923]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:37 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2312236661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:17:38.041 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757302 got 163208757308 for osd.5 2024-04-25T00:17:38.042 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:37 smithi106 ceph-mon[32510]: pgmap v353: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:37 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2312236661' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T00:17:38.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.691+0000 7f91d7800700 1 -- 172.21.15.42:0/4134645532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d01082f0 msgr2=0x7f91d01086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:38.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.691+0000 7f91d7800700 1 --2- 172.21.15.42:0/4134645532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d01082f0 0x7f91d01086d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f91c40098f0 tx=0x7f91c4009c00 comp rx=0 tx=0).stop 2024-04-25T00:17:38.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.692+0000 7f91d7800700 1 -- 172.21.15.42:0/4134645532 shutdown_connections 2024-04-25T00:17:38.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.692+0000 7f91d7800700 1 --2- 172.21.15.42:0/4134645532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0102c50 0x7f91d010f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:38.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.692+0000 7f91d7800700 1 --2- 172.21.15.42:0/4134645532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d01082f0 0x7f91d01086d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:38.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.692+0000 7f91d7800700 1 --2- 172.21.15.42:0/4134645532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91d01022b0 0x7f91d0102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:38.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.692+0000 7f91d7800700 1 -- 172.21.15.42:0/4134645532 >> 172.21.15.42:0/4134645532 conn(0x7f91d0079300 msgr2=0x7f91d0079710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:38.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.692+0000 7f91d7800700 1 -- 172.21.15.42:0/4134645532 shutdown_connections 2024-04-25T00:17:38.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.692+0000 7f91d7800700 1 -- 172.21.15.42:0/4134645532 wait complete. 2024-04-25T00:17:38.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.693+0000 7f91d7800700 1 Processor -- start 2024-04-25T00:17:38.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.693+0000 7f91d7800700 1 -- start start 2024-04-25T00:17:38.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.693+0000 7f91d7800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d01022b0 0x7f91d01a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:38.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.693+0000 7f91d7800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0102c50 0x7f91d01a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:38.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.693+0000 7f91d7800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91d01082f0 0x7f91d019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:38.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d7800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0112580 con 0x7f91d01022b0 2024-04-25T00:17:38.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d7800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0112400 con 0x7f91d01082f0 2024-04-25T00:17:38.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d7800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f91d0112700 con 0x7f91d0102c50 2024-04-25T00:17:38.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d4e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d01022b0 0x7f91d01a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:38.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d4e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d01022b0 0x7f91d01a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59280/0 (socket says 172.21.15.42:59280) 2024-04-25T00:17:38.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d4e00700 1 -- 172.21.15.42:0/171535226 learned_addr learned my addr 172.21.15.42:0/171535226 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:38.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91cfe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0102c50 0x7f91d01a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:38.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d5800700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91d01082f0 0x7f91d019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d4e00700 1 -- 172.21.15.42:0/171535226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0102c50 msgr2=0x7f91d01a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d4e00700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0102c50 0x7f91d01a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d4e00700 1 -- 172.21.15.42:0/171535226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91d01082f0 msgr2=0x7f91d019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d4e00700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91d01082f0 0x7f91d019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d4e00700 1 -- 172.21.15.42:0/171535226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d019b410 con 0x7f91d01022b0 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.694+0000 7f91d5800700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91d01082f0 0x7f91d019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.695+0000 7f91d4e00700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d01022b0 0x7f91d01a0a60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f91c4005950 tx=0x7f91c400fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.695+0000 7f91cd600700 1 -- 172.21.15.42:0/171535226 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4005140 con 0x7f91d01022b0 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.695+0000 7f91cd600700 1 -- 172.21.15.42:0/171535226 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f91c400b560 con 0x7f91d01022b0 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.695+0000 7f91d7800700 1 -- 172.21.15.42:0/171535226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d019b6a0 con 0x7f91d01022b0 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.695+0000 7f91cd600700 1 -- 172.21.15.42:0/171535226 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4003ae0 con 0x7f91d01022b0 2024-04-25T00:17:38.696 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.695+0000 7f91d7800700 1 -- 172.21.15.42:0/171535226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d01a7810 con 0x7f91d01022b0 2024-04-25T00:17:38.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.697+0000 7f91d7800700 1 -- 172.21.15.42:0/171535226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d01039a0 con 0x7f91d01022b0 2024-04-25T00:17:38.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.699+0000 7f91cd600700 1 -- 172.21.15.42:0/171535226 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f91c400ba80 con 0x7f91d01022b0 2024-04-25T00:17:38.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.700+0000 7f91cd600700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91b80a2cb0 0x7f91b80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:38.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.701+0000 7f91cd600700 1 -- 172.21.15.42:0/171535226 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f91c40d02e0 con 0x7f91d01022b0 2024-04-25T00:17:38.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.701+0000 7f91cfe00700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91b80a2cb0 0x7f91b80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:38.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.701+0000 7f91cfe00700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91b80a2cb0 0x7f91b80a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009750 tx=0x7f91bc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:38.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.704+0000 7f91cd600700 1 -- 172.21.15.42:0/171535226 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f91c405e740 con 0x7f91d01022b0 2024-04-25T00:17:38.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:38 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3104193073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:17:38.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.946+0000 7f91d7800700 1 -- 172.21.15.42:0/171535226 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f91d01a7a40 con 0x7f91d01022b0 2024-04-25T00:17:38.946 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.946+0000 7f91cd600700 1 -- 172.21.15.42:0/171535226 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f91c4013030 con 0x7f91d01022b0 2024-04-25T00:17:38.947 INFO:teuthology.orchestra.run.smithi042.stdout:55834574931 2024-04-25T00:17:38.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 -- 172.21.15.42:0/171535226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91b80a2cb0 msgr2=0x7f91b80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:38.949 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91b80a2cb0 0x7f91b80a5170 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f91bc009750 tx=0x7f91bc009310 comp rx=0 tx=0).stop 2024-04-25T00:17:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 -- 172.21.15.42:0/171535226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d01022b0 msgr2=0x7f91d01a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d01022b0 0x7f91d01a0a60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f91c4005950 tx=0x7f91c400fb20 comp rx=0 tx=0).stop 2024-04-25T00:17:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 -- 172.21.15.42:0/171535226 shutdown_connections 2024-04-25T00:17:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91b80a2cb0 0x7f91b80a5170 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f91d01082f0 0x7f91d019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f91d01022b0 0x7f91d01a0a60 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 --2- 172.21.15.42:0/171535226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91d0102c50 0x7f91d01a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 -- 172.21.15.42:0/171535226 >> 172.21.15.42:0/171535226 conn(0x7f91d0079300 msgr2=0x7f91d0100ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 -- 172.21.15.42:0/171535226 shutdown_connections 2024-04-25T00:17:38.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.949+0000 7f91d7800700 1 -- 172.21.15.42:0/171535226 wait complete. 2024-04-25T00:17:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:38 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3104193073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:17:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:38 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3104193073' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.922+0000 7ff945e00700 1 -- 172.21.15.42:0/3380948891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff940074460 msgr2=0x7ff940074840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.922+0000 7ff945e00700 1 --2- 172.21.15.42:0/3380948891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff940074460 0x7ff940074840 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff930009990 tx=0x7ff930009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.922+0000 7ff945e00700 1 -- 172.21.15.42:0/3380948891 shutdown_connections 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.922+0000 7ff945e00700 1 --2- 172.21.15.42:0/3380948891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff94010ec30 0x7ff940111020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.922+0000 7ff945e00700 1 --2- 172.21.15.42:0/3380948891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff940074460 0x7ff940074840 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.922+0000 7ff945e00700 1 --2- 172.21.15.42:0/3380948891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff940074d80 0x7ff94010e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.922+0000 7ff945e00700 1 -- 172.21.15.42:0/3380948891 >> 172.21.15.42:0/3380948891 conn(0x7ff940079390 msgr2=0x7ff9400797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.922+0000 7ff945e00700 1 -- 172.21.15.42:0/3380948891 shutdown_connections 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.922+0000 7ff945e00700 1 -- 172.21.15.42:0/3380948891 wait complete. 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.924+0000 7ff945e00700 1 Processor -- start 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.924+0000 7ff945e00700 1 -- start start 2024-04-25T00:17:39.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff945e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff940074460 0x7ff9401a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff945e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff940074d80 0x7ff9401a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff945e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff94010ec30 0x7ff94019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff945e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff940114730 con 0x7ff940074460 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff945e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401145b0 con 0x7ff94010ec30 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff945e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff9401148b0 con 0x7ff940074d80 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff937e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff940074d80 0x7ff9401a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff937e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff940074d80 0x7ff9401a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:53134/0 (socket says 172.21.15.42:53134) 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff937e00700 1 -- 172.21.15.42:0/348181974 learned_addr learned my addr 172.21.15.42:0/348181974 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff93f400700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff94010ec30 0x7ff94019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff93ea00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff940074460 0x7ff9401a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff937e00700 1 -- 172.21.15.42:0/348181974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff94010ec30 msgr2=0x7ff94019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff937e00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff94010ec30 0x7ff94019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff937e00700 1 -- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff940074460 msgr2=0x7ff9401a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.925+0000 7ff937e00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff940074460 0x7ff9401a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.926+0000 7ff937e00700 1 -- 172.21.15.42:0/348181974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94019b190 con 0x7ff940074d80 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.926+0000 7ff93ea00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff940074460 0x7ff9401a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.926+0000 7ff937e00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff940074d80 0x7ff9401a0d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff92800e970 tx=0x7ff92800ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.926+0000 7ff944a00700 1 -- 172.21.15.42:0/348181974 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92800cbd0 con 0x7ff940074d80 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.926+0000 7ff945e00700 1 -- 172.21.15.42:0/348181974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94019b480 con 0x7ff940074d80 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.926+0000 7ff944a00700 1 -- 172.21.15.42:0/348181974 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff928004500 con 0x7ff940074d80 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.926+0000 7ff944a00700 1 -- 172.21.15.42:0/348181974 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff928003e30 con 0x7ff940074d80 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.926+0000 7ff945e00700 1 -- 172.21.15.42:0/348181974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9401a76f0 con 0x7ff940074d80 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.928+0000 7ff945e00700 1 -- 172.21.15.42:0/348181974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff940068f20 con 0x7ff940074d80 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.931+0000 7ff944a00700 1 -- 172.21.15.42:0/348181974 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff928004000 con 0x7ff940074d80 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.932+0000 7ff944a00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff9240a2d30 0x7ff9240a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.933+0000 7ff944a00700 1 -- 172.21.15.42:0/348181974 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff9280cf4e0 con 0x7ff940074d80 2024-04-25T00:17:39.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.933+0000 7ff93ea00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff9240a2d30 0x7ff9240a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.298 INFO:teuthology.orchestra.run.smithi042.stdout:197568495673 2024-04-25T00:17:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.934+0000 7ff93ea00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff9240a2d30 0x7ff9240a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff930006010 tx=0x7ff93001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:38.937+0000 7ff944a00700 1 -- 172.21.15.42:0/348181974 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff92809c2b0 con 0x7ff940074d80 2024-04-25T00:17:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.177+0000 7ff945e00700 1 -- 172.21.15.42:0/348181974 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7ff94002db30 con 0x7ff940074d80 2024-04-25T00:17:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.178+0000 7ff944a00700 1 -- 172.21.15.42:0/348181974 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff92809bbb0 con 0x7ff940074d80 2024-04-25T00:17:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.182+0000 7ff945e00700 1 -- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff9240a2d30 msgr2=0x7ff9240a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.182+0000 7ff945e00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff9240a2d30 0x7ff9240a51f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff930006010 tx=0x7ff93001a040 comp rx=0 tx=0).stop 2024-04-25T00:17:39.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.182+0000 7ff945e00700 1 -- 172.21.15.42:0/348181974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff940074d80 msgr2=0x7ff9401a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.182+0000 7ff945e00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff940074d80 0x7ff9401a0d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff92800e970 tx=0x7ff92800ec80 comp rx=0 tx=0).stop 2024-04-25T00:17:39.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.182+0000 7ff945e00700 1 -- 172.21.15.42:0/348181974 shutdown_connections 2024-04-25T00:17:39.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.182+0000 7ff945e00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff9240a2d30 0x7ff9240a51f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.182+0000 7ff945e00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff94010ec30 0x7ff94019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.182+0000 7ff945e00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff940074460 0x7ff9401a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.182+0000 7ff945e00700 1 --2- 172.21.15.42:0/348181974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff940074d80 0x7ff9401a0d60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.182+0000 7ff945e00700 1 -- 172.21.15.42:0/348181974 >> 172.21.15.42:0/348181974 conn(0x7ff940079390 msgr2=0x7ff9400ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.183+0000 7ff945e00700 1 -- 172.21.15.42:0/348181974 shutdown_connections 2024-04-25T00:17:39.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:39.183+0000 7ff945e00700 1 -- 172.21.15.42:0/348181974 wait complete. 2024-04-25T00:17:39.639 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.383+0000 7f6a61000700 1 -- 172.21.15.42:0/884616269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c108770 msgr2=0x7f6a5c108bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.383+0000 7f6a61000700 1 --2- 172.21.15.42:0/884616269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c108770 0x7f6a5c108bf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6a44009990 tx=0x7f6a44009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.383+0000 7f6a61000700 1 -- 172.21.15.42:0/884616269 shutdown_connections 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.383+0000 7f6a61000700 1 --2- 172.21.15.42:0/884616269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c109130 0x7f6a5c1155d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.383+0000 7f6a61000700 1 --2- 172.21.15.42:0/884616269 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5c10d640 0x7f6a5c10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.383+0000 7f6a61000700 1 --2- 172.21.15.42:0/884616269 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c108770 0x7f6a5c108bf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.383+0000 7f6a61000700 1 -- 172.21.15.42:0/884616269 >> 172.21.15.42:0/884616269 conn(0x7f6a5c077fc0 msgr2=0x7f6a5c107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.383+0000 7f6a61000700 1 -- 172.21.15.42:0/884616269 shutdown_connections 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.383+0000 7f6a61000700 1 -- 172.21.15.42:0/884616269 wait complete. 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.384+0000 7f6a61000700 1 Processor -- start 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.384+0000 7f6a61000700 1 -- start start 2024-04-25T00:17:39.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a61000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c108770 0x7f6a5c19eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a61000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c109130 0x7f6a5c19f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a61000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5c10d640 0x7f6a5c1a37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a61000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c118aa0 con 0x7f6a5c109130 2024-04-25T00:17:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a61000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c118920 con 0x7f6a5c10d640 2024-04-25T00:17:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a61000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c118c20 con 0x7f6a5c108770 2024-04-25T00:17:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a5b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c108770 0x7f6a5c19eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a5b400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c108770 0x7f6a5c19eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:53094/0 (socket says 172.21.15.42:53094) 2024-04-25T00:17:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a5b400700 1 -- 172.21.15.42:0/3104193073 learned_addr learned my addr 172.21.15.42:0/3104193073 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a52a00700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c109130 0x7f6a5c19f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.641 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a5be00700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5c10d640 0x7f6a5c1a37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.642 INFO:teuthology.orchestra.run.smithi042.stdout:219043332146 2024-04-25T00:17:39.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a5b400700 1 -- 172.21.15.42:0/3104193073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5c10d640 msgr2=0x7f6a5c1a37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a5b400700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5c10d640 0x7f6a5c1a37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a5b400700 1 -- 172.21.15.42:0/3104193073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c109130 msgr2=0x7f6a5c19f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a5b400700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c109130 0x7f6a5c19f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.385+0000 7f6a5b400700 1 -- 172.21.15.42:0/3104193073 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c1a3f00 con 0x7f6a5c108770 2024-04-25T00:17:39.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.386+0000 7f6a5b400700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c108770 0x7f6a5c19eef0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00e9a0 tx=0x7f6a4c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.386+0000 7f6a58c00700 1 -- 172.21.15.42:0/3104193073 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c00ca10 con 0x7f6a5c108770 2024-04-25T00:17:39.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.386+0000 7f6a61000700 1 -- 172.21.15.42:0/3104193073 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5c1a41f0 con 0x7f6a5c108770 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.386+0000 7f6a58c00700 1 -- 172.21.15.42:0/3104193073 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a4c00cbb0 con 0x7f6a5c108770 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.386+0000 7f6a58c00700 1 -- 172.21.15.42:0/3104193073 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4c018680 con 0x7f6a5c108770 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.386+0000 7f6a61000700 1 -- 172.21.15.42:0/3104193073 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5c1aba70 con 0x7f6a5c108770 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.387+0000 7f6a61000700 1 -- 172.21.15.42:0/3104193073 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5c068f20 con 0x7f6a5c108770 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.389+0000 7f6a58c00700 1 -- 172.21.15.42:0/3104193073 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6a4c010430 con 0x7f6a5c108770 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.391+0000 7f6a58c00700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6a400a2cb0 0x7f6a400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.391+0000 7f6a58c00700 1 -- 172.21.15.42:0/3104193073 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f6a4c0cf810 con 0x7f6a5c108770 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.391+0000 7f6a52a00700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6a400a2cb0 0x7f6a400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.392+0000 7f6a52a00700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6a400a2cb0 0x7f6a400a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c108240 tx=0x7f6a4400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.393+0000 7f6a58c00700 1 -- 172.21.15.42:0/3104193073 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6a4c09c5e0 con 0x7f6a5c108770 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.632+0000 7f6a61000700 1 -- 172.21.15.42:0/3104193073 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f6a5c02db30 con 0x7f6a5c108770 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.633+0000 7f6a58c00700 1 -- 172.21.15.42:0/3104193073 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6a4c09bee0 con 0x7f6a5c108770 2024-04-25T00:17:39.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.635+0000 7f6a61000700 1 -- 172.21.15.42:0/3104193073 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6a400a2cb0 msgr2=0x7f6a400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.635+0000 7f6a61000700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6a400a2cb0 0x7f6a400a5170 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c108240 tx=0x7f6a4400b260 comp rx=0 tx=0).stop 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.635+0000 7f6a61000700 1 -- 172.21.15.42:0/3104193073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c108770 msgr2=0x7f6a5c19eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.635+0000 7f6a61000700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c108770 0x7f6a5c19eef0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c00e9a0 tx=0x7f6a4c00ecb0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.636+0000 7f6a61000700 1 -- 172.21.15.42:0/3104193073 shutdown_connections 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.636+0000 7f6a61000700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6a400a2cb0 0x7f6a400a5170 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.636+0000 7f6a61000700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6a5c10d640 0x7f6a5c1a37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.636+0000 7f6a61000700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a5c108770 0x7f6a5c19eef0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.636+0000 7f6a61000700 1 --2- 172.21.15.42:0/3104193073 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6a5c109130 0x7f6a5c19f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.636+0000 7f6a61000700 1 -- 172.21.15.42:0/3104193073 >> 172.21.15.42:0/3104193073 conn(0x7f6a5c077fc0 msgr2=0x7f6a5c078580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.636+0000 7f6a61000700 1 -- 172.21.15.42:0/3104193073 shutdown_connections 2024-04-25T00:17:39.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:37.636+0000 7f6a61000700 1 -- 172.21.15.42:0/3104193073 wait complete. 2024-04-25T00:17:39.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:39 smithi042 ceph-mon[28130]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:39.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:39 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/171535226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:17:39.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:39 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/348181974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:17:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:39 smithi136 ceph-mon[32923]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:40.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:39 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/171535226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:17:40.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:39 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/348181974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:17:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:39 smithi106 ceph-mon[32510]: pgmap v354: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:39 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/171535226' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T00:17:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:39 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/348181974' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T00:17:40.680 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182466 got 107374182472 for osd.3 2024-04-25T00:17:40.680 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:41.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.174+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3538181105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a84106570 msgr2=0x7f3a84106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:41.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.174+0000 7f3a8ac00700 1 --2- 172.21.15.42:0/3538181105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a84106570 0x7f3a84106a20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00b0a0 tx=0x7f3a7c00b3b0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.174+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3538181105 shutdown_connections 2024-04-25T00:17:41.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.174+0000 7f3a8ac00700 1 --2- 172.21.15.42:0/3538181105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a84106570 0x7f3a84106a20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.174+0000 7f3a8ac00700 1 --2- 172.21.15.42:0/3538181105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a84101160 0x7f3a84101540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.174+0000 7f3a8ac00700 1 --2- 172.21.15.42:0/3538181105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a84101a80 0x7f3a84105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.175+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3538181105 >> 172.21.15.42:0/3538181105 conn(0x7f3a840fcbc0 msgr2=0x7f3a840fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:41.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.175+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3538181105 shutdown_connections 2024-04-25T00:17:41.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.175+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3538181105 wait complete. 2024-04-25T00:17:41.179 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.176+0000 7f3a8ac00700 1 Processor -- start 2024-04-25T00:17:41.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.176+0000 7f3a8ac00700 1 -- start start 2024-04-25T00:17:41.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a8ac00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a84101160 0x7f3a8419ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:41.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a8ac00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a84101a80 0x7f3a8419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:41.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a8ac00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a84106570 0x7f3a841a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:41.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a8ac00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3a84118af0 con 0x7f3a84101160 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a8ac00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3a84118970 con 0x7f3a84106570 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a8ac00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3a84118c70 con 0x7f3a84101a80 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a89800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a84101160 0x7f3a8419ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a89800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a84101160 0x7f3a8419ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59346/0 (socket says 172.21.15.42:59346) 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a89800700 1 -- 172.21.15.42:0/3877243933 learned_addr learned my addr 172.21.15.42:0/3877243933 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a88e00700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a84101a80 0x7f3a8419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.177+0000 7f3a8a200700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a84106570 0x7f3a841a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.178+0000 7f3a89800700 1 -- 172.21.15.42:0/3877243933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a84101a80 msgr2=0x7f3a8419f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.178+0000 7f3a89800700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a84101a80 0x7f3a8419f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.178+0000 7f3a89800700 1 -- 172.21.15.42:0/3877243933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a84106570 msgr2=0x7f3a841a37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.178+0000 7f3a89800700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a84106570 0x7f3a841a37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.178+0000 7f3a89800700 1 -- 172.21.15.42:0/3877243933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a841a3f30 con 0x7f3a84101160 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.178+0000 7f3a89800700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a84101160 0x7f3a8419ef20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3a7801e990 tx=0x7f3a7801eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.179+0000 7f3a82000700 1 -- 172.21.15.42:0/3877243933 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78027070 con 0x7f3a84101160 2024-04-25T00:17:41.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.179+0000 7f3a82000700 1 -- 172.21.15.42:0/3877243933 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a78004d10 con 0x7f3a84101160 2024-04-25T00:17:41.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.179+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3877243933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a841a4220 con 0x7f3a84101160 2024-04-25T00:17:41.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.179+0000 7f3a82000700 1 -- 172.21.15.42:0/3877243933 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78005630 con 0x7f3a84101160 2024-04-25T00:17:41.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.179+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3877243933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a84107520 con 0x7f3a84101160 2024-04-25T00:17:41.184 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.183+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3877243933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a84068f20 con 0x7f3a84101160 2024-04-25T00:17:41.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.184+0000 7f3a82000700 1 -- 172.21.15.42:0/3877243933 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3a78004750 con 0x7f3a84101160 2024-04-25T00:17:41.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.187+0000 7f3a82000700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a6c0a2cb0 0x7f3a6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:41.188 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.187+0000 7f3a82000700 1 -- 172.21.15.42:0/3877243933 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3a780dfd80 con 0x7f3a84101160 2024-04-25T00:17:41.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.188+0000 7f3a88e00700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a6c0a2cb0 0x7f3a6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:41.189 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.190+0000 7f3a88e00700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a6c0a2cb0 0x7f3a6c0a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3a84106cb0 tx=0x7f3a70009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:41.193 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.193+0000 7f3a82000700 1 -- 172.21.15.42:0/3877243933 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3a78030030 con 0x7f3a84101160 2024-04-25T00:17:41.198 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.041+0000 7f3de2600700 1 -- 172.21.15.42:0/1877370834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ddc069f40 msgr2=0x7f3ddc06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.041+0000 7f3de2600700 1 --2- 172.21.15.42:0/1877370834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ddc069f40 0x7f3ddc06a320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009990 tx=0x7f3dcc009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.041+0000 7f3de2600700 1 -- 172.21.15.42:0/1877370834 shutdown_connections 2024-04-25T00:17:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.041+0000 7f3de2600700 1 --2- 172.21.15.42:0/1877370834 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ddc10ece0 0x7f3ddc1110d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.041+0000 7f3de2600700 1 --2- 172.21.15.42:0/1877370834 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ddc069f40 0x7f3ddc06a320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.041+0000 7f3de2600700 1 --2- 172.21.15.42:0/1877370834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ddc06a860 0x7f3ddc10e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.041+0000 7f3de2600700 1 -- 172.21.15.42:0/1877370834 >> 172.21.15.42:0/1877370834 conn(0x7f3ddc0780e0 msgr2=0x7f3ddc0784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.041+0000 7f3de2600700 1 -- 172.21.15.42:0/1877370834 shutdown_connections 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.041+0000 7f3de2600700 1 -- 172.21.15.42:0/1877370834 wait complete. 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.042+0000 7f3de2600700 1 Processor -- start 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.042+0000 7f3de2600700 1 -- start start 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3de2600700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ddc069f40 0x7f3ddc1a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3de2600700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ddc06a860 0x7f3ddc1a0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3de2600700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ddc10ece0 0x7f3ddc19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3de2600700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc1147b0 con 0x7f3ddc069f40 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3de2600700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc114630 con 0x7f3ddc06a860 2024-04-25T00:17:41.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3de2600700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc114930 con 0x7f3ddc10ece0 2024-04-25T00:17:41.409 INFO:teuthology.orchestra.run.smithi042.stdout:292057776168 2024-04-25T00:17:41.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3ddb400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ddc069f40 0x7f3ddc1a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:41.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3ddb400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ddc069f40 0x7f3ddc1a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59328/0 (socket says 172.21.15.42:59328) 2024-04-25T00:17:41.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3ddb400700 1 -- 172.21.15.42:0/2846819716 learned_addr learned my addr 172.21.15.42:0/2846819716 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:41.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3ddaa00700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ddc06a860 0x7f3ddc1a0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:41.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.043+0000 7f3ddbe00700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ddc10ece0 0x7f3ddc19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:41.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.044+0000 7f3ddb400700 1 -- 172.21.15.42:0/2846819716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ddc10ece0 msgr2=0x7f3ddc19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:41.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.044+0000 7f3ddb400700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ddc10ece0 0x7f3ddc19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.044+0000 7f3ddb400700 1 -- 172.21.15.42:0/2846819716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ddc06a860 msgr2=0x7f3ddc1a0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:41.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.044+0000 7f3ddb400700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ddc06a860 0x7f3ddc1a0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.044+0000 7f3ddb400700 1 -- 172.21.15.42:0/2846819716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc19b250 con 0x7f3ddc069f40 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.044+0000 7f3ddb400700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ddc069f40 0x7f3ddc1a0930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc000c00 tx=0x7f3dcc00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.044+0000 7f3dd3e00700 1 -- 172.21.15.42:0/2846819716 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc027070 con 0x7f3ddc069f40 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.044+0000 7f3dd3e00700 1 -- 172.21.15.42:0/2846819716 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dcc0056d0 con 0x7f3ddc069f40 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.044+0000 7f3de2600700 1 -- 172.21.15.42:0/2846819716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc19b4e0 con 0x7f3ddc069f40 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.044+0000 7f3dd3e00700 1 -- 172.21.15.42:0/2846819716 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc022e10 con 0x7f3ddc069f40 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.045+0000 7f3de2600700 1 -- 172.21.15.42:0/2846819716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc1a7800 con 0x7f3ddc069f40 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.046+0000 7f3de2600700 1 -- 172.21.15.42:0/2846819716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ddc101750 con 0x7f3ddc069f40 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.048+0000 7f3dd3e00700 1 -- 172.21.15.42:0/2846819716 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3dcc02a070 con 0x7f3ddc069f40 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.049+0000 7f3dd3e00700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3dc00a2cb0 0x7f3dc00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.050+0000 7f3dd3e00700 1 -- 172.21.15.42:0/2846819716 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3dcc0cfe10 con 0x7f3ddc069f40 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.050+0000 7f3ddaa00700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3dc00a2cb0 0x7f3dc00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:41.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.051+0000 7f3ddaa00700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3dc00a2cb0 0x7f3dc00a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc1025b0 tx=0x7f3dc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.054+0000 7f3dd3e00700 1 -- 172.21.15.42:0/2846819716 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3dcc0d3030 con 0x7f3ddc069f40 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.343+0000 7f3de2600700 1 -- 172.21.15.42:0/2846819716 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f3ddc02db30 con 0x7f3ddc069f40 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.344+0000 7f3dd3e00700 1 -- 172.21.15.42:0/2846819716 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3dcc013070 con 0x7f3ddc069f40 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.347+0000 7f3de2600700 1 -- 172.21.15.42:0/2846819716 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3dc00a2cb0 msgr2=0x7f3dc00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.347+0000 7f3de2600700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3dc00a2cb0 0x7f3dc00a5170 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc1025b0 tx=0x7f3dc8009260 comp rx=0 tx=0).stop 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.347+0000 7f3de2600700 1 -- 172.21.15.42:0/2846819716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ddc069f40 msgr2=0x7f3ddc1a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.347+0000 7f3de2600700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ddc069f40 0x7f3ddc1a0930 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc000c00 tx=0x7f3dcc00bab0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.348+0000 7f3de2600700 1 -- 172.21.15.42:0/2846819716 shutdown_connections 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.348+0000 7f3de2600700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3dc00a2cb0 0x7f3dc00a5170 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.348+0000 7f3de2600700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3ddc10ece0 0x7f3ddc19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.348+0000 7f3de2600700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3ddc069f40 0x7f3ddc1a0930 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.348+0000 7f3de2600700 1 --2- 172.21.15.42:0/2846819716 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ddc06a860 0x7f3ddc1a0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.348+0000 7f3de2600700 1 -- 172.21.15.42:0/2846819716 >> 172.21.15.42:0/2846819716 conn(0x7f3ddc0780e0 msgr2=0x7f3ddc110190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:41.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.348+0000 7f3de2600700 1 -- 172.21.15.42:0/2846819716 shutdown_connections 2024-04-25T00:17:41.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.348+0000 7f3de2600700 1 -- 172.21.15.42:0/2846819716 wait complete. 2024-04-25T00:17:41.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.460+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3877243933 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f3a841089c0 con 0x7f3a84101160 2024-04-25T00:17:41.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.461+0000 7f3a82000700 1 -- 172.21.15.42:0/3877243933 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3a780acbb0 con 0x7f3a84101160 2024-04-25T00:17:41.462 INFO:teuthology.orchestra.run.smithi042.stdout:347892351005 2024-04-25T00:17:41.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3877243933 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a6c0a2cb0 msgr2=0x7f3a6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:41.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a6c0a2cb0 0x7f3a6c0a5170 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3a84106cb0 tx=0x7f3a70009260 comp rx=0 tx=0).stop 2024-04-25T00:17:41.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3877243933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a84101160 msgr2=0x7f3a8419ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:41.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a84101160 0x7f3a8419ef20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3a7801e990 tx=0x7f3a7801eca0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3877243933 shutdown_connections 2024-04-25T00:17:41.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3a6c0a2cb0 0x7f3a6c0a5170 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a84106570 0x7f3a841a37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3a84101160 0x7f3a8419ef20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 --2- 172.21.15.42:0/3877243933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3a84101a80 0x7f3a8419f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:41.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3877243933 >> 172.21.15.42:0/3877243933 conn(0x7f3a840fcbc0 msgr2=0x7f3a84103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:41.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3877243933 shutdown_connections 2024-04-25T00:17:41.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:41.464+0000 7f3a8ac00700 1 -- 172.21.15.42:0/3877243933 wait complete. 2024-04-25T00:17:41.663 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:41 smithi042 ceph-mon[28130]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:41.663 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:41 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2846819716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:17:41.663 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:41 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3877243933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:17:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:41 smithi136 ceph-mon[32923]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:41 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2846819716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:17:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:41 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3877243933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:17:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:41 smithi106 ceph-mon[32510]: pgmap v355: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:41 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2846819716' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T00:17:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:41 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3877243933' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T00:17:42.240 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332140 got 219043332146 for osd.7 2024-04-25T00:17:42.240 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:43.672 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495666 got 197568495673 for osd.6 2024-04-25T00:17:43.672 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:43.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:43 smithi042 ceph-mon[28130]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:43 smithi136 ceph-mon[32923]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:43 smithi106 ceph-mon[32510]: pgmap v356: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-04-25T00:17:44.271 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920830 got 141733920835 for osd.4 2024-04-25T00:17:44.271 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:44.871 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574926 got 55834574931 for osd.1 2024-04-25T00:17:44.871 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:45.257 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350999 got 347892351005 for osd.11 2024-04-25T00:17:45.257 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:45.920 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:45 smithi042 ceph-mon[28130]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:45 smithi136 ceph-mon[32923]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:45 smithi106 ceph-mon[32510]: pgmap v357: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:46.101 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776161 got 292057776168 for osd.9 2024-04-25T00:17:46.101 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:47.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.471+0000 7ffaf1e00700 1 -- 172.21.15.42:0/3309054009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec107570 msgr2=0x7ffaec10f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:47.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.471+0000 7ffaf1e00700 1 --2- 172.21.15.42:0/3309054009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec107570 0x7ffaec10f140 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009f90 tx=0x7ffae400b330 comp rx=0 tx=0).stop 2024-04-25T00:17:47.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.471+0000 7ffaf1e00700 1 -- 172.21.15.42:0/3309054009 shutdown_connections 2024-04-25T00:17:47.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.471+0000 7ffaf1e00700 1 --2- 172.21.15.42:0/3309054009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec107570 0x7ffaec10f140 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:47.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.471+0000 7ffaf1e00700 1 --2- 172.21.15.42:0/3309054009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaec1024e0 0x7ffaec1028c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:47.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.471+0000 7ffaf1e00700 1 --2- 172.21.15.42:0/3309054009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaec102e90 0x7ffaec106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:47.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.471+0000 7ffaf1e00700 1 -- 172.21.15.42:0/3309054009 >> 172.21.15.42:0/3309054009 conn(0x7ffaec079390 msgr2=0x7ffaec0797a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:47.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.471+0000 7ffaf1e00700 1 -- 172.21.15.42:0/3309054009 shutdown_connections 2024-04-25T00:17:47.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.471+0000 7ffaf1e00700 1 -- 172.21.15.42:0/3309054009 wait complete. 2024-04-25T00:17:47.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.473+0000 7ffaf1e00700 1 Processor -- start 2024-04-25T00:17:47.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.473+0000 7ffaf1e00700 1 -- start start 2024-04-25T00:17:47.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.473+0000 7ffaf1e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaec1024e0 0x7ffaec074530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:47.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.473+0000 7ffaf1e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec102e90 0x7ffaec074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:47.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaf1e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaec107570 0x7ffaec074fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:47.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaf1e00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1125a0 con 0x7ffaec102e90 2024-04-25T00:17:47.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaf1e00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec112420 con 0x7ffaec1024e0 2024-04-25T00:17:47.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaf1e00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec112720 con 0x7ffaec107570 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaeaa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaec1024e0 0x7ffaec074530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaeaa00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaec1024e0 0x7ffaec074530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40410/0 (socket says 172.21.15.42:40410) 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaeaa00700 1 -- 172.21.15.42:0/2405183267 learned_addr learned my addr 172.21.15.42:0/2405183267 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaea000700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec102e90 0x7ffaec074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaeb400700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaec107570 0x7ffaec074fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaeb400700 1 -- 172.21.15.42:0/2405183267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaec1024e0 msgr2=0x7ffaec074530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaeb400700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaec1024e0 0x7ffaec074530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaeb400700 1 -- 172.21.15.42:0/2405183267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec102e90 msgr2=0x7ffaec074a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaeb400700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec102e90 0x7ffaec074a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.474+0000 7ffaeb400700 1 -- 172.21.15.42:0/2405183267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec1a6f40 con 0x7ffaec107570 2024-04-25T00:17:47.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.475+0000 7ffaeb400700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaec107570 0x7ffaec074fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009f90 tx=0x7ffae400cfa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:47.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.475+0000 7ffaf0a00700 1 -- 172.21.15.42:0/2405183267 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4021070 con 0x7ffaec107570 2024-04-25T00:17:47.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.475+0000 7ffaf1e00700 1 -- 172.21.15.42:0/2405183267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec1a71d0 con 0x7ffaec107570 2024-04-25T00:17:47.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.475+0000 7ffaf0a00700 1 -- 172.21.15.42:0/2405183267 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffae4007ac0 con 0x7ffaec107570 2024-04-25T00:17:47.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.475+0000 7ffaf0a00700 1 -- 172.21.15.42:0/2405183267 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4005040 con 0x7ffaec107570 2024-04-25T00:17:47.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.475+0000 7ffaf1e00700 1 -- 172.21.15.42:0/2405183267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec1a7630 con 0x7ffaec107570 2024-04-25T00:17:47.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.477+0000 7ffaf1e00700 1 -- 172.21.15.42:0/2405183267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaec068f20 con 0x7ffaec107570 2024-04-25T00:17:47.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.480+0000 7ffaf0a00700 1 -- 172.21.15.42:0/2405183267 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffae4024070 con 0x7ffaec107570 2024-04-25T00:17:47.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.482+0000 7ffaf0a00700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffad00a2cb0 0x7ffad00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:47.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.482+0000 7ffaf0a00700 1 -- 172.21.15.42:0/2405183267 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ffae40d0120 con 0x7ffaec107570 2024-04-25T00:17:47.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.482+0000 7ffaeaa00700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffad00a2cb0 0x7ffad00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.483+0000 7ffaeaa00700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffad00a2cb0 0x7ffad00a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009d20 tx=0x7ffadc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:47.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.483+0000 7ffaf0a00700 1 -- 172.21.15.42:0/2405183267 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffae409cf80 con 0x7ffaec107570 2024-04-25T00:17:47.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.719+0000 7ffaf1e00700 1 -- 172.21.15.42:0/2405183267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7ffaec02dcf0 con 0x7ffaec107570 2024-04-25T00:17:47.720 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:47 smithi042 ceph-mon[28130]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:17:47.720 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.720+0000 7ffaf0a00700 1 -- 172.21.15.42:0/2405183267 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ffae409c880 con 0x7ffaec107570 2024-04-25T00:17:47.721 INFO:teuthology.orchestra.run.smithi042.stdout:313532612643 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 -- 172.21.15.42:0/2405183267 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffad00a2cb0 msgr2=0x7ffad00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffad00a2cb0 0x7ffad00a5170 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffadc009d20 tx=0x7ffadc009260 comp rx=0 tx=0).stop 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 -- 172.21.15.42:0/2405183267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaec107570 msgr2=0x7ffaec074fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaec107570 0x7ffaec074fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009f90 tx=0x7ffae400cfa0 comp rx=0 tx=0).stop 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 -- 172.21.15.42:0/2405183267 shutdown_connections 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffad00a2cb0 0x7ffad00a5170 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffaec107570 0x7ffaec074fb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaec1024e0 0x7ffaec074530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 --2- 172.21.15.42:0/2405183267 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffaec102e90 0x7ffaec074a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 -- 172.21.15.42:0/2405183267 >> 172.21.15.42:0/2405183267 conn(0x7ffaec079390 msgr2=0x7ffaec100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 -- 172.21.15.42:0/2405183267 shutdown_connections 2024-04-25T00:17:47.725 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:47.724+0000 7ffaf1e00700 1 -- 172.21.15.42:0/2405183267 wait complete. 2024-04-25T00:17:47.890 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:47 smithi106 ceph-mon[32510]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:17:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:47 smithi136 ceph-mon[32923]: pgmap v358: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:17:48.557 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612637 got 313532612643 for osd.10 2024-04-25T00:17:48.558 DEBUG:teuthology.parallel:result is None 2024-04-25T00:17:48.558 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T00:17:48.558 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T00:17:48.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:48 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2405183267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:17:49.006 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:49.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:48 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2405183267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:17:49.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:48 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2405183267' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T00:17:49.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:49 smithi042 ceph-mon[28130]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:17:49.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:49 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:49.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:49 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:49 smithi136 ceph-mon[32923]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:17:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:49 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:49 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:50.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:49 smithi106 ceph-mon[32510]: pgmap v359: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:17:50.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:49 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:50.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:49 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:50.682 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:50.682 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:50.682 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:51.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:51.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:51.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:51.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:17:51.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:17:51.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:17:51.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.222+0000 7ffb8fe00700 1 -- 172.21.15.42:0/2190754085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9006aa00 msgr2=0x7ffb9006ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:51.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.222+0000 7ffb8fe00700 1 --2- 172.21.15.42:0/2190754085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9006aa00 0x7ffb9006ade0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ffb80009990 tx=0x7ffb80009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.223 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.223+0000 7ffb8fe00700 1 -- 172.21.15.42:0/2190754085 shutdown_connections 2024-04-25T00:17:51.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.223+0000 7ffb8fe00700 1 --2- 172.21.15.42:0/2190754085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb90107560 0x7ffb9010f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.223+0000 7ffb8fe00700 1 --2- 172.21.15.42:0/2190754085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9006aa00 0x7ffb9006ade0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.223+0000 7ffb8fe00700 1 --2- 172.21.15.42:0/2190754085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9006b3b0 0x7ffb90106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.223+0000 7ffb8fe00700 1 -- 172.21.15.42:0/2190754085 >> 172.21.15.42:0/2190754085 conn(0x7ffb90077f70 msgr2=0x7ffb90078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:51.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.223+0000 7ffb8fe00700 1 -- 172.21.15.42:0/2190754085 shutdown_connections 2024-04-25T00:17:51.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.223+0000 7ffb8fe00700 1 -- 172.21.15.42:0/2190754085 wait complete. 2024-04-25T00:17:51.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.225+0000 7ffb8fe00700 1 Processor -- start 2024-04-25T00:17:51.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.225+0000 7ffb8fe00700 1 -- start start 2024-04-25T00:17:51.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb8fe00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9006aa00 0x7ffb9019e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:51.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb8fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9006b3b0 0x7ffb9019eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb8fe00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb90107560 0x7ffb90198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb8fe00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffb901124e0 con 0x7ffb90107560 2024-04-25T00:17:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb8fe00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90112360 con 0x7ffb9006aa00 2024-04-25T00:17:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb8fe00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffb90112660 con 0x7ffb9006b3b0 2024-04-25T00:17:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb8f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb90107560 0x7ffb90198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb8ea00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9006aa00 0x7ffb9019e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:51.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb8f400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb90107560 0x7ffb90198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52778/0 (socket says 172.21.15.42:52778) 2024-04-25T00:17:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb8f400700 1 -- 172.21.15.42:0/550041918 learned_addr learned my addr 172.21.15.42:0/550041918 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.226+0000 7ffb87e00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9006b3b0 0x7ffb9019eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb8f400700 1 -- 172.21.15.42:0/550041918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9006b3b0 msgr2=0x7ffb9019eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb8f400700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9006b3b0 0x7ffb9019eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb8f400700 1 -- 172.21.15.42:0/550041918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9006aa00 msgr2=0x7ffb9019e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb8f400700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9006aa00 0x7ffb9019e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb8f400700 1 -- 172.21.15.42:0/550041918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb90198fe0 con 0x7ffb90107560 2024-04-25T00:17:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb8ea00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9006aa00 0x7ffb9019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb87e00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9006b3b0 0x7ffb9019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:51.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb8f400700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb90107560 0x7ffb90198700 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800efc0 tx=0x7ffb8800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:51.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb94a00700 1 -- 172.21.15.42:0/550041918 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb880092e0 con 0x7ffb90107560 2024-04-25T00:17:51.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb94a00700 1 -- 172.21.15.42:0/550041918 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb88004bb0 con 0x7ffb90107560 2024-04-25T00:17:51.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb8fe00700 1 -- 172.21.15.42:0/550041918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb90199270 con 0x7ffb90107560 2024-04-25T00:17:51.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb94a00700 1 -- 172.21.15.42:0/550041918 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88004020 con 0x7ffb90107560 2024-04-25T00:17:51.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.227+0000 7ffb8fe00700 1 -- 172.21.15.42:0/550041918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb901a55b0 con 0x7ffb90107560 2024-04-25T00:17:51.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.229+0000 7ffb8fe00700 1 -- 172.21.15.42:0/550041918 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb901995e0 con 0x7ffb90107560 2024-04-25T00:17:51.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.231+0000 7ffb94a00700 1 -- 172.21.15.42:0/550041918 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ffb8801b030 con 0x7ffb90107560 2024-04-25T00:17:51.232 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.232+0000 7ffb94a00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb740a2cb0 0x7ffb740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:51.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.232+0000 7ffb94a00700 1 -- 172.21.15.42:0/550041918 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ffb880ceb10 con 0x7ffb90107560 2024-04-25T00:17:51.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.232+0000 7ffb8ea00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb740a2cb0 0x7ffb740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:51.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.233+0000 7ffb8ea00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb740a2cb0 0x7ffb740a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000b2e0 tx=0x7ffb80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:51.237 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.237+0000 7ffb94a00700 1 -- 172.21.15.42:0/550041918 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ffb901995e0 con 0x7ffb90107560 2024-04-25T00:17:51.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.463+0000 7ffb8fe00700 1 -- 172.21.15.42:0/550041918 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffb90199c30 con 0x7ffb740a2cb0 2024-04-25T00:17:51.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.473+0000 7ffb94a00700 1 -- 172.21.15.42:0/550041918 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258691 (secure 0 0 0) 0x7ffb800672a0 con 0x7ffb740a2cb0 2024-04-25T00:17:51.480 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:17:51.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 -- 172.21.15.42:0/550041918 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb740a2cb0 msgr2=0x7ffb740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb740a2cb0 0x7ffb740a5170 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000b2e0 tx=0x7ffb80005fd0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 -- 172.21.15.42:0/550041918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb90107560 msgr2=0x7ffb90198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb90107560 0x7ffb90198700 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffb8800efc0 tx=0x7ffb8800c3a0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 -- 172.21.15.42:0/550041918 shutdown_connections 2024-04-25T00:17:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffb740a2cb0 0x7ffb740a5170 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffb90107560 0x7ffb90198700 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9006aa00 0x7ffb9019e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 --2- 172.21.15.42:0/550041918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffb9006b3b0 0x7ffb9019eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 -- 172.21.15.42:0/550041918 >> 172.21.15.42:0/550041918 conn(0x7ffb90077f70 msgr2=0x7ffb90100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:51.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.480+0000 7ffb8fe00700 1 -- 172.21.15.42:0/550041918 shutdown_connections 2024-04-25T00:17:51.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:51.481+0000 7ffb8fe00700 1 -- 172.21.15.42:0/550041918 wait complete. 2024-04-25T00:17:51.484 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-25T00:17:51.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:51 smithi042 ceph-mon[28130]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:51 smithi136 ceph-mon[32923]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:51 smithi106 ceph-mon[32510]: pgmap v360: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:17:52.429 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-04-25T00:17:49.946456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656108,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":642,"num_read_kb":434,"num_write":340,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358372,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583452,"statfs":{"total":1151940427776,"available":1151573454848,"internally_reserved":0,"allocated":13627392,"data_stored":7504140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005327"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562925+0000","last_change":"2024-04-25T00:16:20.250230+0000","last_active":"2024-04-25T00:17:27.562925+0000","last_peered":"2024-04-25T00:17:27.562925+0000","last_clean":"2024-04-25T00:17:27.562925+0000","last_became_active":"2024-04-25T00:16:20.249751+0000","last_became_peered":"2024-04-25T00:16:20.249751+0000","last_unstale":"2024-04-25T00:17:27.562925+0000","last_undegraded":"2024-04-25T00:17:27.562925+0000","last_fullsized":"2024-04-25T00:17:27.562925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:57.895905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.578052+0000","last_change":"2024-04-25T00:16:20.237934+0000","last_active":"2024-04-25T00:17:27.578052+0000","last_peered":"2024-04-25T00:17:27.578052+0000","last_clean":"2024-04-25T00:17:27.578052+0000","last_became_active":"2024-04-25T00:16:20.236039+0000","last_became_peered":"2024-04-25T00:16:20.236039+0000","last_unstale":"2024-04-25T00:17:27.578052+0000","last_undegraded":"2024-04-25T00:17:27.578052+0000","last_fullsized":"2024-04-25T00:17:27.578052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:07.730977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576639+0000","last_change":"2024-04-25T00:16:20.248923+0000","last_active":"2024-04-25T00:17:27.576639+0000","last_peered":"2024-04-25T00:17:27.576639+0000","last_clean":"2024-04-25T00:17:27.576639+0000","last_became_active":"2024-04-25T00:16:20.248078+0000","last_became_peered":"2024-04-25T00:16:20.248078+0000","last_unstale":"2024-04-25T00:17:27.576639+0000","last_undegraded":"2024-04-25T00:17:27.576639+0000","last_fullsized":"2024-04-25T00:17:27.576639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:25.675449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.578926+0000","last_change":"2024-04-25T00:17:27.577956+0000","last_active":"2024-04-25T00:17:27.578926+0000","last_peered":"2024-04-25T00:17:27.578926+0000","last_clean":"2024-04-25T00:17:27.578926+0000","last_became_active":"2024-04-25T00:17:27.576377+0000","last_became_peered":"2024-04-25T00:17:27.576377+0000","last_unstale":"2024-04-25T00:17:27.578926+0000","last_undegraded":"2024-04-25T00:17:27.578926+0000","last_fullsized":"2024-04-25T00:17:27.578926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:50.097326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565029+0000","last_change":"2024-04-25T00:16:20.238311+0000","last_active":"2024-04-25T00:17:27.565029+0000","last_peered":"2024-04-25T00:17:27.565029+0000","last_clean":"2024-04-25T00:17:27.565029+0000","last_became_active":"2024-04-25T00:16:20.238104+0000","last_became_peered":"2024-04-25T00:16:20.238104+0000","last_unstale":"2024-04-25T00:17:27.565029+0000","last_undegraded":"2024-04-25T00:17:27.565029+0000","last_fullsized":"2024-04-25T00:17:27.565029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:42.459941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563122+0000","last_change":"2024-04-25T00:16:20.228307+0000","last_active":"2024-04-25T00:17:27.563122+0000","last_peered":"2024-04-25T00:17:27.563122+0000","last_clean":"2024-04-25T00:17:27.563122+0000","last_became_active":"2024-04-25T00:16:20.227999+0000","last_became_peered":"2024-04-25T00:16:20.227999+0000","last_unstale":"2024-04-25T00:17:27.563122+0000","last_undegraded":"2024-04-25T00:17:27.563122+0000","last_fullsized":"2024-04-25T00:17:27.563122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:57.910589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663179+0000","last_change":"2024-04-25T00:16:20.224382+0000","last_active":"2024-04-25T00:17:27.663179+0000","last_peered":"2024-04-25T00:17:27.663179+0000","last_clean":"2024-04-25T00:17:27.663179+0000","last_became_active":"2024-04-25T00:16:20.224122+0000","last_became_peered":"2024-04-25T00:16:20.224122+0000","last_unstale":"2024-04-25T00:17:27.663179+0000","last_undegraded":"2024-04-25T00:17:27.663179+0000","last_fullsized":"2024-04-25T00:17:27.663179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:19.651330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.568028+0000","last_change":"2024-04-25T00:16:20.227720+0000","last_active":"2024-04-25T00:17:27.568028+0000","last_peered":"2024-04-25T00:17:27.568028+0000","last_clean":"2024-04-25T00:17:27.568028+0000","last_became_active":"2024-04-25T00:16:20.227263+0000","last_became_peered":"2024-04-25T00:16:20.227263+0000","last_unstale":"2024-04-25T00:17:27.568028+0000","last_undegraded":"2024-04-25T00:17:27.568028+0000","last_fullsized":"2024-04-25T00:17:27.568028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:03.709172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.567448+0000","last_change":"2024-04-25T00:16:20.217673+0000","last_active":"2024-04-25T00:17:27.567448+0000","last_peered":"2024-04-25T00:17:27.567448+0000","last_clean":"2024-04-25T00:17:27.567448+0000","last_became_active":"2024-04-25T00:16:20.216773+0000","last_became_peered":"2024-04-25T00:16:20.216773+0000","last_unstale":"2024-04-25T00:17:27.567448+0000","last_undegraded":"2024-04-25T00:17:27.567448+0000","last_fullsized":"2024-04-25T00:17:27.567448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.623501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563471+0000","last_change":"2024-04-25T00:16:20.218460+0000","last_active":"2024-04-25T00:17:27.563471+0000","last_peered":"2024-04-25T00:17:27.563471+0000","last_clean":"2024-04-25T00:17:27.563471+0000","last_became_active":"2024-04-25T00:16:20.217198+0000","last_became_peered":"2024-04-25T00:16:20.217198+0000","last_unstale":"2024-04-25T00:17:27.563471+0000","last_undegraded":"2024-04-25T00:17:27.563471+0000","last_fullsized":"2024-04-25T00:17:27.563471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:06.511749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663982+0000","last_change":"2024-04-25T00:16:20.221990+0000","last_active":"2024-04-25T00:17:27.663982+0000","last_peered":"2024-04-25T00:17:27.663982+0000","last_clean":"2024-04-25T00:17:27.663982+0000","last_became_active":"2024-04-25T00:16:20.221635+0000","last_became_peered":"2024-04-25T00:16:20.221635+0000","last_unstale":"2024-04-25T00:17:27.663982+0000","last_undegraded":"2024-04-25T00:17:27.663982+0000","last_fullsized":"2024-04-25T00:17:27.663982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:31.205135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664220+0000","last_change":"2024-04-25T00:16:20.219923+0000","last_active":"2024-04-25T00:17:27.664220+0000","last_peered":"2024-04-25T00:17:27.664220+0000","last_clean":"2024-04-25T00:17:27.664220+0000","last_became_active":"2024-04-25T00:16:20.219578+0000","last_became_peered":"2024-04-25T00:16:20.219578+0000","last_unstale":"2024-04-25T00:17:27.664220+0000","last_undegraded":"2024-04-25T00:17:27.664220+0000","last_fullsized":"2024-04-25T00:17:27.664220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:41.811382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576508+0000","last_change":"2024-04-25T00:16:20.240570+0000","last_active":"2024-04-25T00:17:27.576508+0000","last_peered":"2024-04-25T00:17:27.576508+0000","last_clean":"2024-04-25T00:17:27.576508+0000","last_became_active":"2024-04-25T00:16:20.238241+0000","last_became_peered":"2024-04-25T00:16:20.238241+0000","last_unstale":"2024-04-25T00:17:27.576508+0000","last_undegraded":"2024-04-25T00:17:27.576508+0000","last_fullsized":"2024-04-25T00:17:27.576508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:06.981816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576418+0000","last_change":"2024-04-25T00:16:20.234748+0000","last_active":"2024-04-25T00:17:27.576418+0000","last_peered":"2024-04-25T00:17:27.576418+0000","last_clean":"2024-04-25T00:17:27.576418+0000","last_became_active":"2024-04-25T00:16:20.234325+0000","last_became_peered":"2024-04-25T00:16:20.234325+0000","last_unstale":"2024-04-25T00:17:27.576418+0000","last_undegraded":"2024-04-25T00:17:27.576418+0000","last_fullsized":"2024-04-25T00:17:27.576418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:00.383533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576099+0000","last_change":"2024-04-25T00:16:20.242987+0000","last_active":"2024-04-25T00:17:27.576099+0000","last_peered":"2024-04-25T00:17:27.576099+0000","last_clean":"2024-04-25T00:17:27.576099+0000","last_became_active":"2024-04-25T00:16:20.242625+0000","last_became_peered":"2024-04-25T00:16:20.242625+0000","last_unstale":"2024-04-25T00:17:27.576099+0000","last_undegraded":"2024-04-25T00:17:27.576099+0000","last_fullsized":"2024-04-25T00:17:27.576099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:16.618120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566927+0000","last_change":"2024-04-25T00:16:20.241985+0000","last_active":"2024-04-25T00:17:27.566927+0000","last_peered":"2024-04-25T00:17:27.566927+0000","last_clean":"2024-04-25T00:17:27.566927+0000","last_became_active":"2024-04-25T00:16:20.241646+0000","last_became_peered":"2024-04-25T00:16:20.241646+0000","last_unstale":"2024-04-25T00:17:27.566927+0000","last_undegraded":"2024-04-25T00:17:27.566927+0000","last_fullsized":"2024-04-25T00:17:27.566927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:44.094705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563042+0000","last_change":"2024-04-25T00:16:20.250161+0000","last_active":"2024-04-25T00:17:27.563042+0000","last_peered":"2024-04-25T00:17:27.563042+0000","last_clean":"2024-04-25T00:17:27.563042+0000","last_became_active":"2024-04-25T00:16:20.249811+0000","last_became_peered":"2024-04-25T00:16:20.249811+0000","last_unstale":"2024-04-25T00:17:27.563042+0000","last_undegraded":"2024-04-25T00:17:27.563042+0000","last_fullsized":"2024-04-25T00:17:27.563042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:03.222602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.571145+0000","last_change":"2024-04-25T00:16:20.220859+0000","last_active":"2024-04-25T00:17:27.571145+0000","last_peered":"2024-04-25T00:17:27.571145+0000","last_clean":"2024-04-25T00:17:27.571145+0000","last_became_active":"2024-04-25T00:16:20.219864+0000","last_became_peered":"2024-04-25T00:16:20.219864+0000","last_unstale":"2024-04-25T00:17:27.571145+0000","last_undegraded":"2024-04-25T00:17:27.571145+0000","last_fullsized":"2024-04-25T00:17:27.571145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:23.830726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566012+0000","last_change":"2024-04-25T00:16:20.249293+0000","last_active":"2024-04-25T00:17:27.566012+0000","last_peered":"2024-04-25T00:17:27.566012+0000","last_clean":"2024-04-25T00:17:27.566012+0000","last_became_active":"2024-04-25T00:16:20.248952+0000","last_became_peered":"2024-04-25T00:16:20.248952+0000","last_unstale":"2024-04-25T00:17:27.566012+0000","last_undegraded":"2024-04-25T00:17:27.566012+0000","last_fullsized":"2024-04-25T00:17:27.566012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:02.124426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.571590+0000","last_change":"2024-04-25T00:16:19.310971+0000","last_active":"2024-04-25T00:17:27.571590+0000","last_peered":"2024-04-25T00:17:27.571590+0000","last_clean":"2024-04-25T00:17:27.571590+0000","last_became_active":"2024-04-25T00:16:19.309020+0000","last_became_peered":"2024-04-25T00:16:19.309020+0000","last_unstale":"2024-04-25T00:17:27.571590+0000","last_undegraded":"2024-04-25T00:17:27.571590+0000","last_fullsized":"2024-04-25T00:17:27.571590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:56.744138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.574655+0000","last_change":"2024-04-25T00:16:20.250665+0000","last_active":"2024-04-25T00:17:27.574655+0000","last_peered":"2024-04-25T00:17:27.574655+0000","last_clean":"2024-04-25T00:17:27.574655+0000","last_became_active":"2024-04-25T00:16:20.250338+0000","last_became_peered":"2024-04-25T00:16:20.250338+0000","last_unstale":"2024-04-25T00:17:27.574655+0000","last_undegraded":"2024-04-25T00:17:27.574655+0000","last_fullsized":"2024-04-25T00:17:27.574655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:17.018035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.574776+0000","last_change":"2024-04-25T00:16:19.327748+0000","last_active":"2024-04-25T00:17:27.574776+0000","last_peered":"2024-04-25T00:17:27.574776+0000","last_clean":"2024-04-25T00:17:27.574776+0000","last_became_active":"2024-04-25T00:16:19.327264+0000","last_became_peered":"2024-04-25T00:16:19.327264+0000","last_unstale":"2024-04-25T00:17:27.574776+0000","last_undegraded":"2024-04-25T00:17:27.574776+0000","last_fullsized":"2024-04-25T00:17:27.574776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:27.080423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.578504+0000","last_change":"2024-04-25T00:16:20.243061+0000","last_active":"2024-04-25T00:17:27.578504+0000","last_peered":"2024-04-25T00:17:27.578504+0000","last_clean":"2024-04-25T00:17:27.578504+0000","last_became_active":"2024-04-25T00:16:20.242697+0000","last_became_peered":"2024-04-25T00:16:20.242697+0000","last_unstale":"2024-04-25T00:17:27.578504+0000","last_undegraded":"2024-04-25T00:17:27.578504+0000","last_fullsized":"2024-04-25T00:17:27.578504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:57.734902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565495+0000","last_change":"2024-04-25T00:16:19.317122+0000","last_active":"2024-04-25T00:17:27.565495+0000","last_peered":"2024-04-25T00:17:27.565495+0000","last_clean":"2024-04-25T00:17:27.565495+0000","last_became_active":"2024-04-25T00:16:19.316754+0000","last_became_peered":"2024-04-25T00:16:19.316754+0000","last_unstale":"2024-04-25T00:17:27.565495+0000","last_undegraded":"2024-04-25T00:17:27.565495+0000","last_fullsized":"2024-04-25T00:17:27.565495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:28.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566997+0000","last_change":"2024-04-25T00:16:20.238611+0000","last_active":"2024-04-25T00:17:27.566997+0000","last_peered":"2024-04-25T00:17:27.566997+0000","last_clean":"2024-04-25T00:17:27.566997+0000","last_became_active":"2024-04-25T00:16:20.238257+0000","last_became_peered":"2024-04-25T00:16:20.238257+0000","last_unstale":"2024-04-25T00:17:27.566997+0000","last_undegraded":"2024-04-25T00:17:27.566997+0000","last_fullsized":"2024-04-25T00:17:27.566997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:13.490284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565486+0000","last_change":"2024-04-25T00:16:19.308565+0000","last_active":"2024-04-25T00:17:27.565486+0000","last_peered":"2024-04-25T00:17:27.565486+0000","last_clean":"2024-04-25T00:17:27.565486+0000","last_became_active":"2024-04-25T00:16:19.308220+0000","last_became_peered":"2024-04-25T00:16:19.308220+0000","last_unstale":"2024-04-25T00:17:27.565486+0000","last_undegraded":"2024-04-25T00:17:27.565486+0000","last_fullsized":"2024-04-25T00:17:27.565486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:33.215491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.579625+0000","last_change":"2024-04-25T00:17:27.579625+0000","last_active":"2024-04-25T00:17:27.579625+0000","last_peered":"2024-04-25T00:17:27.579625+0000","last_clean":"2024-04-25T00:17:27.579625+0000","last_became_active":"2024-04-25T00:17:27.579200+0000","last_became_peered":"2024-04-25T00:17:27.579200+0000","last_unstale":"2024-04-25T00:17:27.579625+0000","last_undegraded":"2024-04-25T00:17:27.579625+0000","last_fullsized":"2024-04-25T00:17:27.579625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:34.927720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:45.430965+0000","last_change":"2024-04-25T00:16:19.312725+0000","last_active":"2024-04-25T00:17:45.430965+0000","last_peered":"2024-04-25T00:17:45.430965+0000","last_clean":"2024-04-25T00:17:45.430965+0000","last_became_active":"2024-04-25T00:16:19.312115+0000","last_became_peered":"2024-04-25T00:16:19.312115+0000","last_unstale":"2024-04-25T00:17:45.430965+0000","last_undegraded":"2024-04-25T00:17:45.430965+0000","last_fullsized":"2024-04-25T00:17:45.430965+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.661132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":85,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565299+0000","last_change":"2024-04-25T00:16:20.233546+0000","last_active":"2024-04-25T00:17:27.565299+0000","last_peered":"2024-04-25T00:17:27.565299+0000","last_clean":"2024-04-25T00:17:27.565299+0000","last_became_active":"2024-04-25T00:16:20.233029+0000","last_became_peered":"2024-04-25T00:16:20.233029+0000","last_unstale":"2024-04-25T00:17:27.565299+0000","last_undegraded":"2024-04-25T00:17:27.565299+0000","last_fullsized":"2024-04-25T00:17:27.565299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:44.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563038+0000","last_change":"2024-04-25T00:16:19.307512+0000","last_active":"2024-04-25T00:17:27.563038+0000","last_peered":"2024-04-25T00:17:27.563038+0000","last_clean":"2024-04-25T00:17:27.563038+0000","last_became_active":"2024-04-25T00:16:19.307164+0000","last_became_peered":"2024-04-25T00:16:19.307164+0000","last_unstale":"2024-04-25T00:17:27.563038+0000","last_undegraded":"2024-04-25T00:17:27.563038+0000","last_fullsized":"2024-04-25T00:17:27.563038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:37.560178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569282+0000","last_change":"2024-04-25T00:16:20.232758+0000","last_active":"2024-04-25T00:17:27.569282+0000","last_peered":"2024-04-25T00:17:27.569282+0000","last_clean":"2024-04-25T00:17:27.569282+0000","last_became_active":"2024-04-25T00:16:20.232443+0000","last_became_peered":"2024-04-25T00:16:20.232443+0000","last_unstale":"2024-04-25T00:17:27.569282+0000","last_undegraded":"2024-04-25T00:17:27.569282+0000","last_fullsized":"2024-04-25T00:17:27.569282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:31.029585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576842+0000","last_change":"2024-04-25T00:16:19.313924+0000","last_active":"2024-04-25T00:17:27.576842+0000","last_peered":"2024-04-25T00:17:27.576842+0000","last_clean":"2024-04-25T00:17:27.576842+0000","last_became_active":"2024-04-25T00:16:19.313570+0000","last_became_peered":"2024-04-25T00:16:19.313570+0000","last_unstale":"2024-04-25T00:17:27.576842+0000","last_undegraded":"2024-04-25T00:17:27.576842+0000","last_fullsized":"2024-04-25T00:17:27.576842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:42.258910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.574393+0000","last_change":"2024-04-25T00:16:20.244150+0000","last_active":"2024-04-25T00:17:27.574393+0000","last_peered":"2024-04-25T00:17:27.574393+0000","last_clean":"2024-04-25T00:17:27.574393+0000","last_became_active":"2024-04-25T00:16:20.243802+0000","last_became_peered":"2024-04-25T00:16:20.243802+0000","last_unstale":"2024-04-25T00:17:27.574393+0000","last_undegraded":"2024-04-25T00:17:27.574393+0000","last_fullsized":"2024-04-25T00:17:27.574393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:10.192507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569054+0000","last_change":"2024-04-25T00:16:19.312361+0000","last_active":"2024-04-25T00:17:27.569054+0000","last_peered":"2024-04-25T00:17:27.569054+0000","last_clean":"2024-04-25T00:17:27.569054+0000","last_became_active":"2024-04-25T00:16:19.312031+0000","last_became_peered":"2024-04-25T00:16:19.312031+0000","last_unstale":"2024-04-25T00:17:27.569054+0000","last_undegraded":"2024-04-25T00:17:27.569054+0000","last_fullsized":"2024-04-25T00:17:27.569054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:10.062091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566803+0000","last_change":"2024-04-25T00:16:20.231399+0000","last_active":"2024-04-25T00:17:27.566803+0000","last_peered":"2024-04-25T00:17:27.566803+0000","last_clean":"2024-04-25T00:17:27.566803+0000","last_became_active":"2024-04-25T00:16:20.230954+0000","last_became_peered":"2024-04-25T00:16:20.230954+0000","last_unstale":"2024-04-25T00:17:27.566803+0000","last_undegraded":"2024-04-25T00:17:27.566803+0000","last_fullsized":"2024-04-25T00:17:27.566803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:20.934927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":103,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:43.427355+0000","last_change":"2024-04-25T00:16:19.308591+0000","last_active":"2024-04-25T00:17:43.427355+0000","last_peered":"2024-04-25T00:17:43.427355+0000","last_clean":"2024-04-25T00:17:43.427355+0000","last_became_active":"2024-04-25T00:16:19.307987+0000","last_became_peered":"2024-04-25T00:16:19.307987+0000","last_unstale":"2024-04-25T00:17:43.427355+0000","last_undegraded":"2024-04-25T00:17:43.427355+0000","last_fullsized":"2024-04-25T00:17:43.427355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:32.739170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.567391+0000","last_change":"2024-04-25T00:16:20.218545+0000","last_active":"2024-04-25T00:17:27.567391+0000","last_peered":"2024-04-25T00:17:27.567391+0000","last_clean":"2024-04-25T00:17:27.567391+0000","last_became_active":"2024-04-25T00:16:20.218267+0000","last_became_peered":"2024-04-25T00:16:20.218267+0000","last_unstale":"2024-04-25T00:17:27.567391+0000","last_undegraded":"2024-04-25T00:17:27.567391+0000","last_fullsized":"2024-04-25T00:17:27.567391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:47.725985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664114+0000","last_change":"2024-04-25T00:16:19.311918+0000","last_active":"2024-04-25T00:17:27.664114+0000","last_peered":"2024-04-25T00:17:27.664114+0000","last_clean":"2024-04-25T00:17:27.664114+0000","last_became_active":"2024-04-25T00:16:19.311594+0000","last_became_peered":"2024-04-25T00:16:19.311594+0000","last_unstale":"2024-04-25T00:17:27.664114+0000","last_undegraded":"2024-04-25T00:17:27.664114+0000","last_fullsized":"2024-04-25T00:17:27.664114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:04.574190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.577206+0000","last_change":"2024-04-25T00:16:20.248116+0000","last_active":"2024-04-25T00:17:27.577206+0000","last_peered":"2024-04-25T00:17:27.577206+0000","last_clean":"2024-04-25T00:17:27.577206+0000","last_became_active":"2024-04-25T00:16:20.247373+0000","last_became_peered":"2024-04-25T00:16:20.247373+0000","last_unstale":"2024-04-25T00:17:27.577206+0000","last_undegraded":"2024-04-25T00:17:27.577206+0000","last_fullsized":"2024-04-25T00:17:27.577206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:47.777779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569627+0000","last_change":"2024-04-25T00:16:19.320640+0000","last_active":"2024-04-25T00:17:27.569627+0000","last_peered":"2024-04-25T00:17:27.569627+0000","last_clean":"2024-04-25T00:17:27.569627+0000","last_became_active":"2024-04-25T00:16:19.320273+0000","last_became_peered":"2024-04-25T00:16:19.320273+0000","last_unstale":"2024-04-25T00:17:27.569627+0000","last_undegraded":"2024-04-25T00:17:27.569627+0000","last_fullsized":"2024-04-25T00:17:27.569627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:10.843162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.567123+0000","last_change":"2024-04-25T00:16:20.219378+0000","last_active":"2024-04-25T00:17:27.567123+0000","last_peered":"2024-04-25T00:17:27.567123+0000","last_clean":"2024-04-25T00:17:27.567123+0000","last_became_active":"2024-04-25T00:16:20.219069+0000","last_became_peered":"2024-04-25T00:16:20.219069+0000","last_unstale":"2024-04-25T00:17:27.567123+0000","last_undegraded":"2024-04-25T00:17:27.567123+0000","last_fullsized":"2024-04-25T00:17:27.567123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:10.818609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'28","reported_seq":217,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:44.444226+0000","last_change":"2024-04-25T00:16:19.315089+0000","last_active":"2024-04-25T00:17:44.444226+0000","last_peered":"2024-04-25T00:17:44.444226+0000","last_clean":"2024-04-25T00:17:44.444226+0000","last_became_active":"2024-04-25T00:16:19.314416+0000","last_became_peered":"2024-04-25T00:16:19.314416+0000","last_unstale":"2024-04-25T00:17:44.444226+0000","last_undegraded":"2024-04-25T00:17:44.444226+0000","last_fullsized":"2024-04-25T00:17:44.444226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:25.920010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61896,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":170,"num_read_kb":145,"num_write":29,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565245+0000","last_change":"2024-04-25T00:16:20.225678+0000","last_active":"2024-04-25T00:17:27.565245+0000","last_peered":"2024-04-25T00:17:27.565245+0000","last_clean":"2024-04-25T00:17:27.565245+0000","last_became_active":"2024-04-25T00:16:20.225146+0000","last_became_peered":"2024-04-25T00:16:20.225146+0000","last_unstale":"2024-04-25T00:17:27.565245+0000","last_undegraded":"2024-04-25T00:17:27.565245+0000","last_fullsized":"2024-04-25T00:17:27.565245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:56.003701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566115+0000","last_change":"2024-04-25T00:16:19.327877+0000","last_active":"2024-04-25T00:17:27.566115+0000","last_peered":"2024-04-25T00:17:27.566115+0000","last_clean":"2024-04-25T00:17:27.566115+0000","last_became_active":"2024-04-25T00:16:19.327537+0000","last_became_peered":"2024-04-25T00:16:19.327537+0000","last_unstale":"2024-04-25T00:17:27.566115+0000","last_undegraded":"2024-04-25T00:17:27.566115+0000","last_fullsized":"2024-04-25T00:17:27.566115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:14.614232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.667026+0000","last_change":"2024-04-25T00:17:27.667026+0000","last_active":"2024-04-25T00:17:27.667026+0000","last_peered":"2024-04-25T00:17:27.667026+0000","last_clean":"2024-04-25T00:17:27.667026+0000","last_became_active":"2024-04-25T00:17:27.665718+0000","last_became_peered":"2024-04-25T00:17:27.665718+0000","last_unstale":"2024-04-25T00:17:27.667026+0000","last_undegraded":"2024-04-25T00:17:27.667026+0000","last_fullsized":"2024-04-25T00:17:27.667026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:37.297615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569400+0000","last_change":"2024-04-25T00:16:19.328428+0000","last_active":"2024-04-25T00:17:27.569400+0000","last_peered":"2024-04-25T00:17:27.569400+0000","last_clean":"2024-04-25T00:17:27.569400+0000","last_became_active":"2024-04-25T00:16:19.328112+0000","last_became_peered":"2024-04-25T00:16:19.328112+0000","last_unstale":"2024-04-25T00:17:27.569400+0000","last_undegraded":"2024-04-25T00:17:27.569400+0000","last_fullsized":"2024-04-25T00:17:27.569400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:09.130246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663884+0000","last_change":"2024-04-25T00:16:20.237922+0000","last_active":"2024-04-25T00:17:27.663884+0000","last_peered":"2024-04-25T00:17:27.663884+0000","last_clean":"2024-04-25T00:17:27.663884+0000","last_became_active":"2024-04-25T00:16:20.237594+0000","last_became_peered":"2024-04-25T00:16:20.237594+0000","last_unstale":"2024-04-25T00:17:27.663884+0000","last_undegraded":"2024-04-25T00:17:27.663884+0000","last_fullsized":"2024-04-25T00:17:27.663884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:26.517335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":107,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:46.506786+0000","last_change":"2024-04-25T00:16:19.312920+0000","last_active":"2024-04-25T00:17:46.506786+0000","last_peered":"2024-04-25T00:17:46.506786+0000","last_clean":"2024-04-25T00:17:46.506786+0000","last_became_active":"2024-04-25T00:16:19.312548+0000","last_became_peered":"2024-04-25T00:16:19.312548+0000","last_unstale":"2024-04-25T00:17:46.506786+0000","last_undegraded":"2024-04-25T00:17:46.506786+0000","last_fullsized":"2024-04-25T00:17:46.506786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:47.154402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663212+0000","last_change":"2024-04-25T00:16:20.221102+0000","last_active":"2024-04-25T00:17:27.663212+0000","last_peered":"2024-04-25T00:17:27.663212+0000","last_clean":"2024-04-25T00:17:27.663212+0000","last_became_active":"2024-04-25T00:16:20.220753+0000","last_became_peered":"2024-04-25T00:16:20.220753+0000","last_unstale":"2024-04-25T00:17:27.663212+0000","last_undegraded":"2024-04-25T00:17:27.663212+0000","last_fullsized":"2024-04-25T00:17:27.663212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:18.652865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565165+0000","last_change":"2024-04-25T00:16:19.324181+0000","last_active":"2024-04-25T00:17:27.565165+0000","last_peered":"2024-04-25T00:17:27.565165+0000","last_clean":"2024-04-25T00:17:27.565165+0000","last_became_active":"2024-04-25T00:16:19.323821+0000","last_became_peered":"2024-04-25T00:16:19.323821+0000","last_unstale":"2024-04-25T00:17:27.565165+0000","last_undegraded":"2024-04-25T00:17:27.565165+0000","last_fullsized":"2024-04-25T00:17:27.565165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:34.323961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.571552+0000","last_change":"2024-04-25T00:17:27.571552+0000","last_active":"2024-04-25T00:17:27.571552+0000","last_peered":"2024-04-25T00:17:27.571552+0000","last_clean":"2024-04-25T00:17:27.571552+0000","last_became_active":"2024-04-25T00:17:27.571087+0000","last_became_peered":"2024-04-25T00:17:27.571087+0000","last_unstale":"2024-04-25T00:17:27.571552+0000","last_undegraded":"2024-04-25T00:17:27.571552+0000","last_fullsized":"2024-04-25T00:17:27.571552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:10.459526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565067+0000","last_change":"2024-04-25T00:16:19.306005+0000","last_active":"2024-04-25T00:17:27.565067+0000","last_peered":"2024-04-25T00:17:27.565067+0000","last_clean":"2024-04-25T00:17:27.565067+0000","last_became_active":"2024-04-25T00:16:19.305656+0000","last_became_peered":"2024-04-25T00:16:19.305656+0000","last_unstale":"2024-04-25T00:17:27.565067+0000","last_undegraded":"2024-04-25T00:17:27.565067+0000","last_fullsized":"2024-04-25T00:17:27.565067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:16.452352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562723+0000","last_change":"2024-04-25T00:16:20.217886+0000","last_active":"2024-04-25T00:17:27.562723+0000","last_peered":"2024-04-25T00:17:27.562723+0000","last_clean":"2024-04-25T00:17:27.562723+0000","last_became_active":"2024-04-25T00:16:20.217379+0000","last_became_peered":"2024-04-25T00:16:20.217379+0000","last_unstale":"2024-04-25T00:17:27.562723+0000","last_undegraded":"2024-04-25T00:17:27.562723+0000","last_fullsized":"2024-04-25T00:17:27.562723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:22.028791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.568038+0000","last_change":"2024-04-25T00:16:19.328466+0000","last_active":"2024-04-25T00:17:27.568038+0000","last_peered":"2024-04-25T00:17:27.568038+0000","last_clean":"2024-04-25T00:17:27.568038+0000","last_became_active":"2024-04-25T00:16:19.321077+0000","last_became_peered":"2024-04-25T00:16:19.321077+0000","last_unstale":"2024-04-25T00:17:27.568038+0000","last_undegraded":"2024-04-25T00:17:27.568038+0000","last_fullsized":"2024-04-25T00:17:27.568038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:55.296160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563377+0000","last_change":"2024-04-25T00:16:20.249243+0000","last_active":"2024-04-25T00:17:27.563377+0000","last_peered":"2024-04-25T00:17:27.563377+0000","last_clean":"2024-04-25T00:17:27.563377+0000","last_became_active":"2024-04-25T00:16:20.248955+0000","last_became_peered":"2024-04-25T00:16:20.248955+0000","last_unstale":"2024-04-25T00:17:27.563377+0000","last_undegraded":"2024-04-25T00:17:27.563377+0000","last_fullsized":"2024-04-25T00:17:27.563377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:14.195688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664197+0000","last_change":"2024-04-25T00:16:19.316409+0000","last_active":"2024-04-25T00:17:27.664197+0000","last_peered":"2024-04-25T00:17:27.664197+0000","last_clean":"2024-04-25T00:17:27.664197+0000","last_became_active":"2024-04-25T00:16:19.315982+0000","last_became_peered":"2024-04-25T00:16:19.315982+0000","last_unstale":"2024-04-25T00:17:27.664197+0000","last_undegraded":"2024-04-25T00:17:27.664197+0000","last_fullsized":"2024-04-25T00:17:27.664197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.718600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563235+0000","last_change":"2024-04-25T00:16:20.242119+0000","last_active":"2024-04-25T00:17:27.563235+0000","last_peered":"2024-04-25T00:17:27.563235+0000","last_clean":"2024-04-25T00:17:27.563235+0000","last_became_active":"2024-04-25T00:16:20.241702+0000","last_became_peered":"2024-04-25T00:16:20.241702+0000","last_unstale":"2024-04-25T00:17:27.563235+0000","last_undegraded":"2024-04-25T00:17:27.563235+0000","last_fullsized":"2024-04-25T00:17:27.563235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:26.652681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.668759+0000","last_change":"2024-04-25T00:17:27.667027+0000","last_active":"2024-04-25T00:17:27.668759+0000","last_peered":"2024-04-25T00:17:27.668759+0000","last_clean":"2024-04-25T00:17:27.668759+0000","last_became_active":"2024-04-25T00:17:27.665152+0000","last_became_peered":"2024-04-25T00:17:27.665152+0000","last_unstale":"2024-04-25T00:17:27.668759+0000","last_undegraded":"2024-04-25T00:17:27.668759+0000","last_fullsized":"2024-04-25T00:17:27.668759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:10.679692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":104,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:44.500815+0000","last_change":"2024-04-25T00:16:19.328872+0000","last_active":"2024-04-25T00:17:44.500815+0000","last_peered":"2024-04-25T00:17:44.500815+0000","last_clean":"2024-04-25T00:17:44.500815+0000","last_became_active":"2024-04-25T00:16:19.328439+0000","last_became_peered":"2024-04-25T00:16:19.328439+0000","last_unstale":"2024-04-25T00:17:44.500815+0000","last_undegraded":"2024-04-25T00:17:44.500815+0000","last_fullsized":"2024-04-25T00:17:44.500815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:05.832328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":81,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.570225+0000","last_change":"2024-04-25T00:16:20.242857+0000","last_active":"2024-04-25T00:17:27.570225+0000","last_peered":"2024-04-25T00:17:27.570225+0000","last_clean":"2024-04-25T00:17:27.570225+0000","last_became_active":"2024-04-25T00:16:20.240941+0000","last_became_peered":"2024-04-25T00:16:20.240941+0000","last_unstale":"2024-04-25T00:17:27.570225+0000","last_undegraded":"2024-04-25T00:17:27.570225+0000","last_fullsized":"2024-04-25T00:17:27.570225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:36.550571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663465+0000","last_change":"2024-04-25T00:16:19.312914+0000","last_active":"2024-04-25T00:17:27.663465+0000","last_peered":"2024-04-25T00:17:27.663465+0000","last_clean":"2024-04-25T00:17:27.663465+0000","last_became_active":"2024-04-25T00:16:19.312578+0000","last_became_peered":"2024-04-25T00:16:19.312578+0000","last_unstale":"2024-04-25T00:17:27.663465+0000","last_undegraded":"2024-04-25T00:17:27.663465+0000","last_fullsized":"2024-04-25T00:17:27.663465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:37.035904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663384+0000","last_change":"2024-04-25T00:15:03.261320+0000","last_active":"2024-04-25T00:17:27.663384+0000","last_peered":"2024-04-25T00:17:27.663384+0000","last_clean":"2024-04-25T00:17:27.663384+0000","last_became_active":"2024-04-25T00:15:03.260804+0000","last_became_peered":"2024-04-25T00:15:03.260804+0000","last_unstale":"2024-04-25T00:17:27.663384+0000","last_undegraded":"2024-04-25T00:17:27.663384+0000","last_fullsized":"2024-04-25T00:17:27.663384+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:14.473933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:14.473933+0000","last_clean_scrub_stamp":"2024-04-25T00:11:14.473933+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:35.876949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576567+0000","last_change":"2024-04-25T00:16:20.238253+0000","last_active":"2024-04-25T00:17:27.576567+0000","last_peered":"2024-04-25T00:17:27.576567+0000","last_clean":"2024-04-25T00:17:27.576567+0000","last_became_active":"2024-04-25T00:16:20.237824+0000","last_became_peered":"2024-04-25T00:16:20.237824+0000","last_unstale":"2024-04-25T00:17:27.576567+0000","last_undegraded":"2024-04-25T00:17:27.576567+0000","last_fullsized":"2024-04-25T00:17:27.576567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:21.996513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663453+0000","last_change":"2024-04-25T00:16:19.316194+0000","last_active":"2024-04-25T00:17:27.663453+0000","last_peered":"2024-04-25T00:17:27.663453+0000","last_clean":"2024-04-25T00:17:27.663453+0000","last_became_active":"2024-04-25T00:16:19.315625+0000","last_became_peered":"2024-04-25T00:16:19.315625+0000","last_unstale":"2024-04-25T00:17:27.663453+0000","last_undegraded":"2024-04-25T00:17:27.663453+0000","last_fullsized":"2024-04-25T00:17:27.663453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:23.426048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565379+0000","last_change":"2024-04-25T00:16:19.324283+0000","last_active":"2024-04-25T00:17:27.565379+0000","last_peered":"2024-04-25T00:17:27.565379+0000","last_clean":"2024-04-25T00:17:27.565379+0000","last_became_active":"2024-04-25T00:16:19.323800+0000","last_became_peered":"2024-04-25T00:16:19.323800+0000","last_unstale":"2024-04-25T00:17:27.565379+0000","last_undegraded":"2024-04-25T00:17:27.565379+0000","last_fullsized":"2024-04-25T00:17:27.565379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:43.592775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.577063+0000","last_change":"2024-04-25T00:16:20.239205+0000","last_active":"2024-04-25T00:17:27.577063+0000","last_peered":"2024-04-25T00:17:27.577063+0000","last_clean":"2024-04-25T00:17:27.577063+0000","last_became_active":"2024-04-25T00:16:20.237135+0000","last_became_peered":"2024-04-25T00:16:20.237135+0000","last_unstale":"2024-04-25T00:17:27.577063+0000","last_undegraded":"2024-04-25T00:17:27.577063+0000","last_fullsized":"2024-04-25T00:17:27.577063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:37.348227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.571272+0000","last_change":"2024-04-25T00:16:19.325868+0000","last_active":"2024-04-25T00:17:27.571272+0000","last_peered":"2024-04-25T00:17:27.571272+0000","last_clean":"2024-04-25T00:17:27.571272+0000","last_became_active":"2024-04-25T00:16:19.322820+0000","last_became_peered":"2024-04-25T00:16:19.322820+0000","last_unstale":"2024-04-25T00:17:27.571272+0000","last_undegraded":"2024-04-25T00:17:27.571272+0000","last_fullsized":"2024-04-25T00:17:27.571272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:50.982099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.574946+0000","last_change":"2024-04-25T00:16:20.248268+0000","last_active":"2024-04-25T00:17:27.574946+0000","last_peered":"2024-04-25T00:17:27.574946+0000","last_clean":"2024-04-25T00:17:27.574946+0000","last_became_active":"2024-04-25T00:16:20.247274+0000","last_became_peered":"2024-04-25T00:16:20.247274+0000","last_unstale":"2024-04-25T00:17:27.574946+0000","last_undegraded":"2024-04-25T00:17:27.574946+0000","last_fullsized":"2024-04-25T00:17:27.574946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:46.120277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664425+0000","last_change":"2024-04-25T00:16:19.317256+0000","last_active":"2024-04-25T00:17:27.664425+0000","last_peered":"2024-04-25T00:17:27.664425+0000","last_clean":"2024-04-25T00:17:27.664425+0000","last_became_active":"2024-04-25T00:16:19.316931+0000","last_became_peered":"2024-04-25T00:16:19.316931+0000","last_unstale":"2024-04-25T00:17:27.664425+0000","last_undegraded":"2024-04-25T00:17:27.664425+0000","last_fullsized":"2024-04-25T00:17:27.664425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:49.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.577437+0000","last_change":"2024-04-25T00:16:20.252271+0000","last_active":"2024-04-25T00:17:27.577437+0000","last_peered":"2024-04-25T00:17:27.577437+0000","last_clean":"2024-04-25T00:17:27.577437+0000","last_became_active":"2024-04-25T00:16:20.251930+0000","last_became_peered":"2024-04-25T00:16:20.251930+0000","last_unstale":"2024-04-25T00:17:27.577437+0000","last_undegraded":"2024-04-25T00:17:27.577437+0000","last_fullsized":"2024-04-25T00:17:27.577437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:07.313172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.584129+0000","last_change":"2024-04-25T00:17:27.584129+0000","last_active":"2024-04-25T00:17:27.584129+0000","last_peered":"2024-04-25T00:17:27.584129+0000","last_clean":"2024-04-25T00:17:27.584129+0000","last_became_active":"2024-04-25T00:17:27.580604+0000","last_became_peered":"2024-04-25T00:17:27.580604+0000","last_unstale":"2024-04-25T00:17:27.584129+0000","last_undegraded":"2024-04-25T00:17:27.584129+0000","last_fullsized":"2024-04-25T00:17:27.584129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:26.204170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565223+0000","last_change":"2024-04-25T00:16:20.224710+0000","last_active":"2024-04-25T00:17:27.565223+0000","last_peered":"2024-04-25T00:17:27.565223+0000","last_clean":"2024-04-25T00:17:27.565223+0000","last_became_active":"2024-04-25T00:16:20.224385+0000","last_became_peered":"2024-04-25T00:16:20.224385+0000","last_unstale":"2024-04-25T00:17:27.565223+0000","last_undegraded":"2024-04-25T00:17:27.565223+0000","last_fullsized":"2024-04-25T00:17:27.565223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:29.161972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.568285+0000","last_change":"2024-04-25T00:16:19.327498+0000","last_active":"2024-04-25T00:17:27.568285+0000","last_peered":"2024-04-25T00:17:27.568285+0000","last_clean":"2024-04-25T00:17:27.568285+0000","last_became_active":"2024-04-25T00:16:19.326889+0000","last_became_peered":"2024-04-25T00:16:19.326889+0000","last_unstale":"2024-04-25T00:17:27.568285+0000","last_undegraded":"2024-04-25T00:17:27.568285+0000","last_fullsized":"2024-04-25T00:17:27.568285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:05.275999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562551+0000","last_change":"2024-04-25T00:16:20.242150+0000","last_active":"2024-04-25T00:17:27.562551+0000","last_peered":"2024-04-25T00:17:27.562551+0000","last_clean":"2024-04-25T00:17:27.562551+0000","last_became_active":"2024-04-25T00:16:20.241770+0000","last_became_peered":"2024-04-25T00:16:20.241770+0000","last_unstale":"2024-04-25T00:17:27.562551+0000","last_undegraded":"2024-04-25T00:17:27.562551+0000","last_fullsized":"2024-04-25T00:17:27.562551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:38.038980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.579179+0000","last_change":"2024-04-25T00:17:27.578637+0000","last_active":"2024-04-25T00:17:27.579179+0000","last_peered":"2024-04-25T00:17:27.579179+0000","last_clean":"2024-04-25T00:17:27.579179+0000","last_became_active":"2024-04-25T00:17:27.571940+0000","last_became_peered":"2024-04-25T00:17:27.571940+0000","last_unstale":"2024-04-25T00:17:27.579179+0000","last_undegraded":"2024-04-25T00:17:27.579179+0000","last_fullsized":"2024-04-25T00:17:27.579179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:19.798797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.573828+0000","last_change":"2024-04-25T00:16:20.244330+0000","last_active":"2024-04-25T00:17:27.573828+0000","last_peered":"2024-04-25T00:17:27.573828+0000","last_clean":"2024-04-25T00:17:27.573828+0000","last_became_active":"2024-04-25T00:16:20.243946+0000","last_became_peered":"2024-04-25T00:16:20.243946+0000","last_unstale":"2024-04-25T00:17:27.573828+0000","last_undegraded":"2024-04-25T00:17:27.573828+0000","last_fullsized":"2024-04-25T00:17:27.573828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:37.790655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565039+0000","last_change":"2024-04-25T00:16:19.313708+0000","last_active":"2024-04-25T00:17:27.565039+0000","last_peered":"2024-04-25T00:17:27.565039+0000","last_clean":"2024-04-25T00:17:27.565039+0000","last_became_active":"2024-04-25T00:16:19.311700+0000","last_became_peered":"2024-04-25T00:16:19.311700+0000","last_unstale":"2024-04-25T00:17:27.565039+0000","last_undegraded":"2024-04-25T00:17:27.565039+0000","last_fullsized":"2024-04-25T00:17:27.565039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:24.656229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664298+0000","last_change":"2024-04-25T00:16:20.240828+0000","last_active":"2024-04-25T00:17:27.664298+0000","last_peered":"2024-04-25T00:17:27.664298+0000","last_clean":"2024-04-25T00:17:27.664298+0000","last_became_active":"2024-04-25T00:16:20.240495+0000","last_became_peered":"2024-04-25T00:16:20.240495+0000","last_unstale":"2024-04-25T00:17:27.664298+0000","last_undegraded":"2024-04-25T00:17:27.664298+0000","last_fullsized":"2024-04-25T00:17:27.664298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:19.730968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566077+0000","last_change":"2024-04-25T00:16:19.324069+0000","last_active":"2024-04-25T00:17:27.566077+0000","last_peered":"2024-04-25T00:17:27.566077+0000","last_clean":"2024-04-25T00:17:27.566077+0000","last_became_active":"2024-04-25T00:16:19.323604+0000","last_became_peered":"2024-04-25T00:16:19.323604+0000","last_unstale":"2024-04-25T00:17:27.566077+0000","last_undegraded":"2024-04-25T00:17:27.566077+0000","last_fullsized":"2024-04-25T00:17:27.566077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:50.908834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663990+0000","last_change":"2024-04-25T00:16:20.220555+0000","last_active":"2024-04-25T00:17:27.663990+0000","last_peered":"2024-04-25T00:17:27.663990+0000","last_clean":"2024-04-25T00:17:27.663990+0000","last_became_active":"2024-04-25T00:16:20.219986+0000","last_became_peered":"2024-04-25T00:16:20.219986+0000","last_unstale":"2024-04-25T00:17:27.663990+0000","last_undegraded":"2024-04-25T00:17:27.663990+0000","last_fullsized":"2024-04-25T00:17:27.663990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:04.095555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569486+0000","last_change":"2024-04-25T00:16:19.328270+0000","last_active":"2024-04-25T00:17:27.569486+0000","last_peered":"2024-04-25T00:17:27.569486+0000","last_clean":"2024-04-25T00:17:27.569486+0000","last_became_active":"2024-04-25T00:16:19.327929+0000","last_became_peered":"2024-04-25T00:16:19.327929+0000","last_unstale":"2024-04-25T00:17:27.569486+0000","last_undegraded":"2024-04-25T00:17:27.569486+0000","last_fullsized":"2024-04-25T00:17:27.569486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:09.233345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663754+0000","last_change":"2024-04-25T00:16:20.237103+0000","last_active":"2024-04-25T00:17:27.663754+0000","last_peered":"2024-04-25T00:17:27.663754+0000","last_clean":"2024-04-25T00:17:27.663754+0000","last_became_active":"2024-04-25T00:16:20.236811+0000","last_became_peered":"2024-04-25T00:16:20.236811+0000","last_unstale":"2024-04-25T00:17:27.663754+0000","last_undegraded":"2024-04-25T00:17:27.663754+0000","last_fullsized":"2024-04-25T00:17:27.663754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:56.833659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.578061+0000","last_change":"2024-04-25T00:16:19.323227+0000","last_active":"2024-04-25T00:17:27.578061+0000","last_peered":"2024-04-25T00:17:27.578061+0000","last_clean":"2024-04-25T00:17:27.578061+0000","last_became_active":"2024-04-25T00:16:19.322074+0000","last_became_peered":"2024-04-25T00:16:19.322074+0000","last_unstale":"2024-04-25T00:17:27.578061+0000","last_undegraded":"2024-04-25T00:17:27.578061+0000","last_fullsized":"2024-04-25T00:17:27.578061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:53.316171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664221+0000","last_change":"2024-04-25T00:16:20.222049+0000","last_active":"2024-04-25T00:17:27.664221+0000","last_peered":"2024-04-25T00:17:27.664221+0000","last_clean":"2024-04-25T00:17:27.664221+0000","last_became_active":"2024-04-25T00:16:20.221651+0000","last_became_peered":"2024-04-25T00:16:20.221651+0000","last_unstale":"2024-04-25T00:17:27.664221+0000","last_undegraded":"2024-04-25T00:17:27.664221+0000","last_fullsized":"2024-04-25T00:17:27.664221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:31.244463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562619+0000","last_change":"2024-04-25T00:16:20.223864+0000","last_active":"2024-04-25T00:17:27.562619+0000","last_peered":"2024-04-25T00:17:27.562619+0000","last_clean":"2024-04-25T00:17:27.562619+0000","last_became_active":"2024-04-25T00:16:20.223533+0000","last_became_peered":"2024-04-25T00:16:20.223533+0000","last_unstale":"2024-04-25T00:17:27.562619+0000","last_undegraded":"2024-04-25T00:17:27.562619+0000","last_fullsized":"2024-04-25T00:17:27.562619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:14.482038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565715+0000","last_change":"2024-04-25T00:16:20.214232+0000","last_active":"2024-04-25T00:17:27.565715+0000","last_peered":"2024-04-25T00:17:27.565715+0000","last_clean":"2024-04-25T00:17:27.565715+0000","last_became_active":"2024-04-25T00:16:20.213871+0000","last_became_peered":"2024-04-25T00:16:20.213871+0000","last_unstale":"2024-04-25T00:17:27.565715+0000","last_undegraded":"2024-04-25T00:17:27.565715+0000","last_fullsized":"2024-04-25T00:17:27.565715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:07.049327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565818+0000","last_change":"2024-04-25T00:16:20.232935+0000","last_active":"2024-04-25T00:17:27.565818+0000","last_peered":"2024-04-25T00:17:27.565818+0000","last_clean":"2024-04-25T00:17:27.565818+0000","last_became_active":"2024-04-25T00:16:20.232632+0000","last_became_peered":"2024-04-25T00:16:20.232632+0000","last_unstale":"2024-04-25T00:17:27.565818+0000","last_undegraded":"2024-04-25T00:17:27.565818+0000","last_fullsized":"2024-04-25T00:17:27.565818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:36.724694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.577206+0000","last_change":"2024-04-25T00:16:20.244382+0000","last_active":"2024-04-25T00:17:27.577206+0000","last_peered":"2024-04-25T00:17:27.577206+0000","last_clean":"2024-04-25T00:17:27.577206+0000","last_became_active":"2024-04-25T00:16:20.241866+0000","last_became_peered":"2024-04-25T00:16:20.241866+0000","last_unstale":"2024-04-25T00:17:27.577206+0000","last_undegraded":"2024-04-25T00:17:27.577206+0000","last_fullsized":"2024-04-25T00:17:27.577206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:24.170878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.577185+0000","last_change":"2024-04-25T00:16:20.241343+0000","last_active":"2024-04-25T00:17:27.577185+0000","last_peered":"2024-04-25T00:17:27.577185+0000","last_clean":"2024-04-25T00:17:27.577185+0000","last_became_active":"2024-04-25T00:16:20.238417+0000","last_became_peered":"2024-04-25T00:16:20.238417+0000","last_unstale":"2024-04-25T00:17:27.577185+0000","last_undegraded":"2024-04-25T00:17:27.577185+0000","last_fullsized":"2024-04-25T00:17:27.577185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:29.988773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562822+0000","last_change":"2024-04-25T00:16:20.219966+0000","last_active":"2024-04-25T00:17:27.562822+0000","last_peered":"2024-04-25T00:17:27.562822+0000","last_clean":"2024-04-25T00:17:27.562822+0000","last_became_active":"2024-04-25T00:16:20.219656+0000","last_became_peered":"2024-04-25T00:16:20.219656+0000","last_unstale":"2024-04-25T00:17:27.562822+0000","last_undegraded":"2024-04-25T00:17:27.562822+0000","last_fullsized":"2024-04-25T00:17:27.562822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:56.290610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566142+0000","last_change":"2024-04-25T00:16:20.219141+0000","last_active":"2024-04-25T00:17:27.566142+0000","last_peered":"2024-04-25T00:17:27.566142+0000","last_clean":"2024-04-25T00:17:27.566142+0000","last_became_active":"2024-04-25T00:16:20.218842+0000","last_became_peered":"2024-04-25T00:16:20.218842+0000","last_unstale":"2024-04-25T00:17:27.566142+0000","last_undegraded":"2024-04-25T00:17:27.566142+0000","last_fullsized":"2024-04-25T00:17:27.566142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:33.938226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.574814+0000","last_change":"2024-04-25T00:16:20.249183+0000","last_active":"2024-04-25T00:17:27.574814+0000","last_peered":"2024-04-25T00:17:27.574814+0000","last_clean":"2024-04-25T00:17:27.574814+0000","last_became_active":"2024-04-25T00:16:20.248355+0000","last_became_peered":"2024-04-25T00:16:20.248355+0000","last_unstale":"2024-04-25T00:17:27.574814+0000","last_undegraded":"2024-04-25T00:17:27.574814+0000","last_fullsized":"2024-04-25T00:17:27.574814+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:57.738041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569061+0000","last_change":"2024-04-25T00:16:20.227528+0000","last_active":"2024-04-25T00:17:27.569061+0000","last_peered":"2024-04-25T00:17:27.569061+0000","last_clean":"2024-04-25T00:17:27.569061+0000","last_became_active":"2024-04-25T00:16:20.227208+0000","last_became_peered":"2024-04-25T00:16:20.227208+0000","last_unstale":"2024-04-25T00:17:27.569061+0000","last_undegraded":"2024-04-25T00:17:27.569061+0000","last_fullsized":"2024-04-25T00:17:27.569061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:49.920850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.568114+0000","last_change":"2024-04-25T00:16:20.228157+0000","last_active":"2024-04-25T00:17:27.568114+0000","last_peered":"2024-04-25T00:17:27.568114+0000","last_clean":"2024-04-25T00:17:27.568114+0000","last_became_active":"2024-04-25T00:16:20.227659+0000","last_became_peered":"2024-04-25T00:16:20.227659+0000","last_unstale":"2024-04-25T00:17:27.568114+0000","last_undegraded":"2024-04-25T00:17:27.568114+0000","last_fullsized":"2024-04-25T00:17:27.568114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:30.579407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.567192+0000","last_change":"2024-04-25T00:16:20.232237+0000","last_active":"2024-04-25T00:17:27.567192+0000","last_peered":"2024-04-25T00:17:27.567192+0000","last_clean":"2024-04-25T00:17:27.567192+0000","last_became_active":"2024-04-25T00:16:20.231897+0000","last_became_peered":"2024-04-25T00:16:20.231897+0000","last_unstale":"2024-04-25T00:17:27.567192+0000","last_undegraded":"2024-04-25T00:17:27.567192+0000","last_fullsized":"2024-04-25T00:17:27.567192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:23.919515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576373+0000","last_change":"2024-04-25T00:16:20.243454+0000","last_active":"2024-04-25T00:17:27.576373+0000","last_peered":"2024-04-25T00:17:27.576373+0000","last_clean":"2024-04-25T00:17:27.576373+0000","last_became_active":"2024-04-25T00:16:20.243112+0000","last_became_peered":"2024-04-25T00:16:20.243112+0000","last_unstale":"2024-04-25T00:17:27.576373+0000","last_undegraded":"2024-04-25T00:17:27.576373+0000","last_fullsized":"2024-04-25T00:17:27.576373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:58.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562585+0000","last_change":"2024-04-25T00:16:20.220947+0000","last_active":"2024-04-25T00:17:27.562585+0000","last_peered":"2024-04-25T00:17:27.562585+0000","last_clean":"2024-04-25T00:17:27.562585+0000","last_became_active":"2024-04-25T00:16:20.220662+0000","last_became_peered":"2024-04-25T00:16:20.220662+0000","last_unstale":"2024-04-25T00:17:27.562585+0000","last_undegraded":"2024-04-25T00:17:27.562585+0000","last_fullsized":"2024-04-25T00:17:27.562585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:44.840549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65506,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":516,"num_read_kb":325,"num_write":157,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351007,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":461900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1115277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776170,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":970752,"data_stored":462136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037808,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":462056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332148,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1053595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":461525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":523214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":462796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":524564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182475,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":461836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920838,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":461962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36880,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708272,"statfs":{"total":95995035648,"available":95957270528,"internally_reserved":0,"allocated":1576960,"data_stored":1053279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:17:52.434 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T00:17:52.950 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:52 smithi136 ceph-mon[32923]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:17:53.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:52 smithi106 ceph-mon[32510]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:17:53.105 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:52 smithi042 ceph-mon[28130]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:17:53.852 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:53 smithi042 ceph-mon[28130]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:53 smithi136 ceph-mon[32923]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:53 smithi106 ceph-mon[32510]: pgmap v361: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:17:55.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.331+0000 7faf08400700 1 -- 172.21.15.42:0/2282170664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00102ef0 msgr2=0x7faf00106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:55.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.331+0000 7faf08400700 1 --2- 172.21.15.42:0/2282170664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00102ef0 0x7faf00106ec0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7faef0009990 tx=0x7faef0009ca0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.331+0000 7faf08400700 1 -- 172.21.15.42:0/2282170664 shutdown_connections 2024-04-25T00:17:55.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.331+0000 7faf08400700 1 --2- 172.21.15.42:0/2282170664 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf001075a0 0x7faf0010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.331+0000 7faf08400700 1 --2- 172.21.15.42:0/2282170664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00102540 0x7faf00102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.331+0000 7faf08400700 1 --2- 172.21.15.42:0/2282170664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00102ef0 0x7faf00106ec0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.331+0000 7faf08400700 1 -- 172.21.15.42:0/2282170664 >> 172.21.15.42:0/2282170664 conn(0x7faf00079310 msgr2=0x7faf00079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:55.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.331+0000 7faf08400700 1 -- 172.21.15.42:0/2282170664 shutdown_connections 2024-04-25T00:17:55.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.331+0000 7faf08400700 1 -- 172.21.15.42:0/2282170664 wait complete. 2024-04-25T00:17:55.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.333+0000 7faf08400700 1 Processor -- start 2024-04-25T00:17:55.333 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.333+0000 7faf08400700 1 -- start start 2024-04-25T00:17:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.333+0000 7faf08400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00102540 0x7faf001a0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf08400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00102ef0 0x7faf001a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf08400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf001075a0 0x7faf0019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf08400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faf00112560 con 0x7faf00102ef0 2024-04-25T00:17:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf08400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faf001123e0 con 0x7faf00102540 2024-04-25T00:17:55.334 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf08400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf001126e0 con 0x7faf001075a0 2024-04-25T00:17:55.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf05000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00102ef0 0x7faf001a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:55.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf05a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00102540 0x7faf001a0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:55.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf05000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00102ef0 0x7faf001a0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52798/0 (socket says 172.21.15.42:52798) 2024-04-25T00:17:55.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf05000700 1 -- 172.21.15.42:0/734742174 learned_addr learned my addr 172.21.15.42:0/734742174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:55.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf06400700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf001075a0 0x7faf0019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf05000700 1 -- 172.21.15.42:0/734742174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf001075a0 msgr2=0x7faf0019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf05000700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf001075a0 0x7faf0019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf05000700 1 -- 172.21.15.42:0/734742174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00102540 msgr2=0x7faf001a0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf05000700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00102540 0x7faf001a0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf05000700 1 -- 172.21.15.42:0/734742174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0019b140 con 0x7faf00102ef0 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.334+0000 7faf05a00700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00102540 0x7faf001a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.335+0000 7faf05000700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00102ef0 0x7faf001a0d60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faef0000c00 tx=0x7faef000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.335+0000 7faefe000700 1 -- 172.21.15.42:0/734742174 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef0027070 con 0x7faf00102ef0 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.335+0000 7faefe000700 1 -- 172.21.15.42:0/734742174 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef0003d70 con 0x7faf00102ef0 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.335+0000 7faf08400700 1 -- 172.21.15.42:0/734742174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0019b3d0 con 0x7faf00102ef0 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.335+0000 7faefe000700 1 -- 172.21.15.42:0/734742174 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef000f460 con 0x7faf00102ef0 2024-04-25T00:17:55.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.335+0000 7faf08400700 1 -- 172.21.15.42:0/734742174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf001a76f0 con 0x7faf00102ef0 2024-04-25T00:17:55.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.336+0000 7faf08400700 1 -- 172.21.15.42:0/734742174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf00068f20 con 0x7faf00102ef0 2024-04-25T00:17:55.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.339+0000 7faefe000700 1 -- 172.21.15.42:0/734742174 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faef002a070 con 0x7faf00102ef0 2024-04-25T00:17:55.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.340+0000 7faefe000700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2cb0 0x7faee80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:55.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.340+0000 7faefe000700 1 -- 172.21.15.42:0/734742174 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7faef00cffe0 con 0x7faf00102ef0 2024-04-25T00:17:55.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.340+0000 7faf05a00700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2cb0 0x7faee80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:55.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.341+0000 7faf05a00700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2cb0 0x7faee80a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faf0019bb20 tx=0x7faef4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:55.344 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.345+0000 7faefe000700 1 -- 172.21.15.42:0/734742174 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7faef0013070 con 0x7faf00102ef0 2024-04-25T00:17:55.568 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.567+0000 7faf08400700 1 -- 172.21.15.42:0/734742174 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faf00075470 con 0x7faee80a2cb0 2024-04-25T00:17:55.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.577+0000 7faefe000700 1 -- 172.21.15.42:0/734742174 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258691 (secure 0 0 0) 0x7faf00075470 con 0x7faee80a2cb0 2024-04-25T00:17:55.581 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:17:55.583 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 -- 172.21.15.42:0/734742174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2cb0 msgr2=0x7faee80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2cb0 0x7faee80a5170 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7faf0019bb20 tx=0x7faef4009260 comp rx=0 tx=0).stop 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 -- 172.21.15.42:0/734742174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00102ef0 msgr2=0x7faf001a0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00102ef0 0x7faf001a0d60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7faef0000c00 tx=0x7faef000bab0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 -- 172.21.15.42:0/734742174 shutdown_connections 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2cb0 0x7faee80a5170 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf001075a0 0x7faf0019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00102540 0x7faf001a0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 --2- 172.21.15.42:0/734742174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00102ef0 0x7faf001a0d60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 -- 172.21.15.42:0/734742174 >> 172.21.15.42:0/734742174 conn(0x7faf00079310 msgr2=0x7faf001004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 -- 172.21.15.42:0/734742174 shutdown_connections 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:55.582+0000 7faf08400700 1 -- 172.21.15.42:0/734742174 wait complete. 2024-04-25T00:17:55.584 INFO:teuthology.orchestra.run.smithi042.stderr:dumped all 2024-04-25T00:17:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:55 smithi042 ceph-mon[28130]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:55 smithi136 ceph-mon[32923]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:55 smithi106 ceph-mon[32510]: pgmap v362: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:17:56.555 INFO:teuthology.orchestra.run.smithi042.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-25T00:17:53.948250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656108,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":672,"num_read_kb":449,"num_write":340,"num_write_kb":3048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":358372,"kb_used_data":13308,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124583452,"statfs":{"total":1151940427776,"available":1151573454848,"internally_reserved":0,"allocated":13627392,"data_stored":7504140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005494"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562925+0000","last_change":"2024-04-25T00:16:20.250230+0000","last_active":"2024-04-25T00:17:27.562925+0000","last_peered":"2024-04-25T00:17:27.562925+0000","last_clean":"2024-04-25T00:17:27.562925+0000","last_became_active":"2024-04-25T00:16:20.249751+0000","last_became_peered":"2024-04-25T00:16:20.249751+0000","last_unstale":"2024-04-25T00:17:27.562925+0000","last_undegraded":"2024-04-25T00:17:27.562925+0000","last_fullsized":"2024-04-25T00:17:27.562925+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:32:57.895905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.578052+0000","last_change":"2024-04-25T00:16:20.237934+0000","last_active":"2024-04-25T00:17:27.578052+0000","last_peered":"2024-04-25T00:17:27.578052+0000","last_clean":"2024-04-25T00:17:27.578052+0000","last_became_active":"2024-04-25T00:16:20.236039+0000","last_became_peered":"2024-04-25T00:16:20.236039+0000","last_unstale":"2024-04-25T00:17:27.578052+0000","last_undegraded":"2024-04-25T00:17:27.578052+0000","last_fullsized":"2024-04-25T00:17:27.578052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:07.730977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576639+0000","last_change":"2024-04-25T00:16:20.248923+0000","last_active":"2024-04-25T00:17:27.576639+0000","last_peered":"2024-04-25T00:17:27.576639+0000","last_clean":"2024-04-25T00:17:27.576639+0000","last_became_active":"2024-04-25T00:16:20.248078+0000","last_became_peered":"2024-04-25T00:16:20.248078+0000","last_unstale":"2024-04-25T00:17:27.576639+0000","last_undegraded":"2024-04-25T00:17:27.576639+0000","last_fullsized":"2024-04-25T00:17:27.576639+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:29:25.675449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.578926+0000","last_change":"2024-04-25T00:17:27.577956+0000","last_active":"2024-04-25T00:17:27.578926+0000","last_peered":"2024-04-25T00:17:27.578926+0000","last_clean":"2024-04-25T00:17:27.578926+0000","last_became_active":"2024-04-25T00:17:27.576377+0000","last_became_peered":"2024-04-25T00:17:27.576377+0000","last_unstale":"2024-04-25T00:17:27.578926+0000","last_undegraded":"2024-04-25T00:17:27.578926+0000","last_fullsized":"2024-04-25T00:17:27.578926+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:50.097326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,2],"acting":[9,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565029+0000","last_change":"2024-04-25T00:16:20.238311+0000","last_active":"2024-04-25T00:17:27.565029+0000","last_peered":"2024-04-25T00:17:27.565029+0000","last_clean":"2024-04-25T00:17:27.565029+0000","last_became_active":"2024-04-25T00:16:20.238104+0000","last_became_peered":"2024-04-25T00:16:20.238104+0000","last_unstale":"2024-04-25T00:17:27.565029+0000","last_undegraded":"2024-04-25T00:17:27.565029+0000","last_fullsized":"2024-04-25T00:17:27.565029+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:42.459941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563122+0000","last_change":"2024-04-25T00:16:20.228307+0000","last_active":"2024-04-25T00:17:27.563122+0000","last_peered":"2024-04-25T00:17:27.563122+0000","last_clean":"2024-04-25T00:17:27.563122+0000","last_became_active":"2024-04-25T00:16:20.227999+0000","last_became_peered":"2024-04-25T00:16:20.227999+0000","last_unstale":"2024-04-25T00:17:27.563122+0000","last_undegraded":"2024-04-25T00:17:27.563122+0000","last_fullsized":"2024-04-25T00:17:27.563122+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:57.910589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663179+0000","last_change":"2024-04-25T00:16:20.224382+0000","last_active":"2024-04-25T00:17:27.663179+0000","last_peered":"2024-04-25T00:17:27.663179+0000","last_clean":"2024-04-25T00:17:27.663179+0000","last_became_active":"2024-04-25T00:16:20.224122+0000","last_became_peered":"2024-04-25T00:16:20.224122+0000","last_unstale":"2024-04-25T00:17:27.663179+0000","last_undegraded":"2024-04-25T00:17:27.663179+0000","last_fullsized":"2024-04-25T00:17:27.663179+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:30:19.651330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.568028+0000","last_change":"2024-04-25T00:16:20.227720+0000","last_active":"2024-04-25T00:17:27.568028+0000","last_peered":"2024-04-25T00:17:27.568028+0000","last_clean":"2024-04-25T00:17:27.568028+0000","last_became_active":"2024-04-25T00:16:20.227263+0000","last_became_peered":"2024-04-25T00:16:20.227263+0000","last_unstale":"2024-04-25T00:17:27.568028+0000","last_undegraded":"2024-04-25T00:17:27.568028+0000","last_fullsized":"2024-04-25T00:17:27.568028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:03.709172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.567448+0000","last_change":"2024-04-25T00:16:20.217673+0000","last_active":"2024-04-25T00:17:27.567448+0000","last_peered":"2024-04-25T00:17:27.567448+0000","last_clean":"2024-04-25T00:17:27.567448+0000","last_became_active":"2024-04-25T00:16:20.216773+0000","last_became_peered":"2024-04-25T00:16:20.216773+0000","last_unstale":"2024-04-25T00:17:27.567448+0000","last_undegraded":"2024-04-25T00:17:27.567448+0000","last_fullsized":"2024-04-25T00:17:27.567448+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:29.623501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563471+0000","last_change":"2024-04-25T00:16:20.218460+0000","last_active":"2024-04-25T00:17:27.563471+0000","last_peered":"2024-04-25T00:17:27.563471+0000","last_clean":"2024-04-25T00:17:27.563471+0000","last_became_active":"2024-04-25T00:16:20.217198+0000","last_became_peered":"2024-04-25T00:16:20.217198+0000","last_unstale":"2024-04-25T00:17:27.563471+0000","last_undegraded":"2024-04-25T00:17:27.563471+0000","last_fullsized":"2024-04-25T00:17:27.563471+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:06.511749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663982+0000","last_change":"2024-04-25T00:16:20.221990+0000","last_active":"2024-04-25T00:17:27.663982+0000","last_peered":"2024-04-25T00:17:27.663982+0000","last_clean":"2024-04-25T00:17:27.663982+0000","last_became_active":"2024-04-25T00:16:20.221635+0000","last_became_peered":"2024-04-25T00:16:20.221635+0000","last_unstale":"2024-04-25T00:17:27.663982+0000","last_undegraded":"2024-04-25T00:17:27.663982+0000","last_fullsized":"2024-04-25T00:17:27.663982+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:20:31.205135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664220+0000","last_change":"2024-04-25T00:16:20.219923+0000","last_active":"2024-04-25T00:17:27.664220+0000","last_peered":"2024-04-25T00:17:27.664220+0000","last_clean":"2024-04-25T00:17:27.664220+0000","last_became_active":"2024-04-25T00:16:20.219578+0000","last_became_peered":"2024-04-25T00:16:20.219578+0000","last_unstale":"2024-04-25T00:17:27.664220+0000","last_undegraded":"2024-04-25T00:17:27.664220+0000","last_fullsized":"2024-04-25T00:17:27.664220+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:41.811382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576508+0000","last_change":"2024-04-25T00:16:20.240570+0000","last_active":"2024-04-25T00:17:27.576508+0000","last_peered":"2024-04-25T00:17:27.576508+0000","last_clean":"2024-04-25T00:17:27.576508+0000","last_became_active":"2024-04-25T00:16:20.238241+0000","last_became_peered":"2024-04-25T00:16:20.238241+0000","last_unstale":"2024-04-25T00:17:27.576508+0000","last_undegraded":"2024-04-25T00:17:27.576508+0000","last_fullsized":"2024-04-25T00:17:27.576508+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:35:06.981816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576418+0000","last_change":"2024-04-25T00:16:20.234748+0000","last_active":"2024-04-25T00:17:27.576418+0000","last_peered":"2024-04-25T00:17:27.576418+0000","last_clean":"2024-04-25T00:17:27.576418+0000","last_became_active":"2024-04-25T00:16:20.234325+0000","last_became_peered":"2024-04-25T00:16:20.234325+0000","last_unstale":"2024-04-25T00:17:27.576418+0000","last_undegraded":"2024-04-25T00:17:27.576418+0000","last_fullsized":"2024-04-25T00:17:27.576418+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:26:00.383533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576099+0000","last_change":"2024-04-25T00:16:20.242987+0000","last_active":"2024-04-25T00:17:27.576099+0000","last_peered":"2024-04-25T00:17:27.576099+0000","last_clean":"2024-04-25T00:17:27.576099+0000","last_became_active":"2024-04-25T00:16:20.242625+0000","last_became_peered":"2024-04-25T00:16:20.242625+0000","last_unstale":"2024-04-25T00:17:27.576099+0000","last_undegraded":"2024-04-25T00:17:27.576099+0000","last_fullsized":"2024-04-25T00:17:27.576099+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:50:16.618120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566927+0000","last_change":"2024-04-25T00:16:20.241985+0000","last_active":"2024-04-25T00:17:27.566927+0000","last_peered":"2024-04-25T00:17:27.566927+0000","last_clean":"2024-04-25T00:17:27.566927+0000","last_became_active":"2024-04-25T00:16:20.241646+0000","last_became_peered":"2024-04-25T00:16:20.241646+0000","last_unstale":"2024-04-25T00:17:27.566927+0000","last_undegraded":"2024-04-25T00:17:27.566927+0000","last_fullsized":"2024-04-25T00:17:27.566927+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:21:44.094705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563042+0000","last_change":"2024-04-25T00:16:20.250161+0000","last_active":"2024-04-25T00:17:27.563042+0000","last_peered":"2024-04-25T00:17:27.563042+0000","last_clean":"2024-04-25T00:17:27.563042+0000","last_became_active":"2024-04-25T00:16:20.249811+0000","last_became_peered":"2024-04-25T00:16:20.249811+0000","last_unstale":"2024-04-25T00:17:27.563042+0000","last_undegraded":"2024-04-25T00:17:27.563042+0000","last_fullsized":"2024-04-25T00:17:27.563042+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:03.222602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.571145+0000","last_change":"2024-04-25T00:16:20.220859+0000","last_active":"2024-04-25T00:17:27.571145+0000","last_peered":"2024-04-25T00:17:27.571145+0000","last_clean":"2024-04-25T00:17:27.571145+0000","last_became_active":"2024-04-25T00:16:20.219864+0000","last_became_peered":"2024-04-25T00:16:20.219864+0000","last_unstale":"2024-04-25T00:17:27.571145+0000","last_undegraded":"2024-04-25T00:17:27.571145+0000","last_fullsized":"2024-04-25T00:17:27.571145+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:23.830726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566012+0000","last_change":"2024-04-25T00:16:20.249293+0000","last_active":"2024-04-25T00:17:27.566012+0000","last_peered":"2024-04-25T00:17:27.566012+0000","last_clean":"2024-04-25T00:17:27.566012+0000","last_became_active":"2024-04-25T00:16:20.248952+0000","last_became_peered":"2024-04-25T00:16:20.248952+0000","last_unstale":"2024-04-25T00:17:27.566012+0000","last_undegraded":"2024-04-25T00:17:27.566012+0000","last_fullsized":"2024-04-25T00:17:27.566012+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:02.124426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'9","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.571590+0000","last_change":"2024-04-25T00:16:19.310971+0000","last_active":"2024-04-25T00:17:27.571590+0000","last_peered":"2024-04-25T00:17:27.571590+0000","last_clean":"2024-04-25T00:17:27.571590+0000","last_became_active":"2024-04-25T00:16:19.309020+0000","last_became_peered":"2024-04-25T00:16:19.309020+0000","last_unstale":"2024-04-25T00:17:27.571590+0000","last_undegraded":"2024-04-25T00:17:27.571590+0000","last_fullsized":"2024-04-25T00:17:27.571590+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:37:56.744138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.574655+0000","last_change":"2024-04-25T00:16:20.250665+0000","last_active":"2024-04-25T00:17:27.574655+0000","last_peered":"2024-04-25T00:17:27.574655+0000","last_clean":"2024-04-25T00:17:27.574655+0000","last_became_active":"2024-04-25T00:16:20.250338+0000","last_became_peered":"2024-04-25T00:16:20.250338+0000","last_unstale":"2024-04-25T00:17:27.574655+0000","last_undegraded":"2024-04-25T00:17:27.574655+0000","last_fullsized":"2024-04-25T00:17:27.574655+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:28:17.018035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.574776+0000","last_change":"2024-04-25T00:16:19.327748+0000","last_active":"2024-04-25T00:17:27.574776+0000","last_peered":"2024-04-25T00:17:27.574776+0000","last_clean":"2024-04-25T00:17:27.574776+0000","last_became_active":"2024-04-25T00:16:19.327264+0000","last_became_peered":"2024-04-25T00:16:19.327264+0000","last_unstale":"2024-04-25T00:17:27.574776+0000","last_undegraded":"2024-04-25T00:17:27.574776+0000","last_fullsized":"2024-04-25T00:17:27.574776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:45:27.080423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.578504+0000","last_change":"2024-04-25T00:16:20.243061+0000","last_active":"2024-04-25T00:17:27.578504+0000","last_peered":"2024-04-25T00:17:27.578504+0000","last_clean":"2024-04-25T00:17:27.578504+0000","last_became_active":"2024-04-25T00:16:20.242697+0000","last_became_peered":"2024-04-25T00:16:20.242697+0000","last_unstale":"2024-04-25T00:17:27.578504+0000","last_undegraded":"2024-04-25T00:17:27.578504+0000","last_fullsized":"2024-04-25T00:17:27.578504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:57.734902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565495+0000","last_change":"2024-04-25T00:16:19.317122+0000","last_active":"2024-04-25T00:17:27.565495+0000","last_peered":"2024-04-25T00:17:27.565495+0000","last_clean":"2024-04-25T00:17:27.565495+0000","last_became_active":"2024-04-25T00:16:19.316754+0000","last_became_peered":"2024-04-25T00:16:19.316754+0000","last_unstale":"2024-04-25T00:17:27.565495+0000","last_undegraded":"2024-04-25T00:17:27.565495+0000","last_fullsized":"2024-04-25T00:17:27.565495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:47:28.088944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566997+0000","last_change":"2024-04-25T00:16:20.238611+0000","last_active":"2024-04-25T00:17:27.566997+0000","last_peered":"2024-04-25T00:17:27.566997+0000","last_clean":"2024-04-25T00:17:27.566997+0000","last_became_active":"2024-04-25T00:16:20.238257+0000","last_became_peered":"2024-04-25T00:16:20.238257+0000","last_unstale":"2024-04-25T00:17:27.566997+0000","last_undegraded":"2024-04-25T00:17:27.566997+0000","last_fullsized":"2024-04-25T00:17:27.566997+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:13.490284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565486+0000","last_change":"2024-04-25T00:16:19.308565+0000","last_active":"2024-04-25T00:17:27.565486+0000","last_peered":"2024-04-25T00:17:27.565486+0000","last_clean":"2024-04-25T00:17:27.565486+0000","last_became_active":"2024-04-25T00:16:19.308220+0000","last_became_peered":"2024-04-25T00:16:19.308220+0000","last_unstale":"2024-04-25T00:17:27.565486+0000","last_undegraded":"2024-04-25T00:17:27.565486+0000","last_fullsized":"2024-04-25T00:17:27.565486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:33.215491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.579625+0000","last_change":"2024-04-25T00:17:27.579625+0000","last_active":"2024-04-25T00:17:27.579625+0000","last_peered":"2024-04-25T00:17:27.579625+0000","last_clean":"2024-04-25T00:17:27.579625+0000","last_became_active":"2024-04-25T00:17:27.579200+0000","last_became_peered":"2024-04-25T00:17:27.579200+0000","last_unstale":"2024-04-25T00:17:27.579625+0000","last_undegraded":"2024-04-25T00:17:27.579625+0000","last_fullsized":"2024-04-25T00:17:27.579625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:41:34.927720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,1],"acting":[5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":114,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:50.438996+0000","last_change":"2024-04-25T00:16:19.312725+0000","last_active":"2024-04-25T00:17:50.438996+0000","last_peered":"2024-04-25T00:17:50.438996+0000","last_clean":"2024-04-25T00:17:50.438996+0000","last_became_active":"2024-04-25T00:16:19.312115+0000","last_became_peered":"2024-04-25T00:16:19.312115+0000","last_unstale":"2024-04-25T00:17:50.438996+0000","last_undegraded":"2024-04-25T00:17:50.438996+0000","last_fullsized":"2024-04-25T00:17:50.438996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:13.661132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":90,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565299+0000","last_change":"2024-04-25T00:16:20.233546+0000","last_active":"2024-04-25T00:17:27.565299+0000","last_peered":"2024-04-25T00:17:27.565299+0000","last_clean":"2024-04-25T00:17:27.565299+0000","last_became_active":"2024-04-25T00:16:20.233029+0000","last_became_peered":"2024-04-25T00:16:20.233029+0000","last_unstale":"2024-04-25T00:17:27.565299+0000","last_undegraded":"2024-04-25T00:17:27.565299+0000","last_fullsized":"2024-04-25T00:17:27.565299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:42:44.068856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563038+0000","last_change":"2024-04-25T00:16:19.307512+0000","last_active":"2024-04-25T00:17:27.563038+0000","last_peered":"2024-04-25T00:17:27.563038+0000","last_clean":"2024-04-25T00:17:27.563038+0000","last_became_active":"2024-04-25T00:16:19.307164+0000","last_became_peered":"2024-04-25T00:16:19.307164+0000","last_unstale":"2024-04-25T00:17:27.563038+0000","last_undegraded":"2024-04-25T00:17:27.563038+0000","last_fullsized":"2024-04-25T00:17:27.563038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:37.560178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569282+0000","last_change":"2024-04-25T00:16:20.232758+0000","last_active":"2024-04-25T00:17:27.569282+0000","last_peered":"2024-04-25T00:17:27.569282+0000","last_clean":"2024-04-25T00:17:27.569282+0000","last_became_active":"2024-04-25T00:16:20.232443+0000","last_became_peered":"2024-04-25T00:16:20.232443+0000","last_unstale":"2024-04-25T00:17:27.569282+0000","last_undegraded":"2024-04-25T00:17:27.569282+0000","last_fullsized":"2024-04-25T00:17:27.569282+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:26:31.029585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576842+0000","last_change":"2024-04-25T00:16:19.313924+0000","last_active":"2024-04-25T00:17:27.576842+0000","last_peered":"2024-04-25T00:17:27.576842+0000","last_clean":"2024-04-25T00:17:27.576842+0000","last_became_active":"2024-04-25T00:16:19.313570+0000","last_became_peered":"2024-04-25T00:16:19.313570+0000","last_unstale":"2024-04-25T00:17:27.576842+0000","last_undegraded":"2024-04-25T00:17:27.576842+0000","last_fullsized":"2024-04-25T00:17:27.576842+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:42.258910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.574393+0000","last_change":"2024-04-25T00:16:20.244150+0000","last_active":"2024-04-25T00:17:27.574393+0000","last_peered":"2024-04-25T00:17:27.574393+0000","last_clean":"2024-04-25T00:17:27.574393+0000","last_became_active":"2024-04-25T00:16:20.243802+0000","last_became_peered":"2024-04-25T00:16:20.243802+0000","last_unstale":"2024-04-25T00:17:27.574393+0000","last_undegraded":"2024-04-25T00:17:27.574393+0000","last_fullsized":"2024-04-25T00:17:27.574393+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:02:10.192507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569054+0000","last_change":"2024-04-25T00:16:19.312361+0000","last_active":"2024-04-25T00:17:27.569054+0000","last_peered":"2024-04-25T00:17:27.569054+0000","last_clean":"2024-04-25T00:17:27.569054+0000","last_became_active":"2024-04-25T00:16:19.312031+0000","last_became_peered":"2024-04-25T00:16:19.312031+0000","last_unstale":"2024-04-25T00:17:27.569054+0000","last_undegraded":"2024-04-25T00:17:27.569054+0000","last_fullsized":"2024-04-25T00:17:27.569054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:23:10.062091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566803+0000","last_change":"2024-04-25T00:16:20.231399+0000","last_active":"2024-04-25T00:17:27.566803+0000","last_peered":"2024-04-25T00:17:27.566803+0000","last_clean":"2024-04-25T00:17:27.566803+0000","last_became_active":"2024-04-25T00:16:20.230954+0000","last_became_peered":"2024-04-25T00:16:20.230954+0000","last_unstale":"2024-04-25T00:17:27.566803+0000","last_undegraded":"2024-04-25T00:17:27.566803+0000","last_fullsized":"2024-04-25T00:17:27.566803+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:49:20.934927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":108,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:48.435096+0000","last_change":"2024-04-25T00:16:19.308591+0000","last_active":"2024-04-25T00:17:48.435096+0000","last_peered":"2024-04-25T00:17:48.435096+0000","last_clean":"2024-04-25T00:17:48.435096+0000","last_became_active":"2024-04-25T00:16:19.307987+0000","last_became_peered":"2024-04-25T00:16:19.307987+0000","last_unstale":"2024-04-25T00:17:48.435096+0000","last_undegraded":"2024-04-25T00:17:48.435096+0000","last_fullsized":"2024-04-25T00:17:48.435096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:32.739170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.567391+0000","last_change":"2024-04-25T00:16:20.218545+0000","last_active":"2024-04-25T00:17:27.567391+0000","last_peered":"2024-04-25T00:17:27.567391+0000","last_clean":"2024-04-25T00:17:27.567391+0000","last_became_active":"2024-04-25T00:16:20.218267+0000","last_became_peered":"2024-04-25T00:16:20.218267+0000","last_unstale":"2024-04-25T00:17:27.567391+0000","last_undegraded":"2024-04-25T00:17:27.567391+0000","last_fullsized":"2024-04-25T00:17:27.567391+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:36:47.725985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664114+0000","last_change":"2024-04-25T00:16:19.311918+0000","last_active":"2024-04-25T00:17:27.664114+0000","last_peered":"2024-04-25T00:17:27.664114+0000","last_clean":"2024-04-25T00:17:27.664114+0000","last_became_active":"2024-04-25T00:16:19.311594+0000","last_became_peered":"2024-04-25T00:16:19.311594+0000","last_unstale":"2024-04-25T00:17:27.664114+0000","last_undegraded":"2024-04-25T00:17:27.664114+0000","last_fullsized":"2024-04-25T00:17:27.664114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:38:04.574190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.577206+0000","last_change":"2024-04-25T00:16:20.248116+0000","last_active":"2024-04-25T00:17:27.577206+0000","last_peered":"2024-04-25T00:17:27.577206+0000","last_clean":"2024-04-25T00:17:27.577206+0000","last_became_active":"2024-04-25T00:16:20.247373+0000","last_became_peered":"2024-04-25T00:16:20.247373+0000","last_unstale":"2024-04-25T00:17:27.577206+0000","last_undegraded":"2024-04-25T00:17:27.577206+0000","last_fullsized":"2024-04-25T00:17:27.577206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:16:47.777779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569627+0000","last_change":"2024-04-25T00:16:19.320640+0000","last_active":"2024-04-25T00:17:27.569627+0000","last_peered":"2024-04-25T00:17:27.569627+0000","last_clean":"2024-04-25T00:17:27.569627+0000","last_became_active":"2024-04-25T00:16:19.320273+0000","last_became_peered":"2024-04-25T00:16:19.320273+0000","last_unstale":"2024-04-25T00:17:27.569627+0000","last_undegraded":"2024-04-25T00:17:27.569627+0000","last_fullsized":"2024-04-25T00:17:27.569627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:17:10.843162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.567123+0000","last_change":"2024-04-25T00:16:20.219378+0000","last_active":"2024-04-25T00:17:27.567123+0000","last_peered":"2024-04-25T00:17:27.567123+0000","last_clean":"2024-04-25T00:17:27.567123+0000","last_became_active":"2024-04-25T00:16:20.219069+0000","last_became_peered":"2024-04-25T00:16:20.219069+0000","last_unstale":"2024-04-25T00:17:27.567123+0000","last_undegraded":"2024-04-25T00:17:27.567123+0000","last_fullsized":"2024-04-25T00:17:27.567123+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:10.818609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'28","reported_seq":227,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:49.451702+0000","last_change":"2024-04-25T00:16:19.315089+0000","last_active":"2024-04-25T00:17:49.451702+0000","last_peered":"2024-04-25T00:17:49.451702+0000","last_clean":"2024-04-25T00:17:49.451702+0000","last_became_active":"2024-04-25T00:16:19.314416+0000","last_became_peered":"2024-04-25T00:16:19.314416+0000","last_unstale":"2024-04-25T00:17:49.451702+0000","last_undegraded":"2024-04-25T00:17:49.451702+0000","last_fullsized":"2024-04-25T00:17:49.451702+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:56:25.920010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61896,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":180,"num_read_kb":150,"num_write":29,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565245+0000","last_change":"2024-04-25T00:16:20.225678+0000","last_active":"2024-04-25T00:17:27.565245+0000","last_peered":"2024-04-25T00:17:27.565245+0000","last_clean":"2024-04-25T00:17:27.565245+0000","last_became_active":"2024-04-25T00:16:20.225146+0000","last_became_peered":"2024-04-25T00:16:20.225146+0000","last_unstale":"2024-04-25T00:17:27.565245+0000","last_undegraded":"2024-04-25T00:17:27.565245+0000","last_fullsized":"2024-04-25T00:17:27.565245+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:02:56.003701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'6","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566115+0000","last_change":"2024-04-25T00:16:19.327877+0000","last_active":"2024-04-25T00:17:27.566115+0000","last_peered":"2024-04-25T00:17:27.566115+0000","last_clean":"2024-04-25T00:17:27.566115+0000","last_became_active":"2024-04-25T00:16:19.327537+0000","last_became_peered":"2024-04-25T00:16:19.327537+0000","last_unstale":"2024-04-25T00:17:27.566115+0000","last_undegraded":"2024-04-25T00:17:27.566115+0000","last_fullsized":"2024-04-25T00:17:27.566115+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:14.614232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.667026+0000","last_change":"2024-04-25T00:17:27.667026+0000","last_active":"2024-04-25T00:17:27.667026+0000","last_peered":"2024-04-25T00:17:27.667026+0000","last_clean":"2024-04-25T00:17:27.667026+0000","last_became_active":"2024-04-25T00:17:27.665718+0000","last_became_peered":"2024-04-25T00:17:27.665718+0000","last_unstale":"2024-04-25T00:17:27.667026+0000","last_undegraded":"2024-04-25T00:17:27.667026+0000","last_fullsized":"2024-04-25T00:17:27.667026+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:37.297615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,9],"acting":[6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569400+0000","last_change":"2024-04-25T00:16:19.328428+0000","last_active":"2024-04-25T00:17:27.569400+0000","last_peered":"2024-04-25T00:17:27.569400+0000","last_clean":"2024-04-25T00:17:27.569400+0000","last_became_active":"2024-04-25T00:16:19.328112+0000","last_became_peered":"2024-04-25T00:16:19.328112+0000","last_unstale":"2024-04-25T00:17:27.569400+0000","last_undegraded":"2024-04-25T00:17:27.569400+0000","last_fullsized":"2024-04-25T00:17:27.569400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:19:09.130246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663884+0000","last_change":"2024-04-25T00:16:20.237922+0000","last_active":"2024-04-25T00:17:27.663884+0000","last_peered":"2024-04-25T00:17:27.663884+0000","last_clean":"2024-04-25T00:17:27.663884+0000","last_became_active":"2024-04-25T00:16:20.237594+0000","last_became_peered":"2024-04-25T00:16:20.237594+0000","last_unstale":"2024-04-25T00:17:27.663884+0000","last_undegraded":"2024-04-25T00:17:27.663884+0000","last_fullsized":"2024-04-25T00:17:27.663884+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:09:26.517335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'3","reported_seq":112,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:51.517232+0000","last_change":"2024-04-25T00:16:19.312920+0000","last_active":"2024-04-25T00:17:51.517232+0000","last_peered":"2024-04-25T00:17:51.517232+0000","last_clean":"2024-04-25T00:17:51.517232+0000","last_became_active":"2024-04-25T00:16:19.312548+0000","last_became_peered":"2024-04-25T00:16:19.312548+0000","last_unstale":"2024-04-25T00:17:51.517232+0000","last_undegraded":"2024-04-25T00:17:51.517232+0000","last_fullsized":"2024-04-25T00:17:51.517232+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:50:47.154402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":91,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663212+0000","last_change":"2024-04-25T00:16:20.221102+0000","last_active":"2024-04-25T00:17:27.663212+0000","last_peered":"2024-04-25T00:17:27.663212+0000","last_clean":"2024-04-25T00:17:27.663212+0000","last_became_active":"2024-04-25T00:16:20.220753+0000","last_became_peered":"2024-04-25T00:16:20.220753+0000","last_unstale":"2024-04-25T00:17:27.663212+0000","last_undegraded":"2024-04-25T00:17:27.663212+0000","last_fullsized":"2024-04-25T00:17:27.663212+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:18.652865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'15","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565165+0000","last_change":"2024-04-25T00:16:19.324181+0000","last_active":"2024-04-25T00:17:27.565165+0000","last_peered":"2024-04-25T00:17:27.565165+0000","last_clean":"2024-04-25T00:17:27.565165+0000","last_became_active":"2024-04-25T00:16:19.323821+0000","last_became_peered":"2024-04-25T00:16:19.323821+0000","last_unstale":"2024-04-25T00:17:27.565165+0000","last_undegraded":"2024-04-25T00:17:27.565165+0000","last_fullsized":"2024-04-25T00:17:27.565165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:34.323961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.571552+0000","last_change":"2024-04-25T00:17:27.571552+0000","last_active":"2024-04-25T00:17:27.571552+0000","last_peered":"2024-04-25T00:17:27.571552+0000","last_clean":"2024-04-25T00:17:27.571552+0000","last_became_active":"2024-04-25T00:17:27.571087+0000","last_became_peered":"2024-04-25T00:17:27.571087+0000","last_unstale":"2024-04-25T00:17:27.571552+0000","last_undegraded":"2024-04-25T00:17:27.571552+0000","last_fullsized":"2024-04-25T00:17:27.571552+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:27:10.459526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565067+0000","last_change":"2024-04-25T00:16:19.306005+0000","last_active":"2024-04-25T00:17:27.565067+0000","last_peered":"2024-04-25T00:17:27.565067+0000","last_clean":"2024-04-25T00:17:27.565067+0000","last_became_active":"2024-04-25T00:16:19.305656+0000","last_became_peered":"2024-04-25T00:16:19.305656+0000","last_unstale":"2024-04-25T00:17:27.565067+0000","last_undegraded":"2024-04-25T00:17:27.565067+0000","last_fullsized":"2024-04-25T00:17:27.565067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:57:16.452352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562723+0000","last_change":"2024-04-25T00:16:20.217886+0000","last_active":"2024-04-25T00:17:27.562723+0000","last_peered":"2024-04-25T00:17:27.562723+0000","last_clean":"2024-04-25T00:17:27.562723+0000","last_became_active":"2024-04-25T00:16:20.217379+0000","last_became_peered":"2024-04-25T00:16:20.217379+0000","last_unstale":"2024-04-25T00:17:27.562723+0000","last_undegraded":"2024-04-25T00:17:27.562723+0000","last_fullsized":"2024-04-25T00:17:27.562723+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:23:22.028791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.568038+0000","last_change":"2024-04-25T00:16:19.328466+0000","last_active":"2024-04-25T00:17:27.568038+0000","last_peered":"2024-04-25T00:17:27.568038+0000","last_clean":"2024-04-25T00:17:27.568038+0000","last_became_active":"2024-04-25T00:16:19.321077+0000","last_became_peered":"2024-04-25T00:16:19.321077+0000","last_unstale":"2024-04-25T00:17:27.568038+0000","last_undegraded":"2024-04-25T00:17:27.568038+0000","last_fullsized":"2024-04-25T00:17:27.568038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:43:55.296160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563377+0000","last_change":"2024-04-25T00:16:20.249243+0000","last_active":"2024-04-25T00:17:27.563377+0000","last_peered":"2024-04-25T00:17:27.563377+0000","last_clean":"2024-04-25T00:17:27.563377+0000","last_became_active":"2024-04-25T00:16:20.248955+0000","last_became_peered":"2024-04-25T00:16:20.248955+0000","last_unstale":"2024-04-25T00:17:27.563377+0000","last_undegraded":"2024-04-25T00:17:27.563377+0000","last_fullsized":"2024-04-25T00:17:27.563377+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:05:14.195688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664197+0000","last_change":"2024-04-25T00:16:19.316409+0000","last_active":"2024-04-25T00:17:27.664197+0000","last_peered":"2024-04-25T00:17:27.664197+0000","last_clean":"2024-04-25T00:17:27.664197+0000","last_became_active":"2024-04-25T00:16:19.315982+0000","last_became_peered":"2024-04-25T00:16:19.315982+0000","last_unstale":"2024-04-25T00:17:27.664197+0000","last_undegraded":"2024-04-25T00:17:27.664197+0000","last_fullsized":"2024-04-25T00:17:27.664197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:01.718600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.563235+0000","last_change":"2024-04-25T00:16:20.242119+0000","last_active":"2024-04-25T00:17:27.563235+0000","last_peered":"2024-04-25T00:17:27.563235+0000","last_clean":"2024-04-25T00:17:27.563235+0000","last_became_active":"2024-04-25T00:16:20.241702+0000","last_became_peered":"2024-04-25T00:16:20.241702+0000","last_unstale":"2024-04-25T00:17:27.563235+0000","last_undegraded":"2024-04-25T00:17:27.563235+0000","last_fullsized":"2024-04-25T00:17:27.563235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:33:26.652681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.668759+0000","last_change":"2024-04-25T00:17:27.667027+0000","last_active":"2024-04-25T00:17:27.668759+0000","last_peered":"2024-04-25T00:17:27.668759+0000","last_clean":"2024-04-25T00:17:27.668759+0000","last_became_active":"2024-04-25T00:17:27.665152+0000","last_became_peered":"2024-04-25T00:17:27.665152+0000","last_unstale":"2024-04-25T00:17:27.668759+0000","last_undegraded":"2024-04-25T00:17:27.668759+0000","last_fullsized":"2024-04-25T00:17:27.668759+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T07:58:10.679692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"86'4","reported_seq":109,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:49.512077+0000","last_change":"2024-04-25T00:16:19.328872+0000","last_active":"2024-04-25T00:17:49.512077+0000","last_peered":"2024-04-25T00:17:49.512077+0000","last_clean":"2024-04-25T00:17:49.512077+0000","last_became_active":"2024-04-25T00:16:19.328439+0000","last_became_peered":"2024-04-25T00:16:19.328439+0000","last_unstale":"2024-04-25T00:17:49.512077+0000","last_undegraded":"2024-04-25T00:17:49.512077+0000","last_fullsized":"2024-04-25T00:17:49.512077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:05.832328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":86,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.570225+0000","last_change":"2024-04-25T00:16:20.242857+0000","last_active":"2024-04-25T00:17:27.570225+0000","last_peered":"2024-04-25T00:17:27.570225+0000","last_clean":"2024-04-25T00:17:27.570225+0000","last_became_active":"2024-04-25T00:16:20.240941+0000","last_became_peered":"2024-04-25T00:16:20.240941+0000","last_unstale":"2024-04-25T00:17:27.570225+0000","last_undegraded":"2024-04-25T00:17:27.570225+0000","last_fullsized":"2024-04-25T00:17:27.570225+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:30:36.550571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'3","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663465+0000","last_change":"2024-04-25T00:16:19.312914+0000","last_active":"2024-04-25T00:17:27.663465+0000","last_peered":"2024-04-25T00:17:27.663465+0000","last_clean":"2024-04-25T00:17:27.663465+0000","last_became_active":"2024-04-25T00:16:19.312578+0000","last_became_peered":"2024-04-25T00:16:19.312578+0000","last_unstale":"2024-04-25T00:17:27.663465+0000","last_undegraded":"2024-04-25T00:17:27.663465+0000","last_fullsized":"2024-04-25T00:17:27.663465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:15:37.035904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663384+0000","last_change":"2024-04-25T00:15:03.261320+0000","last_active":"2024-04-25T00:17:27.663384+0000","last_peered":"2024-04-25T00:17:27.663384+0000","last_clean":"2024-04-25T00:17:27.663384+0000","last_became_active":"2024-04-25T00:15:03.260804+0000","last_became_peered":"2024-04-25T00:15:03.260804+0000","last_unstale":"2024-04-25T00:17:27.663384+0000","last_undegraded":"2024-04-25T00:17:27.663384+0000","last_fullsized":"2024-04-25T00:17:27.663384+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:11:14.473933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:11:14.473933+0000","last_clean_scrub_stamp":"2024-04-25T00:11:14.473933+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:35.876949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576567+0000","last_change":"2024-04-25T00:16:20.238253+0000","last_active":"2024-04-25T00:17:27.576567+0000","last_peered":"2024-04-25T00:17:27.576567+0000","last_clean":"2024-04-25T00:17:27.576567+0000","last_became_active":"2024-04-25T00:16:20.237824+0000","last_became_peered":"2024-04-25T00:16:20.237824+0000","last_unstale":"2024-04-25T00:17:27.576567+0000","last_undegraded":"2024-04-25T00:17:27.576567+0000","last_fullsized":"2024-04-25T00:17:27.576567+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:12:21.996513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663453+0000","last_change":"2024-04-25T00:16:19.316194+0000","last_active":"2024-04-25T00:17:27.663453+0000","last_peered":"2024-04-25T00:17:27.663453+0000","last_clean":"2024-04-25T00:17:27.663453+0000","last_became_active":"2024-04-25T00:16:19.315625+0000","last_became_peered":"2024-04-25T00:16:19.315625+0000","last_unstale":"2024-04-25T00:17:27.663453+0000","last_undegraded":"2024-04-25T00:17:27.663453+0000","last_fullsized":"2024-04-25T00:17:27.663453+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:23.426048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565379+0000","last_change":"2024-04-25T00:16:19.324283+0000","last_active":"2024-04-25T00:17:27.565379+0000","last_peered":"2024-04-25T00:17:27.565379+0000","last_clean":"2024-04-25T00:17:27.565379+0000","last_became_active":"2024-04-25T00:16:19.323800+0000","last_became_peered":"2024-04-25T00:16:19.323800+0000","last_unstale":"2024-04-25T00:17:27.565379+0000","last_undegraded":"2024-04-25T00:17:27.565379+0000","last_fullsized":"2024-04-25T00:17:27.565379+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:01:43.592775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.577063+0000","last_change":"2024-04-25T00:16:20.239205+0000","last_active":"2024-04-25T00:17:27.577063+0000","last_peered":"2024-04-25T00:17:27.577063+0000","last_clean":"2024-04-25T00:17:27.577063+0000","last_became_active":"2024-04-25T00:16:20.237135+0000","last_became_peered":"2024-04-25T00:16:20.237135+0000","last_unstale":"2024-04-25T00:17:27.577063+0000","last_undegraded":"2024-04-25T00:17:27.577063+0000","last_fullsized":"2024-04-25T00:17:27.577063+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:37.348227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'9","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.571272+0000","last_change":"2024-04-25T00:16:19.325868+0000","last_active":"2024-04-25T00:17:27.571272+0000","last_peered":"2024-04-25T00:17:27.571272+0000","last_clean":"2024-04-25T00:17:27.571272+0000","last_became_active":"2024-04-25T00:16:19.322820+0000","last_became_peered":"2024-04-25T00:16:19.322820+0000","last_unstale":"2024-04-25T00:17:27.571272+0000","last_undegraded":"2024-04-25T00:17:27.571272+0000","last_fullsized":"2024-04-25T00:17:27.571272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:55:50.982099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.574946+0000","last_change":"2024-04-25T00:16:20.248268+0000","last_active":"2024-04-25T00:17:27.574946+0000","last_peered":"2024-04-25T00:17:27.574946+0000","last_clean":"2024-04-25T00:17:27.574946+0000","last_became_active":"2024-04-25T00:16:20.247274+0000","last_became_peered":"2024-04-25T00:16:20.247274+0000","last_unstale":"2024-04-25T00:17:27.574946+0000","last_undegraded":"2024-04-25T00:17:27.574946+0000","last_fullsized":"2024-04-25T00:17:27.574946+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:15:46.120277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664425+0000","last_change":"2024-04-25T00:16:19.317256+0000","last_active":"2024-04-25T00:17:27.664425+0000","last_peered":"2024-04-25T00:17:27.664425+0000","last_clean":"2024-04-25T00:17:27.664425+0000","last_became_active":"2024-04-25T00:16:19.316931+0000","last_became_peered":"2024-04-25T00:16:19.316931+0000","last_unstale":"2024-04-25T00:17:27.664425+0000","last_undegraded":"2024-04-25T00:17:27.664425+0000","last_fullsized":"2024-04-25T00:17:27.664425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:49.685286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.577437+0000","last_change":"2024-04-25T00:16:20.252271+0000","last_active":"2024-04-25T00:17:27.577437+0000","last_peered":"2024-04-25T00:17:27.577437+0000","last_clean":"2024-04-25T00:17:27.577437+0000","last_became_active":"2024-04-25T00:16:20.251930+0000","last_became_peered":"2024-04-25T00:16:20.251930+0000","last_unstale":"2024-04-25T00:17:27.577437+0000","last_undegraded":"2024-04-25T00:17:27.577437+0000","last_fullsized":"2024-04-25T00:17:27.577437+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:07.313172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.584129+0000","last_change":"2024-04-25T00:17:27.584129+0000","last_active":"2024-04-25T00:17:27.584129+0000","last_peered":"2024-04-25T00:17:27.584129+0000","last_clean":"2024-04-25T00:17:27.584129+0000","last_became_active":"2024-04-25T00:17:27.580604+0000","last_became_peered":"2024-04-25T00:17:27.580604+0000","last_unstale":"2024-04-25T00:17:27.584129+0000","last_undegraded":"2024-04-25T00:17:27.584129+0000","last_fullsized":"2024-04-25T00:17:27.584129+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:14:26.204170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565223+0000","last_change":"2024-04-25T00:16:20.224710+0000","last_active":"2024-04-25T00:17:27.565223+0000","last_peered":"2024-04-25T00:17:27.565223+0000","last_clean":"2024-04-25T00:17:27.565223+0000","last_became_active":"2024-04-25T00:16:20.224385+0000","last_became_peered":"2024-04-25T00:16:20.224385+0000","last_unstale":"2024-04-25T00:17:27.565223+0000","last_undegraded":"2024-04-25T00:17:27.565223+0000","last_fullsized":"2024-04-25T00:17:27.565223+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:20:29.161972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.568285+0000","last_change":"2024-04-25T00:16:19.327498+0000","last_active":"2024-04-25T00:17:27.568285+0000","last_peered":"2024-04-25T00:17:27.568285+0000","last_clean":"2024-04-25T00:17:27.568285+0000","last_became_active":"2024-04-25T00:16:19.326889+0000","last_became_peered":"2024-04-25T00:16:19.326889+0000","last_unstale":"2024-04-25T00:17:27.568285+0000","last_undegraded":"2024-04-25T00:17:27.568285+0000","last_fullsized":"2024-04-25T00:17:27.568285+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:41:05.275999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562551+0000","last_change":"2024-04-25T00:16:20.242150+0000","last_active":"2024-04-25T00:17:27.562551+0000","last_peered":"2024-04-25T00:17:27.562551+0000","last_clean":"2024-04-25T00:17:27.562551+0000","last_became_active":"2024-04-25T00:16:20.241770+0000","last_became_peered":"2024-04-25T00:16:20.241770+0000","last_unstale":"2024-04-25T00:17:27.562551+0000","last_undegraded":"2024-04-25T00:17:27.562551+0000","last_fullsized":"2024-04-25T00:17:27.562551+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:38.038980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'1","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.579179+0000","last_change":"2024-04-25T00:17:27.578637+0000","last_active":"2024-04-25T00:17:27.579179+0000","last_peered":"2024-04-25T00:17:27.579179+0000","last_clean":"2024-04-25T00:17:27.579179+0000","last_became_active":"2024-04-25T00:17:27.571940+0000","last_became_peered":"2024-04-25T00:17:27.571940+0000","last_unstale":"2024-04-25T00:17:27.579179+0000","last_undegraded":"2024-04-25T00:17:27.579179+0000","last_fullsized":"2024-04-25T00:17:27.579179+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:19.798797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.573828+0000","last_change":"2024-04-25T00:16:20.244330+0000","last_active":"2024-04-25T00:17:27.573828+0000","last_peered":"2024-04-25T00:17:27.573828+0000","last_clean":"2024-04-25T00:17:27.573828+0000","last_became_active":"2024-04-25T00:16:20.243946+0000","last_became_peered":"2024-04-25T00:16:20.243946+0000","last_unstale":"2024-04-25T00:17:27.573828+0000","last_undegraded":"2024-04-25T00:17:27.573828+0000","last_fullsized":"2024-04-25T00:17:27.573828+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:37.790655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565039+0000","last_change":"2024-04-25T00:16:19.313708+0000","last_active":"2024-04-25T00:17:27.565039+0000","last_peered":"2024-04-25T00:17:27.565039+0000","last_clean":"2024-04-25T00:17:27.565039+0000","last_became_active":"2024-04-25T00:16:19.311700+0000","last_became_peered":"2024-04-25T00:16:19.311700+0000","last_unstale":"2024-04-25T00:17:27.565039+0000","last_undegraded":"2024-04-25T00:17:27.565039+0000","last_fullsized":"2024-04-25T00:17:27.565039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:12:24.656229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664298+0000","last_change":"2024-04-25T00:16:20.240828+0000","last_active":"2024-04-25T00:17:27.664298+0000","last_peered":"2024-04-25T00:17:27.664298+0000","last_clean":"2024-04-25T00:17:27.664298+0000","last_became_active":"2024-04-25T00:16:20.240495+0000","last_became_peered":"2024-04-25T00:16:20.240495+0000","last_unstale":"2024-04-25T00:17:27.664298+0000","last_undegraded":"2024-04-25T00:17:27.664298+0000","last_fullsized":"2024-04-25T00:17:27.664298+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:48:19.730968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566077+0000","last_change":"2024-04-25T00:16:19.324069+0000","last_active":"2024-04-25T00:17:27.566077+0000","last_peered":"2024-04-25T00:17:27.566077+0000","last_clean":"2024-04-25T00:17:27.566077+0000","last_became_active":"2024-04-25T00:16:19.323604+0000","last_became_peered":"2024-04-25T00:16:19.323604+0000","last_unstale":"2024-04-25T00:17:27.566077+0000","last_undegraded":"2024-04-25T00:17:27.566077+0000","last_fullsized":"2024-04-25T00:17:27.566077+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:40:50.908834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663990+0000","last_change":"2024-04-25T00:16:20.220555+0000","last_active":"2024-04-25T00:17:27.663990+0000","last_peered":"2024-04-25T00:17:27.663990+0000","last_clean":"2024-04-25T00:17:27.663990+0000","last_became_active":"2024-04-25T00:16:20.219986+0000","last_became_peered":"2024-04-25T00:16:20.219986+0000","last_unstale":"2024-04-25T00:17:27.663990+0000","last_undegraded":"2024-04-25T00:17:27.663990+0000","last_fullsized":"2024-04-25T00:17:27.663990+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:21:04.095555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'1","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569486+0000","last_change":"2024-04-25T00:16:19.328270+0000","last_active":"2024-04-25T00:17:27.569486+0000","last_peered":"2024-04-25T00:17:27.569486+0000","last_clean":"2024-04-25T00:17:27.569486+0000","last_became_active":"2024-04-25T00:16:19.327929+0000","last_became_peered":"2024-04-25T00:16:19.327929+0000","last_unstale":"2024-04-25T00:17:27.569486+0000","last_undegraded":"2024-04-25T00:17:27.569486+0000","last_fullsized":"2024-04-25T00:17:27.569486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:36:09.233345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.663754+0000","last_change":"2024-04-25T00:16:20.237103+0000","last_active":"2024-04-25T00:17:27.663754+0000","last_peered":"2024-04-25T00:17:27.663754+0000","last_clean":"2024-04-25T00:17:27.663754+0000","last_became_active":"2024-04-25T00:16:20.236811+0000","last_became_peered":"2024-04-25T00:16:20.236811+0000","last_unstale":"2024-04-25T00:17:27.663754+0000","last_undegraded":"2024-04-25T00:17:27.663754+0000","last_fullsized":"2024-04-25T00:17:27.663754+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:56.833659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"85'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.578061+0000","last_change":"2024-04-25T00:16:19.323227+0000","last_active":"2024-04-25T00:17:27.578061+0000","last_peered":"2024-04-25T00:17:27.578061+0000","last_clean":"2024-04-25T00:17:27.578061+0000","last_became_active":"2024-04-25T00:16:19.322074+0000","last_became_peered":"2024-04-25T00:16:19.322074+0000","last_unstale":"2024-04-25T00:17:27.578061+0000","last_undegraded":"2024-04-25T00:17:27.578061+0000","last_fullsized":"2024-04-25T00:17:27.578061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:18.228382+0000","last_clean_scrub_stamp":"2024-04-25T00:16:18.228382+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:53.316171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.664221+0000","last_change":"2024-04-25T00:16:20.222049+0000","last_active":"2024-04-25T00:17:27.664221+0000","last_peered":"2024-04-25T00:17:27.664221+0000","last_clean":"2024-04-25T00:17:27.664221+0000","last_became_active":"2024-04-25T00:16:20.221651+0000","last_became_peered":"2024-04-25T00:16:20.221651+0000","last_unstale":"2024-04-25T00:17:27.664221+0000","last_undegraded":"2024-04-25T00:17:27.664221+0000","last_fullsized":"2024-04-25T00:17:27.664221+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:27:31.244463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562619+0000","last_change":"2024-04-25T00:16:20.223864+0000","last_active":"2024-04-25T00:17:27.562619+0000","last_peered":"2024-04-25T00:17:27.562619+0000","last_clean":"2024-04-25T00:17:27.562619+0000","last_became_active":"2024-04-25T00:16:20.223533+0000","last_became_peered":"2024-04-25T00:16:20.223533+0000","last_unstale":"2024-04-25T00:17:27.562619+0000","last_undegraded":"2024-04-25T00:17:27.562619+0000","last_fullsized":"2024-04-25T00:17:27.562619+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:01:14.482038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565715+0000","last_change":"2024-04-25T00:16:20.214232+0000","last_active":"2024-04-25T00:17:27.565715+0000","last_peered":"2024-04-25T00:17:27.565715+0000","last_clean":"2024-04-25T00:17:27.565715+0000","last_became_active":"2024-04-25T00:16:20.213871+0000","last_became_peered":"2024-04-25T00:16:20.213871+0000","last_unstale":"2024-04-25T00:17:27.565715+0000","last_undegraded":"2024-04-25T00:17:27.565715+0000","last_fullsized":"2024-04-25T00:17:27.565715+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:07.049327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.565818+0000","last_change":"2024-04-25T00:16:20.232935+0000","last_active":"2024-04-25T00:17:27.565818+0000","last_peered":"2024-04-25T00:17:27.565818+0000","last_clean":"2024-04-25T00:17:27.565818+0000","last_became_active":"2024-04-25T00:16:20.232632+0000","last_became_peered":"2024-04-25T00:16:20.232632+0000","last_unstale":"2024-04-25T00:17:27.565818+0000","last_undegraded":"2024-04-25T00:17:27.565818+0000","last_fullsized":"2024-04-25T00:17:27.565818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:36.724694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.577206+0000","last_change":"2024-04-25T00:16:20.244382+0000","last_active":"2024-04-25T00:17:27.577206+0000","last_peered":"2024-04-25T00:17:27.577206+0000","last_clean":"2024-04-25T00:17:27.577206+0000","last_became_active":"2024-04-25T00:16:20.241866+0000","last_became_peered":"2024-04-25T00:16:20.241866+0000","last_unstale":"2024-04-25T00:17:27.577206+0000","last_undegraded":"2024-04-25T00:17:27.577206+0000","last_fullsized":"2024-04-25T00:17:27.577206+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:33:24.170878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.577185+0000","last_change":"2024-04-25T00:16:20.241343+0000","last_active":"2024-04-25T00:17:27.577185+0000","last_peered":"2024-04-25T00:17:27.577185+0000","last_clean":"2024-04-25T00:17:27.577185+0000","last_became_active":"2024-04-25T00:16:20.238417+0000","last_became_peered":"2024-04-25T00:16:20.238417+0000","last_unstale":"2024-04-25T00:17:27.577185+0000","last_undegraded":"2024-04-25T00:17:27.577185+0000","last_fullsized":"2024-04-25T00:17:27.577185+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:48:29.988773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562822+0000","last_change":"2024-04-25T00:16:20.219966+0000","last_active":"2024-04-25T00:17:27.562822+0000","last_peered":"2024-04-25T00:17:27.562822+0000","last_clean":"2024-04-25T00:17:27.562822+0000","last_became_active":"2024-04-25T00:16:20.219656+0000","last_became_peered":"2024-04-25T00:16:20.219656+0000","last_unstale":"2024-04-25T00:17:27.562822+0000","last_undegraded":"2024-04-25T00:17:27.562822+0000","last_fullsized":"2024-04-25T00:17:27.562822+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:56.290610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.566142+0000","last_change":"2024-04-25T00:16:20.219141+0000","last_active":"2024-04-25T00:17:27.566142+0000","last_peered":"2024-04-25T00:17:27.566142+0000","last_clean":"2024-04-25T00:17:27.566142+0000","last_became_active":"2024-04-25T00:16:20.218842+0000","last_became_peered":"2024-04-25T00:16:20.218842+0000","last_unstale":"2024-04-25T00:17:27.566142+0000","last_undegraded":"2024-04-25T00:17:27.566142+0000","last_fullsized":"2024-04-25T00:17:27.566142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:33.938226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.574814+0000","last_change":"2024-04-25T00:16:20.249183+0000","last_active":"2024-04-25T00:17:27.574814+0000","last_peered":"2024-04-25T00:17:27.574814+0000","last_clean":"2024-04-25T00:17:27.574814+0000","last_became_active":"2024-04-25T00:16:20.248355+0000","last_became_peered":"2024-04-25T00:16:20.248355+0000","last_unstale":"2024-04-25T00:17:27.574814+0000","last_undegraded":"2024-04-25T00:17:27.574814+0000","last_fullsized":"2024-04-25T00:17:27.574814+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:57.738041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.569061+0000","last_change":"2024-04-25T00:16:20.227528+0000","last_active":"2024-04-25T00:17:27.569061+0000","last_peered":"2024-04-25T00:17:27.569061+0000","last_clean":"2024-04-25T00:17:27.569061+0000","last_became_active":"2024-04-25T00:16:20.227208+0000","last_became_peered":"2024-04-25T00:16:20.227208+0000","last_unstale":"2024-04-25T00:17:27.569061+0000","last_undegraded":"2024-04-25T00:17:27.569061+0000","last_fullsized":"2024-04-25T00:17:27.569061+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:37:49.920850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.568114+0000","last_change":"2024-04-25T00:16:20.228157+0000","last_active":"2024-04-25T00:17:27.568114+0000","last_peered":"2024-04-25T00:17:27.568114+0000","last_clean":"2024-04-25T00:17:27.568114+0000","last_became_active":"2024-04-25T00:16:20.227659+0000","last_became_peered":"2024-04-25T00:16:20.227659+0000","last_unstale":"2024-04-25T00:17:27.568114+0000","last_undegraded":"2024-04-25T00:17:27.568114+0000","last_fullsized":"2024-04-25T00:17:27.568114+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:48:30.579407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.567192+0000","last_change":"2024-04-25T00:16:20.232237+0000","last_active":"2024-04-25T00:17:27.567192+0000","last_peered":"2024-04-25T00:17:27.567192+0000","last_clean":"2024-04-25T00:17:27.567192+0000","last_became_active":"2024-04-25T00:16:20.231897+0000","last_became_peered":"2024-04-25T00:16:20.231897+0000","last_unstale":"2024-04-25T00:17:27.567192+0000","last_undegraded":"2024-04-25T00:17:27.567192+0000","last_fullsized":"2024-04-25T00:17:27.567192+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:23.919515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.576373+0000","last_change":"2024-04-25T00:16:20.243454+0000","last_active":"2024-04-25T00:17:27.576373+0000","last_peered":"2024-04-25T00:17:27.576373+0000","last_clean":"2024-04-25T00:17:27.576373+0000","last_became_active":"2024-04-25T00:16:20.243112+0000","last_became_peered":"2024-04-25T00:16:20.243112+0000","last_unstale":"2024-04-25T00:17:27.576373+0000","last_undegraded":"2024-04-25T00:17:27.576373+0000","last_fullsized":"2024-04-25T00:17:27.576373+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:57:58.890876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T00:17:27.562585+0000","last_change":"2024-04-25T00:16:20.220947+0000","last_active":"2024-04-25T00:17:27.562585+0000","last_peered":"2024-04-25T00:17:27.562585+0000","last_clean":"2024-04-25T00:17:27.562585+0000","last_became_active":"2024-04-25T00:16:20.220662+0000","last_became_peered":"2024-04-25T00:16:20.220662+0000","last_unstale":"2024-04-25T00:17:27.562585+0000","last_undegraded":"2024-04-25T00:17:27.562585+0000","last_fullsized":"2024-04-25T00:17:27.562585+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T00:16:19.289734+0000","last_clean_scrub_stamp":"2024-04-25T00:16:19.289734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:52:44.840549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":3563520,"data_stored":3542208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":65506,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":546,"num_read_kb":340,"num_write":157,"num_write_kb":108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":196518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":932,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":954368,"data_stored":461900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612644,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28636,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716516,"statfs":{"total":95995035648,"available":95965712384,"internally_reserved":0,"allocated":1650688,"data_stored":1115277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776170,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36288,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708864,"statfs":{"total":95995035648,"available":95957876736,"internally_reserved":0,"allocated":970752,"data_stored":462136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037809,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32128,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713024,"statfs":{"total":95995035648,"available":95962136576,"internally_reserved":0,"allocated":970752,"data_stored":462056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332149,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28548,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716604,"statfs":{"total":95995035648,"available":95965802496,"internally_reserved":0,"allocated":1564672,"data_stored":1053595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":958464,"data_stored":461525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":523214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":462796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1044480,"data_stored":524564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":461836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27960,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717192,"statfs":{"total":95995035648,"available":95966404608,"internally_reserved":0,"allocated":962560,"data_stored":461962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36880,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708272,"statfs":{"total":95995035648,"available":95957270528,"internally_reserved":0,"allocated":1576960,"data_stored":1053279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":61896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T00:17:56.559 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T00:17:56.560 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T00:17:56.560 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T00:17:56.560 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T00:17:56.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:56 smithi042 ceph-mon[28130]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:17:57.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:56 smithi136 ceph-mon[32923]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:17:57.067 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:17:57.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:56 smithi106 ceph-mon[32510]: from='client.14781 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T00:17:57.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:57 smithi042 ceph-mon[28130]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:17:57.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:57 smithi042 ceph-mon[28130]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-25T00:17:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:57 smithi136 ceph-mon[32923]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:17:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:57 smithi136 ceph-mon[32923]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-25T00:17:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:57 smithi106 ceph-mon[32510]: pgmap v363: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:17:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:57 smithi106 ceph-mon[32510]: mgrmap e18: x(active, since 9m), standbys: y, z 2024-04-25T00:17:59.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.335+0000 7f6bab000700 1 -- 172.21.15.42:0/3903225295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba410e860 msgr2=0x7f6ba410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:59.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.335+0000 7f6bab000700 1 --2- 172.21.15.42:0/3903225295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba410e860 0x7f6ba410ec40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6b940098f0 tx=0x7f6b94009c00 comp rx=0 tx=0).stop 2024-04-25T00:17:59.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.335+0000 7f6bab000700 1 -- 172.21.15.42:0/3903225295 shutdown_connections 2024-04-25T00:17:59.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.335+0000 7f6bab000700 1 --2- 172.21.15.42:0/3903225295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4109220 0x7f6ba4115730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:59.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.335+0000 7f6bab000700 1 --2- 172.21.15.42:0/3903225295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba410e860 0x7f6ba410ec40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:59.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.335+0000 7f6bab000700 1 --2- 172.21.15.42:0/3903225295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba4108860 0x7f6ba4108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:59.336 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.335+0000 7f6bab000700 1 -- 172.21.15.42:0/3903225295 >> 172.21.15.42:0/3903225295 conn(0x7f6ba4075a00 msgr2=0x7f6ba4107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.335+0000 7f6bab000700 1 -- 172.21.15.42:0/3903225295 shutdown_connections 2024-04-25T00:17:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.335+0000 7f6bab000700 1 -- 172.21.15.42:0/3903225295 wait complete. 2024-04-25T00:17:59.337 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.336+0000 7f6bab000700 1 Processor -- start 2024-04-25T00:17:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.337+0000 7f6bab000700 1 -- start start 2024-04-25T00:17:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.337+0000 7f6bab000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4108860 0x7f6ba419f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.337+0000 7f6bab000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba4109220 0x7f6ba419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.337+0000 7f6bab000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba410e860 0x7f6ba41a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.337+0000 7f6bab000700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4118c00 con 0x7f6ba4108860 2024-04-25T00:17:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.337+0000 7f6bab000700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4118a80 con 0x7f6ba410e860 2024-04-25T00:17:59.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.337+0000 7f6bab000700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba4118d80 con 0x7f6ba4109220 2024-04-25T00:17:59.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4108860 0x7f6ba419f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:59.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba3e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4108860 0x7f6ba419f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38552/0 (socket says 172.21.15.42:38552) 2024-04-25T00:17:59.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba3e00700 1 -- 172.21.15.42:0/1189726630 learned_addr learned my addr 172.21.15.42:0/1189726630 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:17:59.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba9000700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba410e860 0x7f6ba41a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:59.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba3400700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba4109220 0x7f6ba419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:59.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba9000700 1 -- 172.21.15.42:0/1189726630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba4109220 msgr2=0x7f6ba419f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:59.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba9000700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba4109220 0x7f6ba419f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:59.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba9000700 1 -- 172.21.15.42:0/1189726630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4108860 msgr2=0x7f6ba419f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:59.340 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba9000700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4108860 0x7f6ba419f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:59.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba9000700 1 -- 172.21.15.42:0/1189726630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba41a4160 con 0x7f6ba410e860 2024-04-25T00:17:59.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.338+0000 7f6ba3400700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba4109220 0x7f6ba419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:17:59.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.339+0000 7f6ba9000700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba410e860 0x7f6ba41a3a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009cd0 tx=0x7f6b9c00c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:59.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.339+0000 7f6ba0c00700 1 -- 172.21.15.42:0/1189726630 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c00cd10 con 0x7f6ba410e860 2024-04-25T00:17:59.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.339+0000 7f6ba0c00700 1 -- 172.21.15.42:0/1189726630 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b9c003a00 con 0x7f6ba410e860 2024-04-25T00:17:59.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.339+0000 7f6ba0c00700 1 -- 172.21.15.42:0/1189726630 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c0045a0 con 0x7f6ba410e860 2024-04-25T00:17:59.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.339+0000 7f6bab000700 1 -- 172.21.15.42:0/1189726630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba41a43f0 con 0x7f6ba410e860 2024-04-25T00:17:59.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.339+0000 7f6bab000700 1 -- 172.21.15.42:0/1189726630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba41abcd0 con 0x7f6ba410e860 2024-04-25T00:17:59.341 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.341+0000 7f6bab000700 1 -- 172.21.15.42:0/1189726630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ba4068f20 con 0x7f6ba410e860 2024-04-25T00:17:59.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.342+0000 7f6ba0c00700 1 -- 172.21.15.42:0/1189726630 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b9c01a020 con 0x7f6ba410e860 2024-04-25T00:17:59.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.344+0000 7f6ba0c00700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b880a2ca0 0x7f6b880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:17:59.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.344+0000 7f6ba0c00700 1 -- 172.21.15.42:0/1189726630 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f6b9c0ce860 con 0x7f6ba410e860 2024-04-25T00:17:59.345 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.344+0000 7f6ba3e00700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b880a2ca0 0x7f6b880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:17:59.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.345+0000 7f6ba3e00700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b880a2ca0 0x7f6b880a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c010 tx=0x7f6b94019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:17:59.348 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.348+0000 7f6ba0c00700 1 -- 172.21.15.42:0/1189726630 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f6b9c010040 con 0x7f6ba410e860 2024-04-25T00:17:59.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.625+0000 7f6bab000700 1 -- 172.21.15.42:0/1189726630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f6ba402db30 con 0x7f6ba410e860 2024-04-25T00:17:59.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.626+0000 7f6ba0c00700 1 -- 172.21.15.42:0/1189726630 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f6b9c09b790 con 0x7f6ba410e860 2024-04-25T00:17:59.627 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:17:59.627 INFO:teuthology.orchestra.run.smithi042.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T00:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.629+0000 7f6bab000700 1 -- 172.21.15.42:0/1189726630 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b880a2ca0 msgr2=0x7f6b880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.629+0000 7f6bab000700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b880a2ca0 0x7f6b880a5160 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c010 tx=0x7f6b94019040 comp rx=0 tx=0).stop 2024-04-25T00:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.629+0000 7f6bab000700 1 -- 172.21.15.42:0/1189726630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba410e860 msgr2=0x7f6ba41a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:17:59.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.629+0000 7f6bab000700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba410e860 0x7f6ba41a3a20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c009cd0 tx=0x7f6b9c00c620 comp rx=0 tx=0).stop 2024-04-25T00:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.629+0000 7f6bab000700 1 -- 172.21.15.42:0/1189726630 shutdown_connections 2024-04-25T00:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.629+0000 7f6bab000700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b880a2ca0 0x7f6b880a5160 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.629+0000 7f6bab000700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba410e860 0x7f6ba41a3a20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.629+0000 7f6bab000700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba4108860 0x7f6ba419f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.629+0000 7f6bab000700 1 --2- 172.21.15.42:0/1189726630 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba4109220 0x7f6ba419f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.629+0000 7f6bab000700 1 -- 172.21.15.42:0/1189726630 >> 172.21.15.42:0/1189726630 conn(0x7f6ba4075a00 msgr2=0x7f6ba40760c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:17:59.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.630+0000 7f6bab000700 1 -- 172.21.15.42:0/1189726630 shutdown_connections 2024-04-25T00:17:59.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:17:59.630+0000 7f6bab000700 1 -- 172.21.15.42:0/1189726630 wait complete. 2024-04-25T00:18:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:59 smithi136 ceph-mon[32923]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:17:59 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1189726630' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:18:00.045 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:59 smithi042 ceph-mon[28130]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:00.045 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:17:59 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1189726630' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:18:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:59 smithi106 ceph-mon[32510]: pgmap v364: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:17:59 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1189726630' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T00:18:00.517 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T00:18:00.517 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:18:00.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.740+0000 7f55d7e00640 1 -- 172.21.15.42:0/4044752747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55d8109770 msgr2=0x7f55d8110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:00.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.740+0000 7f55d7e00640 1 --2- 172.21.15.42:0/4044752747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55d8109770 0x7f55d8110000 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009ca0 tx=0x7f55d002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:00.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.740+0000 7f55d7e00640 1 -- 172.21.15.42:0/4044752747 shutdown_connections 2024-04-25T00:18:00.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.740+0000 7f55d7e00640 1 --2- 172.21.15.42:0/4044752747 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55d8109770 0x7f55d8110000 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:00.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.740+0000 7f55d7e00640 1 --2- 172.21.15.42:0/4044752747 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8108db0 0x7f55d8109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:00.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.740+0000 7f55d7e00640 1 --2- 172.21.15.42:0/4044752747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d8107bc0 0x7f55d8107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:00.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.740+0000 7f55d7e00640 1 -- 172.21.15.42:0/4044752747 >> 172.21.15.42:0/4044752747 conn(0x7f55d8076b80 msgr2=0x7f55d8078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:00.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.740+0000 7f55d7e00640 1 -- 172.21.15.42:0/4044752747 shutdown_connections 2024-04-25T00:18:00.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.740+0000 7f55d7e00640 1 -- 172.21.15.42:0/4044752747 wait complete. 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.741+0000 7f55d7e00640 1 Processor -- start 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.741+0000 7f55d7e00640 1 -- start start 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8107bc0 0x7f55d819de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d8108db0 0x7f55d819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55d8109770 0x7f55d81a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8113870 con 0x7f55d8107bc0 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d7e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81136f0 con 0x7f55d8109770 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81139f0 con 0x7f55d8108db0 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8107bc0 0x7f55d819de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8107bc0 0x7f55d819de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38574/0 (socket says 172.21.15.42:38574) 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d6a00640 1 -- 172.21.15.42:0/3331473284 learned_addr learned my addr 172.21.15.42:0/3331473284 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d7400640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55d8109770 0x7f55d81a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:00.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d6000640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d8108db0 0x7f55d819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d6a00640 1 -- 172.21.15.42:0/3331473284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d8108db0 msgr2=0x7f55d819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d6a00640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d8108db0 0x7f55d819e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d6a00640 1 -- 172.21.15.42:0/3331473284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55d8109770 msgr2=0x7f55d81a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d6a00640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55d8109770 0x7f55d81a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d6a00640 1 -- 172.21.15.42:0/3331473284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d81a5aa0 con 0x7f55d8107bc0 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.742+0000 7f55d7400640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55d8109770 0x7f55d81a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.743+0000 7f55d6a00640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8107bc0 0x7f55d819de80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f55c800e9c0 tx=0x7f55c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.743+0000 7f55cf400640 1 -- 172.21.15.42:0/3331473284 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c800cd80 con 0x7f55d8107bc0 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.743+0000 7f55d7e00640 1 -- 172.21.15.42:0/3331473284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d81a5d90 con 0x7f55d8107bc0 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.743+0000 7f55cf400640 1 -- 172.21.15.42:0/3331473284 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55c80035f0 con 0x7f55d8107bc0 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.743+0000 7f55cf400640 1 -- 172.21.15.42:0/3331473284 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8010690 con 0x7f55d8107bc0 2024-04-25T00:18:00.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.743+0000 7f55d7e00640 1 -- 172.21.15.42:0/3331473284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d81a6220 con 0x7f55d8107bc0 2024-04-25T00:18:00.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.744+0000 7f55d7e00640 1 -- 172.21.15.42:0/3331473284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55d810ee80 con 0x7f55d8107bc0 2024-04-25T00:18:00.746 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.746+0000 7f55cf400640 1 -- 172.21.15.42:0/3331473284 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f55c80037a0 con 0x7f55d8107bc0 2024-04-25T00:18:00.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.748+0000 7f55cf400640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f55bc0a2b90 0x7f55bc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:00.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.748+0000 7f55cf400640 1 -- 172.21.15.42:0/3331473284 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f55c80cf440 con 0x7f55d8107bc0 2024-04-25T00:18:00.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.748+0000 7f55d6000640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f55bc0a2b90 0x7f55bc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:00.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.749+0000 7f55d6000640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f55bc0a2b90 0x7f55bc0a5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f55d819f390 tx=0x7f55c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:00.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.753+0000 7f55cf400640 1 -- 172.21.15.42:0/3331473284 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f55c809c370 con 0x7f55d8107bc0 2024-04-25T00:18:00.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.965+0000 7f55d7e00640 1 -- 172.21.15.42:0/3331473284 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f55d8062450 con 0x7f55d8107bc0 2024-04-25T00:18:00.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.968+0000 7f55cf400640 1 -- 172.21.15.42:0/3331473284 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22547 (secure 0 0 0) 0x7f55c809bc70 con 0x7f55d8107bc0 2024-04-25T00:18:00.968 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:00.969 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":88,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:17:27.556239+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.970+0000 7f55d7e00640 1 -- 172.21.15.42:0/3331473284 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f55bc0a2b90 msgr2=0x7f55bc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f55bc0a2b90 0x7f55bc0a5050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f55d819f390 tx=0x7f55c4005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 -- 172.21.15.42:0/3331473284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8107bc0 msgr2=0x7f55d819de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8107bc0 0x7f55d819de80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f55c800e9c0 tx=0x7f55c800ee80 comp rx=0 tx=0).stop 2024-04-25T00:18:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 -- 172.21.15.42:0/3331473284 shutdown_connections 2024-04-25T00:18:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f55bc0a2b90 0x7f55bc0a5050 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55d8109770 0x7f55d81a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55d8108db0 0x7f55d819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 --2- 172.21.15.42:0/3331473284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55d8107bc0 0x7f55d819de80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:00.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 -- 172.21.15.42:0/3331473284 >> 172.21.15.42:0/3331473284 conn(0x7f55d8076b80 msgr2=0x7f55d810c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 -- 172.21.15.42:0/3331473284 shutdown_connections 2024-04-25T00:18:00.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:00.971+0000 7f55d7e00640 1 -- 172.21.15.42:0/3331473284 wait complete. 2024-04-25T00:18:00.992 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:11:14.472869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:16:18.168243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:16:19.289585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:18:00.992 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:18:01.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.209+0000 7f886b200640 1 -- 172.21.15.42:0/2593380621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886c104a70 msgr2=0x7f886c104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:01.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.209+0000 7f886b200640 1 --2- 172.21.15.42:0/2593380621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886c104a70 0x7f886c104ef0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8854009860 tx=0x7f885402a340 comp rx=0 tx=0).stop 2024-04-25T00:18:01.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.209+0000 7f886b200640 1 -- 172.21.15.42:0/2593380621 shutdown_connections 2024-04-25T00:18:01.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.209+0000 7f886b200640 1 --2- 172.21.15.42:0/2593380621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f886c105430 0x7f886c10bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.209 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.209+0000 7f886b200640 1 --2- 172.21.15.42:0/2593380621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886c104a70 0x7f886c104ef0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.209+0000 7f886b200640 1 --2- 172.21.15.42:0/2593380621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886c103880 0x7f886c103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.209+0000 7f886b200640 1 -- 172.21.15.42:0/2593380621 >> 172.21.15.42:0/2593380621 conn(0x7f886c0fedf0 msgr2=0x7f886c101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:01.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.209+0000 7f886b200640 1 -- 172.21.15.42:0/2593380621 shutdown_connections 2024-04-25T00:18:01.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.209+0000 7f886b200640 1 -- 172.21.15.42:0/2593380621 wait complete. 2024-04-25T00:18:01.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.210+0000 7f886b200640 1 Processor -- start 2024-04-25T00:18:01.210 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.210+0000 7f886b200640 1 -- start start 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f886b200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886c103880 0x7f886c199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f886b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886c104a70 0x7f886c19a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f886b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f886c105430 0x7f886c1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f886b200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f886c10f4e0 con 0x7f886c103880 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f886b200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f886c10f360 con 0x7f886c105430 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f886b200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f886c10f660 con 0x7f886c104a70 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f8869e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886c103880 0x7f886c199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f8869e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886c103880 0x7f886c199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38602/0 (socket says 172.21.15.42:38602) 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f8869e00640 1 -- 172.21.15.42:0/2465281673 learned_addr learned my addr 172.21.15.42:0/2465281673 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f8861400640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886c104a70 0x7f886c19a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.211+0000 7f886a800640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f886c105430 0x7f886c1a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f8869e00640 1 -- 172.21.15.42:0/2465281673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886c104a70 msgr2=0x7f886c19a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:01.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f8869e00640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886c104a70 0x7f886c19a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f8869e00640 1 -- 172.21.15.42:0/2465281673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f886c105430 msgr2=0x7f886c1a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:01.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f8869e00640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f886c105430 0x7f886c1a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f8869e00640 1 -- 172.21.15.42:0/2465281673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f886c1a1750 con 0x7f886c103880 2024-04-25T00:18:01.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f8869e00640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886c103880 0x7f886c199b30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f885c00ebc0 tx=0x7f885c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:01.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f8863400640 1 -- 172.21.15.42:0/2465281673 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c003970 con 0x7f886c103880 2024-04-25T00:18:01.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f8863400640 1 -- 172.21.15.42:0/2465281673 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f885c004030 con 0x7f886c103880 2024-04-25T00:18:01.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f886b200640 1 -- 172.21.15.42:0/2465281673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f886c1a1a40 con 0x7f886c103880 2024-04-25T00:18:01.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f8863400640 1 -- 172.21.15.42:0/2465281673 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885c0197b0 con 0x7f886c103880 2024-04-25T00:18:01.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.212+0000 7f886b200640 1 -- 172.21.15.42:0/2465281673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f886c1a1f60 con 0x7f886c103880 2024-04-25T00:18:01.214 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.213+0000 7f886b200640 1 -- 172.21.15.42:0/2465281673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f886c0688d0 con 0x7f886c103880 2024-04-25T00:18:01.215 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.215+0000 7f8863400640 1 -- 172.21.15.42:0/2465281673 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f885c0041e0 con 0x7f886c103880 2024-04-25T00:18:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.218+0000 7f8863400640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f88500a2b90 0x7f88500a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.218+0000 7f8863400640 1 -- 172.21.15.42:0/2465281673 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f885c0cf970 con 0x7f886c103880 2024-04-25T00:18:01.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.218+0000 7f8861400640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f88500a2b90 0x7f88500a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:01.219 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.219+0000 7f8861400640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f88500a2b90 0x7f88500a5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8854009830 tx=0x7f8854030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:01.222 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.222+0000 7f8863400640 1 -- 172.21.15.42:0/2465281673 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f885c09c8a0 con 0x7f886c103880 2024-04-25T00:18:01.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.426+0000 7f886b200640 1 -- 172.21.15.42:0/2465281673 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f886c062340 con 0x7f886c103880 2024-04-25T00:18:01.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.427+0000 7f8863400640 1 -- 172.21.15.42:0/2465281673 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f885c09c1a0 con 0x7f886c103880 2024-04-25T00:18:01.428 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 -- 172.21.15.42:0/2465281673 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f88500a2b90 msgr2=0x7f88500a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f88500a2b90 0x7f88500a5050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8854009830 tx=0x7f8854030550 comp rx=0 tx=0).stop 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 -- 172.21.15.42:0/2465281673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886c103880 msgr2=0x7f886c199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886c103880 0x7f886c199b30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f885c00ebc0 tx=0x7f885c00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 -- 172.21.15.42:0/2465281673 shutdown_connections 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f88500a2b90 0x7f88500a5050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f886c105430 0x7f886c1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f886c104a70 0x7f886c19a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 --2- 172.21.15.42:0/2465281673 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f886c103880 0x7f886c199b30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 -- 172.21.15.42:0/2465281673 >> 172.21.15.42:0/2465281673 conn(0x7f886c0fedf0 msgr2=0x7f886c107dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 -- 172.21.15.42:0/2465281673 shutdown_connections 2024-04-25T00:18:01.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.430+0000 7f886b200640 1 -- 172.21.15.42:0/2465281673 wait complete. 2024-04-25T00:18:01.451 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:18:01.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.668+0000 7fadad400640 1 -- 172.21.15.42:0/353451021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fada8100830 msgr2=0x7fada8100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:01.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.668+0000 7fadad400640 1 --2- 172.21.15.42:0/353451021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fada8100830 0x7fada8100c90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad90009ca0 tx=0x7fad9002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.668+0000 7fadad400640 1 -- 172.21.15.42:0/353451021 shutdown_connections 2024-04-25T00:18:01.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.669+0000 7fadad400640 1 --2- 172.21.15.42:0/353451021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada81011d0 0x7fada8101650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.669+0000 7fadad400640 1 --2- 172.21.15.42:0/353451021 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fada8100830 0x7fada8100c90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.669 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.669+0000 7fadad400640 1 --2- 172.21.15.42:0/353451021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fada80ffed0 0x7fada81002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.669+0000 7fadad400640 1 -- 172.21.15.42:0/353451021 >> 172.21.15.42:0/353451021 conn(0x7fada80fbaf0 msgr2=0x7fada80fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:01.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.669+0000 7fadad400640 1 -- 172.21.15.42:0/353451021 shutdown_connections 2024-04-25T00:18:01.670 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.669+0000 7fadad400640 1 -- 172.21.15.42:0/353451021 wait complete. 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fadad400640 1 Processor -- start 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fadad400640 1 -- start start 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fadad400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fada80ffed0 0x7fada8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fadad400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fada8100830 0x7fada819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fadad400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada81011d0 0x7fada81a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fadad400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fada810f520 con 0x7fada80ffed0 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fadad400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fada810f3a0 con 0x7fada8100830 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fadad400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fada810f6a0 con 0x7fada81011d0 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fada7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fada80ffed0 0x7fada8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fada7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fada80ffed0 0x7fada8199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38622/0 (socket says 172.21.15.42:38622) 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fada7e00640 1 -- 172.21.15.42:0/3002912709 learned_addr learned my addr 172.21.15.42:0/3002912709 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:01.672 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.670+0000 7fadaca00640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada81011d0 0x7fada81a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.671+0000 7fada7400640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fada8100830 0x7fada819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.671+0000 7fada7e00640 1 -- 172.21.15.42:0/3002912709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada81011d0 msgr2=0x7fada81a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.671+0000 7fada7e00640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada81011d0 0x7fada81a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.671+0000 7fada7e00640 1 -- 172.21.15.42:0/3002912709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fada8100830 msgr2=0x7fada819a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.671+0000 7fada7e00640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fada8100830 0x7fada819a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.671+0000 7fada7e00640 1 -- 172.21.15.42:0/3002912709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fada81a1750 con 0x7fada80ffed0 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.671+0000 7fadaca00640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada81011d0 0x7fada81a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.671+0000 7fada7e00640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fada80ffed0 0x7fada8199b30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad9800ea60 tx=0x7fad9800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.671+0000 7fada4c00640 1 -- 172.21.15.42:0/3002912709 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad9800cde0 con 0x7fada80ffed0 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.672+0000 7fada4c00640 1 -- 172.21.15.42:0/3002912709 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fad980035f0 con 0x7fada80ffed0 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.672+0000 7fada4c00640 1 -- 172.21.15.42:0/3002912709 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98010670 con 0x7fada80ffed0 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.672+0000 7fadad400640 1 -- 172.21.15.42:0/3002912709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fada81a1a40 con 0x7fada80ffed0 2024-04-25T00:18:01.673 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.672+0000 7fadad400640 1 -- 172.21.15.42:0/3002912709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fada81a1f10 con 0x7fada80ffed0 2024-04-25T00:18:01.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.673+0000 7fadad400640 1 -- 172.21.15.42:0/3002912709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fada810ad60 con 0x7fada80ffed0 2024-04-25T00:18:01.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.675+0000 7fada4c00640 1 -- 172.21.15.42:0/3002912709 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad980037a0 con 0x7fada80ffed0 2024-04-25T00:18:01.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.676+0000 7fada4c00640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fad880a2b90 0x7fad880a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:01.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.677+0000 7fada4c00640 1 -- 172.21.15.42:0/3002912709 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fad980cf340 con 0x7fada80ffed0 2024-04-25T00:18:01.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.677+0000 7fada7400640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fad880a2b90 0x7fad880a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:01.677 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.677+0000 7fada7400640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fad880a2b90 0x7fad880a5050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad90005e30 tx=0x7fad90005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:01.682 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.680+0000 7fada4c00640 1 -- 172.21.15.42:0/3002912709 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fad9809c270 con 0x7fada80ffed0 2024-04-25T00:18:01.887 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.887+0000 7fadad400640 1 -- 172.21.15.42:0/3002912709 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fada802d3f0 con 0x7fada80ffed0 2024-04-25T00:18:01.888 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.888+0000 7fada4c00640 1 -- 172.21.15.42:0/3002912709 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7fad9809bb70 con 0x7fada80ffed0 2024-04-25T00:18:01.888 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-25T00:18:01.890 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.890+0000 7fadad400640 1 -- 172.21.15.42:0/3002912709 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fad880a2b90 msgr2=0x7fad880a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:01.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.890+0000 7fadad400640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fad880a2b90 0x7fad880a5050 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad90005e30 tx=0x7fad90005f50 comp rx=0 tx=0).stop 2024-04-25T00:18:01.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.890+0000 7fadad400640 1 -- 172.21.15.42:0/3002912709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fada80ffed0 msgr2=0x7fada8199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:01.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.890+0000 7fadad400640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fada80ffed0 0x7fada8199b30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad9800ea60 tx=0x7fad9800ef20 comp rx=0 tx=0).stop 2024-04-25T00:18:01.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.890+0000 7fadad400640 1 -- 172.21.15.42:0/3002912709 shutdown_connections 2024-04-25T00:18:01.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.890+0000 7fadad400640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fad880a2b90 0x7fad880a5050 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.890+0000 7fadad400640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fada81011d0 0x7fada81a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.890+0000 7fadad400640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fada8100830 0x7fada819a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.890+0000 7fadad400640 1 --2- 172.21.15.42:0/3002912709 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fada80ffed0 0x7fada8199b30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:01.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.890+0000 7fadad400640 1 -- 172.21.15.42:0/3002912709 >> 172.21.15.42:0/3002912709 conn(0x7fada80fbaf0 msgr2=0x7fada80fd130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:01.891 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.891+0000 7fadad400640 1 -- 172.21.15.42:0/3002912709 shutdown_connections 2024-04-25T00:18:01.892 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:01.891+0000 7fadad400640 1 -- 172.21.15.42:0/3002912709 wait complete. 2024-04-25T00:18:01.912 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:18:01.978 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:01 smithi042 ceph-mon[28130]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:18:01.978 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:01 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3331473284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:01.978 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:01 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2465281673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:01 smithi136 ceph-mon[32923]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:18:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:01 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3331473284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:01 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2465281673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:02.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:01 smithi106 ceph-mon[32510]: pgmap v365: 97 pgs: 97 active+clean; 641 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:18:02.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:01 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3331473284' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:02.091 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:01 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2465281673' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:02.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.121+0000 7f54c5c00640 1 -- 172.21.15.42:0/1173448566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54c00ffec0 msgr2=0x7f54c01022f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:02.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.121+0000 7f54c5c00640 1 --2- 172.21.15.42:0/1173448566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54c00ffec0 0x7f54c01022f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009900 tx=0x7f54b802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.122+0000 7f54c5c00640 1 -- 172.21.15.42:0/1173448566 shutdown_connections 2024-04-25T00:18:02.122 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.122+0000 7f54c5c00640 1 --2- 172.21.15.42:0/1173448566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c01051e0 0x7f54c0105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.122+0000 7f54c5c00640 1 --2- 172.21.15.42:0/1173448566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c0102830 0x7f54c0104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.122+0000 7f54c5c00640 1 --2- 172.21.15.42:0/1173448566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54c00ffec0 0x7f54c01022f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.122+0000 7f54c5c00640 1 -- 172.21.15.42:0/1173448566 >> 172.21.15.42:0/1173448566 conn(0x7f54c00fbae0 msgr2=0x7f54c00fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:02.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.122+0000 7f54c5c00640 1 -- 172.21.15.42:0/1173448566 shutdown_connections 2024-04-25T00:18:02.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.122+0000 7f54c5c00640 1 -- 172.21.15.42:0/1173448566 wait complete. 2024-04-25T00:18:02.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.123+0000 7f54c5c00640 1 Processor -- start 2024-04-25T00:18:02.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.123+0000 7f54c5c00640 1 -- start start 2024-04-25T00:18:02.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.123+0000 7f54c5c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54c00ffec0 0x7f54c019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:02.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.123+0000 7f54c5c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c0102830 0x7f54c019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:02.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.123+0000 7f54c5c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c01051e0 0x7f54c01a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:02.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.123+0000 7f54c5c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f54c01138c0 con 0x7f54c01051e0 2024-04-25T00:18:02.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.123+0000 7f54c5c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f54c0113740 con 0x7f54c0102830 2024-04-25T00:18:02.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.123+0000 7f54c5c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f54c0113a40 con 0x7f54c00ffec0 2024-04-25T00:18:02.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54bfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54c00ffec0 0x7f54c019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:02.125 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54bfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54c00ffec0 0x7f54c019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39194/0 (socket says 172.21.15.42:39194) 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54bfe00640 1 -- 172.21.15.42:0/2701744416 learned_addr learned my addr 172.21.15.42:0/2701744416 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54c5200640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c01051e0 0x7f54c01a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54bf400640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c0102830 0x7f54c019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54bfe00640 1 -- 172.21.15.42:0/2701744416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c0102830 msgr2=0x7f54c019e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54bfe00640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c0102830 0x7f54c019e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54bfe00640 1 -- 172.21.15.42:0/2701744416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c01051e0 msgr2=0x7f54c01a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54bfe00640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c01051e0 0x7f54c01a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54bfe00640 1 -- 172.21.15.42:0/2701744416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c01a5b60 con 0x7f54c00ffec0 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.124+0000 7f54bfe00640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54c00ffec0 0x7f54c019df90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009230 tx=0x7f54b80042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.125+0000 7f54bcc00640 1 -- 172.21.15.42:0/2701744416 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b803f030 con 0x7f54c00ffec0 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.125+0000 7f54c5c00640 1 -- 172.21.15.42:0/2701744416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c01a5df0 con 0x7f54c00ffec0 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.125+0000 7f54bcc00640 1 -- 172.21.15.42:0/2701744416 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f54b8003ce0 con 0x7f54c00ffec0 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.125+0000 7f54bcc00640 1 -- 172.21.15.42:0/2701744416 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8035070 con 0x7f54c00ffec0 2024-04-25T00:18:02.126 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.125+0000 7f54c5c00640 1 -- 172.21.15.42:0/2701744416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c01a6250 con 0x7f54c00ffec0 2024-04-25T00:18:02.127 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.126+0000 7f54c5c00640 1 -- 172.21.15.42:0/2701744416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c00fd5d0 con 0x7f54c00ffec0 2024-04-25T00:18:02.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.128+0000 7f54bcc00640 1 -- 172.21.15.42:0/2701744416 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f54b8042030 con 0x7f54c00ffec0 2024-04-25T00:18:02.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.130+0000 7f54bcc00640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f54a40a2c10 0x7f54a40a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:02.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.130+0000 7f54bcc00640 1 -- 172.21.15.42:0/2701744416 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f54b80ef440 con 0x7f54c00ffec0 2024-04-25T00:18:02.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.130+0000 7f54bf400640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f54a40a2c10 0x7f54a40a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:02.130 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.131+0000 7f54bf400640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f54a40a2c10 0x7f54a40a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f54c019f450 tx=0x7f54a8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:02.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.134+0000 7f54bcc00640 1 -- 172.21.15.42:0/2701744416 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f54b80bbd20 con 0x7f54c00ffec0 2024-04-25T00:18:02.342 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.342+0000 7f54c5c00640 1 -- 172.21.15.42:0/2701744416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f54c002d3b0 con 0x7f54c00ffec0 2024-04-25T00:18:02.343 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.343+0000 7f54bcc00640 1 -- 172.21.15.42:0/2701744416 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f54b80c0020 con 0x7f54c00ffec0 2024-04-25T00:18:02.343 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-25T00:18:02.346 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 -- 172.21.15.42:0/2701744416 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f54a40a2c10 msgr2=0x7f54a40a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f54a40a2c10 0x7f54a40a50d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f54c019f450 tx=0x7f54a8009500 comp rx=0 tx=0).stop 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 -- 172.21.15.42:0/2701744416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54c00ffec0 msgr2=0x7f54c019df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54c00ffec0 0x7f54c019df90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f54b8009230 tx=0x7f54b80042f0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 -- 172.21.15.42:0/2701744416 shutdown_connections 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f54a40a2c10 0x7f54a40a50d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f54c01051e0 0x7f54c01a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c0102830 0x7f54c019e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 --2- 172.21.15.42:0/2701744416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54c00ffec0 0x7f54c019df90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 -- 172.21.15.42:0/2701744416 >> 172.21.15.42:0/2701744416 conn(0x7f54c00fbae0 msgr2=0x7f54c0103140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 -- 172.21.15.42:0/2701744416 shutdown_connections 2024-04-25T00:18:02.347 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.346+0000 7f54c5c00640 1 -- 172.21.15.42:0/2701744416 wait complete. 2024-04-25T00:18:02.364 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:18:02.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.576+0000 7f73d0a00640 1 -- 172.21.15.42:0/2455807382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73cc1049b0 msgr2=0x7f73cc104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:02.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.576+0000 7f73d0a00640 1 --2- 172.21.15.42:0/2455807382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73cc1049b0 0x7f73cc104e30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f73b4009860 tx=0x7f73b402a340 comp rx=0 tx=0).stop 2024-04-25T00:18:02.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.576+0000 7f73d0a00640 1 -- 172.21.15.42:0/2455807382 shutdown_connections 2024-04-25T00:18:02.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.576+0000 7f73d0a00640 1 --2- 172.21.15.42:0/2455807382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73cc105370 0x7f73cc10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.576+0000 7f73d0a00640 1 --2- 172.21.15.42:0/2455807382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73cc1049b0 0x7f73cc104e30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.576+0000 7f73d0a00640 1 --2- 172.21.15.42:0/2455807382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73cc1037c0 0x7f73cc103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.576+0000 7f73d0a00640 1 -- 172.21.15.42:0/2455807382 >> 172.21.15.42:0/2455807382 conn(0x7f73cc0fed50 msgr2=0x7f73cc101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:02.577 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.577+0000 7f73d0a00640 1 -- 172.21.15.42:0/2455807382 shutdown_connections 2024-04-25T00:18:02.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.577+0000 7f73d0a00640 1 -- 172.21.15.42:0/2455807382 wait complete. 2024-04-25T00:18:02.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.577+0000 7f73d0a00640 1 Processor -- start 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73d0a00640 1 -- start start 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73d0a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73cc1037c0 0x7f73cc199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73d0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73cc1049b0 0x7f73cc19a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73d0a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73cc105370 0x7f73cc1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73d0a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc10f440 con 0x7f73cc1049b0 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73d0a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc10f2c0 con 0x7f73cc105370 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73d0a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f73cc10f5c0 con 0x7f73cc1037c0 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73ca800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73cc1037c0 0x7f73cc199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73c1e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73cc1049b0 0x7f73cc19a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73ca800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73cc1037c0 0x7f73cc199b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39204/0 (socket says 172.21.15.42:39204) 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73ca800640 1 -- 172.21.15.42:0/1318212219 learned_addr learned my addr 172.21.15.42:0/1318212219 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.578+0000 7f73cb200640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73cc105370 0x7f73cc1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73ca800640 1 -- 172.21.15.42:0/1318212219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73cc105370 msgr2=0x7f73cc1a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:02.580 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73ca800640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73cc105370 0x7f73cc1a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73ca800640 1 -- 172.21.15.42:0/1318212219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73cc1049b0 msgr2=0x7f73cc19a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:02.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73ca800640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73cc1049b0 0x7f73cc19a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73ca800640 1 -- 172.21.15.42:0/1318212219 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73cc1a1690 con 0x7f73cc1037c0 2024-04-25T00:18:02.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73ca800640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73cc1037c0 0x7f73cc199b10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e9f0 tx=0x7f73bc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:02.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73c3e00640 1 -- 172.21.15.42:0/1318212219 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00ccc0 con 0x7f73cc1037c0 2024-04-25T00:18:02.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73d0a00640 1 -- 172.21.15.42:0/1318212219 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73cc1a1980 con 0x7f73cc1037c0 2024-04-25T00:18:02.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73c3e00640 1 -- 172.21.15.42:0/1318212219 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f73bc0035f0 con 0x7f73cc1037c0 2024-04-25T00:18:02.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73c3e00640 1 -- 172.21.15.42:0/1318212219 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc00ce60 con 0x7f73cc1037c0 2024-04-25T00:18:02.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.579+0000 7f73d0a00640 1 -- 172.21.15.42:0/1318212219 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73cc1a1ea0 con 0x7f73cc1037c0 2024-04-25T00:18:02.581 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.580+0000 7f73d0a00640 1 -- 172.21.15.42:0/1318212219 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73cc10aae0 con 0x7f73cc1037c0 2024-04-25T00:18:02.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.583+0000 7f73c3e00640 1 -- 172.21.15.42:0/1318212219 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f73bc0037a0 con 0x7f73cc1037c0 2024-04-25T00:18:02.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.584+0000 7f73c3e00640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f73b00a2b90 0x7f73b00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:02.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.584+0000 7f73c3e00640 1 -- 172.21.15.42:0/1318212219 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f73bc0cf2b0 con 0x7f73cc1037c0 2024-04-25T00:18:02.584 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.584+0000 7f73c1e00640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f73b00a2b90 0x7f73b00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:02.585 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.585+0000 7f73c1e00640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f73b00a2b90 0x7f73b00a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f73b402a930 tx=0x7f73b4030570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:02.588 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.588+0000 7f73c3e00640 1 -- 172.21.15.42:0/1318212219 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f73bc09c1e0 con 0x7f73cc1037c0 2024-04-25T00:18:02.836 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.835+0000 7f73d0a00640 1 -- 172.21.15.42:0/1318212219 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73cc02d170 con 0x7f73cc1037c0 2024-04-25T00:18:02.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.837+0000 7f73c3e00640 1 -- 172.21.15.42:0/1318212219 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11868 (secure 0 0 0) 0x7f73bc09bae0 con 0x7f73cc1037c0 2024-04-25T00:18:02.837 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:02.837 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:16:19.771762+0000","modified":"2024-04-25T00:16:25.317872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24568,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24568":{"gid":24568,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6841/4068466248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4068466248},{"type":"v1","addr":"172.21.15.42:6841","nonce":4068466248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3349564106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3349564106},{"type":"v1","addr":"172.21.15.106:6839","nonce":3349564106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24517,24568,24569]},"id":1}]} 2024-04-25T00:18:02.838 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-04-25T00:18:02.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 -- 172.21.15.42:0/1318212219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f73b00a2b90 msgr2=0x7f73b00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:02.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f73b00a2b90 0x7f73b00a5050 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f73b402a930 tx=0x7f73b4030570 comp rx=0 tx=0).stop 2024-04-25T00:18:02.840 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 -- 172.21.15.42:0/1318212219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73cc1037c0 msgr2=0x7f73cc199b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:02.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73cc1037c0 0x7f73cc199b10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00e9f0 tx=0x7f73bc00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 -- 172.21.15.42:0/1318212219 shutdown_connections 2024-04-25T00:18:02.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f73b00a2b90 0x7f73b00a5050 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f73cc105370 0x7f73cc1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f73cc1049b0 0x7f73cc19a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 --2- 172.21.15.42:0/1318212219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f73cc1037c0 0x7f73cc199b10 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:02.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 -- 172.21.15.42:0/1318212219 >> 172.21.15.42:0/1318212219 conn(0x7f73cc0fed50 msgr2=0x7f73cc107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:02.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 -- 172.21.15.42:0/1318212219 shutdown_connections 2024-04-25T00:18:02.841 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:02.840+0000 7f73d0a00640 1 -- 172.21.15.42:0/1318212219 wait complete. 2024-04-25T00:18:02.859 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:18:03.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:02 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3002912709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:03.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:02 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2701744416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:03.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.071+0000 7f769aa00640 1 -- 172.21.15.42:0/2805190838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76941037c0 msgr2=0x7f7694103be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.071+0000 7f769aa00640 1 --2- 172.21.15.42:0/2805190838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76941037c0 0x7f7694103be0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7688009860 tx=0x7f768802a340 comp rx=0 tx=0).stop 2024-04-25T00:18:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.071+0000 7f769aa00640 1 -- 172.21.15.42:0/2805190838 shutdown_connections 2024-04-25T00:18:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.071+0000 7f769aa00640 1 --2- 172.21.15.42:0/2805190838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7694105370 0x7f769410bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.071+0000 7f769aa00640 1 --2- 172.21.15.42:0/2805190838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76941049b0 0x7f7694104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.071+0000 7f769aa00640 1 --2- 172.21.15.42:0/2805190838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76941037c0 0x7f7694103be0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.071+0000 7f769aa00640 1 -- 172.21.15.42:0/2805190838 >> 172.21.15.42:0/2805190838 conn(0x7f76940fed50 msgr2=0x7f7694101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.071+0000 7f769aa00640 1 -- 172.21.15.42:0/2805190838 shutdown_connections 2024-04-25T00:18:03.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.071+0000 7f769aa00640 1 -- 172.21.15.42:0/2805190838 wait complete. 2024-04-25T00:18:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.072+0000 7f769aa00640 1 Processor -- start 2024-04-25T00:18:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.072+0000 7f769aa00640 1 -- start start 2024-04-25T00:18:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f769aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76941037c0 0x7f7694199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f769aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76941049b0 0x7f7694199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f769aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7694105370 0x7f76941a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f769aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f769410f410 con 0x7f76941037c0 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f769aa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f769410f290 con 0x7f7694105370 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f769aa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f769410f590 con 0x7f76941049b0 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f7699600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76941037c0 0x7f7694199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f7699600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76941037c0 0x7f7694199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38660/0 (socket says 172.21.15.42:38660) 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f7699600640 1 -- 172.21.15.42:0/3026861105 learned_addr learned my addr 172.21.15.42:0/3026861105 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f7698c00640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76941049b0 0x7f7694199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f769a000640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7694105370 0x7f76941a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f7699600640 1 -- 172.21.15.42:0/3026861105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76941049b0 msgr2=0x7f7694199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f7699600640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76941049b0 0x7f7694199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f7699600640 1 -- 172.21.15.42:0/3026861105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7694105370 msgr2=0x7f76941a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f7699600640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7694105370 0x7f76941a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f7699600640 1 -- 172.21.15.42:0/3026861105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76941a16d0 con 0x7f76941037c0 2024-04-25T00:18:03.075 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.073+0000 7f769a000640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7694105370 0x7f76941a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.074+0000 7f7699600640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76941037c0 0x7f7694199ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7688035040 tx=0x7f7688033d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.074+0000 7f7692000640 1 -- 172.21.15.42:0/3026861105 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688037040 con 0x7f76941037c0 2024-04-25T00:18:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.074+0000 7f769aa00640 1 -- 172.21.15.42:0/3026861105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76941a1960 con 0x7f76941037c0 2024-04-25T00:18:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.074+0000 7f7692000640 1 -- 172.21.15.42:0/3026861105 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7688030850 con 0x7f76941037c0 2024-04-25T00:18:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.074+0000 7f7692000640 1 -- 172.21.15.42:0/3026861105 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7688004920 con 0x7f76941037c0 2024-04-25T00:18:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.074+0000 7f769aa00640 1 -- 172.21.15.42:0/3026861105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76941a1ea0 con 0x7f76941037c0 2024-04-25T00:18:03.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.075+0000 7f769aa00640 1 -- 172.21.15.42:0/3026861105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f769410ab60 con 0x7f76941037c0 2024-04-25T00:18:03.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.077+0000 7f7692000640 1 -- 172.21.15.42:0/3026861105 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7688004050 con 0x7f76941037c0 2024-04-25T00:18:03.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.079+0000 7f7692000640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f767c0a2c10 0x7f767c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.079+0000 7f7692000640 1 -- 172.21.15.42:0/3026861105 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f76880eefc0 con 0x7f76941037c0 2024-04-25T00:18:03.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.079+0000 7f7698c00640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f767c0a2c10 0x7f767c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.079 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.079+0000 7f7698c00640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f767c0a2c10 0x7f767c0a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f769419afc0 tx=0x7f768000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:03.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.083+0000 7f7692000640 1 -- 172.21.15.42:0/3026861105 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f76880bbef0 con 0x7f76941037c0 2024-04-25T00:18:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:02 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3002912709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:02 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2701744416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:03.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:02 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3002912709' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:03.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:02 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2701744416' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:03.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.288+0000 7f769aa00640 1 -- 172.21.15.42:0/3026861105 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7694062340 con 0x7f76941037c0 2024-04-25T00:18:03.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.292+0000 7f7692000640 1 -- 172.21.15.42:0/3026861105 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22547 (secure 0 0 0) 0x7f76880bb7f0 con 0x7f76941037c0 2024-04-25T00:18:03.293 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:03.293 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":88,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:17:27.556239+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:03.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 -- 172.21.15.42:0/3026861105 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f767c0a2c10 msgr2=0x7f767c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f767c0a2c10 0x7f767c0a50d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f769419afc0 tx=0x7f768000b410 comp rx=0 tx=0).stop 2024-04-25T00:18:03.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 -- 172.21.15.42:0/3026861105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76941037c0 msgr2=0x7f7694199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76941037c0 0x7f7694199ab0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7688035040 tx=0x7f7688033d90 comp rx=0 tx=0).stop 2024-04-25T00:18:03.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 -- 172.21.15.42:0/3026861105 shutdown_connections 2024-04-25T00:18:03.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f767c0a2c10 0x7f767c0a50d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7694105370 0x7f76941a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76941049b0 0x7f7694199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 --2- 172.21.15.42:0/3026861105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76941037c0 0x7f7694199ab0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 -- 172.21.15.42:0/3026861105 >> 172.21.15.42:0/3026861105 conn(0x7f76940fed50 msgr2=0x7f7694107d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:03.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 -- 172.21.15.42:0/3026861105 shutdown_connections 2024-04-25T00:18:03.297 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.296+0000 7f769aa00640 1 -- 172.21.15.42:0/3026861105 wait complete. 2024-04-25T00:18:03.316 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:11:14.472869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:16:18.168243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:16:19.289585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:18:03.317 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:18:03.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.532+0000 7fdecd600640 1 -- 172.21.15.42:0/1878699399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8109830 msgr2=0x7fdec81100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.532+0000 7fdecd600640 1 --2- 172.21.15.42:0/1878699399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8109830 0x7fdec81100c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009ca0 tx=0x7fdec002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.533+0000 7fdecd600640 1 -- 172.21.15.42:0/1878699399 shutdown_connections 2024-04-25T00:18:03.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.533+0000 7fdecd600640 1 --2- 172.21.15.42:0/1878699399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8109830 0x7fdec81100c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.533+0000 7fdecd600640 1 --2- 172.21.15.42:0/1878699399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec8108e70 0x7fdec81092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.533+0000 7fdecd600640 1 --2- 172.21.15.42:0/1878699399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec8107c80 0x7fdec81080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.533 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.533+0000 7fdecd600640 1 -- 172.21.15.42:0/1878699399 >> 172.21.15.42:0/1878699399 conn(0x7fdec8076c10 msgr2=0x7fdec8079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:03.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.533+0000 7fdecd600640 1 -- 172.21.15.42:0/1878699399 shutdown_connections 2024-04-25T00:18:03.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.533+0000 7fdecd600640 1 -- 172.21.15.42:0/1878699399 wait complete. 2024-04-25T00:18:03.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.534+0000 7fdecd600640 1 Processor -- start 2024-04-25T00:18:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.534+0000 7fdecd600640 1 -- start start 2024-04-25T00:18:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.534+0000 7fdecd600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8107c80 0x7fdec819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdecd600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec8108e70 0x7fdec819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdecd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec8109830 0x7fdec81a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdecd600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8113950 con 0x7fdec8109830 2024-04-25T00:18:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdecd600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdec81137d0 con 0x7fdec8108e70 2024-04-25T00:18:03.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdecd600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8113ad0 con 0x7fdec8107c80 2024-04-25T00:18:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdebfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec8108e70 0x7fdec819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdebfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec8108e70 0x7fdec819e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46886/0 (socket says 172.21.15.42:46886) 2024-04-25T00:18:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdebfe00640 1 -- 172.21.15.42:0/3054139044 learned_addr learned my addr 172.21.15.42:0/3054139044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdec7e00640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8107c80 0x7fdec819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdeccc00640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec8109830 0x7fdec81a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdebfe00640 1 -- 172.21.15.42:0/3054139044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8107c80 msgr2=0x7fdec819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdebfe00640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8107c80 0x7fdec819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdebfe00640 1 -- 172.21.15.42:0/3054139044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec8109830 msgr2=0x7fdec81a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdebfe00640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec8109830 0x7fdec81a5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdebfe00640 1 -- 172.21.15.42:0/3054139044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec81a5af0 con 0x7fdec8108e70 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.535+0000 7fdec7e00640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8107c80 0x7fdec819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.536+0000 7fdebfe00640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec8108e70 0x7fdec819e410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000e9f0 tx=0x7fdeb000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.536+0000 7fdec5600640 1 -- 172.21.15.42:0/3054139044 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb000cd80 con 0x7fdec8108e70 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.536+0000 7fdec5600640 1 -- 172.21.15.42:0/3054139044 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdeb0010910 con 0x7fdec8108e70 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.536+0000 7fdecd600640 1 -- 172.21.15.42:0/3054139044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec81a5de0 con 0x7fdec8108e70 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.536+0000 7fdec5600640 1 -- 172.21.15.42:0/3054139044 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0018990 con 0x7fdec8108e70 2024-04-25T00:18:03.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.536+0000 7fdecd600640 1 -- 172.21.15.42:0/3054139044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec81a62a0 con 0x7fdec8108e70 2024-04-25T00:18:03.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.537+0000 7fdecd600640 1 -- 172.21.15.42:0/3054139044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec810eed0 con 0x7fdec8108e70 2024-04-25T00:18:03.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.539+0000 7fdec5600640 1 -- 172.21.15.42:0/3054139044 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdeb0010ac0 con 0x7fdec8108e70 2024-04-25T00:18:03.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.541+0000 7fdec5600640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdeac0a2b90 0x7fdeac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.541+0000 7fdec5600640 1 -- 172.21.15.42:0/3054139044 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fdeb00cf850 con 0x7fdec8108e70 2024-04-25T00:18:03.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.541+0000 7fdec7e00640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdeac0a2b90 0x7fdeac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.542+0000 7fdec7e00640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdeac0a2b90 0x7fdeac0a5050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdec8108cd0 tx=0x7fdeb8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:03.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.543+0000 7fdec5600640 1 -- 172.21.15.42:0/3054139044 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdeb009c130 con 0x7fdec8108e70 2024-04-25T00:18:03.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.748+0000 7fdecd600640 1 -- 172.21.15.42:0/3054139044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdec802d170 con 0x7fdec8108e70 2024-04-25T00:18:03.749 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.749+0000 7fdec5600640 1 -- 172.21.15.42:0/3054139044 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fdeb00a0430 con 0x7fdec8108e70 2024-04-25T00:18:03.749 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-25T00:18:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 -- 172.21.15.42:0/3054139044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdeac0a2b90 msgr2=0x7fdeac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdeac0a2b90 0x7fdeac0a5050 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdec8108cd0 tx=0x7fdeb8005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 -- 172.21.15.42:0/3054139044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec8108e70 msgr2=0x7fdec819e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec8108e70 0x7fdec819e410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000e9f0 tx=0x7fdeb000eeb0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 -- 172.21.15.42:0/3054139044 shutdown_connections 2024-04-25T00:18:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdeac0a2b90 0x7fdeac0a5050 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdec8109830 0x7fdec81a5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdec8108e70 0x7fdec819e410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 --2- 172.21.15.42:0/3054139044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec8107c80 0x7fdec819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 -- 172.21.15.42:0/3054139044 >> 172.21.15.42:0/3054139044 conn(0x7fdec8076c10 msgr2=0x7fdec810c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 -- 172.21.15.42:0/3054139044 shutdown_connections 2024-04-25T00:18:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.752+0000 7fdecd600640 1 -- 172.21.15.42:0/3054139044 wait complete. 2024-04-25T00:18:03.771 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:18:03.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.988+0000 7f08fe200640 1 -- 172.21.15.42:0/3958555986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f810dd50 msgr2=0x7f08f8110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.988+0000 7f08fe200640 1 --2- 172.21.15.42:0/3958555986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f810dd50 0x7f08f8110140 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009c10 tx=0x7f08f002a6f0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.989+0000 7f08fe200640 1 -- 172.21.15.42:0/3958555986 shutdown_connections 2024-04-25T00:18:03.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.989+0000 7f08fe200640 1 --2- 172.21.15.42:0/3958555986 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f810dd50 0x7f08f8110140 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.989+0000 7f08fe200640 1 --2- 172.21.15.42:0/3958555986 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f80741a0 0x7f08f8074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.989+0000 7f08fe200640 1 --2- 172.21.15.42:0/3958555986 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08f8075780 0x7f08f8073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.989+0000 7f08fe200640 1 -- 172.21.15.42:0/3958555986 >> 172.21.15.42:0/3958555986 conn(0x7f08f80fccf0 msgr2=0x7f08f80ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.989+0000 7f08fe200640 1 -- 172.21.15.42:0/3958555986 shutdown_connections 2024-04-25T00:18:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.989+0000 7f08fe200640 1 -- 172.21.15.42:0/3958555986 wait complete. 2024-04-25T00:18:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.990+0000 7f08fe200640 1 Processor -- start 2024-04-25T00:18:03.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.990+0000 7f08fe200640 1 -- start start 2024-04-25T00:18:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08fe200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f80741a0 0x7f08f819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08fe200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08f8075780 0x7f08f819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08fe200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f810dd50 0x7f08f81a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08fe200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f08f81139d0 con 0x7f08f810dd50 2024-04-25T00:18:03.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08fe200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8113850 con 0x7f08f80741a0 2024-04-25T00:18:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08fe200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f08f8113b50 con 0x7f08f8075780 2024-04-25T00:18:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08f7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f810dd50 0x7f08f81a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08f8075780 0x7f08f819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08f7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f80741a0 0x7f08f819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08f7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f810dd50 0x7f08f81a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38722/0 (socket says 172.21.15.42:38722) 2024-04-25T00:18:03.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.991+0000 7f08f7e00640 1 -- 172.21.15.42:0/4170812698 learned_addr learned my addr 172.21.15.42:0/4170812698 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08f7e00640 1 -- 172.21.15.42:0/4170812698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08f8075780 msgr2=0x7f08f819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08f7e00640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08f8075780 0x7f08f819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08f7e00640 1 -- 172.21.15.42:0/4170812698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f80741a0 msgr2=0x7f08f819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08f7e00640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f80741a0 0x7f08f819e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08f7e00640 1 -- 172.21.15.42:0/4170812698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f81a5c80 con 0x7f08f810dd50 2024-04-25T00:18:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08f7400640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f80741a0 0x7f08f819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08f7e00640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f810dd50 0x7f08f81a5610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009b30 tx=0x7f08f00305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08efe00640 1 -- 172.21.15.42:0/4170812698 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0041020 con 0x7f08f810dd50 2024-04-25T00:18:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08efe00640 1 -- 172.21.15.42:0/4170812698 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f08f0030740 con 0x7f08f810dd50 2024-04-25T00:18:03.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08efe00640 1 -- 172.21.15.42:0/4170812698 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08f0036040 con 0x7f08f810dd50 2024-04-25T00:18:03.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.992+0000 7f08fe200640 1 -- 172.21.15.42:0/4170812698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08f81a5f10 con 0x7f08f810dd50 2024-04-25T00:18:03.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.993+0000 7f08fe200640 1 -- 172.21.15.42:0/4170812698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08f81a6370 con 0x7f08f810dd50 2024-04-25T00:18:03.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.994+0000 7f08fe200640 1 -- 172.21.15.42:0/4170812698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08f8103f20 con 0x7f08f810dd50 2024-04-25T00:18:03.996 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.996+0000 7f08efe00640 1 -- 172.21.15.42:0/4170812698 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f08f0044020 con 0x7f08f810dd50 2024-04-25T00:18:03.997 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.997+0000 7f08efe00640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f08dc0a2b90 0x7f08dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:03.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.997+0000 7f08efe00640 1 -- 172.21.15.42:0/4170812698 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f08f00efde0 con 0x7f08f810dd50 2024-04-25T00:18:03.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.998+0000 7f08f7400640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f08dc0a2b90 0x7f08dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:03.998 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:03.998+0000 7f08f7400640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f08dc0a2b90 0x7f08dc0a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009fc0 tx=0x7f08e8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:04.003 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.003+0000 7f08efe00640 1 -- 172.21.15.42:0/4170812698 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f08f004b020 con 0x7f08f810dd50 2024-04-25T00:18:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:03 smithi136 ceph-mon[32923]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:18:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:03 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1318212219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:03 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3026861105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:03 smithi106 ceph-mon[32510]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:18:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:03 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1318212219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:03 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3026861105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:03 smithi042 ceph-mon[28130]: pgmap v366: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:18:04.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:03 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1318212219' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:04.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:03 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3026861105' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:04.212 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.212+0000 7f08fe200640 1 -- 172.21.15.42:0/4170812698 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f08f80623e0 con 0x7f08f810dd50 2024-04-25T00:18:04.213 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.213+0000 7f08efe00640 1 -- 172.21.15.42:0/4170812698 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f08f00bcd10 con 0x7f08f810dd50 2024-04-25T00:18:04.213 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 -- 172.21.15.42:0/4170812698 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f08dc0a2b90 msgr2=0x7f08dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f08dc0a2b90 0x7f08dc0a5050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009fc0 tx=0x7f08e8005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 -- 172.21.15.42:0/4170812698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f810dd50 msgr2=0x7f08f81a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f810dd50 0x7f08f81a5610 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f08f0009b30 tx=0x7f08f00305d0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 -- 172.21.15.42:0/4170812698 shutdown_connections 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f08dc0a2b90 0x7f08dc0a5050 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f08f810dd50 0x7f08f81a5610 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f08f8075780 0x7f08f819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 --2- 172.21.15.42:0/4170812698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f80741a0 0x7f08f819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 -- 172.21.15.42:0/4170812698 >> 172.21.15.42:0/4170812698 conn(0x7f08f80fccf0 msgr2=0x7f08f81010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.217+0000 7f08fe200640 1 -- 172.21.15.42:0/4170812698 shutdown_connections 2024-04-25T00:18:04.218 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.218+0000 7f08fe200640 1 -- 172.21.15.42:0/4170812698 wait complete. 2024-04-25T00:18:04.245 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:18:04.452 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.452+0000 7f8809600640 1 -- 172.21.15.42:0/3513476897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804102840 msgr2=0x7f8804102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.452+0000 7f8809600640 1 --2- 172.21.15.42:0/3513476897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804102840 0x7f8804102cc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f87f0009860 tx=0x7f87f002a340 comp rx=0 tx=0).stop 2024-04-25T00:18:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.452+0000 7f8809600640 1 -- 172.21.15.42:0/3513476897 shutdown_connections 2024-04-25T00:18:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.452+0000 7f8809600640 1 --2- 172.21.15.42:0/3513476897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8804103200 0x7f8804109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.452+0000 7f8809600640 1 --2- 172.21.15.42:0/3513476897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804102840 0x7f8804102cc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.452+0000 7f8809600640 1 --2- 172.21.15.42:0/3513476897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8804101650 0x7f8804101a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.452+0000 7f8809600640 1 -- 172.21.15.42:0/3513476897 >> 172.21.15.42:0/3513476897 conn(0x7f88040fcbe0 msgr2=0x7f88040ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.452+0000 7f8809600640 1 -- 172.21.15.42:0/3513476897 shutdown_connections 2024-04-25T00:18:04.453 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.452+0000 7f8809600640 1 -- 172.21.15.42:0/3513476897 wait complete. 2024-04-25T00:18:04.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.453+0000 7f8809600640 1 Processor -- start 2024-04-25T00:18:04.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8809600640 1 -- start start 2024-04-25T00:18:04.454 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8809600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804101650 0x7f8804199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8809600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8804102840 0x7f880419a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8809600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8804103200 0x7f88041a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8809600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f880410d2e0 con 0x7f8804102840 2024-04-25T00:18:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8809600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f880410d160 con 0x7f8804101650 2024-04-25T00:18:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8809600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f880410d460 con 0x7f8804103200 2024-04-25T00:18:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8803400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8804102840 0x7f880419a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8803400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8804102840 0x7f880419a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38744/0 (socket says 172.21.15.42:38744) 2024-04-25T00:18:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8803400640 1 -- 172.21.15.42:0/1487788318 learned_addr learned my addr 172.21.15.42:0/1487788318 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8808c00640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8804103200 0x7f88041a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:04.455 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.454+0000 7f8803e00640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804101650 0x7f8804199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.455+0000 7f8808c00640 1 -- 172.21.15.42:0/1487788318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804101650 msgr2=0x7f8804199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.455+0000 7f8808c00640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804101650 0x7f8804199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.455+0000 7f8808c00640 1 -- 172.21.15.42:0/1487788318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8804102840 msgr2=0x7f880419a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.455+0000 7f8808c00640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8804102840 0x7f880419a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.455+0000 7f8808c00640 1 -- 172.21.15.42:0/1487788318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88041a1760 con 0x7f8804103200 2024-04-25T00:18:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.455+0000 7f8803400640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8804102840 0x7f880419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:04.456 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.455+0000 7f8808c00640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8804103200 0x7f88041a10f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00ef00 tx=0x7f87fc00c740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:04.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.455+0000 7f8800c00640 1 -- 172.21.15.42:0/1487788318 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc004b20 con 0x7f8804103200 2024-04-25T00:18:04.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.456+0000 7f8809600640 1 -- 172.21.15.42:0/1487788318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88041a1a50 con 0x7f8804103200 2024-04-25T00:18:04.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.456+0000 7f8800c00640 1 -- 172.21.15.42:0/1487788318 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f87fc003e50 con 0x7f8804103200 2024-04-25T00:18:04.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.456+0000 7f8800c00640 1 -- 172.21.15.42:0/1487788318 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00b570 con 0x7f8804103200 2024-04-25T00:18:04.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.456+0000 7f8809600640 1 -- 172.21.15.42:0/1487788318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88041a1f50 con 0x7f8804103200 2024-04-25T00:18:04.457 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.457+0000 7f8809600640 1 -- 172.21.15.42:0/1487788318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88041a2180 con 0x7f8804103200 2024-04-25T00:18:04.459 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.459+0000 7f8800c00640 1 -- 172.21.15.42:0/1487788318 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87fc01d020 con 0x7f8804103200 2024-04-25T00:18:04.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.460+0000 7f8800c00640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f87e80a2b90 0x7f87e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:04.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.460+0000 7f8800c00640 1 -- 172.21.15.42:0/1487788318 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f87fc0ceef0 con 0x7f8804103200 2024-04-25T00:18:04.460 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.460+0000 7f8803e00640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f87e80a2b90 0x7f87e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:04.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.461+0000 7f8803e00640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f87e80a2b90 0x7f87e80a5050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f88041026a0 tx=0x7f87f4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:04.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.465+0000 7f8800c00640 1 -- 172.21.15.42:0/1487788318 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f87fc09be20 con 0x7f8804103200 2024-04-25T00:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.671+0000 7f8809600640 1 -- 172.21.15.42:0/1487788318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f88041a2510 con 0x7f8804103200 2024-04-25T00:18:04.671 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.672+0000 7f8800c00640 1 -- 172.21.15.42:0/1487788318 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f87fc09b720 con 0x7f8804103200 2024-04-25T00:18:04.672 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-25T00:18:04.674 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.674+0000 7f8809600640 1 -- 172.21.15.42:0/1487788318 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f87e80a2b90 msgr2=0x7f87e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.674+0000 7f8809600640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f87e80a2b90 0x7f87e80a5050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f88041026a0 tx=0x7f87f4005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.674+0000 7f8809600640 1 -- 172.21.15.42:0/1487788318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8804103200 msgr2=0x7f88041a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.674+0000 7f8809600640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8804103200 0x7f88041a10f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00ef00 tx=0x7f87fc00c740 comp rx=0 tx=0).stop 2024-04-25T00:18:04.675 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.674+0000 7f8809600640 1 -- 172.21.15.42:0/1487788318 shutdown_connections 2024-04-25T00:18:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.674+0000 7f8809600640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f87e80a2b90 0x7f87e80a5050 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.674+0000 7f8809600640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8804103200 0x7f88041a10f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.675+0000 7f8809600640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8804102840 0x7f880419a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.675+0000 7f8809600640 1 --2- 172.21.15.42:0/1487788318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804101650 0x7f8804199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.675+0000 7f8809600640 1 -- 172.21.15.42:0/1487788318 >> 172.21.15.42:0/1487788318 conn(0x7f88040fcbe0 msgr2=0x7f8804073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.675+0000 7f8809600640 1 -- 172.21.15.42:0/1487788318 shutdown_connections 2024-04-25T00:18:04.676 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.675+0000 7f8809600640 1 -- 172.21.15.42:0/1487788318 wait complete. 2024-04-25T00:18:04.695 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:18:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.919+0000 7fa448c00640 1 -- 172.21.15.42:0/257886720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa444102200 msgr2=0x7fa4441045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.919+0000 7fa448c00640 1 --2- 172.21.15.42:0/257886720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa444102200 0x7fa4441045f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa434009900 tx=0x7fa43402a410 comp rx=0 tx=0).stop 2024-04-25T00:18:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.919+0000 7fa448c00640 1 -- 172.21.15.42:0/257886720 shutdown_connections 2024-04-25T00:18:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.919+0000 7fa448c00640 1 --2- 172.21.15.42:0/257886720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa444107460 0x7fa4441098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.919+0000 7fa448c00640 1 --2- 172.21.15.42:0/257886720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa444104b30 0x7fa444106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.919+0000 7fa448c00640 1 --2- 172.21.15.42:0/257886720 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa444102200 0x7fa4441045f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.919+0000 7fa448c00640 1 -- 172.21.15.42:0/257886720 >> 172.21.15.42:0/257886720 conn(0x7fa4440fbbf0 msgr2=0x7fa4440fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:04.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.919+0000 7fa448c00640 1 -- 172.21.15.42:0/257886720 shutdown_connections 2024-04-25T00:18:04.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.919+0000 7fa448c00640 1 -- 172.21.15.42:0/257886720 wait complete. 2024-04-25T00:18:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.920+0000 7fa448c00640 1 Processor -- start 2024-04-25T00:18:04.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.920+0000 7fa448c00640 1 -- start start 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa448c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa444102200 0x7fa444101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa448c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa444104b30 0x7fa4441001b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa448c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa444107460 0x7fa4441006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa448c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410d480 con 0x7fa444102200 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa448c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410d300 con 0x7fa444107460 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa448c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa44410d600 con 0x7fa444104b30 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa441e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa444102200 0x7fa444101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa441e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa444102200 0x7fa444101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38756/0 (socket says 172.21.15.42:38756) 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa441e00640 1 -- 172.21.15.42:0/3006079818 learned_addr learned my addr 172.21.15.42:0/3006079818 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa441400640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa444104b30 0x7fa4441001b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa442800640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa444107460 0x7fa4441006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa441400640 1 -- 172.21.15.42:0/3006079818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa444107460 msgr2=0x7fa4441006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa441400640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa444107460 0x7fa4441006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa441400640 1 -- 172.21.15.42:0/3006079818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa444102200 msgr2=0x7fa444101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa441400640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa444102200 0x7fa444101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa441400640 1 -- 172.21.15.42:0/3006079818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa444100f80 con 0x7fa444104b30 2024-04-25T00:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.921+0000 7fa442800640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa444107460 0x7fa4441006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.922+0000 7fa441400640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa444104b30 0x7fa4441001b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa43000e9c0 tx=0x7fa43000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.922+0000 7fa43aa00640 1 -- 172.21.15.42:0/3006079818 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43000ccc0 con 0x7fa444104b30 2024-04-25T00:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.922+0000 7fa43aa00640 1 -- 172.21.15.42:0/3006079818 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa430005020 con 0x7fa444104b30 2024-04-25T00:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.922+0000 7fa448c00640 1 -- 172.21.15.42:0/3006079818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4441a1bf0 con 0x7fa444104b30 2024-04-25T00:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.922+0000 7fa43aa00640 1 -- 172.21.15.42:0/3006079818 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430004a00 con 0x7fa444104b30 2024-04-25T00:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.922+0000 7fa448c00640 1 -- 172.21.15.42:0/3006079818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4441a2140 con 0x7fa444104b30 2024-04-25T00:18:04.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.924+0000 7fa448c00640 1 -- 172.21.15.42:0/3006079818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4440fd6e0 con 0x7fa444104b30 2024-04-25T00:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.928+0000 7fa43aa00640 1 -- 172.21.15.42:0/3006079818 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa4300051d0 con 0x7fa444104b30 2024-04-25T00:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.929+0000 7fa43aa00640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4280a2c10 0x7fa4280a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.930+0000 7fa43aa00640 1 -- 172.21.15.42:0/3006079818 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa4300cefa0 con 0x7fa444104b30 2024-04-25T00:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.930+0000 7fa441e00640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4280a2c10 0x7fa4280a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:04.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.930+0000 7fa441e00640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4280a2c10 0x7fa4280a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa4340096c0 tx=0x7fa4340315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:04.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:04.934+0000 7fa43aa00640 1 -- 172.21.15.42:0/3006079818 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fa43009bed0 con 0x7fa444104b30 2024-04-25T00:18:05.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:04 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3054139044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:05.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:04 smithi136 ceph-mon[32923]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:18:05.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:04 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/4170812698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:05.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:04 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1487788318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:05.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:04 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3054139044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:05.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:04 smithi106 ceph-mon[32510]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:18:05.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:04 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/4170812698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:05.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:04 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1487788318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:05.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:04 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3054139044' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:05.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:04 smithi042 ceph-mon[28130]: pgmap v367: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:18:05.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:04 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/4170812698' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:05.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:04 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1487788318' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:05.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.177+0000 7fa448c00640 1 -- 172.21.15.42:0/3006079818 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa444062340 con 0x7fa444104b30 2024-04-25T00:18:05.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.178+0000 7fa43aa00640 1 -- 172.21.15.42:0/3006079818 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11868 (secure 0 0 0) 0x7fa43009b7d0 con 0x7fa444104b30 2024-04-25T00:18:05.179 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:05.179 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:16:19.771762+0000","modified":"2024-04-25T00:16:25.317872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24568,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24568":{"gid":24568,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6841/4068466248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4068466248},{"type":"v1","addr":"172.21.15.42:6841","nonce":4068466248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3349564106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3349564106},{"type":"v1","addr":"172.21.15.106:6839","nonce":3349564106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24517,24568,24569]},"id":1}]} 2024-04-25T00:18:05.179 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-04-25T00:18:05.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 -- 172.21.15.42:0/3006079818 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4280a2c10 msgr2=0x7fa4280a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4280a2c10 0x7fa4280a50d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa4340096c0 tx=0x7fa4340315b0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 -- 172.21.15.42:0/3006079818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa444104b30 msgr2=0x7fa4441001b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa444104b30 0x7fa4441001b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa43000e9c0 tx=0x7fa43000ee80 comp rx=0 tx=0).stop 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 -- 172.21.15.42:0/3006079818 shutdown_connections 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4280a2c10 0x7fa4280a50d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa444107460 0x7fa4441006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa444104b30 0x7fa4441001b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 --2- 172.21.15.42:0/3006079818 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa444102200 0x7fa444101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 -- 172.21.15.42:0/3006079818 >> 172.21.15.42:0/3006079818 conn(0x7fa4440fbbf0 msgr2=0x7fa444102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 -- 172.21.15.42:0/3006079818 shutdown_connections 2024-04-25T00:18:05.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.181+0000 7fa448c00640 1 -- 172.21.15.42:0/3006079818 wait complete. 2024-04-25T00:18:05.200 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:18:05.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.406+0000 7fd355800640 1 -- 172.21.15.42:0/3304055611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd350109770 msgr2=0x7fd350110000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:05.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.406+0000 7fd355800640 1 --2- 172.21.15.42:0/3304055611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd350109770 0x7fd350110000 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd348009ca0 tx=0x7fd34802a8b0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.406+0000 7fd355800640 1 -- 172.21.15.42:0/3304055611 shutdown_connections 2024-04-25T00:18:05.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.406+0000 7fd355800640 1 --2- 172.21.15.42:0/3304055611 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd350109770 0x7fd350110000 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.406+0000 7fd355800640 1 --2- 172.21.15.42:0/3304055611 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd350108db0 0x7fd350109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.406+0000 7fd355800640 1 --2- 172.21.15.42:0/3304055611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd350107bc0 0x7fd350107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.406+0000 7fd355800640 1 -- 172.21.15.42:0/3304055611 >> 172.21.15.42:0/3304055611 conn(0x7fd350076b80 msgr2=0x7fd350078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:05.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.406+0000 7fd355800640 1 -- 172.21.15.42:0/3304055611 shutdown_connections 2024-04-25T00:18:05.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.406+0000 7fd355800640 1 -- 172.21.15.42:0/3304055611 wait complete. 2024-04-25T00:18:05.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.407+0000 7fd355800640 1 Processor -- start 2024-04-25T00:18:05.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.407+0000 7fd355800640 1 -- start start 2024-04-25T00:18:05.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.407+0000 7fd355800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd350107bc0 0x7fd35019de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:05.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.407+0000 7fd355800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd350108db0 0x7fd35019e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:05.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.407+0000 7fd355800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd350109770 0x7fd3501a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:05.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.407+0000 7fd355800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd350113820 con 0x7fd350107bc0 2024-04-25T00:18:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.407+0000 7fd355800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd3501136a0 con 0x7fd350108db0 2024-04-25T00:18:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.407+0000 7fd355800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3501139a0 con 0x7fd350109770 2024-04-25T00:18:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd34fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd350107bc0 0x7fd35019de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd34fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd350107bc0 0x7fd35019de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38762/0 (socket says 172.21.15.42:38762) 2024-04-25T00:18:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd34fe00640 1 -- 172.21.15.42:0/404379398 learned_addr learned my addr 172.21.15.42:0/404379398 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd347e00640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd350108db0 0x7fd35019e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd354e00640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd350109770 0x7fd3501a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd34fe00640 1 -- 172.21.15.42:0/404379398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd350109770 msgr2=0x7fd3501a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:05.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd34fe00640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd350109770 0x7fd3501a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd34fe00640 1 -- 172.21.15.42:0/404379398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd350108db0 msgr2=0x7fd35019e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd34fe00640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd350108db0 0x7fd35019e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd34fe00640 1 -- 172.21.15.42:0/404379398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3501a5a80 con 0x7fd350107bc0 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd354e00640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd350109770 0x7fd3501a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd347e00640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd350108db0 0x7fd35019e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.408+0000 7fd34fe00640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd350107bc0 0x7fd35019de60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd34000e9c0 tx=0x7fd34000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.409+0000 7fd34d600640 1 -- 172.21.15.42:0/404379398 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34000cd80 con 0x7fd350107bc0 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.409+0000 7fd34d600640 1 -- 172.21.15.42:0/404379398 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3400035f0 con 0x7fd350107bc0 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.409+0000 7fd355800640 1 -- 172.21.15.42:0/404379398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3501a5d70 con 0x7fd350107bc0 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.409+0000 7fd34d600640 1 -- 172.21.15.42:0/404379398 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340010690 con 0x7fd350107bc0 2024-04-25T00:18:05.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.409+0000 7fd355800640 1 -- 172.21.15.42:0/404379398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3501a6230 con 0x7fd350107bc0 2024-04-25T00:18:05.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.410+0000 7fd355800640 1 -- 172.21.15.42:0/404379398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd35010ee80 con 0x7fd350107bc0 2024-04-25T00:18:05.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.412+0000 7fd34d600640 1 -- 172.21.15.42:0/404379398 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd3400037a0 con 0x7fd350107bc0 2024-04-25T00:18:05.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.413+0000 7fd34d600640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3340a2c10 0x7fd3340a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:05.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.413+0000 7fd34d600640 1 -- 172.21.15.42:0/404379398 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd3400cf500 con 0x7fd350107bc0 2024-04-25T00:18:05.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.414+0000 7fd347e00640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3340a2c10 0x7fd3340a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:05.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.414+0000 7fd347e00640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3340a2c10 0x7fd3340a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd35019f370 tx=0x7fd338005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:05.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.416+0000 7fd34d600640 1 -- 172.21.15.42:0/404379398 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd34009c4c0 con 0x7fd350107bc0 2024-04-25T00:18:05.626 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.625+0000 7fd355800640 1 -- 172.21.15.42:0/404379398 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd35002d3d0 con 0x7fd350107bc0 2024-04-25T00:18:05.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.630+0000 7fd34d600640 1 -- 172.21.15.42:0/404379398 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22547 (secure 0 0 0) 0x7fd34009bdc0 con 0x7fd350107bc0 2024-04-25T00:18:05.630 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:05.631 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":88,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:17:27.556239+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:05.632 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.632+0000 7fd355800640 1 -- 172.21.15.42:0/404379398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3340a2c10 msgr2=0x7fd3340a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.632+0000 7fd355800640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3340a2c10 0x7fd3340a50d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd35019f370 tx=0x7fd338005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.632+0000 7fd355800640 1 -- 172.21.15.42:0/404379398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd350107bc0 msgr2=0x7fd35019de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.632+0000 7fd355800640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd350107bc0 0x7fd35019de60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd34000e9c0 tx=0x7fd34000ee80 comp rx=0 tx=0).stop 2024-04-25T00:18:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.633+0000 7fd355800640 1 -- 172.21.15.42:0/404379398 shutdown_connections 2024-04-25T00:18:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.633+0000 7fd355800640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3340a2c10 0x7fd3340a50d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.633 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.633+0000 7fd355800640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd350109770 0x7fd3501a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.633+0000 7fd355800640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd350108db0 0x7fd35019e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.633+0000 7fd355800640 1 --2- 172.21.15.42:0/404379398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd350107bc0 0x7fd35019de60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.633+0000 7fd355800640 1 -- 172.21.15.42:0/404379398 >> 172.21.15.42:0/404379398 conn(0x7fd350076b80 msgr2=0x7fd35010c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.633+0000 7fd355800640 1 -- 172.21.15.42:0/404379398 shutdown_connections 2024-04-25T00:18:05.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.633+0000 7fd355800640 1 -- 172.21.15.42:0/404379398 wait complete. 2024-04-25T00:18:05.652 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:18:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.871+0000 7faeb2400640 1 -- 172.21.15.42:0/3386412489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faeac101880 msgr2=0x7faeac101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.871+0000 7faeb2400640 1 --2- 172.21.15.42:0/3386412489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faeac101880 0x7faeac101d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae98009860 tx=0x7fae9802a340 comp rx=0 tx=0).stop 2024-04-25T00:18:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.871+0000 7faeb2400640 1 -- 172.21.15.42:0/3386412489 shutdown_connections 2024-04-25T00:18:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.871+0000 7faeb2400640 1 --2- 172.21.15.42:0/3386412489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faeac102240 0x7faeac108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.871+0000 7faeb2400640 1 --2- 172.21.15.42:0/3386412489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faeac101880 0x7faeac101d00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.871+0000 7faeb2400640 1 --2- 172.21.15.42:0/3386412489 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faeac100690 0x7faeac100ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.871+0000 7faeb2400640 1 -- 172.21.15.42:0/3386412489 >> 172.21.15.42:0/3386412489 conn(0x7faeac0fbc20 msgr2=0x7faeac0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.872+0000 7faeb2400640 1 -- 172.21.15.42:0/3386412489 shutdown_connections 2024-04-25T00:18:05.872 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.872+0000 7faeb2400640 1 -- 172.21.15.42:0/3386412489 wait complete. 2024-04-25T00:18:05.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.873+0000 7faeb2400640 1 Processor -- start 2024-04-25T00:18:05.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.873+0000 7faeb2400640 1 -- start start 2024-04-25T00:18:05.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.873+0000 7faeb2400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faeac100690 0x7faeac10b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:05.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.873+0000 7faeb2400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faeac101880 0x7faeac109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.873+0000 7faeb2400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faeac102240 0x7faeac109b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.873+0000 7faeb2400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faeac10d430 con 0x7faeac100690 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.873+0000 7faeb2400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faeac10d2b0 con 0x7faeac102240 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.873+0000 7faeb2400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faeac10d5b0 con 0x7faeac101880 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.874+0000 7faeaaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faeac101880 0x7faeac109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.874+0000 7faeab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faeac100690 0x7faeac10b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.874+0000 7faeaaa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faeac101880 0x7faeac109650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39336/0 (socket says 172.21.15.42:39336) 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.874+0000 7faeaaa00640 1 -- 172.21.15.42:0/3798609846 learned_addr learned my addr 172.21.15.42:0/3798609846 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.874+0000 7faeabe00640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faeac102240 0x7faeac109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.874+0000 7faeaaa00640 1 -- 172.21.15.42:0/3798609846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faeac102240 msgr2=0x7faeac109b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.874+0000 7faeaaa00640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faeac102240 0x7faeac109b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.874+0000 7faeaaa00640 1 -- 172.21.15.42:0/3798609846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faeac100690 msgr2=0x7faeac10b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.874+0000 7faeaaa00640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faeac100690 0x7faeac10b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.874+0000 7faeaaa00640 1 -- 172.21.15.42:0/3798609846 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeac10a420 con 0x7faeac101880 2024-04-25T00:18:05.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.875+0000 7faeaaa00640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faeac101880 0x7faeac109650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae98035040 tx=0x7fae98033d90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:05.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.875+0000 7faea3e00640 1 -- 172.21.15.42:0/3798609846 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98037040 con 0x7faeac101880 2024-04-25T00:18:05.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.875+0000 7faeb2400640 1 -- 172.21.15.42:0/3798609846 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeac1a1b40 con 0x7faeac101880 2024-04-25T00:18:05.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.875+0000 7faea3e00640 1 -- 172.21.15.42:0/3798609846 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae98032770 con 0x7faeac101880 2024-04-25T00:18:05.876 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.875+0000 7faea3e00640 1 -- 172.21.15.42:0/3798609846 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae980046c0 con 0x7faeac101880 2024-04-25T00:18:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.875+0000 7faeb2400640 1 -- 172.21.15.42:0/3798609846 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeac1a20e0 con 0x7faeac101880 2024-04-25T00:18:05.877 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.876+0000 7faeb2400640 1 -- 172.21.15.42:0/3798609846 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeac1079b0 con 0x7faeac101880 2024-04-25T00:18:05.878 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.878+0000 7faea3e00640 1 -- 172.21.15.42:0/3798609846 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae98004050 con 0x7faeac101880 2024-04-25T00:18:05.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.880+0000 7faea3e00640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fae900a2c10 0x7fae900a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:05.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.880+0000 7faea3e00640 1 -- 172.21.15.42:0/3798609846 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fae980ef170 con 0x7faeac101880 2024-04-25T00:18:05.880 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.880+0000 7faeab400640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fae900a2c10 0x7fae900a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:05.881 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.881+0000 7faeab400640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fae900a2c10 0x7fae900a50d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faeac1016e0 tx=0x7fae9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:05.882 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:05.882+0000 7faea3e00640 1 -- 172.21.15.42:0/3798609846 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fae980bc0a0 con 0x7faeac101880 2024-04-25T00:18:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:05 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3006079818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:05 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/404379398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:05 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3006079818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:05 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/404379398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:05 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3006079818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:06.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:05 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/404379398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:06.128 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.128+0000 7faeb2400640 1 -- 172.21.15.42:0/3798609846 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faeac02d170 con 0x7faeac101880 2024-04-25T00:18:06.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.129+0000 7faea3e00640 1 -- 172.21.15.42:0/3798609846 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11868 (secure 0 0 0) 0x7fae980bb9a0 con 0x7faeac101880 2024-04-25T00:18:06.129 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:06.129 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:16:19.771762+0000","modified":"2024-04-25T00:16:25.317872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24568,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24568":{"gid":24568,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6841/4068466248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4068466248},{"type":"v1","addr":"172.21.15.42:6841","nonce":4068466248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3349564106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3349564106},{"type":"v1","addr":"172.21.15.106:6839","nonce":3349564106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24517,24568,24569]},"id":1}]} 2024-04-25T00:18:06.130 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-04-25T00:18:06.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.131+0000 7faeb2400640 1 -- 172.21.15.42:0/3798609846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fae900a2c10 msgr2=0x7fae900a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fae900a2c10 0x7fae900a50d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7faeac1016e0 tx=0x7fae9c009260 comp rx=0 tx=0).stop 2024-04-25T00:18:06.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 -- 172.21.15.42:0/3798609846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faeac101880 msgr2=0x7faeac109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faeac101880 0x7faeac109650 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fae98035040 tx=0x7fae98033d90 comp rx=0 tx=0).stop 2024-04-25T00:18:06.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 -- 172.21.15.42:0/3798609846 shutdown_connections 2024-04-25T00:18:06.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fae900a2c10 0x7fae900a50d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faeac102240 0x7faeac109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faeac101880 0x7faeac109650 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 --2- 172.21.15.42:0/3798609846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faeac100690 0x7faeac10b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 -- 172.21.15.42:0/3798609846 >> 172.21.15.42:0/3798609846 conn(0x7faeac0fbc20 msgr2=0x7faeac104b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 -- 172.21.15.42:0/3798609846 shutdown_connections 2024-04-25T00:18:06.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.132+0000 7faeb2400640 1 -- 172.21.15.42:0/3798609846 wait complete. 2024-04-25T00:18:06.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T00:16:19.771762+0000', 'modified': '2024-04-25T00:16:25.317872+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24569, 'mds_1': 24568, 'mds_2': 24517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/12913503', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 12913503}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 12913503}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.136:6837/222488442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 222488442}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 222488442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/1331229112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 1331229112}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 1331229112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6839/501637536', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 501637536}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 501637536}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24568': {'gid': 24568, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.42:6841/4068466248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6840', 'nonce': 4068466248}, {'type': 'v1', 'addr': '172.21.15.42:6841', 'nonce': 4068466248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/3349564106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 3349564106}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 3349564106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24569, 'qdb_cluster': [24517, 24568, 24569]} 2024-04-25T00:18:06.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T00:18:06.153 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:18:06.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.362+0000 7fa31be00640 1 -- 172.21.15.42:0/3529260907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa31c109830 msgr2=0x7fa31c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.362+0000 7fa31be00640 1 --2- 172.21.15.42:0/3529260907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa31c109830 0x7fa31c1100c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa314009ca0 tx=0x7fa31402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.363+0000 7fa31be00640 1 -- 172.21.15.42:0/3529260907 shutdown_connections 2024-04-25T00:18:06.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.363+0000 7fa31be00640 1 --2- 172.21.15.42:0/3529260907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa31c109830 0x7fa31c1100c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.363 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.363+0000 7fa31be00640 1 --2- 172.21.15.42:0/3529260907 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31c108e70 0x7fa31c1092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.363+0000 7fa31be00640 1 --2- 172.21.15.42:0/3529260907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31c107c80 0x7fa31c1080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.363+0000 7fa31be00640 1 -- 172.21.15.42:0/3529260907 >> 172.21.15.42:0/3529260907 conn(0x7fa31c076c10 msgr2=0x7fa31c079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.363+0000 7fa31be00640 1 -- 172.21.15.42:0/3529260907 shutdown_connections 2024-04-25T00:18:06.364 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.363+0000 7fa31be00640 1 -- 172.21.15.42:0/3529260907 wait complete. 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31be00640 1 Processor -- start 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31be00640 1 -- start start 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31c107c80 0x7fa31c19ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa31c108e70 0x7fa31c19e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31c109830 0x7fa31c1a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c113950 con 0x7fa31c109830 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c1137d0 con 0x7fa31c107c80 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa31c113ad0 con 0x7fa31c108e70 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31c109830 0x7fa31c1a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31c109830 0x7fa31c1a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38784/0 (socket says 172.21.15.42:38784) 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.364+0000 7fa31b400640 1 -- 172.21.15.42:0/1549729241 learned_addr learned my addr 172.21.15.42:0/1549729241 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.366 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.365+0000 7fa31aa00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31c107c80 0x7fa31c19ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.365+0000 7fa312000640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa31c108e70 0x7fa31c19e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.365+0000 7fa31b400640 1 -- 172.21.15.42:0/1549729241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa31c108e70 msgr2=0x7fa31c19e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.365+0000 7fa31b400640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa31c108e70 0x7fa31c19e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.365+0000 7fa31b400640 1 -- 172.21.15.42:0/1549729241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31c107c80 msgr2=0x7fa31c19ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.365+0000 7fa31b400640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31c107c80 0x7fa31c19ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.365+0000 7fa31b400640 1 -- 172.21.15.42:0/1549729241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31c1a5af0 con 0x7fa31c109830 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.365+0000 7fa31aa00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31c107c80 0x7fa31c19ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.365+0000 7fa31b400640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31c109830 0x7fa31c1a5480 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa314009430 tx=0x7fa3140319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.366+0000 7fa313e00640 1 -- 172.21.15.42:0/1549729241 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314003b70 con 0x7fa31c109830 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.366+0000 7fa31be00640 1 -- 172.21.15.42:0/1549729241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31c1a5d80 con 0x7fa31c109830 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.366+0000 7fa313e00640 1 -- 172.21.15.42:0/1549729241 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa314003d10 con 0x7fa31c109830 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.366+0000 7fa313e00640 1 -- 172.21.15.42:0/1549729241 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31402fc50 con 0x7fa31c109830 2024-04-25T00:18:06.367 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.366+0000 7fa31be00640 1 -- 172.21.15.42:0/1549729241 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa31c1a6270 con 0x7fa31c109830 2024-04-25T00:18:06.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.369+0000 7fa313e00640 1 -- 172.21.15.42:0/1549729241 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa314004040 con 0x7fa31c109830 2024-04-25T00:18:06.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.369+0000 7fa32479a740 1 Processor -- start 2024-04-25T00:18:06.369 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.369+0000 7fa32479a740 1 -- start start 2024-04-25T00:18:06.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa32479a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad401d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa32479a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad402360 0x55b8ad406e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa32479a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad409960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa32479a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad301ed0 con 0x55b8ad4074d0 2024-04-25T00:18:06.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa32479a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad364df0 con 0x55b8ad402360 2024-04-25T00:18:06.371 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa32479a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad30d8d0 con 0x55b8ad401970 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa312000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad409960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa312000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad401d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa312000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad401d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39346/0 (socket says 172.21.15.42:39346) 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa312000640 1 -- 172.21.15.42:0/2979135715 learned_addr learned my addr 172.21.15.42:0/2979135715 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa31aa00640 1 --2- 172.21.15.42:0/2979135715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad402360 0x55b8ad406e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa312000640 1 -- 172.21.15.42:0/2979135715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad402360 msgr2=0x55b8ad406e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa312000640 1 --2- 172.21.15.42:0/2979135715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad402360 0x55b8ad406e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa312000640 1 -- 172.21.15.42:0/2979135715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 msgr2=0x55b8ad409960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa312000640 1 --2- 172.21.15.42:0/2979135715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad409960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.370+0000 7fa312000640 1 -- 172.21.15.42:0/2979135715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ad0792d0 con 0x55b8ad401970 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.371+0000 7fa312000640 1 --2- 172.21.15.42:0/2979135715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad401d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa304012790 tx=0x7fa30400b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68bbdd62e8995b4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.371+0000 7fa313e00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa3000a2b90 0x7fa3000a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.371+0000 7fa30be00640 1 -- 172.21.15.42:0/2979135715 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304004050 con 0x55b8ad401970 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.371+0000 7fa30be00640 1 -- 172.21.15.42:0/2979135715 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa304005500 con 0x55b8ad401970 2024-04-25T00:18:06.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.371+0000 7fa30be00640 1 -- 172.21.15.42:0/2979135715 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa304004ce0 con 0x55b8ad401970 2024-04-25T00:18:06.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.371+0000 7fa313e00640 1 -- 172.21.15.42:0/1549729241 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa3140f04f0 con 0x7fa31c109830 2024-04-25T00:18:06.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.371+0000 7fa31aa00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa3000a2b90 0x7fa3000a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa32479a740 1 -- 172.21.15.42:0/2979135715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 msgr2=0x55b8ad401d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa32479a740 1 --2- 172.21.15.42:0/2979135715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad401d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa304012790 tx=0x7fa30400b390 comp rx=0 tx=0).stop 2024-04-25T00:18:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa31aa00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa3000a2b90 0x7fa3000a5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55b8ad402760 tx=0x7fa30c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa32479a740 1 -- 172.21.15.42:0/2979135715 shutdown_connections 2024-04-25T00:18:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa32479a740 1 --2- 172.21.15.42:0/2979135715 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad409960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa32479a740 1 --2- 172.21.15.42:0/2979135715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad402360 0x55b8ad406e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa32479a740 1 --2- 172.21.15.42:0/2979135715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad401d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa32479a740 1 -- 172.21.15.42:0/2979135715 >> 172.21.15.42:0/2979135715 conn(0x55b8ad3fcfe0 msgr2=0x55b8ad3fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa32479a740 1 -- 172.21.15.42:0/2979135715 shutdown_connections 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa32479a740 1 -- 172.21.15.42:0/2979135715 wait complete. 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.372+0000 7fa32479a740 1 -- start start 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa32479a740 1 Processor -- start 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa32479a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad40bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa32479a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad402360 0x55b8ad40c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa32479a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad413220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa32479a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad320d20 con 0x55b8ad4074d0 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa32479a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad360db0 con 0x55b8ad402360 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa32479a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad40a480 con 0x55b8ad401970 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa32479a740 10 client.0 _ll_register_callbacks cb 0x55b8ad35bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa31aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad40bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa31b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad413220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa31aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad40bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39362/0 (socket says 172.21.15.42:39362) 2024-04-25T00:18:06.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.373+0000 7fa31aa00640 1 -- 172.21.15.42:0/3234257335 learned_addr learned my addr 172.21.15.42:0/3234257335 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.374+0000 7fa312000640 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad402360 0x55b8ad40c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.374+0000 7fa31aa00640 1 -- 172.21.15.42:0/3234257335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad402360 msgr2=0x55b8ad40c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.374+0000 7fa31aa00640 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad402360 0x55b8ad40c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.374+0000 7fa31aa00640 1 -- 172.21.15.42:0/3234257335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 msgr2=0x55b8ad413220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.374+0000 7fa31aa00640 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad413220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.374+0000 7fa31aa00640 1 -- 172.21.15.42:0/3234257335 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ad1436b0 con 0x55b8ad401970 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.374+0000 7fa31aa00640 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad40bcb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa30c004630 tx=0x7fa30c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ffe11bee2bcf04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.374+0000 7fa308c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.374+0000 7fa308c00640 1 -- 172.21.15.42:0/3234257335 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00f8f0 con 0x55b8ad401970 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.375+0000 7fa32479a740 1 -- 172.21.15.42:0/3234257335 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa304005500 con 0x55b8ad401970 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.375+0000 7fa308c00640 1 -- 172.21.15.42:0/3234257335 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa30c00fa90 con 0x55b8ad401970 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.375+0000 7fa308c00640 1 -- 172.21.15.42:0/3234257335 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa30c00fc90 con 0x55b8ad401970 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.375+0000 7fa32479a740 10 client.24985 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.375+0000 7fa32479a740 1 -- 172.21.15.42:0/3234257335 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b8ad416660 con 0x55b8ad401970 2024-04-25T00:18:06.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.375+0000 7fa308c00640 1 -- 172.21.15.42:0/3234257335 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa304005500 con 0x55b8ad401970 2024-04-25T00:18:06.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.376+0000 7fa308c00640 1 -- 172.21.15.42:0/3234257335 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fa30c027070 con 0x55b8ad401970 2024-04-25T00:18:06.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.376+0000 7fa32479a740 10 client.24985 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:06.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.376+0000 7fa32479a740 10 client.24985 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:18:06.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.376+0000 7fa32479a740 20 client.24985 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:06.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.376+0000 7fa32479a740 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55b8ad418d50 0x55b8ad41a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.376+0000 7fa32479a740 4 client.24985 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:06.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.376+0000 7fa32479a740 1 -- 172.21.15.42:0/3234257335 --> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8ad372330 con 0x55b8ad418d50 2024-04-25T00:18:06.379 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.376+0000 7fa312000640 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55b8ad418d50 0x55b8ad41a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.377+0000 7fa312000640 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55b8ad418d50 0x55b8ad41a1c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e9d1a8addff804b server_cookie=cb7439b0c9215fb3 in_seq=0 out_seq=0 2024-04-25T00:18:06.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.377+0000 7fa308c00640 10 client.24985 ms_handle_connect on v2:172.21.15.136:6836/222488442 2024-04-25T00:18:06.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa308c00640 1 -- 172.21.15.42:0/3234257335 <== mds.2 v2:172.21.15.136:6836/222488442 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8ad372330 con 0x55b8ad418d50 2024-04-25T00:18:06.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa308c00640 10 client.24985 handle_command_reply: tid=0 2024-04-25T00:18:06.380 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 client.24985 shutdown 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 -- 172.21.15.42:0/3234257335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 msgr2=0x55b8ad40bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad40bcb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa30c004630 tx=0x7fa30c004730 comp rx=0 tx=0).stop 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 -- 172.21.15.42:0/3234257335 shutdown_connections 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55b8ad418d50 0x55b8ad41a1c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad413220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad402360 0x55b8ad40c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa308c00640 0 client.24985 ms_handle_reset on v2:172.21.15.136:6836/222488442 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 --2- 172.21.15.42:0/3234257335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad401970 0x55b8ad40bcb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 -- 172.21.15.42:0/3234257335 >> 172.21.15.42:0/3234257335 conn(0x55b8ad3fcfe0 msgr2=0x55b8ad3fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 -- 172.21.15.42:0/3234257335 shutdown_connections 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 1 -- 172.21.15.42:0/3234257335 wait complete. 2024-04-25T00:18:06.381 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.378+0000 7fa32479a740 20 client.24985 trim_cache size 0 max 16384 2024-04-25T00:18:06.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa32479a740 1 Processor -- start 2024-04-25T00:18:06.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa32479a740 1 -- start start 2024-04-25T00:18:06.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa32479a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad401970 0x55b8ad4098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa32479a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad402360 0x55b8ad4130f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa32479a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad4074d0 0x55b8ad40b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa32479a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad418500 con 0x55b8ad401970 2024-04-25T00:18:06.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa32479a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad418bd0 con 0x55b8ad4074d0 2024-04-25T00:18:06.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa32479a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad4192a0 con 0x55b8ad402360 2024-04-25T00:18:06.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa31aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad401970 0x55b8ad4098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa31aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad401970 0x55b8ad4098c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38796/0 (socket says 172.21.15.42:38796) 2024-04-25T00:18:06.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa31aa00640 1 -- 172.21.15.42:0/1983671280 learned_addr learned my addr 172.21.15.42:0/1983671280 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.399+0000 7fa312000640 1 --2- 172.21.15.42:0/1983671280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad402360 0x55b8ad4130f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa31b400640 1 --2- 172.21.15.42:0/1983671280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad4074d0 0x55b8ad40b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa31aa00640 1 -- 172.21.15.42:0/1983671280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad402360 msgr2=0x55b8ad4130f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa31aa00640 1 --2- 172.21.15.42:0/1983671280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad402360 0x55b8ad4130f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa31aa00640 1 -- 172.21.15.42:0/1983671280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad4074d0 msgr2=0x55b8ad40b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa31aa00640 1 --2- 172.21.15.42:0/1983671280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad4074d0 0x55b8ad40b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa31aa00640 1 -- 172.21.15.42:0/1983671280 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ad3fb620 con 0x55b8ad401970 2024-04-25T00:18:06.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa31b400640 1 --2- 172.21.15.42:0/1983671280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad4074d0 0x55b8ad40b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:06.402 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa31aa00640 1 --2- 172.21.15.42:0/1983671280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad401970 0x55b8ad4098c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009f70 tx=0x7fa30c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08e013b93b1862c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa30b400640 1 -- 172.21.15.42:0/1983671280 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00fe30 con 0x55b8ad401970 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa30b400640 1 -- 172.21.15.42:0/1983671280 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c00f8f0 con 0x55b8ad401970 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.400+0000 7fa30b400640 1 -- 172.21.15.42:0/1983671280 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c01e6d0 con 0x55b8ad401970 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.401+0000 7fa32479a740 1 -- 172.21.15.42:0/1983671280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad401970 msgr2=0x55b8ad4098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.401+0000 7fa32479a740 1 --2- 172.21.15.42:0/1983671280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad401970 0x55b8ad4098c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa30c009f70 tx=0x7fa30c003f60 comp rx=0 tx=0).stop 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.401+0000 7fa32479a740 1 -- 172.21.15.42:0/1983671280 shutdown_connections 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.401+0000 7fa32479a740 1 --2- 172.21.15.42:0/1983671280 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad4074d0 0x55b8ad40b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.401+0000 7fa32479a740 1 --2- 172.21.15.42:0/1983671280 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad402360 0x55b8ad4130f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.401+0000 7fa32479a740 1 --2- 172.21.15.42:0/1983671280 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad401970 0x55b8ad4098c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.401+0000 7fa32479a740 1 -- 172.21.15.42:0/1983671280 >> 172.21.15.42:0/1983671280 conn(0x55b8ad3fcfe0 msgr2=0x55b8ad3fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.401+0000 7fa32479a740 1 -- 172.21.15.42:0/1983671280 shutdown_connections 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.401+0000 7fa32479a740 1 -- 172.21.15.42:0/1983671280 wait complete. 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.402+0000 7fa32479a740 1 -- start start 2024-04-25T00:18:06.403 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.402+0000 7fa32479a740 1 Processor -- start 2024-04-25T00:18:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.402+0000 7fa32479a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad401970 0x55b8ad40f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.402+0000 7fa32479a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad402360 0x55b8ad40f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.402+0000 7fa32479a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad4129a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.402+0000 7fa32479a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad41a040 con 0x55b8ad4074d0 2024-04-25T00:18:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.402+0000 7fa32479a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad419970 con 0x55b8ad401970 2024-04-25T00:18:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.402+0000 7fa32479a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8ad41a710 con 0x55b8ad402360 2024-04-25T00:18:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.402+0000 7fa32479a740 10 client.0 _ll_register_callbacks cb 0x7fa30c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad4129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad4129a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38812/0 (socket says 172.21.15.42:38812) 2024-04-25T00:18:06.407 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31b400640 1 -- 172.21.15.42:0/305422278 learned_addr learned my addr 172.21.15.42:0/305422278 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31aa00640 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad401970 0x55b8ad40f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa312000640 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad402360 0x55b8ad40f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31aa00640 1 -- 172.21.15.42:0/305422278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad402360 msgr2=0x55b8ad40f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31aa00640 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad402360 0x55b8ad40f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31aa00640 1 -- 172.21.15.42:0/305422278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 msgr2=0x55b8ad4129a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31aa00640 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad4129a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31aa00640 1 -- 172.21.15.42:0/305422278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ad412f90 con 0x55b8ad401970 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31b400640 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad4129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa31aa00640 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad401970 0x55b8ad40f490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0040a0 tx=0x7fa30c005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30b36b85e3e35bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.403+0000 7fa308c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.404+0000 7fa308c00640 1 -- 172.21.15.42:0/305422278 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c012020 con 0x55b8ad401970 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.404+0000 7fa308c00640 1 -- 172.21.15.42:0/305422278 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fa30c0121c0 con 0x55b8ad401970 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.404+0000 7fa32479a740 1 -- 172.21.15.42:0/305422278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa30c00f8f0 con 0x55b8ad401970 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.404+0000 7fa308c00640 1 -- 172.21.15.42:0/305422278 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fa30c01e680 con 0x55b8ad401970 2024-04-25T00:18:06.408 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.404+0000 7fa32479a740 10 client.24983 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.404+0000 7fa32479a740 1 -- 172.21.15.42:0/305422278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55b8ad47f780 con 0x55b8ad401970 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.404+0000 7fa308c00640 1 -- 172.21.15.42:0/305422278 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c00f8f0 con 0x55b8ad401970 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.404+0000 7fa308c00640 1 -- 172.21.15.42:0/305422278 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fa30c042020 con 0x55b8ad401970 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.405+0000 7fa32479a740 10 client.24983 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.405+0000 7fa32479a740 10 client.24983 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.405+0000 7fa32479a740 20 client.24983 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.405+0000 7fa32479a740 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55b8ad483f90 0x55b8ad482ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.405+0000 7fa32479a740 4 client.24983 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.405+0000 7fa32479a740 1 -- 172.21.15.42:0/305422278 --> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] -- command(tid 0: {"prefix": "status"}) -- 0x55b8ad36c680 con 0x55b8ad483f90 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.405+0000 7fa312000640 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55b8ad483f90 0x55b8ad482ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.405+0000 7fa312000640 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55b8ad483f90 0x55b8ad482ca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e58bc3e9dfaadb96 server_cookie=76c47b06dfc1e718 in_seq=0 out_seq=0 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.405+0000 7fa308c00640 10 client.24983 ms_handle_connect on v2:172.21.15.136:6836/222488442 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa308c00640 1 -- 172.21.15.42:0/305422278 <== mds.2 v2:172.21.15.136:6836/222488442 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55b8ad36c680 con 0x55b8ad483f90 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa308c00640 10 client.24983 handle_command_reply: tid=0 2024-04-25T00:18:06.409 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 client.24983 shutdown 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 -- 172.21.15.42:0/305422278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad401970 msgr2=0x55b8ad40f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad401970 0x55b8ad40f490 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0040a0 tx=0x7fa30c005730 comp rx=0 tx=0).stop 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 -- 172.21.15.42:0/305422278 shutdown_connections 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55b8ad483f90 0x55b8ad482ca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8ad4074d0 0x55b8ad4129a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8ad402360 0x55b8ad40f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 --2- 172.21.15.42:0/305422278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8ad401970 0x55b8ad40f490 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa308c00640 0 client.24983 ms_handle_reset on v2:172.21.15.136:6836/222488442 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 -- 172.21.15.42:0/305422278 >> 172.21.15.42:0/305422278 conn(0x55b8ad3fcfe0 msgr2=0x55b8ad40b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 -- 172.21.15.42:0/305422278 shutdown_connections 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.406+0000 7fa32479a740 1 -- 172.21.15.42:0/305422278 wait complete. 2024-04-25T00:18:06.410 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa32479a740 20 client.24983 trim_cache size 0 max 16384 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "9d95543e-0297-11ef-bc93-c7b262605968", 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24517, 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 102.096183262, 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 23, 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 88, 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 127.91269156 2024-04-25T00:18:06.411 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:18:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 -- 172.21.15.42:0/1549729241 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa3000a2b90 msgr2=0x7fa3000a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa3000a2b90 0x7fa3000a5050 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55b8ad402760 tx=0x7fa30c005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 -- 172.21.15.42:0/1549729241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31c109830 msgr2=0x7fa31c1a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31c109830 0x7fa31c1a5480 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa314009430 tx=0x7fa3140319f0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 -- 172.21.15.42:0/1549729241 shutdown_connections 2024-04-25T00:18:06.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa3000a2b90 0x7fa3000a5050 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa31c109830 0x7fa31c1a5480 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa31c108e70 0x7fa31c19e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 --2- 172.21.15.42:0/1549729241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31c107c80 0x7fa31c19ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 -- 172.21.15.42:0/1549729241 >> 172.21.15.42:0/1549729241 conn(0x7fa31c076c10 msgr2=0x7fa31c10c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.407+0000 7fa31be00640 1 -- 172.21.15.42:0/1549729241 shutdown_connections 2024-04-25T00:18:06.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.408+0000 7fa31be00640 1 -- 172.21.15.42:0/1549729241 wait complete. 2024-04-25T00:18:06.426 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.642+0000 7f76dde00640 1 -- 172.21.15.42:0/3656459843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d8104a70 msgr2=0x7f76d8104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.642+0000 7f76dde00640 1 --2- 172.21.15.42:0/3656459843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d8104a70 0x7f76d8104ef0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009900 tx=0x7f76c402a410 comp rx=0 tx=0).stop 2024-04-25T00:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.642+0000 7f76dde00640 1 -- 172.21.15.42:0/3656459843 shutdown_connections 2024-04-25T00:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.642+0000 7f76dde00640 1 --2- 172.21.15.42:0/3656459843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d8105430 0x7f76d810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.642+0000 7f76dde00640 1 --2- 172.21.15.42:0/3656459843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d8104a70 0x7f76d8104ef0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.642+0000 7f76dde00640 1 --2- 172.21.15.42:0/3656459843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8103880 0x7f76d8103ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.642+0000 7f76dde00640 1 -- 172.21.15.42:0/3656459843 >> 172.21.15.42:0/3656459843 conn(0x7f76d80fedf0 msgr2=0x7f76d8101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.642+0000 7f76dde00640 1 -- 172.21.15.42:0/3656459843 shutdown_connections 2024-04-25T00:18:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.642+0000 7f76dde00640 1 -- 172.21.15.42:0/3656459843 wait complete. 2024-04-25T00:18:06.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.643+0000 7f76dde00640 1 Processor -- start 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dde00640 1 -- start start 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d8103880 0x7f76d8199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8104a70 0x7f76d819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dde00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d8105430 0x7f76d81a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dde00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f76d810f550 con 0x7f76d8103880 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dde00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f76d810f3d0 con 0x7f76d8105430 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dde00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76d810f6d0 con 0x7f76d8104a70 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d8103880 0x7f76d8199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d8103880 0x7f76d8199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38838/0 (socket says 172.21.15.42:38838) 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dca00640 1 -- 172.21.15.42:0/3213493774 learned_addr learned my addr 172.21.15.42:0/3213493774 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76dd400640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d8105430 0x7f76d81a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.644+0000 7f76d7e00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8104a70 0x7f76d819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76dca00640 1 -- 172.21.15.42:0/3213493774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8104a70 msgr2=0x7f76d819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76dca00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8104a70 0x7f76d819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76dca00640 1 -- 172.21.15.42:0/3213493774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d8105430 msgr2=0x7f76d81a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76dca00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d8105430 0x7f76d81a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76dca00640 1 -- 172.21.15.42:0/3213493774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76d81a17e0 con 0x7f76d8103880 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76dca00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d8103880 0x7f76d8199bc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00e9c0 tx=0x7f76cc00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76d5600640 1 -- 172.21.15.42:0/3213493774 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00ccc0 con 0x7f76d8103880 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76d5600640 1 -- 172.21.15.42:0/3213493774 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76cc0035f0 con 0x7f76d8103880 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76d5600640 1 -- 172.21.15.42:0/3213493774 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76cc00ce60 con 0x7f76d8103880 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76dde00640 1 -- 172.21.15.42:0/3213493774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76d81a1ad0 con 0x7f76d8103880 2024-04-25T00:18:06.647 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.645+0000 7f76dde00640 1 -- 172.21.15.42:0/3213493774 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76d81a1ff0 con 0x7f76d8103880 2024-04-25T00:18:06.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.648+0000 7f76e2180740 1 Processor -- start 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76e2180740 1 -- start start 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76d5600640 1 -- 172.21.15.42:0/3213493774 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f76cc0037a0 con 0x7f76d8103880 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76e2180740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 0x55a70bf19d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76e2180740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf1ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76e2180740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1f4d0 0x55a70bf21960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76e2180740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a70be19ed0 con 0x55a70bf19970 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76e2180740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a70be7cdf0 con 0x55a70bf1a360 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76e2180740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a70be258d0 con 0x55a70bf1f4d0 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76dd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 0x55a70bf19d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76dd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 0x55a70bf19d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38842/0 (socket says 172.21.15.42:38842) 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76dd400640 1 -- 172.21.15.42:0/1457769977 learned_addr learned my addr 172.21.15.42:0/1457769977 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76dca00640 1 --2- 172.21.15.42:0/1457769977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf1ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.650 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.649+0000 7f76d7e00640 1 --2- 172.21.15.42:0/1457769977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1f4d0 0x55a70bf21960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.650+0000 7f76dd400640 1 -- 172.21.15.42:0/1457769977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1f4d0 msgr2=0x55a70bf21960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.650+0000 7f76dd400640 1 --2- 172.21.15.42:0/1457769977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1f4d0 0x55a70bf21960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.650+0000 7f76dd400640 1 -- 172.21.15.42:0/1457769977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 msgr2=0x55a70bf1ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.650+0000 7f76dd400640 1 --2- 172.21.15.42:0/1457769977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf1ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.650+0000 7f76dd400640 1 -- 172.21.15.42:0/1457769977 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70bb912d0 con 0x55a70bf19970 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.650+0000 7f76dd400640 1 --2- 172.21.15.42:0/1457769977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 0x55a70bf19d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76d0011440 tx=0x7f76d0011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb95fb9a7956027 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.650+0000 7f76c9c00640 1 -- 172.21.15.42:0/1457769977 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0003c30 con 0x55a70bf19970 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.651+0000 7f76c9c00640 1 -- 172.21.15.42:0/1457769977 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d0003dd0 con 0x55a70bf19970 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.651+0000 7f76c9c00640 1 -- 172.21.15.42:0/1457769977 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d000fb10 con 0x55a70bf19970 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.651+0000 7f76d5600640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f76c00a2c10 0x7f76c00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.651+0000 7f76dca00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f76c00a2c10 0x7f76c00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.651+0000 7f76d5600640 1 -- 172.21.15.42:0/3213493774 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f76cc0cf360 con 0x7f76d8103880 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.651+0000 7f76e2180740 1 -- 172.21.15.42:0/1457769977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 msgr2=0x55a70bf19d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.651+0000 7f76e2180740 1 --2- 172.21.15.42:0/1457769977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 0x55a70bf19d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f76d0011440 tx=0x7f76d0011900 comp rx=0 tx=0).stop 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.652+0000 7f76dca00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f76c00a2c10 0x7f76c00a50d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55a70bf1a760 tx=0x7f76cc0d2da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.652+0000 7f76e2180740 1 -- 172.21.15.42:0/1457769977 shutdown_connections 2024-04-25T00:18:06.653 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.652+0000 7f76e2180740 1 --2- 172.21.15.42:0/1457769977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1f4d0 0x55a70bf21960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.652+0000 7f76e2180740 1 --2- 172.21.15.42:0/1457769977 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf1ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.652+0000 7f76e2180740 1 --2- 172.21.15.42:0/1457769977 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 0x55a70bf19d90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.652+0000 7f76e2180740 1 -- 172.21.15.42:0/1457769977 >> 172.21.15.42:0/1457769977 conn(0x55a70bf14fe0 msgr2=0x55a70bf153e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.652+0000 7f76e2180740 1 -- 172.21.15.42:0/1457769977 shutdown_connections 2024-04-25T00:18:06.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.652+0000 7f76e2180740 1 -- 172.21.15.42:0/1457769977 wait complete. 2024-04-25T00:18:06.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.652+0000 7f76e2180740 1 -- start start 2024-04-25T00:18:06.654 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.652+0000 7f76e2180740 1 Processor -- start 2024-04-25T00:18:06.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76e2180740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 0x55a70bf23c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.657 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76e2180740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1a360 0x55a70bf241a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76e2180740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76e2180740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a70be38d20 con 0x55a70bf19970 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76e2180740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a70be78db0 con 0x55a70bf1f4d0 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76e2180740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a70bf22480 con 0x55a70bf1a360 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76e2180740 10 client.0 _ll_register_callbacks cb 0x55a70be73f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76d7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1a360 0x55a70bf241a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76dd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 0x55a70bf23c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76d7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1a360 0x55a70bf241a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39422/0 (socket says 172.21.15.42:39422) 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76d7e00640 1 -- 172.21.15.42:0/3010183366 learned_addr learned my addr 172.21.15.42:0/3010183366 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76dd400640 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.653+0000 7f76d7e00640 1 -- 172.21.15.42:0/3010183366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1f4d0 msgr2=0x55a70bf2b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.658 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.654+0000 7f76d7e00640 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.654+0000 7f76d7e00640 1 -- 172.21.15.42:0/3010183366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 msgr2=0x55a70bf23c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.654+0000 7f76d7e00640 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 0x55a70bf23c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.654+0000 7f76d7e00640 1 -- 172.21.15.42:0/3010183366 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70bc5b6b0 con 0x55a70bf1a360 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.654+0000 7f76dd400640 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.654+0000 7f76d7e00640 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1a360 0x55a70bf241a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009900 tx=0x7f76c40043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ed7732f8a39f1b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.654+0000 7f76be000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.654+0000 7f76be000640 1 -- 172.21.15.42:0/3010183366 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76c4035040 con 0x55a70bf1a360 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.654+0000 7f76be000640 1 -- 172.21.15.42:0/3010183366 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f76c4005450 con 0x55a70bf1a360 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.654+0000 7f76e2180740 1 -- 172.21.15.42:0/3010183366 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76d0003dd0 con 0x55a70bf1a360 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76be000640 1 -- 172.21.15.42:0/3010183366 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f76c404d070 con 0x55a70bf1a360 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76e2180740 10 client.25003 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76e2180740 1 -- 172.21.15.42:0/3010183366 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a70bf2e5a0 con 0x55a70bf1a360 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76be000640 1 -- 172.21.15.42:0/3010183366 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d0003dd0 con 0x55a70bf1a360 2024-04-25T00:18:06.659 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76be000640 1 -- 172.21.15.42:0/3010183366 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f76c4031a20 con 0x55a70bf1a360 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76e2180740 10 client.25003 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76e2180740 10 client.25003 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76e2180740 20 client.25003 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76e2180740 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55a70bf30bc0 0x55a70bf32030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76e2180740 4 client.25003 mds_command: new command op to 24568 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.655+0000 7f76e2180740 1 -- 172.21.15.42:0/3010183366 --> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a70be8a330 con 0x55a70bf30bc0 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.656+0000 7f76dd400640 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55a70bf30bc0 0x55a70bf32030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.656+0000 7f76dd400640 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55a70bf30bc0 0x55a70bf32030 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b52d69589c24e75 server_cookie=e36e633e3ab52558 in_seq=0 out_seq=0 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.656+0000 7f76be000640 10 client.25003 ms_handle_connect on v2:172.21.15.42:6840/4068466248 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.657+0000 7f76be000640 1 -- 172.21.15.42:0/3010183366 <== mds.1 v2:172.21.15.42:6840/4068466248 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a70be8a330 con 0x55a70bf30bc0 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.657+0000 7f76be000640 10 client.25003 handle_command_reply: tid=0 2024-04-25T00:18:06.660 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 client.25003 shutdown 2024-04-25T00:18:06.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 -- 172.21.15.42:0/3010183366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1a360 msgr2=0x55a70bf241a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1a360 0x55a70bf241a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f76c4009900 tx=0x7f76c40043b0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 -- 172.21.15.42:0/3010183366 shutdown_connections 2024-04-25T00:18:06.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55a70bf30bc0 0x55a70bf32030 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf1a360 0x55a70bf241a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.661 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 --2- 172.21.15.42:0/3010183366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf19970 0x55a70bf23c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76be000640 0 client.25003 ms_handle_reset on v2:172.21.15.42:6840/4068466248 2024-04-25T00:18:06.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 -- 172.21.15.42:0/3010183366 >> 172.21.15.42:0/3010183366 conn(0x55a70bf14fe0 msgr2=0x55a70bf14120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 -- 172.21.15.42:0/3010183366 shutdown_connections 2024-04-25T00:18:06.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 1 -- 172.21.15.42:0/3010183366 wait complete. 2024-04-25T00:18:06.662 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.658+0000 7f76e2180740 20 client.25003 trim_cache size 0 max 16384 2024-04-25T00:18:06.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.678+0000 7f76e2180740 1 Processor -- start 2024-04-25T00:18:06.679 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.678+0000 7f76e2180740 1 -- start start 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.678+0000 7f76e2180740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf19970 0x55a70bf1ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76e2180740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf2b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76e2180740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf239f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76e2180740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a70bf30370 con 0x55a70bf1f4d0 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76e2180740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a70bf30a40 con 0x55a70bf1a360 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76e2180740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a70bf31110 con 0x55a70bf19970 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76dd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf239f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76dd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf239f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38860/0 (socket says 172.21.15.42:38860) 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76dd400640 1 -- 172.21.15.42:0/1714722314 learned_addr learned my addr 172.21.15.42:0/1714722314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76dd400640 1 --2- 172.21.15.42:0/1714722314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf19970 0x55a70bf1ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76d7e00640 1 --2- 172.21.15.42:0/1714722314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf2b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76dd400640 1 -- 172.21.15.42:0/1714722314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf19970 msgr2=0x55a70bf1ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76dd400640 1 --2- 172.21.15.42:0/1714722314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf19970 0x55a70bf1ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76dd400640 1 -- 172.21.15.42:0/1714722314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 msgr2=0x55a70bf2b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76dd400640 1 --2- 172.21.15.42:0/1714722314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf2b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.679+0000 7f76dd400640 1 -- 172.21.15.42:0/1714722314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a70bf1ea10 con 0x55a70bf1f4d0 2024-04-25T00:18:06.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.680+0000 7f76dd400640 1 --2- 172.21.15.42:0/1714722314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf239f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009560 tx=0x7f76d0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922206bea74bd001 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.680+0000 7f76c9200640 1 -- 172.21.15.42:0/1714722314 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d0004250 con 0x55a70bf1f4d0 2024-04-25T00:18:06.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.680+0000 7f76c9200640 1 -- 172.21.15.42:0/1714722314 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d000f600 con 0x55a70bf1f4d0 2024-04-25T00:18:06.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.680+0000 7f76c9200640 1 -- 172.21.15.42:0/1714722314 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d002d2a0 con 0x55a70bf1f4d0 2024-04-25T00:18:06.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 -- 172.21.15.42:0/1714722314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 msgr2=0x55a70bf239f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 --2- 172.21.15.42:0/1714722314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf239f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f76d0009560 tx=0x7f76d0000f40 comp rx=0 tx=0).stop 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 -- 172.21.15.42:0/1714722314 shutdown_connections 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 --2- 172.21.15.42:0/1714722314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf239f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 --2- 172.21.15.42:0/1714722314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf2b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 --2- 172.21.15.42:0/1714722314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf19970 0x55a70bf1ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 -- 172.21.15.42:0/1714722314 >> 172.21.15.42:0/1714722314 conn(0x55a70bf14fe0 msgr2=0x55a70bf153c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 -- 172.21.15.42:0/1714722314 shutdown_connections 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 -- 172.21.15.42:0/1714722314 wait complete. 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 -- start start 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.681+0000 7f76e2180740 1 Processor -- start 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76e2180740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf19970 0x55a70bf26ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76e2180740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf27220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76e2180740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76e2180740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a70bf31eb0 con 0x55a70bf1f4d0 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76e2180740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a70bf317e0 con 0x55a70bf1a360 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76e2180740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a70bf32580 con 0x55a70bf19970 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76e2180740 10 client.0 _ll_register_callbacks cb 0x7f76c4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76dd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76dd400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38874/0 (socket says 172.21.15.42:38874) 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76dd400640 1 -- 172.21.15.42:0/501060323 learned_addr learned my addr 172.21.15.42:0/501060323 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76dd400640 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf19970 0x55a70bf26ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76d7e00640 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf27220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76dd400640 1 -- 172.21.15.42:0/501060323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf19970 msgr2=0x55a70bf26ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76dd400640 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf19970 0x55a70bf26ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76dd400640 1 -- 172.21.15.42:0/501060323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 msgr2=0x55a70bf27220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76dd400640 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf27220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.682+0000 7f76dd400640 1 -- 172.21.15.42:0/501060323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a70bf2a7e0 con 0x55a70bf1f4d0 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.683+0000 7f76d7e00640 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf27220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.683+0000 7f76dd400640 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2a1f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f76d0006010 tx=0x7f76d0009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e1c0622eb77f3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.683+0000 7f76be000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:18:06.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.683+0000 7f76be000640 1 -- 172.21.15.42:0/501060323 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76d000f9c0 con 0x55a70bf1f4d0 2024-04-25T00:18:06.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.683+0000 7f76e2180740 1 -- 172.21.15.42:0/501060323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76d000f600 con 0x55a70bf1f4d0 2024-04-25T00:18:06.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.683+0000 7f76be000640 1 -- 172.21.15.42:0/501060323 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f76d000fb60 con 0x55a70bf1f4d0 2024-04-25T00:18:06.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.683+0000 7f76be000640 1 -- 172.21.15.42:0/501060323 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f76d0004050 con 0x55a70bf1f4d0 2024-04-25T00:18:06.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.683+0000 7f76e2180740 10 client.14895 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:06.686 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.683+0000 7f76e2180740 1 -- 172.21.15.42:0/501060323 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55a70bf97050 con 0x55a70bf1f4d0 2024-04-25T00:18:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.684+0000 7f76be000640 1 -- 172.21.15.42:0/501060323 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76d000f600 con 0x55a70bf1f4d0 2024-04-25T00:18:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.684+0000 7f76be000640 1 -- 172.21.15.42:0/501060323 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f76d002d540 con 0x55a70bf1f4d0 2024-04-25T00:18:06.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.684+0000 7f76e2180740 10 client.14895 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.684+0000 7f76e2180740 10 client.14895 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:18:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.684+0000 7f76e2180740 20 client.14895 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.684+0000 7f76e2180740 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55a70bf9b520 0x55a70bf9a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.684+0000 7f76e2180740 4 client.14895 mds_command: new command op to 24568 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:18:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.684+0000 7f76e2180740 1 -- 172.21.15.42:0/501060323 --> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] -- command(tid 0: {"prefix": "status"}) -- 0x55a70be84680 con 0x55a70bf9b520 2024-04-25T00:18:06.690 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.684+0000 7f76d7e00640 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55a70bf9b520 0x55a70bf9a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.685+0000 7f76d7e00640 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55a70bf9b520 0x55a70bf9a230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ac81bfa8c2975e58 server_cookie=e45d23942c24c129 in_seq=0 out_seq=0 2024-04-25T00:18:06.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.685+0000 7f76be000640 10 client.14895 ms_handle_connect on v2:172.21.15.42:6840/4068466248 2024-04-25T00:18:06.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.685+0000 7f76be000640 1 -- 172.21.15.42:0/501060323 <== mds.1 v2:172.21.15.42:6840/4068466248 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a70be84680 con 0x55a70bf9b520 2024-04-25T00:18:06.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.685+0000 7f76be000640 10 client.14895 handle_command_reply: tid=0 2024-04-25T00:18:06.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.685+0000 7f76e2180740 1 client.14895 shutdown 2024-04-25T00:18:06.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.685+0000 7f76e2180740 1 -- 172.21.15.42:0/501060323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 msgr2=0x55a70bf2a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.685+0000 7f76e2180740 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2a1f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f76d0006010 tx=0x7f76d0009900 comp rx=0 tx=0).stop 2024-04-25T00:18:06.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76e2180740 1 -- 172.21.15.42:0/501060323 shutdown_connections 2024-04-25T00:18:06.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76e2180740 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55a70bf9b520 0x55a70bf9a230 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76e2180740 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a70bf1f4d0 0x55a70bf2a1f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.692 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76e2180740 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a70bf1a360 0x55a70bf27220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76e2180740 1 --2- 172.21.15.42:0/501060323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a70bf19970 0x55a70bf26ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76e2180740 1 -- 172.21.15.42:0/501060323 >> 172.21.15.42:0/501060323 conn(0x55a70bf14fe0 msgr2=0x55a70bf239f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76be000640 0 client.14895 ms_handle_reset on v2:172.21.15.42:6840/4068466248 2024-04-25T00:18:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76e2180740 1 -- 172.21.15.42:0/501060323 shutdown_connections 2024-04-25T00:18:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76e2180740 1 -- 172.21.15.42:0/501060323 wait complete. 2024-04-25T00:18:06.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76e2180740 20 client.14895 trim_cache size 0 max 16384 2024-04-25T00:18:06.694 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:18:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "9d95543e-0297-11ef-bc93-c7b262605968", 2024-04-25T00:18:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2024-04-25T00:18:06.694 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24568, 2024-04-25T00:18:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-04-25T00:18:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-04-25T00:18:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-04-25T00:18:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 103.32347545499999, 2024-04-25T00:18:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 23, 2024-04-25T00:18:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 88, 2024-04-25T00:18:06.695 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T00:18:06.696 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 114.53953626400001 2024-04-25T00:18:06.696 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:18:06.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76dde00640 1 -- 172.21.15.42:0/3213493774 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f76c00a2c10 msgr2=0x7f76c00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76dde00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f76c00a2c10 0x7f76c00a50d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55a70bf1a760 tx=0x7f76cc0d2da0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76dde00640 1 -- 172.21.15.42:0/3213493774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d8103880 msgr2=0x7f76d8199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.686+0000 7f76dde00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d8103880 0x7f76d8199bc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f76cc00e9c0 tx=0x7f76cc00edc0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.687+0000 7f76dde00640 1 -- 172.21.15.42:0/3213493774 shutdown_connections 2024-04-25T00:18:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.687+0000 7f76dde00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f76c00a2c10 0x7f76c00a50d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.687+0000 7f76dde00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f76d8105430 0x7f76d81a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.687+0000 7f76dde00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f76d8104a70 0x7f76d819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.687+0000 7f76dde00640 1 --2- 172.21.15.42:0/3213493774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f76d8103880 0x7f76d8199bc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.687+0000 7f76dde00640 1 -- 172.21.15.42:0/3213493774 >> 172.21.15.42:0/3213493774 conn(0x7f76d80fedf0 msgr2=0x7f76d8107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.687+0000 7f76dde00640 1 -- 172.21.15.42:0/3213493774 shutdown_connections 2024-04-25T00:18:06.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.687+0000 7f76dde00640 1 -- 172.21.15.42:0/3213493774 wait complete. 2024-04-25T00:18:06.705 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:18:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.921+0000 7fc983e00640 1 -- 172.21.15.42:0/3035295394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc984102830 msgr2=0x7fc984104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.921+0000 7fc983e00640 1 --2- 172.21.15.42:0/3035295394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc984102830 0x7fc984104ca0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc970009900 tx=0x7fc97002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.921+0000 7fc983e00640 1 -- 172.21.15.42:0/3035295394 shutdown_connections 2024-04-25T00:18:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.921+0000 7fc983e00640 1 --2- 172.21.15.42:0/3035295394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9841051e0 0x7fc984105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.921+0000 7fc983e00640 1 --2- 172.21.15.42:0/3035295394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc984102830 0x7fc984104ca0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.921+0000 7fc983e00640 1 --2- 172.21.15.42:0/3035295394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9840ffec0 0x7fc9841022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.922 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.921+0000 7fc983e00640 1 -- 172.21.15.42:0/3035295394 >> 172.21.15.42:0/3035295394 conn(0x7fc9840fbae0 msgr2=0x7fc9840fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.922+0000 7fc983e00640 1 -- 172.21.15.42:0/3035295394 shutdown_connections 2024-04-25T00:18:06.923 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.922+0000 7fc983e00640 1 -- 172.21.15.42:0/3035295394 wait complete. 2024-04-25T00:18:06.924 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.923+0000 7fc983e00640 1 Processor -- start 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.923+0000 7fc983e00640 1 -- start start 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.923+0000 7fc983e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9840ffec0 0x7fc98419df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.923+0000 7fc983e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc984102830 0x7fc98419e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.923+0000 7fc983e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9841051e0 0x7fc9841a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc983e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9841138a0 con 0x7fc984102830 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc983e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc984113720 con 0x7fc9841051e0 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc983e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc984113a20 con 0x7fc9840ffec0 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc982000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc984102830 0x7fc98419e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc982000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc984102830 0x7fc98419e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42130/0 (socket says 172.21.15.42:42130) 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc982000640 1 -- 172.21.15.42:0/3879194056 learned_addr learned my addr 172.21.15.42:0/3879194056 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc982a00640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9840ffec0 0x7fc98419df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc983400640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9841051e0 0x7fc9841a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc982000640 1 -- 172.21.15.42:0/3879194056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9840ffec0 msgr2=0x7fc98419df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.925 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc982000640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9840ffec0 0x7fc98419df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc982000640 1 -- 172.21.15.42:0/3879194056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9841051e0 msgr2=0x7fc9841a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc982000640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9841051e0 0x7fc9841a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc982000640 1 -- 172.21.15.42:0/3879194056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9841a5b10 con 0x7fc984102830 2024-04-25T00:18:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.924+0000 7fc983400640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9841051e0 0x7fc9841a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:06.926 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.925+0000 7fc982000640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc984102830 0x7fc98419e480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc970009230 tx=0x7fc9700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.925+0000 7fc97b400640 1 -- 172.21.15.42:0/3879194056 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97003f030 con 0x7fc984102830 2024-04-25T00:18:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.925+0000 7fc97b400640 1 -- 172.21.15.42:0/3879194056 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc970003ce0 con 0x7fc984102830 2024-04-25T00:18:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.925+0000 7fc983e00640 1 -- 172.21.15.42:0/3879194056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9841a5da0 con 0x7fc984102830 2024-04-25T00:18:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.925+0000 7fc97b400640 1 -- 172.21.15.42:0/3879194056 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970035070 con 0x7fc984102830 2024-04-25T00:18:06.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.925+0000 7fc983e00640 1 -- 172.21.15.42:0/3879194056 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9841a6230 con 0x7fc984102830 2024-04-25T00:18:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.928+0000 7fc98c629740 1 Processor -- start 2024-04-25T00:18:06.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.928+0000 7fc98c629740 1 -- start start 2024-04-25T00:18:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc98c629740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc98c629740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3ece60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc98c629740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3ef960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc98c629740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55649f2e7ed0 con 0x55649f3e8360 2024-04-25T00:18:06.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc98c629740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55649f34adf0 con 0x55649f3e7970 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc98c629740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55649f2f38d0 con 0x55649f3ed4d0 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc97b400640 1 -- 172.21.15.42:0/3879194056 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc970042030 con 0x7fc984102830 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc983400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc982a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3ece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc983400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3e7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:48198/0 (socket says 172.21.15.42:48198) 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.929+0000 7fc983400640 1 -- 172.21.15.42:0/1061148861 learned_addr learned my addr 172.21.15.42:0/1061148861 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc983400640 1 -- 172.21.15.42:0/1061148861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 msgr2=0x55649f3ef960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc982000640 1 --2- 172.21.15.42:0/1061148861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3ef960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc983400640 1 --2- 172.21.15.42:0/1061148861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3ef960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc983400640 1 -- 172.21.15.42:0/1061148861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 msgr2=0x55649f3ece60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc983400640 1 --2- 172.21.15.42:0/1061148861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3ece60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc983400640 1 -- 172.21.15.42:0/1061148861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55649f05f2d0 con 0x55649f3e7970 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc982a00640 1 --2- 172.21.15.42:0/1061148861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3ece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:06.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc982000640 1 --2- 172.21.15.42:0/1061148861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3ef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc983400640 1 --2- 172.21.15.42:0/1061148861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3e7d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0117e0 tx=0x7fc97c011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1555b41305ec1e07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc96fe00640 1 -- 172.21.15.42:0/1061148861 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c011310 con 0x55649f3e7970 2024-04-25T00:18:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc96fe00640 1 -- 172.21.15.42:0/1061148861 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97c003e50 con 0x55649f3e7970 2024-04-25T00:18:06.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.930+0000 7fc96fe00640 1 -- 172.21.15.42:0/1061148861 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97c004620 con 0x55649f3e7970 2024-04-25T00:18:06.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc97b400640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9680a2b90 0x7fc9680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc98c629740 1 -- 172.21.15.42:0/1061148861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 msgr2=0x55649f3e7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc98c629740 1 --2- 172.21.15.42:0/1061148861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3e7d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc97c0117e0 tx=0x7fc97c011ca0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc97b400640 1 -- 172.21.15.42:0/3879194056 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc9700ef140 con 0x7fc984102830 2024-04-25T00:18:06.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc98c629740 1 -- 172.21.15.42:0/1061148861 shutdown_connections 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc98c629740 1 --2- 172.21.15.42:0/1061148861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3ef960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc98c629740 1 --2- 172.21.15.42:0/1061148861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3ece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc98c629740 1 --2- 172.21.15.42:0/1061148861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3e7d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc982a00640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9680a2b90 0x7fc9680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc98c629740 1 -- 172.21.15.42:0/1061148861 >> 172.21.15.42:0/1061148861 conn(0x55649f3e2fe0 msgr2=0x55649f3e33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc98c629740 1 -- 172.21.15.42:0/1061148861 shutdown_connections 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.931+0000 7fc98c629740 1 -- 172.21.15.42:0/1061148861 wait complete. 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.932+0000 7fc98c629740 1 -- start start 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.932+0000 7fc982a00640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9680a2b90 0x7fc9680a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55649f3e3f50 tx=0x7fc974005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.932+0000 7fc98c629740 1 Processor -- start 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc98c629740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3f1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc98c629740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc98c629740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3f9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc98c629740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55649f306d20 con 0x55649f3e8360 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc98c629740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55649f346db0 con 0x55649f3e7970 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc98c629740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55649f3f0480 con 0x55649f3ed4d0 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc98c629740 10 client.0 _ll_register_callbacks cb 0x55649f341f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc982000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc982a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3f1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc982000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42144/0 (socket says 172.21.15.42:42144) 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc982000640 1 -- 172.21.15.42:0/3080185229 learned_addr learned my addr 172.21.15.42:0/3080185229 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc982a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3f1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:48202/0 (socket says 172.21.15.42:48202) 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc983400640 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc982000640 1 -- 172.21.15.42:0/3080185229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 msgr2=0x55649f3f9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc982000640 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3f9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc982000640 1 -- 172.21.15.42:0/3080185229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 msgr2=0x55649f3f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc982000640 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3f1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.933+0000 7fc982000640 1 -- 172.21.15.42:0/3080185229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55649f1296b0 con 0x55649f3e8360 2024-04-25T00:18:06.937 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.934+0000 7fc982000640 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3f21e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9700c4110 tx=0x7fc9700bb910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2e60e54100931f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.934+0000 7fc96cc00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.934+0000 7fc96cc00640 1 -- 172.21.15.42:0/3080185229 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970049070 con 0x55649f3e8360 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.934+0000 7fc96cc00640 1 -- 172.21.15.42:0/3080185229 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc9700b80f0 con 0x55649f3e8360 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.934+0000 7fc98c629740 1 -- 172.21.15.42:0/3080185229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc97c003e50 con 0x55649f3e8360 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.934+0000 7fc96cc00640 1 -- 172.21.15.42:0/3080185229 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc9700bfd80 con 0x55649f3e8360 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.934+0000 7fc98c629740 10 client.14907 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.934+0000 7fc98c629740 1 -- 172.21.15.42:0/3080185229 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55649f3fc4f0 con 0x55649f3e8360 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.934+0000 7fc96cc00640 1 -- 172.21.15.42:0/3080185229 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97c003e50 con 0x55649f3e8360 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.935+0000 7fc96cc00640 1 -- 172.21.15.42:0/3080185229 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc97004c060 con 0x55649f3e8360 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.935+0000 7fc98c629740 10 client.14907 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.935+0000 7fc98c629740 10 client.14907 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.935+0000 7fc98c629740 20 client.14907 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.935+0000 7fc98c629740 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55649f3febd0 0x55649f400040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.938 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.935+0000 7fc98c629740 4 client.14907 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.935+0000 7fc98c629740 1 -- 172.21.15.42:0/3080185229 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55649f358330 con 0x55649f3febd0 2024-04-25T00:18:06.939 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.935+0000 7fc982a00640 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55649f3febd0 0x55649f400040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.936+0000 7fc982a00640 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55649f3febd0 0x55649f400040 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e279a0edb498c1a server_cookie=e328d888109fea75 in_seq=0 out_seq=0 2024-04-25T00:18:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.936+0000 7fc96cc00640 10 client.14907 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:18:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc96cc00640 1 -- 172.21.15.42:0/3080185229 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55649f358330 con 0x55649f3febd0 2024-04-25T00:18:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc96cc00640 10 client.14907 handle_command_reply: tid=0 2024-04-25T00:18:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 client.14907 shutdown 2024-04-25T00:18:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 -- 172.21.15.42:0/3080185229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 msgr2=0x55649f3f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3f21e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9700c4110 tx=0x7fc9700bb910 comp rx=0 tx=0).stop 2024-04-25T00:18:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 -- 172.21.15.42:0/3080185229 shutdown_connections 2024-04-25T00:18:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55649f3febd0 0x55649f400040 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.940 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3f9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e8360 0x55649f3f21e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc96cc00640 0 client.14907 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:18:06.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 --2- 172.21.15.42:0/3080185229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e7970 0x55649f3f1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 -- 172.21.15.42:0/3080185229 >> 172.21.15.42:0/3080185229 conn(0x55649f3e2fe0 msgr2=0x55649f3e2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 -- 172.21.15.42:0/3080185229 shutdown_connections 2024-04-25T00:18:06.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.937+0000 7fc98c629740 1 -- 172.21.15.42:0/3080185229 wait complete. 2024-04-25T00:18:06.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.938+0000 7fc98c629740 20 client.14907 trim_cache size 0 max 16384 2024-04-25T00:18:06.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.957+0000 7fc98c629740 1 Processor -- start 2024-04-25T00:18:06.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc98c629740 1 -- start start 2024-04-25T00:18:06.959 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc98c629740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3e28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc98c629740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc98c629740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3fd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc98c629740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55649f3fe380 con 0x55649f3e7970 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc98c629740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55649f3fea50 con 0x55649f3e8360 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc98c629740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55649f3ff120 con 0x55649f3ed4d0 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc982a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3e28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc982a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3e28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42146/0 (socket says 172.21.15.42:42146) 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc982a00640 1 -- 172.21.15.42:0/2388240701 learned_addr learned my addr 172.21.15.42:0/2388240701 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc982000640 1 --2- 172.21.15.42:0/2388240701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.958+0000 7fc983400640 1 --2- 172.21.15.42:0/2388240701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.959+0000 7fc982000640 1 -- 172.21.15.42:0/2388240701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 msgr2=0x55649f3fd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.959+0000 7fc982000640 1 --2- 172.21.15.42:0/2388240701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3fd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.959+0000 7fc982000640 1 -- 172.21.15.42:0/2388240701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 msgr2=0x55649f3e28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.959+0000 7fc982000640 1 --2- 172.21.15.42:0/2388240701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3e28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.959+0000 7fc982000640 1 -- 172.21.15.42:0/2388240701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55649f3e8bf0 con 0x55649f3e8360 2024-04-25T00:18:06.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.959+0000 7fc982a00640 1 --2- 172.21.15.42:0/2388240701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:06.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.959+0000 7fc982000640 1 --2- 172.21.15.42:0/2388240701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f91d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc9700314e0 tx=0x7fc97002a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf5ae62d741336d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.959+0000 7fc96f400640 1 -- 172.21.15.42:0/2388240701 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc970049070 con 0x55649f3e8360 2024-04-25T00:18:06.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.959+0000 7fc96f400640 1 -- 172.21.15.42:0/2388240701 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97003b460 con 0x55649f3e8360 2024-04-25T00:18:06.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.959+0000 7fc96f400640 1 -- 172.21.15.42:0/2388240701 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9700bb130 con 0x55649f3e8360 2024-04-25T00:18:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.960+0000 7fc98c629740 1 -- 172.21.15.42:0/2388240701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 msgr2=0x55649f3f91d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.960+0000 7fc98c629740 1 --2- 172.21.15.42:0/2388240701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f91d0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc9700314e0 tx=0x7fc97002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.960+0000 7fc98c629740 1 -- 172.21.15.42:0/2388240701 shutdown_connections 2024-04-25T00:18:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.960+0000 7fc98c629740 1 --2- 172.21.15.42:0/2388240701 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3fd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.960+0000 7fc98c629740 1 --2- 172.21.15.42:0/2388240701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f91d0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.960+0000 7fc98c629740 1 --2- 172.21.15.42:0/2388240701 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3e28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.960+0000 7fc98c629740 1 -- 172.21.15.42:0/2388240701 >> 172.21.15.42:0/2388240701 conn(0x55649f3e2fe0 msgr2=0x55649f3e33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.960+0000 7fc98c629740 1 -- 172.21.15.42:0/2388240701 shutdown_connections 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.960+0000 7fc98c629740 1 -- 172.21.15.42:0/2388240701 wait complete. 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.961+0000 7fc98c629740 1 -- start start 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.961+0000 7fc98c629740 1 Processor -- start 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.961+0000 7fc98c629740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3f5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.961+0000 7fc98c629740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.961+0000 7fc98c629740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3f85f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.961+0000 7fc98c629740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55649f3ffec0 con 0x55649f3e7970 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.961+0000 7fc98c629740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55649f3ff7f0 con 0x55649f3e8360 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.961+0000 7fc98c629740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55649f400590 con 0x55649f3ed4d0 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.961+0000 7fc98c629740 10 client.0 _ll_register_callbacks cb 0x7fc970005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc983400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3f85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:48226/0 (socket says 172.21.15.42:48226) 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982000640 1 -- 172.21.15.42:0/1771169525 learned_addr learned my addr 172.21.15.42:0/1771169525 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982000640 1 -- 172.21.15.42:0/1771169525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 msgr2=0x55649f3f85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982000640 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3f85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982000640 1 -- 172.21.15.42:0/1771169525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 msgr2=0x55649f3f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982000640 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982000640 1 -- 172.21.15.42:0/1771169525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55649f3f8c30 con 0x55649f3e8360 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982a00640 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc982000640 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f5940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc970000c00 tx=0x7fc9700bb910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d38c81300ccf4b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.962+0000 7fc96cc00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc96cc00640 1 -- 172.21.15.42:0/1771169525 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc97009f060 con 0x55649f3e8360 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc96cc00640 1 -- 172.21.15.42:0/1771169525 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc9700d4bd0 con 0x55649f3e8360 2024-04-25T00:18:06.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc98c629740 1 -- 172.21.15.42:0/1771169525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc97003b460 con 0x55649f3e8360 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc96cc00640 1 -- 172.21.15.42:0/1771169525 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc970049030 con 0x55649f3e8360 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc98c629740 10 client.25016 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc98c629740 1 -- 172.21.15.42:0/1771169525 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55649f3f8c30 con 0x55649f3e8360 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc96cc00640 1 -- 172.21.15.42:0/1771169525 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc97003b460 con 0x55649f3e8360 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc96cc00640 1 -- 172.21.15.42:0/1771169525 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7fc970031b10 con 0x55649f3e8360 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc98c629740 10 client.25016 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc98c629740 10 client.25016 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc98c629740 20 client.25016 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc98c629740 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55649f4697b0 0x55649f4684c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc98c629740 4 client.25016 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.963+0000 7fc98c629740 1 -- 172.21.15.42:0/1771169525 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "status"}) -- 0x55649f352680 con 0x55649f4697b0 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.964+0000 7fc982a00640 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55649f4697b0 0x55649f4684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.964+0000 7fc982a00640 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55649f4697b0 0x55649f4684c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2e088f97b02ff7a server_cookie=47e9bb7fe484035d in_seq=0 out_seq=0 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.964+0000 7fc96cc00640 10 client.25016 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:18:06.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.964+0000 7fc96cc00640 1 -- 172.21.15.42:0/1771169525 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55649f352680 con 0x55649f4697b0 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.964+0000 7fc96cc00640 10 client.25016 handle_command_reply: tid=0 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 client.25016 shutdown 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 -- 172.21.15.42:0/1771169525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 msgr2=0x55649f3f5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f5940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc970000c00 tx=0x7fc9700bb910 comp rx=0 tx=0).stop 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 -- 172.21.15.42:0/1771169525 shutdown_connections 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55649f4697b0 0x55649f4684c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55649f3ed4d0 0x55649f3f85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55649f3e8360 0x55649f3f5940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 --2- 172.21.15.42:0/1771169525 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55649f3e7970 0x55649f3f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc96cc00640 0 client.25016 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 -- 172.21.15.42:0/1771169525 >> 172.21.15.42:0/1771169525 conn(0x55649f3e2fe0 msgr2=0x55649f3fd840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 -- 172.21.15.42:0/1771169525 shutdown_connections 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 1 -- 172.21.15.42:0/1771169525 wait complete. 2024-04-25T00:18:06.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.965+0000 7fc98c629740 20 client.25016 trim_cache size 0 max 16384 2024-04-25T00:18:06.970 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:18:06.970 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "9d95543e-0297-11ef-bc93-c7b262605968", 2024-04-25T00:18:06.970 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-04-25T00:18:06.970 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24569, 2024-04-25T00:18:06.970 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-04-25T00:18:06.970 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-04-25T00:18:06.970 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-04-25T00:18:06.970 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 106.646263, 2024-04-25T00:18:06.970 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 23, 2024-04-25T00:18:06.970 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 88, 2024-04-25T00:18:06.971 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T00:18:06.971 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 112.094677711 2024-04-25T00:18:06.971 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:18:06.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 -- 172.21.15.42:0/3879194056 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9680a2b90 msgr2=0x7fc9680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9680a2b90 0x7fc9680a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55649f3e3f50 tx=0x7fc974005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 -- 172.21.15.42:0/3879194056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc984102830 msgr2=0x7fc98419e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc984102830 0x7fc98419e480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc970009230 tx=0x7fc9700042f0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 -- 172.21.15.42:0/3879194056 shutdown_connections 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9680a2b90 0x7fc9680a5050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9841051e0 0x7fc9841a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc984102830 0x7fc98419e480 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 --2- 172.21.15.42:0/3879194056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9840ffec0 0x7fc98419df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 -- 172.21.15.42:0/3879194056 >> 172.21.15.42:0/3879194056 conn(0x7fc9840fbae0 msgr2=0x7fc984100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 -- 172.21.15.42:0/3879194056 shutdown_connections 2024-04-25T00:18:06.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:06.966+0000 7fc983e00640 1 -- 172.21.15.42:0/3879194056 wait complete. 2024-04-25T00:18:06.982 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:06 smithi042 ceph-mon[28130]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:18:06.982 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:06 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3798609846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:06.986 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T00:18:06.987 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T00:18:06.997 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-04-25T00:18:06.997 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T00:18:07.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:06 smithi136 ceph-mon[32923]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:18:07.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:06 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3798609846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:07.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:06 smithi106 ceph-mon[32510]: pgmap v368: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:18:07.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:06 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3798609846' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:07.475 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:09.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:09 smithi136 ceph-mon[32923]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:18:09.267 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:09 smithi042 ceph-mon[28130]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:18:09.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:09 smithi106 ceph-mon[32510]: pgmap v369: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:18:09.814 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.813+0000 7fec5cc00700 1 -- 172.21.15.42:0/3701761475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec58102d20 msgr2=0x7fec58105110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.813+0000 7fec5cc00700 1 --2- 172.21.15.42:0/3701761475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec58102d20 0x7fec58105110 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fec480098f0 tx=0x7fec48009c00 comp rx=0 tx=0).stop 2024-04-25T00:18:09.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.814+0000 7fec5cc00700 1 -- 172.21.15.42:0/3701761475 shutdown_connections 2024-04-25T00:18:09.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.814+0000 7fec5cc00700 1 --2- 172.21.15.42:0/3701761475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec58107f80 0x7fec5810a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.814+0000 7fec5cc00700 1 --2- 172.21.15.42:0/3701761475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec58102d20 0x7fec58105110 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.814+0000 7fec5cc00700 1 --2- 172.21.15.42:0/3701761475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec58105650 0x7fec58107a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.814+0000 7fec5cc00700 1 -- 172.21.15.42:0/3701761475 >> 172.21.15.42:0/3701761475 conn(0x7fec580fc730 msgr2=0x7fec580feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:09.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.814+0000 7fec5cc00700 1 -- 172.21.15.42:0/3701761475 shutdown_connections 2024-04-25T00:18:09.815 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.814+0000 7fec5cc00700 1 -- 172.21.15.42:0/3701761475 wait complete. 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.815+0000 7fec5cc00700 1 Processor -- start 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.815+0000 7fec5cc00700 1 -- start start 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.815+0000 7fec5cc00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec58102d20 0x7fec5819a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.815+0000 7fec5cc00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec58105650 0x7fec5819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.815+0000 7fec5cc00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec58107f80 0x7fec581a1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.815+0000 7fec5cc00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fec5810de60 con 0x7fec58102d20 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.815+0000 7fec5cc00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fec5810dce0 con 0x7fec58105650 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.815+0000 7fec5cc00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fec5810dfe0 con 0x7fec58107f80 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec55e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec58102d20 0x7fec5819a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.817 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec55e00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec58102d20 0x7fec5819a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42182/0 (socket says 172.21.15.42:42182) 2024-04-25T00:18:09.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec55e00700 1 -- 172.21.15.42:0/3639722574 learned_addr learned my addr 172.21.15.42:0/3639722574 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:09.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec55400700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec58105650 0x7fec5819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec56800700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec58107f80 0x7fec581a1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec55e00700 1 -- 172.21.15.42:0/3639722574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec58107f80 msgr2=0x7fec581a1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec55e00700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec58107f80 0x7fec581a1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec55e00700 1 -- 172.21.15.42:0/3639722574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec58105650 msgr2=0x7fec5819acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:09.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec55e00700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec58105650 0x7fec5819acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:09.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec55e00700 1 -- 172.21.15.42:0/3639722574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec581a2410 con 0x7fec58102d20 2024-04-25T00:18:09.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.816+0000 7fec55400700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec58105650 0x7fec5819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:09.818 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.817+0000 7fec55e00700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec58102d20 0x7fec5819a7b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec480059b0 tx=0x7fec480050f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:09.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.817+0000 7fec4ea00700 1 -- 172.21.15.42:0/3639722574 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec480055d0 con 0x7fec58102d20 2024-04-25T00:18:09.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.817+0000 7fec4ea00700 1 -- 172.21.15.42:0/3639722574 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec4800ba40 con 0x7fec58102d20 2024-04-25T00:18:09.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.817+0000 7fec4ea00700 1 -- 172.21.15.42:0/3639722574 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec48017400 con 0x7fec58102d20 2024-04-25T00:18:09.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.817+0000 7fec5cc00700 1 -- 172.21.15.42:0/3639722574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec581a26a0 con 0x7fec58102d20 2024-04-25T00:18:09.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.817+0000 7fec5cc00700 1 -- 172.21.15.42:0/3639722574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec581a2af0 con 0x7fec58102d20 2024-04-25T00:18:09.819 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.819+0000 7fec5cc00700 1 -- 172.21.15.42:0/3639722574 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec58068f20 con 0x7fec58102d20 2024-04-25T00:18:09.820 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.820+0000 7fec4ea00700 1 -- 172.21.15.42:0/3639722574 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec48004760 con 0x7fec58102d20 2024-04-25T00:18:09.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.822+0000 7fec4ea00700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fec3c0a2c20 0x7fec3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:09.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.822+0000 7fec4ea00700 1 -- 172.21.15.42:0/3639722574 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fec480d0240 con 0x7fec58102d20 2024-04-25T00:18:09.822 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.822+0000 7fec55400700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fec3c0a2c20 0x7fec3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:09.823 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.823+0000 7fec55400700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fec3c0a2c20 0x7fec3c0a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fec44009cb0 tx=0x7fec44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:09.828 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:09.828+0000 7fec4ea00700 1 -- 172.21.15.42:0/3639722574 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fec48013030 con 0x7fec58102d20 2024-04-25T00:18:10.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.077+0000 7fec5cc00700 1 -- 172.21.15.42:0/3639722574 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fec58062670 con 0x7fec3c0a2c20 2024-04-25T00:18:10.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.081+0000 7fec4ea00700 1 -- 172.21.15.42:0/3639722574 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fec58062670 con 0x7fec3c0a2c20 2024-04-25T00:18:10.082 INFO:teuthology.orchestra.run.smithi042.stdout:Backend: cephadm 2024-04-25T00:18:10.082 INFO:teuthology.orchestra.run.smithi042.stdout:Available: Yes 2024-04-25T00:18:10.082 INFO:teuthology.orchestra.run.smithi042.stdout:Paused: No 2024-04-25T00:18:10.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.084+0000 7fec5cc00700 1 -- 172.21.15.42:0/3639722574 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fec3c0a2c20 msgr2=0x7fec3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.084+0000 7fec5cc00700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fec3c0a2c20 0x7fec3c0a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fec44009cb0 tx=0x7fec44009260 comp rx=0 tx=0).stop 2024-04-25T00:18:10.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.084+0000 7fec5cc00700 1 -- 172.21.15.42:0/3639722574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec58102d20 msgr2=0x7fec5819a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:10.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.084+0000 7fec5cc00700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec58102d20 0x7fec5819a7b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec480059b0 tx=0x7fec480050f0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.084+0000 7fec5cc00700 1 -- 172.21.15.42:0/3639722574 shutdown_connections 2024-04-25T00:18:10.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.085+0000 7fec5cc00700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fec3c0a2c20 0x7fec3c0a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.085+0000 7fec5cc00700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec58107f80 0x7fec581a1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.085+0000 7fec5cc00700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fec58102d20 0x7fec5819a7b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.085+0000 7fec5cc00700 1 --2- 172.21.15.42:0/3639722574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fec58105650 0x7fec5819acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:10.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.085+0000 7fec5cc00700 1 -- 172.21.15.42:0/3639722574 >> 172.21.15.42:0/3639722574 conn(0x7fec580fc730 msgr2=0x7fec581024e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:10.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.085+0000 7fec5cc00700 1 -- 172.21.15.42:0/3639722574 shutdown_connections 2024-04-25T00:18:10.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:10.085+0000 7fec5cc00700 1 -- 172.21.15.42:0/3639722574 wait complete. 2024-04-25T00:18:11.062 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T00:18:11.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:11 smithi042 ceph-mon[28130]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:18:11.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:11 smithi042 ceph-mon[28130]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:11.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:11 smithi106 ceph-mon[32510]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:18:11.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:11 smithi106 ceph-mon[32510]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:11.508 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:11 smithi136 ceph-mon[32923]: pgmap v370: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:18:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:11 smithi136 ceph-mon[32923]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:13.243 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:13 smithi042 ceph-mon[28130]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:18:13.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:13 smithi106 ceph-mon[32510]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:18:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:13 smithi136 ceph-mon[32923]: pgmap v371: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:18:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.775+0000 7f2e17400700 1 -- 172.21.15.42:0/2817568576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e10105640 msgr2=0x7f2e10107a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.775+0000 7f2e17400700 1 --2- 172.21.15.42:0/2817568576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e10105640 0x7f2e10107a30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0098f0 tx=0x7f2dfc009c00 comp rx=0 tx=0).stop 2024-04-25T00:18:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.775+0000 7f2e17400700 1 -- 172.21.15.42:0/2817568576 shutdown_connections 2024-04-25T00:18:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.775+0000 7f2e17400700 1 --2- 172.21.15.42:0/2817568576 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e10107f70 0x7f2e1010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.775+0000 7f2e17400700 1 --2- 172.21.15.42:0/2817568576 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e10102d10 0x7f2e10105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.775+0000 7f2e17400700 1 --2- 172.21.15.42:0/2817568576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e10105640 0x7f2e10107a30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.775+0000 7f2e17400700 1 -- 172.21.15.42:0/2817568576 >> 172.21.15.42:0/2817568576 conn(0x7f2e100fc720 msgr2=0x7f2e100feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.775+0000 7f2e17400700 1 -- 172.21.15.42:0/2817568576 shutdown_connections 2024-04-25T00:18:13.776 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.775+0000 7f2e17400700 1 -- 172.21.15.42:0/2817568576 wait complete. 2024-04-25T00:18:13.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.776+0000 7f2e17400700 1 Processor -- start 2024-04-25T00:18:13.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.777+0000 7f2e17400700 1 -- start start 2024-04-25T00:18:13.777 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.777+0000 7f2e17400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e10102d10 0x7f2e101984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:13.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.777+0000 7f2e17400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e10105640 0x7f2e10198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:13.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.777+0000 7f2e17400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e10107f70 0x7f2e1019fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:13.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.777+0000 7f2e17400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1010de90 con 0x7f2e10107f70 2024-04-25T00:18:13.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.777+0000 7f2e17400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1010dd10 con 0x7f2e10102d10 2024-04-25T00:18:13.778 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.777+0000 7f2e17400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2e1010e010 con 0x7f2e10105640 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e15400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e10107f70 0x7f2e1019fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e14a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e10102d10 0x7f2e101984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e0fe00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e10105640 0x7f2e10198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e15400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e10107f70 0x7f2e1019fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42202/0 (socket says 172.21.15.42:42202) 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e15400700 1 -- 172.21.15.42:0/876742417 learned_addr learned my addr 172.21.15.42:0/876742417 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e14a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e10102d10 0x7f2e101984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:48260/0 (socket says 172.21.15.42:48260) 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e15400700 1 -- 172.21.15.42:0/876742417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e10105640 msgr2=0x7f2e10198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e15400700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e10105640 0x7f2e10198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e15400700 1 -- 172.21.15.42:0/876742417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e10102d10 msgr2=0x7f2e101984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e15400700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e10102d10 0x7f2e101984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e15400700 1 -- 172.21.15.42:0/876742417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e101a0120 con 0x7f2e10107f70 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.778+0000 7f2e14a00700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e10102d10 0x7f2e101984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:13.779 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.779+0000 7f2e15400700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e10107f70 0x7f2e1019fa80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800efc0 tx=0x7f2e0800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:13.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.779+0000 7f2e0d600700 1 -- 172.21.15.42:0/876742417 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e080092e0 con 0x7f2e10107f70 2024-04-25T00:18:13.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.779+0000 7f2e17400700 1 -- 172.21.15.42:0/876742417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e101a03b0 con 0x7f2e10107f70 2024-04-25T00:18:13.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.779+0000 7f2e0d600700 1 -- 172.21.15.42:0/876742417 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e08004bb0 con 0x7f2e10107f70 2024-04-25T00:18:13.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.779+0000 7f2e0d600700 1 -- 172.21.15.42:0/876742417 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e08004080 con 0x7f2e10107f70 2024-04-25T00:18:13.780 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.779+0000 7f2e17400700 1 -- 172.21.15.42:0/876742417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e101a08f0 con 0x7f2e10107f70 2024-04-25T00:18:13.781 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.781+0000 7f2e17400700 1 -- 172.21.15.42:0/876742417 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e10068f20 con 0x7f2e10107f70 2024-04-25T00:18:13.783 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.783+0000 7f2e0d600700 1 -- 172.21.15.42:0/876742417 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2e0801b030 con 0x7f2e10107f70 2024-04-25T00:18:13.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.784+0000 7f2e0d600700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2df80a2c20 0x7f2df80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:13.784 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.784+0000 7f2e0d600700 1 -- 172.21.15.42:0/876742417 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f2e080ceaf0 con 0x7f2e10107f70 2024-04-25T00:18:13.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.784+0000 7f2e14a00700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2df80a2c20 0x7f2df80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:13.785 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.785+0000 7f2e14a00700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2df80a2c20 0x7f2df80a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009680 tx=0x7f2e04009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:13.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:13.788+0000 7f2e0d600700 1 -- 172.21.15.42:0/876742417 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2e08022020 con 0x7f2e10107f70 2024-04-25T00:18:14.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.028+0000 7f2e17400700 1 -- 172.21.15.42:0/876742417 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2e10062670 con 0x7f2df80a2c20 2024-04-25T00:18:14.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.069+0000 7f2e0d600700 1 -- 172.21.15.42:0/876742417 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f2e10062670 con 0x7f2df80a2c20 2024-04-25T00:18:14.070 INFO:teuthology.orchestra.run.smithi042.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T00:18:14.070 INFO:teuthology.orchestra.run.smithi042.stdout:mds.a smithi042 running (2m) 25s ago 2m 16.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 640f6360ec46 2024-04-25T00:18:14.070 INFO:teuthology.orchestra.run.smithi042.stdout:mds.b smithi106 running (119s) 50s ago 119s 25.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef 132e3dfb6c26 2024-04-25T00:18:14.070 INFO:teuthology.orchestra.run.smithi042.stdout:mds.c smithi136 running (2m) 50s ago 2m 17.8M - 19.0.0-2445-gd2c45d97 f68e37b807ef dc40c9ab2d53 2024-04-25T00:18:14.070 INFO:teuthology.orchestra.run.smithi042.stdout:mds.d smithi042 running (2m) 25s ago 2m 12.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef 2e34cbe51038 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mds.e smithi106 running (2m) 50s ago 2m 13.5M - 19.0.0-2445-gd2c45d97 f68e37b807ef 6b97b9550016 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mds.f smithi136 running (2m) 50s ago 2m 16.7M - 19.0.0-2445-gd2c45d97 f68e37b807ef 2c248d79ecf8 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mds.g smithi042 running (2m) 25s ago 2m 12.2M - 19.0.0-2445-gd2c45d97 f68e37b807ef 2960e809835a 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mds.h smithi106 running (2m) 50s ago 2m 19.3M - 19.0.0-2445-gd2c45d97 f68e37b807ef b047dea94f74 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mds.i smithi136 running (2m) 50s ago 2m 17.9M - 19.0.0-2445-gd2c45d97 f68e37b807ef 29203519efc7 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mds.j smithi042 running (2m) 25s ago 2m 12.4M - 19.0.0-2445-gd2c45d97 f68e37b807ef cc36f7531251 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mds.k smithi106 running (2m) 50s ago 2m 16.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 9af6b9f04f64 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mds.l smithi136 running (2m) 50s ago 2m 14.1M - 19.0.0-2445-gd2c45d97 f68e37b807ef 0b8ea76fe113 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.x smithi042 *:9283,8765 running (10m) 25s ago 10m 516M - 19.0.0-2445-gd2c45d97 f68e37b807ef c05d3001fc48 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.y smithi106 *:8443,8765 running (8m) 50s ago 8m 452M - 19.0.0-2445-gd2c45d97 f68e37b807ef 5077203a74ec 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mgr.z smithi136 *:8443,8765 running (8m) 50s ago 8m 452M - 19.0.0-2445-gd2c45d97 f68e37b807ef d8b0fe708cd7 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mon.a smithi042 running (10m) 25s ago 10m 48.6M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 7bf7a651ead3 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mon.b smithi106 running (8m) 50s ago 8m 44.0M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 959ee9e9d9b3 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:mon.c smithi136 running (8m) 50s ago 8m 46.4M 2048M 19.0.0-2445-gd2c45d97 f68e37b807ef 02072d7cff8c 2024-04-25T00:18:14.071 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 smithi042 running (7m) 25s ago 7m 47.7M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef 2c8d9311c283 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 smithi106 running (7m) 50s ago 7m 49.1M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef e1d5e5fe39db 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 smithi136 running (7m) 50s ago 7m 49.7M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 444626d16979 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 smithi042 running (6m) 25s ago 6m 45.7M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef d715bea2b313 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 smithi106 running (6m) 50s ago 6m 45.9M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 9a00358cb526 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 smithi136 running (5m) 50s ago 5m 53.8M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef adc0a9c524c1 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 smithi042 running (5m) 25s ago 5m 44.0M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef dd0437b5cb7c 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 smithi106 running (4m) 50s ago 4m 46.3M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef eef0f07bd7a3 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 smithi136 running (4m) 50s ago 4m 54.3M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 3129ebf7f49d 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 smithi042 running (3m) 25s ago 3m 49.2M 4273M 19.0.0-2445-gd2c45d97 f68e37b807ef e2fe7e09bc24 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 smithi106 running (3m) 50s ago 3m 49.5M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef b2f9b2872929 2024-04-25T00:18:14.072 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 smithi136 running (2m) 50s ago 2m 45.5M 4272M 19.0.0-2445-gd2c45d97 f68e37b807ef 963cb7636e71 2024-04-25T00:18:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.072+0000 7f2e17400700 1 -- 172.21.15.42:0/876742417 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2df80a2c20 msgr2=0x7f2df80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.072+0000 7f2e17400700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2df80a2c20 0x7f2df80a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2e04009680 tx=0x7f2e04009260 comp rx=0 tx=0).stop 2024-04-25T00:18:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.072+0000 7f2e17400700 1 -- 172.21.15.42:0/876742417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e10107f70 msgr2=0x7f2e1019fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.072+0000 7f2e17400700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e10107f70 0x7f2e1019fa80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2e0800efc0 tx=0x7f2e0800c3a0 comp rx=0 tx=0).stop 2024-04-25T00:18:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.072+0000 7f2e17400700 1 -- 172.21.15.42:0/876742417 shutdown_connections 2024-04-25T00:18:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.072+0000 7f2e17400700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2df80a2c20 0x7f2df80a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:14.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.072+0000 7f2e17400700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2e10107f70 0x7f2e1019fa80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.073+0000 7f2e17400700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2e10102d10 0x7f2e101984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.073+0000 7f2e17400700 1 --2- 172.21.15.42:0/876742417 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e10105640 0x7f2e10198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.073+0000 7f2e17400700 1 -- 172.21.15.42:0/876742417 >> 172.21.15.42:0/876742417 conn(0x7f2e100fc720 msgr2=0x7f2e10103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.073+0000 7f2e17400700 1 -- 172.21.15.42:0/876742417 shutdown_connections 2024-04-25T00:18:14.074 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:14.073+0000 7f2e17400700 1 -- 172.21.15.42:0/876742417 wait complete. 2024-04-25T00:18:15.012 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T00:18:15.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:15 smithi106 ceph-mon[32510]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:15.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:15 smithi106 ceph-mon[32510]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:15.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:15 smithi042 ceph-mon[28130]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:15.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:15 smithi042 ceph-mon[28130]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:15.470 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:15 smithi136 ceph-mon[32923]: pgmap v372: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:15 smithi136 ceph-mon[32923]: from='client.14925 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:17.221 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:17 smithi042 ceph-mon[28130]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:17.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:17 smithi106 ceph-mon[32510]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:17.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:17 smithi136 ceph-mon[32923]: pgmap v373: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.795+0000 7f2c12c00700 1 -- 172.21.15.42:0/2853984724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c108010 msgr2=0x7f2c0c10a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.795+0000 7f2c12c00700 1 --2- 172.21.15.42:0/2853984724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c108010 0x7f2c0c10a400 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c0400b0a0 tx=0x7f2c0400b3b0 comp rx=0 tx=0).stop 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.795+0000 7f2c12c00700 1 -- 172.21.15.42:0/2853984724 shutdown_connections 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.795+0000 7f2c12c00700 1 --2- 172.21.15.42:0/2853984724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c108010 0x7f2c0c10a400 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.795+0000 7f2c12c00700 1 --2- 172.21.15.42:0/2853984724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c102db0 0x7f2c0c1051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.795+0000 7f2c12c00700 1 --2- 172.21.15.42:0/2853984724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c1056e0 0x7f2c0c107ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.795+0000 7f2c12c00700 1 -- 172.21.15.42:0/2853984724 >> 172.21.15.42:0/2853984724 conn(0x7f2c0c0fc720 msgr2=0x7f2c0c0feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.795+0000 7f2c12c00700 1 -- 172.21.15.42:0/2853984724 shutdown_connections 2024-04-25T00:18:17.796 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.795+0000 7f2c12c00700 1 -- 172.21.15.42:0/2853984724 wait complete. 2024-04-25T00:18:17.797 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.796+0000 7f2c12c00700 1 Processor -- start 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.796+0000 7f2c12c00700 1 -- start start 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c12c00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c102db0 0x7f2c0c198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c12c00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c1056e0 0x7f2c0c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c12c00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c108010 0x7f2c0c19fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c12c00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10dea0 con 0x7f2c0c108010 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c12c00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10dd20 con 0x7f2c0c102db0 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c12c00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0c10e020 con 0x7f2c0c1056e0 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c0be00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c102db0 0x7f2c0c198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c0be00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c102db0 0x7f2c0c198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51198/0 (socket says 172.21.15.42:51198) 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c0be00700 1 -- 172.21.15.42:0/3497146472 learned_addr learned my addr 172.21.15.42:0/3497146472 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c10c00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c108010 0x7f2c0c19fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.797+0000 7f2c0b400700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c1056e0 0x7f2c0c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:17.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c0be00700 1 -- 172.21.15.42:0/3497146472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c1056e0 msgr2=0x7f2c0c1989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c0be00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c1056e0 0x7f2c0c1989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c0be00700 1 -- 172.21.15.42:0/3497146472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c108010 msgr2=0x7f2c0c19fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c0be00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c108010 0x7f2c0c19fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:17.799 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c0be00700 1 -- 172.21.15.42:0/3497146472 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0c1a00c0 con 0x7f2c0c102db0 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c0be00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c102db0 0x7f2c0c198460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00e9c0 tx=0x7f2bfc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c08c00700 1 -- 172.21.15.42:0/3497146472 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc00cbd0 con 0x7f2c0c102db0 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c08c00700 1 -- 172.21.15.42:0/3497146472 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bfc00ee60 con 0x7f2c0c102db0 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c12c00700 1 -- 172.21.15.42:0/3497146472 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0c1a03b0 con 0x7f2c0c102db0 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c08c00700 1 -- 172.21.15.42:0/3497146472 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bfc005680 con 0x7f2c0c102db0 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.798+0000 7f2c12c00700 1 -- 172.21.15.42:0/3497146472 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0c1a08f0 con 0x7f2c0c102db0 2024-04-25T00:18:17.800 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.799+0000 7f2c12c00700 1 -- 172.21.15.42:0/3497146472 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c0c068f20 con 0x7f2c0c102db0 2024-04-25T00:18:17.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.801+0000 7f2c08c00700 1 -- 172.21.15.42:0/3497146472 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2bfc004750 con 0x7f2c0c102db0 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.803+0000 7f2c08c00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf00a2ca0 0x7f2bf00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.803+0000 7f2c08c00700 1 -- 172.21.15.42:0/3497146472 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f2bfc0cf240 con 0x7f2c0c102db0 2024-04-25T00:18:17.803 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.803+0000 7f2c0b400700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf00a2ca0 0x7f2bf00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:17.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.804+0000 7f2c0b400700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf00a2ca0 0x7f2bf00a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009680 tx=0x7f2bf8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:17.807 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:17.807+0000 7f2c08c00700 1 -- 172.21.15.42:0/3497146472 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2bfc09c170 con 0x7f2c0c102db0 2024-04-25T00:18:18.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.048+0000 7f2c12c00700 1 -- 172.21.15.42:0/3497146472 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f2c0c062670 con 0x7f2bf00a2ca0 2024-04-25T00:18:18.066 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.066+0000 7f2c08c00700 1 -- 172.21.15.42:0/3497146472 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f2c0c062670 con 0x7f2bf00a2ca0 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi042.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi042.stdout:mds.all 12/12 54s ago 2m smithi042=a;smithi042=d;smithi042=g;smithi042=j;smithi106=b;smithi106=e;smithi106=h;smithi106=k;smithi136=c;smithi136=f;smithi136=i;smithi136=l;count:12 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi042.stdout:mgr 3/3 54s ago 8m smithi042=x;smithi106=y;smithi136=z;count:3 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi042.stdout:mon 3/3 54s ago 9m smithi042:172.21.15.42=a;smithi106:172.21.15.106=b;smithi136:172.21.15.136=c;count:3 2024-04-25T00:18:18.067 INFO:teuthology.orchestra.run.smithi042.stdout:osd 12 54s ago - 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 -- 172.21.15.42:0/3497146472 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf00a2ca0 msgr2=0x7f2bf00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf00a2ca0 0x7f2bf00a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009680 tx=0x7f2bf8009260 comp rx=0 tx=0).stop 2024-04-25T00:18:18.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 -- 172.21.15.42:0/3497146472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c102db0 msgr2=0x7f2c0c198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c102db0 0x7f2c0c198460 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc00e9c0 tx=0x7f2bfc00ecd0 comp rx=0 tx=0).stop 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 -- 172.21.15.42:0/3497146472 shutdown_connections 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2bf00a2ca0 0x7f2bf00a5160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c0c108010 0x7f2c0c19fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c0c102db0 0x7f2c0c198460 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 --2- 172.21.15.42:0/3497146472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c0c1056e0 0x7f2c0c1989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 -- 172.21.15.42:0/3497146472 >> 172.21.15.42:0/3497146472 conn(0x7f2c0c0fc720 msgr2=0x7f2c0c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:18.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 -- 172.21.15.42:0/3497146472 shutdown_connections 2024-04-25T00:18:18.071 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:18.069+0000 7f2c12c00700 1 -- 172.21.15.42:0/3497146472 wait complete. 2024-04-25T00:18:18.884 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T00:18:19.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:19 smithi042 ceph-mon[28130]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:19.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:19 smithi042 ceph-mon[28130]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:19.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:19 smithi106 ceph-mon[32510]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:19.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:19 smithi106 ceph-mon[32510]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:19.353 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:19.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:19 smithi136 ceph-mon[32923]: pgmap v374: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:19.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:19 smithi136 ceph-mon[32923]: from='client.25034 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:21.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:21 smithi106 ceph-mon[32510]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:18:21.369 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:21 smithi042 ceph-mon[28130]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:18:21.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:21 smithi136 ceph-mon[32923]: pgmap v375: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:18:21.545 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.545+0000 7f4b4d800700 1 -- 172.21.15.42:0/1483877642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b48106070 msgr2=0x7f4b4810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:21.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.545+0000 7f4b4d800700 1 --2- 172.21.15.42:0/1483877642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b48106070 0x7f4b4810c920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009920 tx=0x7f4b40009c30 comp rx=0 tx=0).stop 2024-04-25T00:18:21.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.545+0000 7f4b4d800700 1 -- 172.21.15.42:0/1483877642 shutdown_connections 2024-04-25T00:18:21.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.545+0000 7f4b4d800700 1 --2- 172.21.15.42:0/1483877642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b48106070 0x7f4b4810c920 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:21.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.545+0000 7f4b4d800700 1 --2- 172.21.15.42:0/1483877642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b481044d0 0x7f4b481048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:21.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.545+0000 7f4b4d800700 1 --2- 172.21.15.42:0/1483877642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b481056d0 0x7f4b48105b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:21.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.545+0000 7f4b4d800700 1 -- 172.21.15.42:0/1483877642 >> 172.21.15.42:0/1483877642 conn(0x7f4b480ffa90 msgr2=0x7f4b48101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:21.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.545+0000 7f4b4d800700 1 -- 172.21.15.42:0/1483877642 shutdown_connections 2024-04-25T00:18:21.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.545+0000 7f4b4d800700 1 -- 172.21.15.42:0/1483877642 wait complete. 2024-04-25T00:18:21.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.547+0000 7f4b4d800700 1 Processor -- start 2024-04-25T00:18:21.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.547+0000 7f4b4d800700 1 -- start start 2024-04-25T00:18:21.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.547+0000 7f4b4d800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b481044d0 0x7f4b4819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:21.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.547+0000 7f4b4d800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b481056d0 0x7f4b4819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:21.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.547+0000 7f4b4d800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b48106070 0x7f4b481a1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:21.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.547+0000 7f4b4d800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481100f0 con 0x7f4b481056d0 2024-04-25T00:18:21.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.547+0000 7f4b4d800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810ff70 con 0x7f4b481044d0 2024-04-25T00:18:21.548 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.547+0000 7f4b4d800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b48110270 con 0x7f4b48106070 2024-04-25T00:18:21.549 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.548+0000 7f4b46000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b481056d0 0x7f4b4819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.548+0000 7f4b46000700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b481056d0 0x7f4b4819ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45028/0 (socket says 172.21.15.42:45028) 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.548+0000 7f4b46a00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b481044d0 0x7f4b4819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.548+0000 7f4b46000700 1 -- 172.21.15.42:0/532112604 learned_addr learned my addr 172.21.15.42:0/532112604 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.548+0000 7f4b47400700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b48106070 0x7f4b481a1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.548+0000 7f4b46a00700 1 -- 172.21.15.42:0/532112604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b48106070 msgr2=0x7f4b481a1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.548+0000 7f4b46a00700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b48106070 0x7f4b481a1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.548+0000 7f4b46a00700 1 -- 172.21.15.42:0/532112604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b481056d0 msgr2=0x7f4b4819ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.548+0000 7f4b46a00700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b481056d0 0x7f4b4819ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.548+0000 7f4b46a00700 1 -- 172.21.15.42:0/532112604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b481a2360 con 0x7f4b481044d0 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.549+0000 7f4b46a00700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b481044d0 0x7f4b4819a700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800eee0 tx=0x7f4b3800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.549+0000 7f4b3f400700 1 -- 172.21.15.42:0/532112604 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b380092e0 con 0x7f4b481044d0 2024-04-25T00:18:21.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.549+0000 7f4b3f400700 1 -- 172.21.15.42:0/532112604 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b38003a00 con 0x7f4b481044d0 2024-04-25T00:18:21.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.549+0000 7f4b3f400700 1 -- 172.21.15.42:0/532112604 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b380045e0 con 0x7f4b481044d0 2024-04-25T00:18:21.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.549+0000 7f4b4d800700 1 -- 172.21.15.42:0/532112604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b481a25f0 con 0x7f4b481044d0 2024-04-25T00:18:21.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.549+0000 7f4b4d800700 1 -- 172.21.15.42:0/532112604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b481a2b80 con 0x7f4b481044d0 2024-04-25T00:18:21.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.550+0000 7f4b4d800700 1 -- 172.21.15.42:0/532112604 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b48068f20 con 0x7f4b481044d0 2024-04-25T00:18:21.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.552+0000 7f4b3f400700 1 -- 172.21.15.42:0/532112604 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b3801b030 con 0x7f4b481044d0 2024-04-25T00:18:21.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.554+0000 7f4b3f400700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4b2c0a2ca0 0x7f4b2c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:21.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.554+0000 7f4b3f400700 1 -- 172.21.15.42:0/532112604 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f4b380cefe0 con 0x7f4b481044d0 2024-04-25T00:18:21.554 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.554+0000 7f4b46000700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4b2c0a2ca0 0x7f4b2c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:21.555 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.555+0000 7f4b46000700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4b2c0a2ca0 0x7f4b2c0a5160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005950 tx=0x7f4b34009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:21.558 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.558+0000 7f4b3f400700 1 -- 172.21.15.42:0/532112604 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f4b38022020 con 0x7f4b481044d0 2024-04-25T00:18:21.795 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.795+0000 7f4b4d800700 1 -- 172.21.15.42:0/532112604 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f4b48062670 con 0x7f4b2c0a2ca0 2024-04-25T00:18:21.798 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.798+0000 7f4b3f400700 1 -- 172.21.15.42:0/532112604 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f4b48062670 con 0x7f4b2c0a2ca0 2024-04-25T00:18:21.799 INFO:teuthology.orchestra.run.smithi042.stdout:HOST ADDR LABELS STATUS 2024-04-25T00:18:21.799 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 172.21.15.42 2024-04-25T00:18:21.799 INFO:teuthology.orchestra.run.smithi042.stdout:smithi106 172.21.15.106 2024-04-25T00:18:21.799 INFO:teuthology.orchestra.run.smithi042.stdout:smithi136 172.21.15.136 2024-04-25T00:18:21.799 INFO:teuthology.orchestra.run.smithi042.stdout:3 hosts in cluster 2024-04-25T00:18:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 -- 172.21.15.42:0/532112604 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4b2c0a2ca0 msgr2=0x7f4b2c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4b2c0a2ca0 0x7f4b2c0a5160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4b34005950 tx=0x7f4b34009310 comp rx=0 tx=0).stop 2024-04-25T00:18:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 -- 172.21.15.42:0/532112604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b481044d0 msgr2=0x7f4b4819a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b481044d0 0x7f4b4819a700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800eee0 tx=0x7f4b3800c3a0 comp rx=0 tx=0).stop 2024-04-25T00:18:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 -- 172.21.15.42:0/532112604 shutdown_connections 2024-04-25T00:18:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f4b2c0a2ca0 0x7f4b2c0a5160 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b48106070 0x7f4b481a1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b481044d0 0x7f4b4819a700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:21.801 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 --2- 172.21.15.42:0/532112604 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4b481056d0 0x7f4b4819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 -- 172.21.15.42:0/532112604 >> 172.21.15.42:0/532112604 conn(0x7f4b480ffa90 msgr2=0x7f4b48108970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 -- 172.21.15.42:0/532112604 shutdown_connections 2024-04-25T00:18:21.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:21.801+0000 7f4b4d800700 1 -- 172.21.15.42:0/532112604 wait complete. 2024-04-25T00:18:22.686 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T00:18:23.165 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:23.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:23 smithi042 ceph-mon[28130]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:23.326 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:23 smithi042 ceph-mon[28130]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:18:23.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:23 smithi106 ceph-mon[32510]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:23.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:23 smithi106 ceph-mon[32510]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:18:23.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:23 smithi136 ceph-mon[32923]: from='client.25040 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:23.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:23 smithi136 ceph-mon[32923]: pgmap v376: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.283+0000 7f9f5e400700 1 -- 172.21.15.42:0/2051116019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f58103d70 msgr2=0x7f9f5810a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.283+0000 7f9f5e400700 1 --2- 172.21.15.42:0/2051116019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f58103d70 0x7f9f5810a620 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000b0d0 tx=0x7f9f5000b3e0 comp rx=0 tx=0).stop 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.283+0000 7f9f5e400700 1 -- 172.21.15.42:0/2051116019 shutdown_connections 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.283+0000 7f9f5e400700 1 --2- 172.21.15.42:0/2051116019 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f58103d70 0x7f9f5810a620 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.283+0000 7f9f5e400700 1 --2- 172.21.15.42:0/2051116019 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f581021d0 0x7f9f581025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.283+0000 7f9f5e400700 1 --2- 172.21.15.42:0/2051116019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f581033d0 0x7f9f58103830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.283+0000 7f9f5e400700 1 -- 172.21.15.42:0/2051116019 >> 172.21.15.42:0/2051116019 conn(0x7f9f580fd770 msgr2=0x7f9f580ffbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.283+0000 7f9f5e400700 1 -- 172.21.15.42:0/2051116019 shutdown_connections 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.283+0000 7f9f5e400700 1 -- 172.21.15.42:0/2051116019 wait complete. 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.284+0000 7f9f5e400700 1 Processor -- start 2024-04-25T00:18:25.285 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5e400700 1 -- start start 2024-04-25T00:18:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5e400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f581021d0 0x7f9f58198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5e400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f581033d0 0x7f9f58198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5e400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f58103d70 0x7f9f5819f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5e400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5810ddd0 con 0x7f9f58103d70 2024-04-25T00:18:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5e400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5810dc50 con 0x7f9f581021d0 2024-04-25T00:18:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5e400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9f5810df50 con 0x7f9f581033d0 2024-04-25T00:18:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5d000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f581021d0 0x7f9f58198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5d000700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f581021d0 0x7f9f58198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51236/0 (socket says 172.21.15.42:51236) 2024-04-25T00:18:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5d000700 1 -- 172.21.15.42:0/3951017032 learned_addr learned my addr 172.21.15.42:0/3951017032 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:25.287 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f57e00700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f581033d0 0x7f9f58198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.285+0000 7f9f5da00700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f58103d70 0x7f9f5819f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.286+0000 7f9f57e00700 1 -- 172.21.15.42:0/3951017032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f581021d0 msgr2=0x7f9f58198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.286+0000 7f9f57e00700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f581021d0 0x7f9f58198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.286+0000 7f9f57e00700 1 -- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f58103d70 msgr2=0x7f9f5819f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.286+0000 7f9f57e00700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f58103d70 0x7f9f5819f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.286+0000 7f9f57e00700 1 -- 172.21.15.42:0/3951017032 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5819ff80 con 0x7f9f581033d0 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.286+0000 7f9f5d000700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f581021d0 0x7f9f58198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.286+0000 7f9f5da00700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f58103d70 0x7f9f5819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.286+0000 7f9f57e00700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f581033d0 0x7f9f58198860 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400ea20 tx=0x7f9f4400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.287+0000 7f9f55600700 1 -- 172.21.15.42:0/3951017032 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400cbd0 con 0x7f9f581033d0 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.287+0000 7f9f55600700 1 -- 172.21.15.42:0/3951017032 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f44004500 con 0x7f9f581033d0 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.287+0000 7f9f5e400700 1 -- 172.21.15.42:0/3951017032 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f581a0270 con 0x7f9f581033d0 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.287+0000 7f9f55600700 1 -- 172.21.15.42:0/3951017032 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44005020 con 0x7f9f581033d0 2024-04-25T00:18:25.288 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.287+0000 7f9f5e400700 1 -- 172.21.15.42:0/3951017032 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f581a07b0 con 0x7f9f581033d0 2024-04-25T00:18:25.289 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.288+0000 7f9f5e400700 1 -- 172.21.15.42:0/3951017032 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f58068f20 con 0x7f9f581033d0 2024-04-25T00:18:25.290 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.290+0000 7f9f55600700 1 -- 172.21.15.42:0/3951017032 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9f44005200 con 0x7f9f581033d0 2024-04-25T00:18:25.291 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.291+0000 7f9f55600700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f400a2ca0 0x7f9f400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:25.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.291+0000 7f9f55600700 1 -- 172.21.15.42:0/3951017032 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f9f440ceeb0 con 0x7f9f581033d0 2024-04-25T00:18:25.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.291+0000 7f9f5d000700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f400a2ca0 0x7f9f400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:25.292 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.292+0000 7f9f5d000700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f400a2ca0 0x7f9f400a5160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c005950 tx=0x7f9f4c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:25.296 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.296+0000 7f9f55600700 1 -- 172.21.15.42:0/3951017032 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f9f4409bde0 con 0x7f9f581033d0 2024-04-25T00:18:25.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:25 smithi106 ceph-mon[32510]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:25.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:25 smithi042 ceph-mon[28130]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:25.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:25 smithi136 ceph-mon[32923]: pgmap v377: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:25.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.542+0000 7f9f5e400700 1 -- 172.21.15.42:0/3951017032 --> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f9f58062670 con 0x7f9f400a2ca0 2024-04-25T00:18:25.547 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.547+0000 7f9f55600700 1 -- 172.21.15.42:0/3951017032 <== mgr.14152 v2:172.21.15.42:6800/384891702 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f9f58062670 con 0x7f9f400a2ca0 2024-04-25T00:18:25.548 INFO:teuthology.orchestra.run.smithi042.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T00:18:25.548 INFO:teuthology.orchestra.run.smithi042.stdout:smithi042 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130021480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:18:25.548 INFO:teuthology.orchestra.run.smithi042.stdout:smithi106 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013R400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:18:25.548 INFO:teuthology.orchestra.run.smithi042.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 3m ago Has a FileSystem, LVM detected 2024-04-25T00:18:25.550 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.550+0000 7f9f5e400700 1 -- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f400a2ca0 msgr2=0x7f9f400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.550+0000 7f9f5e400700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f400a2ca0 0x7f9f400a5160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c005950 tx=0x7f9f4c009260 comp rx=0 tx=0).stop 2024-04-25T00:18:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.550+0000 7f9f5e400700 1 -- 172.21.15.42:0/3951017032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f581033d0 msgr2=0x7f9f58198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.550+0000 7f9f5e400700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f581033d0 0x7f9f58198860 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400ea20 tx=0x7f9f4400ed30 comp rx=0 tx=0).stop 2024-04-25T00:18:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.550+0000 7f9f5e400700 1 -- 172.21.15.42:0/3951017032 shutdown_connections 2024-04-25T00:18:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.550+0000 7f9f5e400700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9f400a2ca0 0x7f9f400a5160 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.550+0000 7f9f5e400700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9f58103d70 0x7f9f5819f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:25.551 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.550+0000 7f9f5e400700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9f581021d0 0x7f9f58198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.550+0000 7f9f5e400700 1 --2- 172.21.15.42:0/3951017032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9f581033d0 0x7f9f58198860 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.550+0000 7f9f5e400700 1 -- 172.21.15.42:0/3951017032 >> 172.21.15.42:0/3951017032 conn(0x7f9f580fd770 msgr2=0x7f9f58106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.551+0000 7f9f5e400700 1 -- 172.21.15.42:0/3951017032 shutdown_connections 2024-04-25T00:18:25.552 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:25.551+0000 7f9f5e400700 1 -- 172.21.15.42:0/3951017032 wait complete. 2024-04-25T00:18:26.262 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:26 smithi042 ceph-mon[28130]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:26.335 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T00:18:26.346 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-04-25T00:18:26.346 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T00:18:26.346 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:26 smithi106 ceph-mon[32510]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:26.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:26 smithi136 ceph-mon[32923]: from='client.25054 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T00:18:26.822 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:27.316 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:27 smithi042 ceph-mon[28130]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:27.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:27 smithi106 ceph-mon[32510]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:27.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:27 smithi136 ceph-mon[32923]: pgmap v378: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:29.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.042+0000 7f36cca00700 1 -- 172.21.15.42:0/2504691814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c81040e0 msgr2=0x7f36c8104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:29.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.042+0000 7f36cca00700 1 --2- 172.21.15.42:0/2504691814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c81040e0 0x7f36c8104540 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f36b40099c0 tx=0x7f36b4009cd0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.042+0000 7f36cca00700 1 -- 172.21.15.42:0/2504691814 shutdown_connections 2024-04-25T00:18:29.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.042+0000 7f36cca00700 1 --2- 172.21.15.42:0/2504691814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c8104a80 0x7f36c810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.042+0000 7f36cca00700 1 --2- 172.21.15.42:0/2504691814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36c8105ba0 0x7f36c8105fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.042+0000 7f36cca00700 1 --2- 172.21.15.42:0/2504691814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c81040e0 0x7f36c8104540 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.042+0000 7f36cca00700 1 -- 172.21.15.42:0/2504691814 >> 172.21.15.42:0/2504691814 conn(0x7f36c80ffa80 msgr2=0x7f36c8101ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:29.043 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.042+0000 7f36cca00700 1 -- 172.21.15.42:0/2504691814 shutdown_connections 2024-04-25T00:18:29.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.043+0000 7f36cca00700 1 -- 172.21.15.42:0/2504691814 wait complete. 2024-04-25T00:18:29.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.044+0000 7f36cca00700 1 Processor -- start 2024-04-25T00:18:29.044 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.044+0000 7f36cca00700 1 -- start start 2024-04-25T00:18:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.044+0000 7f36cca00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36c81040e0 0x7f36c819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.044+0000 7f36cca00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c8104a80 0x7f36c819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.044+0000 7f36cca00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c8105ba0 0x7f36c81a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.044+0000 7f36cca00700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36c81101a0 con 0x7f36c8105ba0 2024-04-25T00:18:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.044+0000 7f36cca00700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8110020 con 0x7f36c8104a80 2024-04-25T00:18:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.044+0000 7f36cca00700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f36c8110320 con 0x7f36c81040e0 2024-04-25T00:18:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c5e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36c81040e0 0x7f36c819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c6800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c8105ba0 0x7f36c81a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c5e00700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36c81040e0 0x7f36c819a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:51534/0 (socket says 172.21.15.42:51534) 2024-04-25T00:18:29.045 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c5e00700 1 -- 172.21.15.42:0/1204049026 learned_addr learned my addr 172.21.15.42:0/1204049026 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c5400700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c8104a80 0x7f36c819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:29.046 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c5e00700 1 -- 172.21.15.42:0/1204049026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c8104a80 msgr2=0x7f36c819aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c5e00700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c8104a80 0x7f36c819aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c5e00700 1 -- 172.21.15.42:0/1204049026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c8105ba0 msgr2=0x7f36c81a1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c5e00700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c8105ba0 0x7f36c81a1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c5e00700 1 -- 172.21.15.42:0/1204049026 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c81a23c0 con 0x7f36c81040e0 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.045+0000 7f36c6800700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c8105ba0 0x7f36c81a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.046+0000 7f36c5e00700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36c81040e0 0x7f36c819a760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f36b800e980 tx=0x7f36b800ec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.046+0000 7f36bea00700 1 -- 172.21.15.42:0/1204049026 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8004230 con 0x7f36c81040e0 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.046+0000 7f36cca00700 1 -- 172.21.15.42:0/1204049026 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c81a26b0 con 0x7f36c81040e0 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.046+0000 7f36bea00700 1 -- 172.21.15.42:0/1204049026 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36b80043d0 con 0x7f36c81040e0 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.046+0000 7f36bea00700 1 -- 172.21.15.42:0/1204049026 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b8003dc0 con 0x7f36c81040e0 2024-04-25T00:18:29.047 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.046+0000 7f36cca00700 1 -- 172.21.15.42:0/1204049026 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c81a2bf0 con 0x7f36c81040e0 2024-04-25T00:18:29.048 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.047+0000 7f36cca00700 1 -- 172.21.15.42:0/1204049026 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c8068f20 con 0x7f36c81040e0 2024-04-25T00:18:29.049 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.049+0000 7f36bea00700 1 -- 172.21.15.42:0/1204049026 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36b800c810 con 0x7f36c81040e0 2024-04-25T00:18:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.051+0000 7f36bea00700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36ac0a2ca0 0x7f36ac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.051+0000 7f36bea00700 1 -- 172.21.15.42:0/1204049026 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f36b80cf310 con 0x7f36c81040e0 2024-04-25T00:18:29.051 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.051+0000 7f36c5400700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36ac0a2ca0 0x7f36ac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:29.052 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.052+0000 7f36c5400700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36ac0a2ca0 0x7f36ac0a5160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f36b4000c00 tx=0x7f36b4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:29.053 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.053+0000 7f36bea00700 1 -- 172.21.15.42:0/1204049026 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f36b809c240 con 0x7f36c81040e0 2024-04-25T00:18:29.332 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:29 smithi042 ceph-mon[28130]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:29.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.331+0000 7f36cca00700 1 -- 172.21.15.42:0/1204049026 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f36c802db30 con 0x7f36c81040e0 2024-04-25T00:18:29.332 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.332+0000 7f36bea00700 1 -- 172.21.15.42:0/1204049026 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) ==== 76+0+3048 (secure 0 0 0) 0x7f36b809bb40 con 0x7f36c81040e0 2024-04-25T00:18:29.333 INFO:teuthology.orchestra.run.smithi042.stdout:e23 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:legacy client fscid: 1 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:Filesystem 'cephfs' (1) 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:fs_name cephfs 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 23 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-04-25T00:16:19.771762+0000 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-04-25T00:16:25.317872+0000 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:tableserver 0 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:root 0 2024-04-25T00:18:29.334 INFO:teuthology.orchestra.run.smithi042.stdout:session_timeout 300 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:session_autoclose 300 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:max_file_size 1099511627776 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:max_xattr_size 65536 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:required_client_features {} 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure 0 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:last_failure_osd_epoch 0 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:max_mds 3 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:in 0,1,2 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:up {0=24569,1=24568,2=24517} 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:failed 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:damaged 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:stopped 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:data_pools [3] 2024-04-25T00:18:29.335 INFO:teuthology.orchestra.run.smithi042.stdout:metadata_pool 2 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:inline_data disabled 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:balancer 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:bal_rank_mask -1 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:standby_count_wanted 0 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:qdb_cluster leader: 24569 members: 24517,24568,24569 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.b{0:24569} state up:active seq 3 addr [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.i{0:24538} state up:standby-replay seq 1 addr [v2:172.21.15.136:6838/501637536,v1:172.21.15.136:6839/501637536] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.a{1:24568} state up:active seq 4 addr [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.h{1:24521} state up:standby-replay seq 1 addr [v2:172.21.15.106:6836/1331229112,v1:172.21.15.106:6837/1331229112] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.c{2:24517} state up:active seq 8 addr [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.f{2:24505} state up:standby-replay seq 1 addr [v2:172.21.15.136:6834/12913503,v1:172.21.15.136:6835/12913503] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout:Standby daemons: 2024-04-25T00:18:29.336 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:29.337 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.j{-1:14385} state up:standby seq 1 addr [v2:172.21.15.42:6834/1318501714,v1:172.21.15.42:6835/1318501714] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.337 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.d{-1:14403} state up:standby seq 1 addr [v2:172.21.15.42:6836/3589701125,v1:172.21.15.42:6837/3589701125] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.337 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.g{-1:14451} state up:standby seq 1 addr [v2:172.21.15.42:6838/3053633590,v1:172.21.15.42:6839/3053633590] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.337 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.l{-1:24442} state up:standby seq 1 addr [v2:172.21.15.136:6832/756794270,v1:172.21.15.136:6833/756794270] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.337 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.k{-1:24449} state up:standby seq 1 addr [v2:172.21.15.106:6832/1706914889,v1:172.21.15.106:6833/1706914889] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.337 INFO:teuthology.orchestra.run.smithi042.stdout:[mds.e{-1:24494} state up:standby seq 1 addr [v2:172.21.15.106:6834/1507277236,v1:172.21.15.106:6835/1507277236] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T00:18:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 -- 172.21.15.42:0/1204049026 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36ac0a2ca0 msgr2=0x7f36ac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36ac0a2ca0 0x7f36ac0a5160 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f36b4000c00 tx=0x7f36b4005fb0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 -- 172.21.15.42:0/1204049026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36c81040e0 msgr2=0x7f36c819a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36c81040e0 0x7f36c819a760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f36b800e980 tx=0x7f36b800ec90 comp rx=0 tx=0).stop 2024-04-25T00:18:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 -- 172.21.15.42:0/1204049026 shutdown_connections 2024-04-25T00:18:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36ac0a2ca0 0x7f36ac0a5160 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36c8105ba0 0x7f36c81a1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36c81040e0 0x7f36c819a760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 --2- 172.21.15.42:0/1204049026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36c8104a80 0x7f36c819aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:29.338 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 -- 172.21.15.42:0/1204049026 >> 172.21.15.42:0/1204049026 conn(0x7f36c80ffa80 msgr2=0x7f36c81089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:29.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.337+0000 7f36cca00700 1 -- 172.21.15.42:0/1204049026 shutdown_connections 2024-04-25T00:18:29.339 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:29.338+0000 7f36cca00700 1 -- 172.21.15.42:0/1204049026 wait complete. 2024-04-25T00:18:29.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:29 smithi106 ceph-mon[32510]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:29.339 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-04-25T00:18:29.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:29 smithi136 ceph-mon[32923]: pgmap v379: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:30.137 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T00:18:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:30 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1204049026' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:18:30.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:30 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1204049026' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:18:30.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:30 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1204049026' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T00:18:30.597 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:31 smithi106 ceph-mon[32510]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:18:31.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:31 smithi042 ceph-mon[28130]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:18:31.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:31 smithi136 ceph-mon[32923]: pgmap v380: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:18:32.866 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.866+0000 7f2fbc800700 1 -- 172.21.15.42:0/3332749003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb4076260 msgr2=0x7f2fb40746c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.866+0000 7f2fbc800700 1 --2- 172.21.15.42:0/3332749003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb4076260 0x7f2fb40746c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8009990 tx=0x7f2fa8009ca0 comp rx=0 tx=0).stop 2024-04-25T00:18:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.866+0000 7f2fbc800700 1 -- 172.21.15.42:0/3332749003 shutdown_connections 2024-04-25T00:18:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.866+0000 7f2fbc800700 1 --2- 172.21.15.42:0/3332749003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb410e880 0x7f2fb4110c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.866+0000 7f2fbc800700 1 --2- 172.21.15.42:0/3332749003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb4076260 0x7f2fb40746c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.866+0000 7f2fbc800700 1 --2- 172.21.15.42:0/3332749003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fb4074c90 0x7f2fb4075110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.866+0000 7f2fbc800700 1 -- 172.21.15.42:0/3332749003 >> 172.21.15.42:0/3332749003 conn(0x7f2fb40fd840 msgr2=0x7f2fb40ffc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.866+0000 7f2fbc800700 1 -- 172.21.15.42:0/3332749003 shutdown_connections 2024-04-25T00:18:32.867 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.866+0000 7f2fbc800700 1 -- 172.21.15.42:0/3332749003 wait complete. 2024-04-25T00:18:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.867+0000 7f2fbc800700 1 Processor -- start 2024-04-25T00:18:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.867+0000 7f2fbc800700 1 -- start start 2024-04-25T00:18:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.867+0000 7f2fbc800700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4074c90 0x7f2fb419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fbc800700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb4076260 0x7f2fb419f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fbc800700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fb410e880 0x7f2fb41a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:32.868 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fbc800700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb41144d0 con 0x7f2fb4076260 2024-04-25T00:18:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fbc800700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4114350 con 0x7f2fb4074c90 2024-04-25T00:18:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fbc800700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb4114650 con 0x7f2fb410e880 2024-04-25T00:18:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fb9400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb4076260 0x7f2fb419f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fb9e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4074c90 0x7f2fb419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fb9400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb4076260 0x7f2fb419f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40452/0 (socket says 172.21.15.42:40452) 2024-04-25T00:18:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fb9400700 1 -- 172.21.15.42:0/1874561955 learned_addr learned my addr 172.21.15.42:0/1874561955 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:32.869 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fba800700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fb410e880 0x7f2fb41a60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fb9400700 1 -- 172.21.15.42:0/1874561955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fb410e880 msgr2=0x7f2fb41a60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fb9400700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fb410e880 0x7f2fb41a60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fb9400700 1 -- 172.21.15.42:0/1874561955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4074c90 msgr2=0x7f2fb419eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fb9400700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4074c90 0x7f2fb419eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fb9400700 1 -- 172.21.15.42:0/1874561955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb41a6750 con 0x7f2fb4076260 2024-04-25T00:18:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.868+0000 7f2fb9e00700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4074c90 0x7f2fb419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.869+0000 7f2fb9400700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb4076260 0x7f2fb419f030 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400e980 tx=0x7f2fa400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.869+0000 7f2faea00700 1 -- 172.21.15.42:0/1874561955 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4004230 con 0x7f2fb4076260 2024-04-25T00:18:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.869+0000 7f2faea00700 1 -- 172.21.15.42:0/1874561955 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fa40043d0 con 0x7f2fb4076260 2024-04-25T00:18:32.870 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.869+0000 7f2fbc800700 1 -- 172.21.15.42:0/1874561955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb41a6a40 con 0x7f2fb4076260 2024-04-25T00:18:32.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.869+0000 7f2faea00700 1 -- 172.21.15.42:0/1874561955 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4003dc0 con 0x7f2fb4076260 2024-04-25T00:18:32.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.869+0000 7f2fbc800700 1 -- 172.21.15.42:0/1874561955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb41a6f80 con 0x7f2fb4076260 2024-04-25T00:18:32.871 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.871+0000 7f2fbc800700 1 -- 172.21.15.42:0/1874561955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb4068f20 con 0x7f2fb4076260 2024-04-25T00:18:32.873 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.873+0000 7f2faea00700 1 -- 172.21.15.42:0/1874561955 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2fa400c810 con 0x7f2fb4076260 2024-04-25T00:18:32.874 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.874+0000 7f2faea00700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f9c0a2ca0 0x7f2f9c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:32.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.875+0000 7f2faea00700 1 -- 172.21.15.42:0/1874561955 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f2fa40cf280 con 0x7f2fb4076260 2024-04-25T00:18:32.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.875+0000 7f2fb9e00700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f9c0a2ca0 0x7f2f9c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:32.875 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.876+0000 7f2fb9e00700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f9c0a2ca0 0x7f2f9c0a5160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800b7b0 tx=0x7f2fa8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:32.879 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:32.879+0000 7f2faea00700 1 -- 172.21.15.42:0/1874561955 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f2fa409c1b0 con 0x7f2fb4076260 2024-04-25T00:18:33.112 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.111+0000 7f2fbc800700 1 -- 172.21.15.42:0/1874561955 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f2fb4062e40 con 0x7f2fb4076260 2024-04-25T00:18:33.114 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.114+0000 7f2faea00700 1 -- 172.21.15.42:0/1874561955 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) ==== 56+0+5255 (secure 0 0 0) 0x7f2fa409bab0 con 0x7f2fb4076260 2024-04-25T00:18:33.115 INFO:teuthology.orchestra.run.smithi042.stdout:epoch 88 2024-04-25T00:18:33.115 INFO:teuthology.orchestra.run.smithi042.stdout:fsid 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:18:33.115 INFO:teuthology.orchestra.run.smithi042.stdout:created 2024-04-25T00:07:16.359664+0000 2024-04-25T00:18:33.115 INFO:teuthology.orchestra.run.smithi042.stdout:modified 2024-04-25T00:17:27.556239+0000 2024-04-25T00:18:33.115 INFO:teuthology.orchestra.run.smithi042.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T00:18:33.115 INFO:teuthology.orchestra.run.smithi042.stdout:crush_version 26 2024-04-25T00:18:33.115 INFO:teuthology.orchestra.run.smithi042.stdout:full_ratio 0.95 2024-04-25T00:18:33.115 INFO:teuthology.orchestra.run.smithi042.stdout:backfillfull_ratio 0.9 2024-04-25T00:18:33.115 INFO:teuthology.orchestra.run.smithi042.stdout:nearfull_ratio 0.85 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:require_min_compat_client luminous 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:min_compat_client luminous 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:require_osd_release squid 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:stretch_mode_enabled false 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:max_osd 12 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6802/2959872296,v1:172.21.15.42:6803/2959872296] [v2:172.21.15.42:6804/2959872296,v1:172.21.15.42:6805/2959872296] exists,up 9e126226-45cb-411a-ac44-64c6f80f8eaa 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6800/2488433173,v1:172.21.15.106:6801/2488433173] [v2:172.21.15.106:6802/2488433173,v1:172.21.15.106:6803/2488433173] exists,up df2777b5-5078-4e63-b3f9-6f20ea649a3d 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/930826162,v1:172.21.15.136:6801/930826162] [v2:172.21.15.136:6802/930826162,v1:172.21.15.136:6803/930826162] exists,up 0f53ac73-5268-4b0f-9256-bf92ba350ce5 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6810/2791262023,v1:172.21.15.42:6811/2791262023] [v2:172.21.15.42:6812/2791262023,v1:172.21.15.42:6813/2791262023] exists,up a8044c10-17a2-4a89-926c-2fe500f36943 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6808/2825344351,v1:172.21.15.106:6809/2825344351] [v2:172.21.15.106:6810/2825344351,v1:172.21.15.106:6811/2825344351] exists,up 61b5a912-f971-4678-98fb-5f95190f8c46 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:osd.5 up in weight 1 up_from 38 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/3211315252,v1:172.21.15.136:6809/3211315252] [v2:172.21.15.136:6810/3211315252,v1:172.21.15.136:6811/3211315252] exists,up 978af6c3-4b15-4a19-bc5b-587c384520ef 2024-04-25T00:18:33.116 INFO:teuthology.orchestra.run.smithi042.stdout:osd.6 up in weight 1 up_from 46 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6818/2564442522,v1:172.21.15.42:6819/2564442522] [v2:172.21.15.42:6820/2564442522,v1:172.21.15.42:6821/2564442522] exists,up 0ac3ab28-29e1-4316-af39-4ca7e9785cf3 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:osd.7 up in weight 1 up_from 51 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6816/178560796,v1:172.21.15.106:6817/178560796] [v2:172.21.15.106:6818/178560796,v1:172.21.15.106:6819/178560796] exists,up c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/3281651567,v1:172.21.15.136:6817/3281651567] [v2:172.21.15.136:6818/3281651567,v1:172.21.15.136:6819/3281651567] exists,up ff7307f4-b84a-4297-941f-544e5e06b1d0 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.42:6826/876648842,v1:172.21.15.42:6827/876648842] [v2:172.21.15.42:6828/876648842,v1:172.21.15.42:6829/876648842] exists,up 179b1772-e33e-4970-9a88-6af3a1eb19dd 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.106:6824/3446161379,v1:172.21.15.106:6825/3446161379] [v2:172.21.15.106:6826/3446161379,v1:172.21.15.106:6827/3446161379] exists,up fc443ab7-2aea-427e-9bfd-d25f4f4dc239 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/3705953158,v1:172.21.15.136:6825/3705953158] [v2:172.21.15.136:6826/3705953158,v1:172.21.15.136:6827/3705953158] exists,up bbdadbeb-aa71-49d8-bffb-4977f2dbaa44 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.8 [2,11] 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 2.a [2,11] 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.0 [5,9] 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.11 [4,8] 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.12 [3,9] 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.1d [4,6] 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:pg_upmap_items 3.3c [4,9] 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/1811796714 expires 2024-04-26T00:08:25.763656+0000 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/3681869279 expires 2024-04-26T00:08:25.763656+0000 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/3681869279 expires 2024-04-26T00:08:25.763656+0000 2024-04-25T00:18:33.117 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/3081484237 expires 2024-04-26T00:08:25.763656+0000 2024-04-25T00:18:33.118 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/4222574490 expires 2024-04-26T00:07:50.514990+0000 2024-04-25T00:18:33.118 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/15970351 expires 2024-04-26T00:07:50.514990+0000 2024-04-25T00:18:33.118 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/1603265964 expires 2024-04-26T00:07:50.514990+0000 2024-04-25T00:18:33.118 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6800/174637222 expires 2024-04-26T00:07:50.514990+0000 2024-04-25T00:18:33.118 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:6801/174637222 expires 2024-04-26T00:07:50.514990+0000 2024-04-25T00:18:33.118 INFO:teuthology.orchestra.run.smithi042.stdout:blocklist 172.21.15.42:0/1839487361 expires 2024-04-26T00:08:25.763656+0000 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 -- 172.21.15.42:0/1874561955 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f9c0a2ca0 msgr2=0x7f2f9c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f9c0a2ca0 0x7f2f9c0a5160 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f2fa800b7b0 tx=0x7f2fa8005fd0 comp rx=0 tx=0).stop 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 -- 172.21.15.42:0/1874561955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb4076260 msgr2=0x7f2fb419f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb4076260 0x7f2fb419f030 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400e980 tx=0x7f2fa400ec90 comp rx=0 tx=0).stop 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 -- 172.21.15.42:0/1874561955 shutdown_connections 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2f9c0a2ca0 0x7f2f9c0a5160 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2fb410e880 0x7f2fb41a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4074c90 0x7f2fb419eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 --2- 172.21.15.42:0/1874561955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2fb4076260 0x7f2fb419f030 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 -- 172.21.15.42:0/1874561955 >> 172.21.15.42:0/1874561955 conn(0x7f2fb40fd840 msgr2=0x7f2fb4101c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 -- 172.21.15.42:0/1874561955 shutdown_connections 2024-04-25T00:18:33.119 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:33.117+0000 7f2fbc800700 1 -- 172.21.15.42:0/1874561955 wait complete. 2024-04-25T00:18:33.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:33 smithi106 ceph-mon[32510]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:18:33.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:33 smithi042 ceph-mon[28130]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:18:33.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:33 smithi136 ceph-mon[32923]: pgmap v381: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:18:33.954 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T00:18:33.967 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:18:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.194+0000 7f0028400640 1 -- 172.21.15.42:0/2322028987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0020100a10 msgr2=0x7f0020100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:34.194 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.194+0000 7f0028400640 1 --2- 172.21.15.42:0/2322028987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0020100a10 0x7f0020100e70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f000c009900 tx=0x7f000c02a480 comp rx=0 tx=0).stop 2024-04-25T00:18:34.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.194+0000 7f0028400640 1 -- 172.21.15.42:0/2322028987 shutdown_connections 2024-04-25T00:18:34.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.194+0000 7f0028400640 1 --2- 172.21.15.42:0/2322028987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00201013b0 0x7f0020101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.194+0000 7f0028400640 1 --2- 172.21.15.42:0/2322028987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0020100a10 0x7f0020100e70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.194+0000 7f0028400640 1 --2- 172.21.15.42:0/2322028987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00201000b0 0x7f00201004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.194+0000 7f0028400640 1 -- 172.21.15.42:0/2322028987 >> 172.21.15.42:0/2322028987 conn(0x7f00200fbc30 msgr2=0x7f00200fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:34.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.194+0000 7f0028400640 1 -- 172.21.15.42:0/2322028987 shutdown_connections 2024-04-25T00:18:34.195 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.194+0000 7f0028400640 1 -- 172.21.15.42:0/2322028987 wait complete. 2024-04-25T00:18:34.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.195+0000 7f0028400640 1 Processor -- start 2024-04-25T00:18:34.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.195+0000 7f0028400640 1 -- start start 2024-04-25T00:18:34.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.195+0000 7f0028400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00201000b0 0x7f0020199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:34.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.195+0000 7f0028400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0020100a10 0x7f002019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:34.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.195+0000 7f0028400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00201013b0 0x7f00201a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:34.197 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.195+0000 7f0028400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f002010f690 con 0x7f0020100a10 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.195+0000 7f0028400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f002010f510 con 0x7f00201000b0 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.195+0000 7f0028400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f002010f810 con 0x7f00201013b0 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0025000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0020100a10 0x7f002019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0025000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0020100a10 0x7f002019a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40470/0 (socket says 172.21.15.42:40470) 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0025000640 1 -- 172.21.15.42:0/1195467807 learned_addr learned my addr 172.21.15.42:0/1195467807 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0025a00640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00201000b0 0x7f0020199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0026400640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00201013b0 0x7f00201a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0025000640 1 -- 172.21.15.42:0/1195467807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00201013b0 msgr2=0x7f00201a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0025000640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00201013b0 0x7f00201a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0025000640 1 -- 172.21.15.42:0/1195467807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00201000b0 msgr2=0x7f0020199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0025000640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00201000b0 0x7f0020199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0025000640 1 -- 172.21.15.42:0/1195467807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00201a1960 con 0x7f0020100a10 2024-04-25T00:18:34.198 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.196+0000 7f0025000640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0020100a10 0x7f002019a280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f000c0314e0 tx=0x7f000c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.197+0000 7f001e000640 1 -- 172.21.15.42:0/1195467807 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c03f030 con 0x7f0020100a10 2024-04-25T00:18:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.197+0000 7f0028400640 1 -- 172.21.15.42:0/1195467807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00201a1bf0 con 0x7f0020100a10 2024-04-25T00:18:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.197+0000 7f001e000640 1 -- 172.21.15.42:0/1195467807 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000c003ce0 con 0x7f0020100a10 2024-04-25T00:18:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.197+0000 7f001e000640 1 -- 172.21.15.42:0/1195467807 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c035070 con 0x7f0020100a10 2024-04-25T00:18:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.197+0000 7f0028400640 1 -- 172.21.15.42:0/1195467807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00201a20e0 con 0x7f0020100a10 2024-04-25T00:18:34.199 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.198+0000 7f0028400640 1 -- 172.21.15.42:0/1195467807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f002010afe0 con 0x7f0020100a10 2024-04-25T00:18:34.200 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.200+0000 7f001e000640 1 -- 172.21.15.42:0/1195467807 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f000c042030 con 0x7f0020100a10 2024-04-25T00:18:34.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.202+0000 7f001e000640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00040a2b90 0x7f00040a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:34.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.202+0000 7f001e000640 1 -- 172.21.15.42:0/1195467807 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f000c0ef7f0 con 0x7f0020100a10 2024-04-25T00:18:34.202 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.202+0000 7f0025a00640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00040a2b90 0x7f00040a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:34.203 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.203+0000 7f0025a00640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00040a2b90 0x7f00040a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0010005fd0 tx=0x7f0010009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:34.204 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.204+0000 7f001e000640 1 -- 172.21.15.42:0/1195467807 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f000c0bc0d0 con 0x7f0020100a10 2024-04-25T00:18:34.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:34 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1874561955' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:18:34.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:34 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1874561955' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:18:34.437 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.436+0000 7f0028400640 1 -- 172.21.15.42:0/1195467807 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f002002d330 con 0x7f0020100a10 2024-04-25T00:18:34.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.438+0000 7f001e000640 1 -- 172.21.15.42:0/1195467807 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22547 (secure 0 0 0) 0x7f000c0c03d0 con 0x7f0020100a10 2024-04-25T00:18:34.439 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:34.439 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":88,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:17:27.556239+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:34.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 -- 172.21.15.42:0/1195467807 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00040a2b90 msgr2=0x7f00040a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:34.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00040a2b90 0x7f00040a5050 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0010005fd0 tx=0x7f0010009500 comp rx=0 tx=0).stop 2024-04-25T00:18:34.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 -- 172.21.15.42:0/1195467807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0020100a10 msgr2=0x7f002019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:34.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0020100a10 0x7f002019a280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f000c0314e0 tx=0x7f000c032ce0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 -- 172.21.15.42:0/1195467807 shutdown_connections 2024-04-25T00:18:34.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f00040a2b90 0x7f00040a5050 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00201013b0 0x7f00201a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0020100a10 0x7f002019a280 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 --2- 172.21.15.42:0/1195467807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00201000b0 0x7f0020199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 -- 172.21.15.42:0/1195467807 >> 172.21.15.42:0/1195467807 conn(0x7f00200fbc30 msgr2=0x7f00200fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:34.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 -- 172.21.15.42:0/1195467807 shutdown_connections 2024-04-25T00:18:34.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.442+0000 7f0028400640 1 -- 172.21.15.42:0/1195467807 wait complete. 2024-04-25T00:18:34.466 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:11:14.472869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:16:18.168243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:16:19.289585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:18:34.466 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:18:34.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:34 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1874561955' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T00:18:34.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.680+0000 7f4964a00640 1 -- 172.21.15.42:0/3175954441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4960109830 msgr2=0x7f49601100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:34.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.680+0000 7f4964a00640 1 --2- 172.21.15.42:0/3175954441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4960109830 0x7f49601100c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4958009860 tx=0x7f495802a340 comp rx=0 tx=0).stop 2024-04-25T00:18:34.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.680+0000 7f4964a00640 1 -- 172.21.15.42:0/3175954441 shutdown_connections 2024-04-25T00:18:34.680 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.680+0000 7f4964a00640 1 --2- 172.21.15.42:0/3175954441 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4960109830 0x7f49601100c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.680+0000 7f4964a00640 1 --2- 172.21.15.42:0/3175954441 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4960108e70 0x7f49601092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.680+0000 7f4964a00640 1 --2- 172.21.15.42:0/3175954441 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4960107c80 0x7f49601080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.680+0000 7f4964a00640 1 -- 172.21.15.42:0/3175954441 >> 172.21.15.42:0/3175954441 conn(0x7f4960076c10 msgr2=0x7f4960079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.680+0000 7f4964a00640 1 -- 172.21.15.42:0/3175954441 shutdown_connections 2024-04-25T00:18:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.680+0000 7f4964a00640 1 -- 172.21.15.42:0/3175954441 wait complete. 2024-04-25T00:18:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.681+0000 7f4964a00640 1 Processor -- start 2024-04-25T00:18:34.681 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.681+0000 7f4964a00640 1 -- start start 2024-04-25T00:18:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f4964a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4960107c80 0x7f496019dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f4964a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4960108e70 0x7f496019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f4964a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4960109830 0x7f49601a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f4964a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49601138e0 con 0x7f4960107c80 2024-04-25T00:18:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f4964a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4960113760 con 0x7f4960108e70 2024-04-25T00:18:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f4964a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4960113a60 con 0x7f4960109830 2024-04-25T00:18:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f495f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4960107c80 0x7f496019dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:34.683 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f495f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4960107c80 0x7f496019dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40486/0 (socket says 172.21.15.42:40486) 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f495f400640 1 -- 172.21.15.42:0/337887242 learned_addr learned my addr 172.21.15.42:0/337887242 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f495ea00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4960108e70 0x7f496019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.682+0000 7f495fe00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4960109830 0x7f49601a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f495ea00640 1 -- 172.21.15.42:0/337887242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4960109830 msgr2=0x7f49601a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f495ea00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4960109830 0x7f49601a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f495ea00640 1 -- 172.21.15.42:0/337887242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4960107c80 msgr2=0x7f496019dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f495ea00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4960107c80 0x7f496019dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f495ea00640 1 -- 172.21.15.42:0/337887242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49601a5ae0 con 0x7f4960108e70 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f495f400640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4960107c80 0x7f496019dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f495fe00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4960109830 0x7f49601a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:34.684 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f495ea00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4960108e70 0x7f496019e400 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f494800e9f0 tx=0x7f494800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:34.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f4957e00640 1 -- 172.21.15.42:0/337887242 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494800ccc0 con 0x7f4960108e70 2024-04-25T00:18:34.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f4957e00640 1 -- 172.21.15.42:0/337887242 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4948010910 con 0x7f4960108e70 2024-04-25T00:18:34.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f4957e00640 1 -- 172.21.15.42:0/337887242 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948018960 con 0x7f4960108e70 2024-04-25T00:18:34.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.683+0000 7f4964a00640 1 -- 172.21.15.42:0/337887242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49601a5dd0 con 0x7f4960108e70 2024-04-25T00:18:34.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.684+0000 7f4964a00640 1 -- 172.21.15.42:0/337887242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49601a6290 con 0x7f4960108e70 2024-04-25T00:18:34.685 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.685+0000 7f4964a00640 1 -- 172.21.15.42:0/337887242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f496010ef40 con 0x7f4960108e70 2024-04-25T00:18:34.687 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.687+0000 7f4957e00640 1 -- 172.21.15.42:0/337887242 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4948010ac0 con 0x7f4960108e70 2024-04-25T00:18:34.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.688+0000 7f4957e00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49440a2b90 0x7f49440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:34.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.688+0000 7f4957e00640 1 -- 172.21.15.42:0/337887242 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f49480cf760 con 0x7f4960108e70 2024-04-25T00:18:34.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.689+0000 7f495f400640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49440a2b90 0x7f49440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:34.689 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.689+0000 7f495f400640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49440a2b90 0x7f49440a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4950005c40 tx=0x7f49500095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:34.691 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.691+0000 7f4957e00640 1 -- 172.21.15.42:0/337887242 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f494809c040 con 0x7f4960108e70 2024-04-25T00:18:34.896 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.896+0000 7f4964a00640 1 -- 172.21.15.42:0/337887242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f496002d210 con 0x7f4960108e70 2024-04-25T00:18:34.897 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.897+0000 7f4957e00640 1 -- 172.21.15.42:0/337887242 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7f49480a0340 con 0x7f4960108e70 2024-04-25T00:18:34.897 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-25T00:18:34.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 -- 172.21.15.42:0/337887242 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49440a2b90 msgr2=0x7f49440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:34.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49440a2b90 0x7f49440a5050 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4950005c40 tx=0x7f49500095c0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.900 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 -- 172.21.15.42:0/337887242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4960108e70 msgr2=0x7f496019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:34.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4960108e70 0x7f496019e400 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f494800e9f0 tx=0x7f494800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 -- 172.21.15.42:0/337887242 shutdown_connections 2024-04-25T00:18:34.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49440a2b90 0x7f49440a5050 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4960109830 0x7f49601a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4960108e70 0x7f496019e400 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 --2- 172.21.15.42:0/337887242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4960107c80 0x7f496019dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:34.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 -- 172.21.15.42:0/337887242 >> 172.21.15.42:0/337887242 conn(0x7f4960076c10 msgr2=0x7f496010c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:34.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 -- 172.21.15.42:0/337887242 shutdown_connections 2024-04-25T00:18:34.901 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:34.900+0000 7f4964a00640 1 -- 172.21.15.42:0/337887242 wait complete. 2024-04-25T00:18:34.920 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:18:35.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.149+0000 7f70c7e00640 1 -- 172.21.15.42:0/3064729557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c81049f0 msgr2=0x7f70c8106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:35.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.149+0000 7f70c7e00640 1 --2- 172.21.15.42:0/3064729557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c81049f0 0x7f70c8106de0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009860 tx=0x7f70b402a340 comp rx=0 tx=0).stop 2024-04-25T00:18:35.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.149+0000 7f70c7e00640 1 -- 172.21.15.42:0/3064729557 shutdown_connections 2024-04-25T00:18:35.149 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.149+0000 7f70c7e00640 1 --2- 172.21.15.42:0/3064729557 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c8107320 0x7f70c8109770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.149+0000 7f70c7e00640 1 --2- 172.21.15.42:0/3064729557 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c81049f0 0x7f70c8106de0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.149+0000 7f70c7e00640 1 --2- 172.21.15.42:0/3064729557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c81020c0 0x7f70c81044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.149+0000 7f70c7e00640 1 -- 172.21.15.42:0/3064729557 >> 172.21.15.42:0/3064729557 conn(0x7f70c80fbaf0 msgr2=0x7f70c80fdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:35.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.149+0000 7f70c7e00640 1 -- 172.21.15.42:0/3064729557 shutdown_connections 2024-04-25T00:18:35.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.149+0000 7f70c7e00640 1 -- 172.21.15.42:0/3064729557 wait complete. 2024-04-25T00:18:35.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.150+0000 7f70c7e00640 1 Processor -- start 2024-04-25T00:18:35.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.150+0000 7f70c7e00640 1 -- start start 2024-04-25T00:18:35.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.150+0000 7f70c7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c81020c0 0x7f70c8199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:35.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.150+0000 7f70c7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c81049f0 0x7f70c819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:35.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.150+0000 7f70c7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c8107320 0x7f70c81a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:35.151 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.150+0000 7f70c7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f70c810d2a0 con 0x7f70c8107320 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.150+0000 7f70c7e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f70c810d120 con 0x7f70c81049f0 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.150+0000 7f70c7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70c810d420 con 0x7f70c81020c0 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c81020c0 0x7f70c8199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c81020c0 0x7f70c8199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:51602/0 (socket says 172.21.15.42:51602) 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c6a00640 1 -- 172.21.15.42:0/2373193528 learned_addr learned my addr 172.21.15.42:0/2373193528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c7400640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c8107320 0x7f70c81a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c6000640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c81049f0 0x7f70c819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c6a00640 1 -- 172.21.15.42:0/2373193528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c81049f0 msgr2=0x7f70c819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c6a00640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c81049f0 0x7f70c819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c6a00640 1 -- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c8107320 msgr2=0x7f70c81a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c6a00640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c8107320 0x7f70c81a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c6a00640 1 -- 172.21.15.42:0/2373193528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c81a17c0 con 0x7f70c81020c0 2024-04-25T00:18:35.152 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c6000640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c81049f0 0x7f70c819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:35.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.151+0000 7f70c7400640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c8107320 0x7f70c81a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:35.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.152+0000 7f70c6a00640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c81020c0 0x7f70c8199ba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f70b800e9c0 tx=0x7f70b800edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:35.153 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.152+0000 7f70cca00640 1 -- 172.21.15.42:0/2373193528 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b800cd80 con 0x7f70c81020c0 2024-04-25T00:18:35.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.152+0000 7f70cca00640 1 -- 172.21.15.42:0/2373193528 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70b80035f0 con 0x7f70c81020c0 2024-04-25T00:18:35.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.152+0000 7f70cca00640 1 -- 172.21.15.42:0/2373193528 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b8010670 con 0x7f70c81020c0 2024-04-25T00:18:35.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.152+0000 7f70c7e00640 1 -- 172.21.15.42:0/2373193528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c81a1ab0 con 0x7f70c81020c0 2024-04-25T00:18:35.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.152+0000 7f70c7e00640 1 -- 172.21.15.42:0/2373193528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c81a1fd0 con 0x7f70c81020c0 2024-04-25T00:18:35.154 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.153+0000 7f70c7e00640 1 -- 172.21.15.42:0/2373193528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c80688d0 con 0x7f70c81020c0 2024-04-25T00:18:35.155 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.155+0000 7f70cca00640 1 -- 172.21.15.42:0/2373193528 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f70b80037a0 con 0x7f70c81020c0 2024-04-25T00:18:35.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.156+0000 7f70cca00640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f70ac0a2b90 0x7f70ac0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:35.156 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.156+0000 7f70cca00640 1 -- 172.21.15.42:0/2373193528 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f70b80cf290 con 0x7f70c81020c0 2024-04-25T00:18:35.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.157+0000 7f70c6000640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f70ac0a2b90 0x7f70ac0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:35.157 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.157+0000 7f70c6000640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f70ac0a2b90 0x7f70ac0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f70b4030410 tx=0x7f70b4030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:35.161 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.161+0000 7f70cca00640 1 -- 172.21.15.42:0/2373193528 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f70b809c1c0 con 0x7f70c81020c0 2024-04-25T00:18:35.372 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.372+0000 7f70c7e00640 1 -- 172.21.15.42:0/2373193528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f70c8062340 con 0x7f70c81020c0 2024-04-25T00:18:35.373 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.373+0000 7f70cca00640 1 -- 172.21.15.42:0/2373193528 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f70b809bac0 con 0x7f70c81020c0 2024-04-25T00:18:35.373 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-25T00:18:35.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 -- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f70ac0a2b90 msgr2=0x7f70ac0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:35.376 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f70ac0a2b90 0x7f70ac0a5050 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f70b4030410 tx=0x7f70b4030550 comp rx=0 tx=0).stop 2024-04-25T00:18:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 -- 172.21.15.42:0/2373193528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c81020c0 msgr2=0x7f70c8199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c81020c0 0x7f70c8199ba0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f70b800e9c0 tx=0x7f70b800edc0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 -- 172.21.15.42:0/2373193528 shutdown_connections 2024-04-25T00:18:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f70ac0a2b90 0x7f70ac0a5050 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f70c8107320 0x7f70c81a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70c81049f0 0x7f70c819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 --2- 172.21.15.42:0/2373193528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70c81020c0 0x7f70c8199ba0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.377 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 -- 172.21.15.42:0/2373193528 >> 172.21.15.42:0/2373193528 conn(0x7f70c80fbaf0 msgr2=0x7f70c81017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:35.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:35 smithi042 ceph-mon[28130]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:35.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:35 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1195467807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:35.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:35 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/337887242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:35.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 -- 172.21.15.42:0/2373193528 shutdown_connections 2024-04-25T00:18:35.378 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.376+0000 7f70c7e00640 1 -- 172.21.15.42:0/2373193528 wait complete. 2024-04-25T00:18:35.396 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:18:35.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:35 smithi136 ceph-mon[32923]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:35.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:35 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1195467807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:35.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:35 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/337887242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:35.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:35 smithi106 ceph-mon[32510]: pgmap v382: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:35.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:35 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1195467807' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:35.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:35 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/337887242' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:35.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.611+0000 7f6308000640 1 -- 172.21.15.42:0/3933825144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63001039a0 msgr2=0x7f6300103dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:35.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.611+0000 7f6308000640 1 --2- 172.21.15.42:0/3933825144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63001039a0 0x7f6300103dc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009900 tx=0x7f62f402a410 comp rx=0 tx=0).stop 2024-04-25T00:18:35.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.611+0000 7f6308000640 1 -- 172.21.15.42:0/3933825144 shutdown_connections 2024-04-25T00:18:35.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.611+0000 7f6308000640 1 --2- 172.21.15.42:0/3933825144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6300105550 0x7f630010bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.611+0000 7f6308000640 1 --2- 172.21.15.42:0/3933825144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6300104b90 0x7f6300105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.611+0000 7f6308000640 1 --2- 172.21.15.42:0/3933825144 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63001039a0 0x7f6300103dc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.611+0000 7f6308000640 1 -- 172.21.15.42:0/3933825144 >> 172.21.15.42:0/3933825144 conn(0x7f63000fef30 msgr2=0x7f6300101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:35.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.611+0000 7f6308000640 1 -- 172.21.15.42:0/3933825144 shutdown_connections 2024-04-25T00:18:35.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.611+0000 7f6308000640 1 -- 172.21.15.42:0/3933825144 wait complete. 2024-04-25T00:18:35.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.612+0000 7f6308000640 1 Processor -- start 2024-04-25T00:18:35.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.612+0000 7f6308000640 1 -- start start 2024-04-25T00:18:35.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.612+0000 7f6308000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63001039a0 0x7f630019e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:35.613 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.612+0000 7f6308000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6300104b90 0x7f630019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.612+0000 7f6308000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6300105550 0x7f63001a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.612+0000 7f6308000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f630010f670 con 0x7f6300104b90 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.612+0000 7f6308000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f630010f4f0 con 0x7f63001039a0 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.612+0000 7f6308000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f630010f7f0 con 0x7f6300105550 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.613+0000 7f6305600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63001039a0 0x7f630019e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.613+0000 7f6305600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63001039a0 0x7f630019e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51102/0 (socket says 172.21.15.42:51102) 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.613+0000 7f6305600640 1 -- 172.21.15.42:0/82038014 learned_addr learned my addr 172.21.15.42:0/82038014 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.613+0000 7f6304c00640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6300104b90 0x7f630019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.613+0000 7f6306000640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6300105550 0x7f63001a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.613+0000 7f6305600640 1 -- 172.21.15.42:0/82038014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6300105550 msgr2=0x7f63001a5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:35.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.613+0000 7f6305600640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6300105550 0x7f63001a5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.613+0000 7f6305600640 1 -- 172.21.15.42:0/82038014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6300104b90 msgr2=0x7f630019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:35.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.613+0000 7f6305600640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6300104b90 0x7f630019e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.618 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.613+0000 7f6305600640 1 -- 172.21.15.42:0/82038014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63001a5d00 con 0x7f63001039a0 2024-04-25T00:18:35.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.614+0000 7f6305600640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63001039a0 0x7f630019e0e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f62f4031440 tx=0x7f62f4032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:35.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.614+0000 7f62fe000640 1 -- 172.21.15.42:0/82038014 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f403f030 con 0x7f63001039a0 2024-04-25T00:18:35.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.614+0000 7f62fe000640 1 -- 172.21.15.42:0/82038014 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62f4003ce0 con 0x7f63001039a0 2024-04-25T00:18:35.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.614+0000 7f6308000640 1 -- 172.21.15.42:0/82038014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63001a5f90 con 0x7f63001039a0 2024-04-25T00:18:35.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.614+0000 7f62fe000640 1 -- 172.21.15.42:0/82038014 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62f4035070 con 0x7f63001039a0 2024-04-25T00:18:35.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.614+0000 7f6308000640 1 -- 172.21.15.42:0/82038014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63001a63a0 con 0x7f63001039a0 2024-04-25T00:18:35.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.615+0000 7f6308000640 1 -- 172.21.15.42:0/82038014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63001a65d0 con 0x7f63001039a0 2024-04-25T00:18:35.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.617+0000 7f62fe000640 1 -- 172.21.15.42:0/82038014 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f62f4042030 con 0x7f63001039a0 2024-04-25T00:18:35.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.619+0000 7f62fe000640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62e80a2b90 0x7f62e80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:35.620 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.619+0000 7f62fe000640 1 -- 172.21.15.42:0/82038014 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f62f40ef9a0 con 0x7f63001039a0 2024-04-25T00:18:35.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.619+0000 7f6304c00640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62e80a2b90 0x7f62e80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:35.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.620+0000 7f6304c00640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62e80a2b90 0x7f62e80a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f630019f5f0 tx=0x7f62ec00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:35.621 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.621+0000 7f62fe000640 1 -- 172.21.15.42:0/82038014 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f62f40bc280 con 0x7f63001039a0 2024-04-25T00:18:35.826 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.826+0000 7f6308000640 1 -- 172.21.15.42:0/82038014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f63001a67a0 con 0x7f63001039a0 2024-04-25T00:18:35.827 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.826+0000 7f62fe000640 1 -- 172.21.15.42:0/82038014 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f62f40c0580 con 0x7f63001039a0 2024-04-25T00:18:35.827 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.829+0000 7f6308000640 1 -- 172.21.15.42:0/82038014 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62e80a2b90 msgr2=0x7f62e80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.829+0000 7f6308000640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62e80a2b90 0x7f62e80a5050 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f630019f5f0 tx=0x7f62ec00b390 comp rx=0 tx=0).stop 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.829+0000 7f6308000640 1 -- 172.21.15.42:0/82038014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63001039a0 msgr2=0x7f630019e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.829+0000 7f6308000640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63001039a0 0x7f630019e0e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f62f4031440 tx=0x7f62f4032ce0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.829+0000 7f6308000640 1 -- 172.21.15.42:0/82038014 shutdown_connections 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.829+0000 7f6308000640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62e80a2b90 0x7f62e80a5050 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.829+0000 7f6308000640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6300105550 0x7f63001a5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.829+0000 7f6308000640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6300104b90 0x7f630019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.829+0000 7f6308000640 1 --2- 172.21.15.42:0/82038014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63001039a0 0x7f630019e0e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.829+0000 7f6308000640 1 -- 172.21.15.42:0/82038014 >> 172.21.15.42:0/82038014 conn(0x7f63000fef30 msgr2=0x7f6300077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:35.830 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.830+0000 7f6308000640 1 -- 172.21.15.42:0/82038014 shutdown_connections 2024-04-25T00:18:35.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:35.830+0000 7f6308000640 1 -- 172.21.15.42:0/82038014 wait complete. 2024-04-25T00:18:35.852 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:18:36.064 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.064+0000 7fd2fea00640 1 -- 172.21.15.42:0/1715594313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2f8076100 msgr2=0x7fd2f8076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.064+0000 7fd2fea00640 1 --2- 172.21.15.42:0/1715594313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2f8076100 0x7fd2f8076520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0009ca0 tx=0x7fd2f002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.064+0000 7fd2fea00640 1 -- 172.21.15.42:0/1715594313 shutdown_connections 2024-04-25T00:18:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.064+0000 7fd2fea00640 1 --2- 172.21.15.42:0/1715594313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2f8077ce0 0x7fd2f8078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.064+0000 7fd2fea00640 1 --2- 172.21.15.42:0/1715594313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2f8077340 0x7fd2f80777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.064+0000 7fd2fea00640 1 --2- 172.21.15.42:0/1715594313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2f8076100 0x7fd2f8076520 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.064+0000 7fd2fea00640 1 -- 172.21.15.42:0/1715594313 >> 172.21.15.42:0/1715594313 conn(0x7fd2f80fef30 msgr2=0x7fd2f8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.064+0000 7fd2fea00640 1 -- 172.21.15.42:0/1715594313 shutdown_connections 2024-04-25T00:18:36.065 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.064+0000 7fd2fea00640 1 -- 172.21.15.42:0/1715594313 wait complete. 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.065+0000 7fd2fea00640 1 Processor -- start 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.065+0000 7fd2fea00640 1 -- start start 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2fea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2f8076100 0x7fd2f819e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2fea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2f8077340 0x7fd2f819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2fea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2f8077ce0 0x7fd2f81a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2fea00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f8113a30 con 0x7fd2f8077340 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2fea00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81138b0 con 0x7fd2f8076100 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2fea00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f8113bb0 con 0x7fd2f8077ce0 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2f7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2f8076100 0x7fd2f819e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2f7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2f8077340 0x7fd2f819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2f7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2f8076100 0x7fd2f819e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51114/0 (socket says 172.21.15.42:51114) 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2f7e00640 1 -- 172.21.15.42:0/2306995438 learned_addr learned my addr 172.21.15.42:0/2306995438 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2fca00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2f8077ce0 0x7fd2f81a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:36.067 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2f7e00640 1 -- 172.21.15.42:0/2306995438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2f8077ce0 msgr2=0x7fd2f81a5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:36.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2f7e00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2f8077ce0 0x7fd2f81a5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2f7e00640 1 -- 172.21.15.42:0/2306995438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2f8077340 msgr2=0x7fd2f819e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:36.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2f7e00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2f8077340 0x7fd2f819e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.066+0000 7fd2f7e00640 1 -- 172.21.15.42:0/2306995438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f81a5cc0 con 0x7fd2f8076100 2024-04-25T00:18:36.068 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.067+0000 7fd2f7e00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2f8076100 0x7fd2f819e0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0000f80 tx=0x7fd2f00319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:36.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.067+0000 7fd2f4c00640 1 -- 172.21.15.42:0/2306995438 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0003b70 con 0x7fd2f8076100 2024-04-25T00:18:36.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.067+0000 7fd2f4c00640 1 -- 172.21.15.42:0/2306995438 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2f0003d10 con 0x7fd2f8076100 2024-04-25T00:18:36.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.067+0000 7fd2fea00640 1 -- 172.21.15.42:0/2306995438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f81a5f50 con 0x7fd2f8076100 2024-04-25T00:18:36.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.067+0000 7fd2f4c00640 1 -- 172.21.15.42:0/2306995438 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f002fc00 con 0x7fd2f8076100 2024-04-25T00:18:36.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.067+0000 7fd2fea00640 1 -- 172.21.15.42:0/2306995438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f81a63b0 con 0x7fd2f8076100 2024-04-25T00:18:36.069 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.068+0000 7fd2fea00640 1 -- 172.21.15.42:0/2306995438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f81104a0 con 0x7fd2f8076100 2024-04-25T00:18:36.070 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.070+0000 7fd2f4c00640 1 -- 172.21.15.42:0/2306995438 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd2f0004040 con 0x7fd2f8076100 2024-04-25T00:18:36.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.072+0000 7fd2f4c00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2dc0a2b90 0x7fd2dc0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:36.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.072+0000 7fd2f4c00640 1 -- 172.21.15.42:0/2306995438 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd2f00f0530 con 0x7fd2f8076100 2024-04-25T00:18:36.072 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.072+0000 7fd2f7400640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2dc0a2b90 0x7fd2dc0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:36.073 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.073+0000 7fd2f7400640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2dc0a2b90 0x7fd2dc0a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00095f0 tx=0x7fd2e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:36.076 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.076+0000 7fd2f4c00640 1 -- 172.21.15.42:0/2306995438 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd2f00bd460 con 0x7fd2f8076100 2024-04-25T00:18:36.330 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.330+0000 7fd2fea00640 1 -- 172.21.15.42:0/2306995438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2f802d170 con 0x7fd2f8076100 2024-04-25T00:18:36.331 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.331+0000 7fd2f4c00640 1 -- 172.21.15.42:0/2306995438 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11868 (secure 0 0 0) 0x7fd2f00bcd60 con 0x7fd2f8076100 2024-04-25T00:18:36.332 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:36.332 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:16:19.771762+0000","modified":"2024-04-25T00:16:25.317872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24568,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24568":{"gid":24568,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6841/4068466248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4068466248},{"type":"v1","addr":"172.21.15.42:6841","nonce":4068466248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3349564106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3349564106},{"type":"v1","addr":"172.21.15.106:6839","nonce":3349564106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24517,24568,24569]},"id":1}]} 2024-04-25T00:18:36.332 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 -- 172.21.15.42:0/2306995438 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2dc0a2b90 msgr2=0x7fd2dc0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2dc0a2b90 0x7fd2dc0a5050 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd2e00095f0 tx=0x7fd2e0005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 -- 172.21.15.42:0/2306995438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2f8076100 msgr2=0x7fd2f819e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2f8076100 0x7fd2f819e0a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd2f0000f80 tx=0x7fd2f00319f0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 -- 172.21.15.42:0/2306995438 shutdown_connections 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd2dc0a2b90 0x7fd2dc0a5050 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd2f8077ce0 0x7fd2f81a5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd2f8077340 0x7fd2f819e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 --2- 172.21.15.42:0/2306995438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd2f8076100 0x7fd2f819e0a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 -- 172.21.15.42:0/2306995438 >> 172.21.15.42:0/2306995438 conn(0x7fd2f80fef30 msgr2=0x7fd2f81005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 -- 172.21.15.42:0/2306995438 shutdown_connections 2024-04-25T00:18:36.335 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.334+0000 7fd2fea00640 1 -- 172.21.15.42:0/2306995438 wait complete. 2024-04-25T00:18:36.356 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:18:36.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:36 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2373193528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:36.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:36 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/82038014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:36.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:36 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2373193528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:36.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:36 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/82038014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:36.570 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.570+0000 7fc443200640 1 -- 172.21.15.42:0/1502401477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc43c104970 msgr2=0x7fc43c106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.570+0000 7fc443200640 1 --2- 172.21.15.42:0/1502401477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc43c104970 0x7fc43c106d60 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc42c009900 tx=0x7fc42c02a410 comp rx=0 tx=0).stop 2024-04-25T00:18:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.570+0000 7fc443200640 1 -- 172.21.15.42:0/1502401477 shutdown_connections 2024-04-25T00:18:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.570+0000 7fc443200640 1 --2- 172.21.15.42:0/1502401477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc43c1072a0 0x7fc43c1096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.570+0000 7fc443200640 1 --2- 172.21.15.42:0/1502401477 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc43c104970 0x7fc43c106d60 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.570+0000 7fc443200640 1 --2- 172.21.15.42:0/1502401477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc43c102040 0x7fc43c104430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.570+0000 7fc443200640 1 -- 172.21.15.42:0/1502401477 >> 172.21.15.42:0/1502401477 conn(0x7fc43c0fba10 msgr2=0x7fc43c0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.570+0000 7fc443200640 1 -- 172.21.15.42:0/1502401477 shutdown_connections 2024-04-25T00:18:36.571 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.570+0000 7fc443200640 1 -- 172.21.15.42:0/1502401477 wait complete. 2024-04-25T00:18:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.571+0000 7fc443200640 1 Processor -- start 2024-04-25T00:18:36.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.571+0000 7fc443200640 1 -- start start 2024-04-25T00:18:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.571+0000 7fc443200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc43c102040 0x7fc43c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc443200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc43c104970 0x7fc43c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc443200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc43c1072a0 0x7fc43c1a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc443200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c10d290 con 0x7fc43c102040 2024-04-25T00:18:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc443200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c10d110 con 0x7fc43c1072a0 2024-04-25T00:18:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc443200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc43c10d410 con 0x7fc43c104970 2024-04-25T00:18:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc441e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc43c102040 0x7fc43c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc442800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc43c1072a0 0x7fc43c1a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:36.574 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc441e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc43c102040 0x7fc43c199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:40560/0 (socket says 172.21.15.42:40560) 2024-04-25T00:18:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc441e00640 1 -- 172.21.15.42:0/1633803356 learned_addr learned my addr 172.21.15.42:0/1633803356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc441400640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc43c104970 0x7fc43c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc441e00640 1 -- 172.21.15.42:0/1633803356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc43c104970 msgr2=0x7fc43c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc441e00640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc43c104970 0x7fc43c19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc441e00640 1 -- 172.21.15.42:0/1633803356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc43c1072a0 msgr2=0x7fc43c1a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc441e00640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc43c1072a0 0x7fc43c1a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.572+0000 7fc441e00640 1 -- 172.21.15.42:0/1633803356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43c1a1760 con 0x7fc43c102040 2024-04-25T00:18:36.575 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.573+0000 7fc441e00640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc43c102040 0x7fc43c199b40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc43000e9f0 tx=0x7fc43000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.573+0000 7fc436a00640 1 -- 172.21.15.42:0/1633803356 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43000cd80 con 0x7fc43c102040 2024-04-25T00:18:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.573+0000 7fc436a00640 1 -- 172.21.15.42:0/1633803356 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc430005500 con 0x7fc43c102040 2024-04-25T00:18:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.573+0000 7fc443200640 1 -- 172.21.15.42:0/1633803356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc43c1a1a50 con 0x7fc43c102040 2024-04-25T00:18:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.573+0000 7fc436a00640 1 -- 172.21.15.42:0/1633803356 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430010690 con 0x7fc43c102040 2024-04-25T00:18:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.573+0000 7fc443200640 1 -- 172.21.15.42:0/1633803356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc43c1a1f70 con 0x7fc43c102040 2024-04-25T00:18:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.574+0000 7fc443200640 1 -- 172.21.15.42:0/1633803356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc43c108df0 con 0x7fc43c102040 2024-04-25T00:18:36.576 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.576+0000 7fc436a00640 1 -- 172.21.15.42:0/1633803356 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc4300046c0 con 0x7fc43c102040 2024-04-25T00:18:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.578+0000 7fc436a00640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc4240a2c10 0x7fc4240a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.578+0000 7fc436a00640 1 -- 172.21.15.42:0/1633803356 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fc4300cf3f0 con 0x7fc43c102040 2024-04-25T00:18:36.578 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.578+0000 7fc441400640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc4240a2c10 0x7fc4240a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:36.579 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.579+0000 7fc441400640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc4240a2c10 0x7fc4240a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc42c02aa00 tx=0x7fc42c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:36.582 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.582+0000 7fc436a00640 1 -- 172.21.15.42:0/1633803356 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fc43009c320 con 0x7fc43c102040 2024-04-25T00:18:36.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:36 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2373193528' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:36.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:36 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/82038014' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:36.788 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.787+0000 7fc443200640 1 -- 172.21.15.42:0/1633803356 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc43c062370 con 0x7fc43c102040 2024-04-25T00:18:36.789 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.789+0000 7fc436a00640 1 -- 172.21.15.42:0/1633803356 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22547 (secure 0 0 0) 0x7fc430086070 con 0x7fc43c102040 2024-04-25T00:18:36.790 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:36.790 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":88,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:17:27.556239+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:36.792 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.792+0000 7fc443200640 1 -- 172.21.15.42:0/1633803356 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc4240a2c10 msgr2=0x7fc4240a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.792+0000 7fc443200640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc4240a2c10 0x7fc4240a50d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc42c02aa00 tx=0x7fc42c0315b0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.792+0000 7fc443200640 1 -- 172.21.15.42:0/1633803356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc43c102040 msgr2=0x7fc43c199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.792+0000 7fc443200640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc43c102040 0x7fc43c199b40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc43000e9f0 tx=0x7fc43000edf0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.792+0000 7fc443200640 1 -- 172.21.15.42:0/1633803356 shutdown_connections 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.792+0000 7fc443200640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc4240a2c10 0x7fc4240a50d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.792+0000 7fc443200640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc43c1072a0 0x7fc43c1a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.792+0000 7fc443200640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc43c104970 0x7fc43c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.792+0000 7fc443200640 1 --2- 172.21.15.42:0/1633803356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc43c102040 0x7fc43c199b40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.792+0000 7fc443200640 1 -- 172.21.15.42:0/1633803356 >> 172.21.15.42:0/1633803356 conn(0x7fc43c0fba10 msgr2=0x7fc43c101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.793+0000 7fc443200640 1 -- 172.21.15.42:0/1633803356 shutdown_connections 2024-04-25T00:18:36.793 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:36.793+0000 7fc443200640 1 -- 172.21.15.42:0/1633803356 wait complete. 2024-04-25T00:18:36.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:11:14.472869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:16:18.168243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:16:19.289585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:18:36.812 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:18:37.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.076+0000 7fd16d600640 1 -- 172.21.15.42:0/1681617027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103880 msgr2=0x7fd168103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.076+0000 7fd16d600640 1 --2- 172.21.15.42:0/1681617027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103880 0x7fd168103ca0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fd158009900 tx=0x7fd15802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.076+0000 7fd16d600640 1 -- 172.21.15.42:0/1681617027 shutdown_connections 2024-04-25T00:18:37.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.076+0000 7fd16d600640 1 --2- 172.21.15.42:0/1681617027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168105430 0x7fd16810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.076+0000 7fd16d600640 1 --2- 172.21.15.42:0/1681617027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd168104a70 0x7fd168104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.077 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.076+0000 7fd16d600640 1 --2- 172.21.15.42:0/1681617027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103880 0x7fd168103ca0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.076+0000 7fd16d600640 1 -- 172.21.15.42:0/1681617027 >> 172.21.15.42:0/1681617027 conn(0x7fd1680fedf0 msgr2=0x7fd168101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:37.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.077+0000 7fd16d600640 1 -- 172.21.15.42:0/1681617027 shutdown_connections 2024-04-25T00:18:37.078 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.077+0000 7fd16d600640 1 -- 172.21.15.42:0/1681617027 wait complete. 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.077+0000 7fd16d600640 1 Processor -- start 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd16d600640 1 -- start start 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd16d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103880 0x7fd168199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd16d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168104a70 0x7fd16819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd16d600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd168105430 0x7fd1681a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd16d600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810f550 con 0x7fd168103880 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd16d600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810f3d0 con 0x7fd168105430 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd16d600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd16810f6d0 con 0x7fd168104a70 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd167e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103880 0x7fd168199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd167e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103880 0x7fd168199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52078/0 (socket says 172.21.15.42:52078) 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd167e00640 1 -- 172.21.15.42:0/2064498357 learned_addr learned my addr 172.21.15.42:0/2064498357 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:37.080 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.078+0000 7fd16cc00640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd168105430 0x7fd1681a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.079+0000 7fd167400640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168104a70 0x7fd16819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.079+0000 7fd167e00640 1 -- 172.21.15.42:0/2064498357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168104a70 msgr2=0x7fd16819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.079+0000 7fd167e00640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168104a70 0x7fd16819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.079+0000 7fd167e00640 1 -- 172.21.15.42:0/2064498357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd168105430 msgr2=0x7fd1681a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.079+0000 7fd167e00640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd168105430 0x7fd1681a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.079+0000 7fd167e00640 1 -- 172.21.15.42:0/2064498357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1681a1830 con 0x7fd168103880 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.079+0000 7fd167e00640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103880 0x7fd168199c20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd158031440 tx=0x7fd158032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.079+0000 7fd164c00640 1 -- 172.21.15.42:0/2064498357 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15803f030 con 0x7fd168103880 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.079+0000 7fd164c00640 1 -- 172.21.15.42:0/2064498357 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd158003ce0 con 0x7fd168103880 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.080+0000 7fd16d600640 1 -- 172.21.15.42:0/2064498357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1681a1ac0 con 0x7fd168103880 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.080+0000 7fd164c00640 1 -- 172.21.15.42:0/2064498357 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158035070 con 0x7fd168103880 2024-04-25T00:18:37.081 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.080+0000 7fd16d600640 1 -- 172.21.15.42:0/2064498357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a1fb0 con 0x7fd168103880 2024-04-25T00:18:37.082 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.081+0000 7fd16d600640 1 -- 172.21.15.42:0/2064498357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd16810ab40 con 0x7fd168103880 2024-04-25T00:18:37.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.083+0000 7fd164c00640 1 -- 172.21.15.42:0/2064498357 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd158042030 con 0x7fd168103880 2024-04-25T00:18:37.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.084+0000 7fd164c00640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd14c0a2b90 0x7fd14c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:37.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.084+0000 7fd164c00640 1 -- 172.21.15.42:0/2064498357 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fd1580ef430 con 0x7fd168103880 2024-04-25T00:18:37.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.084+0000 7fd167400640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd14c0a2b90 0x7fd14c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:37.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.085+0000 7fd167400640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd14c0a2b90 0x7fd14c0a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd1540095f0 tx=0x7fd154005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:37.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.089+0000 7fd164c00640 1 -- 172.21.15.42:0/2064498357 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fd1580bbd10 con 0x7fd168103880 2024-04-25T00:18:37.294 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.294+0000 7fd16d600640 1 -- 172.21.15.42:0/2064498357 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd168062340 con 0x7fd168103880 2024-04-25T00:18:37.295 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.295+0000 7fd164c00640 1 -- 172.21.15.42:0/2064498357 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) ==== 93+0+10 (secure 0 0 0) 0x7fd1580c0010 con 0x7fd168103880 2024-04-25T00:18:37.295 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-25T00:18:37.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.297+0000 7fd16d600640 1 -- 172.21.15.42:0/2064498357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd14c0a2b90 msgr2=0x7fd14c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.297+0000 7fd16d600640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd14c0a2b90 0x7fd14c0a5050 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd1540095f0 tx=0x7fd154005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:37.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.297+0000 7fd16d600640 1 -- 172.21.15.42:0/2064498357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103880 msgr2=0x7fd168199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.298 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.297+0000 7fd16d600640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103880 0x7fd168199c20 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd158031440 tx=0x7fd158032ce0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.298+0000 7fd16d600640 1 -- 172.21.15.42:0/2064498357 shutdown_connections 2024-04-25T00:18:37.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.298+0000 7fd16d600640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd14c0a2b90 0x7fd14c0a5050 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.298+0000 7fd16d600640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd168105430 0x7fd1681a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.298+0000 7fd16d600640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168104a70 0x7fd16819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.298+0000 7fd16d600640 1 --2- 172.21.15.42:0/2064498357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168103880 0x7fd168199c20 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.298+0000 7fd16d600640 1 -- 172.21.15.42:0/2064498357 >> 172.21.15.42:0/2064498357 conn(0x7fd1680fedf0 msgr2=0x7fd168107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:37.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.298+0000 7fd16d600640 1 -- 172.21.15.42:0/2064498357 shutdown_connections 2024-04-25T00:18:37.299 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.298+0000 7fd16d600640 1 -- 172.21.15.42:0/2064498357 wait complete. 2024-04-25T00:18:37.319 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:18:37.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:37 smithi042 ceph-mon[28130]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:37.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:37 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2306995438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:37.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:37 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1633803356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:37.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:37 smithi136 ceph-mon[32923]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:37.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:37 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2306995438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:37.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:37 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1633803356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.534+0000 7f1873200640 1 -- 172.21.15.42:0/2026180365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18740ffec0 msgr2=0x7f18741002e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.534+0000 7f1873200640 1 --2- 172.21.15.42:0/2026180365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18740ffec0 0x7f18741002e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1864009ca0 tx=0x7f186402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.535+0000 7f1873200640 1 -- 172.21.15.42:0/2026180365 shutdown_connections 2024-04-25T00:18:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.535+0000 7f1873200640 1 --2- 172.21.15.42:0/2026180365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18741011c0 0x7f1874101640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.535+0000 7f1873200640 1 --2- 172.21.15.42:0/2026180365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874100820 0x7f1874100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.535+0000 7f1873200640 1 --2- 172.21.15.42:0/2026180365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18740ffec0 0x7f18741002e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.535+0000 7f1873200640 1 -- 172.21.15.42:0/2026180365 >> 172.21.15.42:0/2026180365 conn(0x7f18740fbae0 msgr2=0x7f18740fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.535+0000 7f1873200640 1 -- 172.21.15.42:0/2026180365 shutdown_connections 2024-04-25T00:18:37.536 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.535+0000 7f1873200640 1 -- 172.21.15.42:0/2026180365 wait complete. 2024-04-25T00:18:37.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1873200640 1 Processor -- start 2024-04-25T00:18:37.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1873200640 1 -- start start 2024-04-25T00:18:37.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1873200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18740ffec0 0x7f1874199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:37.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1873200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874100820 0x7f187419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:37.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1873200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18741011c0 0x7f18741a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:37.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1873200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f187410f4e0 con 0x7f18740ffec0 2024-04-25T00:18:37.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1873200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f187410f360 con 0x7f18741011c0 2024-04-25T00:18:37.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1873200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f187410f660 con 0x7f1874100820 2024-04-25T00:18:37.538 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1871e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18740ffec0 0x7f1874199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1871e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18740ffec0 0x7f1874199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52096/0 (socket says 172.21.15.42:52096) 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1871e00640 1 -- 172.21.15.42:0/2072368552 learned_addr learned my addr 172.21.15.42:0/2072368552 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.536+0000 7f1869400640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874100820 0x7f187419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.537+0000 7f1872800640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18741011c0 0x7f18741a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.537+0000 7f1871e00640 1 -- 172.21.15.42:0/2072368552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874100820 msgr2=0x7f187419a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.537+0000 7f1871e00640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874100820 0x7f187419a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.537+0000 7f1871e00640 1 -- 172.21.15.42:0/2072368552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18741011c0 msgr2=0x7f18741a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.537+0000 7f1871e00640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18741011c0 0x7f18741a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.537+0000 7f1871e00640 1 -- 172.21.15.42:0/2072368552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18741a1710 con 0x7f18740ffec0 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.537+0000 7f1871e00640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18740ffec0 0x7f1874199af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1864000f80 tx=0x7f18640319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.538+0000 7f186b400640 1 -- 172.21.15.42:0/2072368552 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864003bf0 con 0x7f18740ffec0 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.538+0000 7f186b400640 1 -- 172.21.15.42:0/2072368552 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1864003d90 con 0x7f18740ffec0 2024-04-25T00:18:37.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.538+0000 7f186b400640 1 -- 172.21.15.42:0/2072368552 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186402fd50 con 0x7f18740ffec0 2024-04-25T00:18:37.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.538+0000 7f1873200640 1 -- 172.21.15.42:0/2072368552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18741a19a0 con 0x7f18740ffec0 2024-04-25T00:18:37.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.538+0000 7f1873200640 1 -- 172.21.15.42:0/2072368552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741a1ea0 con 0x7f18740ffec0 2024-04-25T00:18:37.540 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.539+0000 7f1873200640 1 -- 172.21.15.42:0/2072368552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f187410ad50 con 0x7f18740ffec0 2024-04-25T00:18:37.541 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.541+0000 7f186b400640 1 -- 172.21.15.42:0/2072368552 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1864004040 con 0x7f18740ffec0 2024-04-25T00:18:37.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.542+0000 7f186b400640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18580a2c10 0x7f18580a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:37.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.542+0000 7f186b400640 1 -- 172.21.15.42:0/2072368552 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f18640f0410 con 0x7f18740ffec0 2024-04-25T00:18:37.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.543+0000 7f1869400640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18580a2c10 0x7f18580a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:37.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.543+0000 7f1869400640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18580a2c10 0x7f18580a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f187419b000 tx=0x7f185c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:37.546 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.547+0000 7f186b400640 1 -- 172.21.15.42:0/2072368552 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f18640bd3d0 con 0x7f18740ffec0 2024-04-25T00:18:37.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:37 smithi106 ceph-mon[32510]: pgmap v383: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:37.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:37 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2306995438' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:37.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:37 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1633803356' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:37.766 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.766+0000 7f1873200640 1 -- 172.21.15.42:0/2072368552 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1874062340 con 0x7f18740ffec0 2024-04-25T00:18:37.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.767+0000 7f186b400640 1 -- 172.21.15.42:0/2072368552 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) ==== 104+0+11 (secure 0 0 0) 0x7f18640bccd0 con 0x7f18740ffec0 2024-04-25T00:18:37.767 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-25T00:18:37.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 -- 172.21.15.42:0/2072368552 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18580a2c10 msgr2=0x7f18580a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.770 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18580a2c10 0x7f18580a50d0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f187419b000 tx=0x7f185c005f90 comp rx=0 tx=0).stop 2024-04-25T00:18:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 -- 172.21.15.42:0/2072368552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18740ffec0 msgr2=0x7f1874199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18740ffec0 0x7f1874199af0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1864000f80 tx=0x7f18640319f0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 -- 172.21.15.42:0/2072368552 shutdown_connections 2024-04-25T00:18:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18580a2c10 0x7f18580a50d0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18741011c0 0x7f18741a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874100820 0x7f187419a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 --2- 172.21.15.42:0/2072368552 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18740ffec0 0x7f1874199af0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 -- 172.21.15.42:0/2072368552 >> 172.21.15.42:0/2072368552 conn(0x7f18740fbae0 msgr2=0x7f18740fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 -- 172.21.15.42:0/2072368552 shutdown_connections 2024-04-25T00:18:37.771 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:37.770+0000 7f1873200640 1 -- 172.21.15.42:0/2072368552 wait complete. 2024-04-25T00:18:37.790 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:18:38.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.011+0000 7f5c58c00640 1 -- 172.21.15.42:0/2190776850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c54101790 msgr2=0x7f5c54101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.011+0000 7f5c58c00640 1 --2- 172.21.15.42:0/2190776850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c54101790 0x7f5c54101bb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009900 tx=0x7f5c4402a410 comp rx=0 tx=0).stop 2024-04-25T00:18:38.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.011+0000 7f5c58c00640 1 -- 172.21.15.42:0/2190776850 shutdown_connections 2024-04-25T00:18:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.011+0000 7f5c58c00640 1 --2- 172.21.15.42:0/2190776850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54103340 0x7f5c54109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.011+0000 7f5c58c00640 1 --2- 172.21.15.42:0/2190776850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c54102980 0x7f5c54102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.011+0000 7f5c58c00640 1 --2- 172.21.15.42:0/2190776850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c54101790 0x7f5c54101bb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.011+0000 7f5c58c00640 1 -- 172.21.15.42:0/2190776850 >> 172.21.15.42:0/2190776850 conn(0x7f5c540fcd20 msgr2=0x7f5c540ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.011+0000 7f5c58c00640 1 -- 172.21.15.42:0/2190776850 shutdown_connections 2024-04-25T00:18:38.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.011+0000 7f5c58c00640 1 -- 172.21.15.42:0/2190776850 wait complete. 2024-04-25T00:18:38.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.012+0000 7f5c58c00640 1 Processor -- start 2024-04-25T00:18:38.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.012+0000 7f5c58c00640 1 -- start start 2024-04-25T00:18:38.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.012+0000 7f5c58c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54101790 0x7f5c54199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.012+0000 7f5c58c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c54102980 0x7f5c5419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.012+0000 7f5c58c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c54103340 0x7f5c541a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.012+0000 7f5c58c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410d430 con 0x7f5c54102980 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.012+0000 7f5c58c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410d2b0 con 0x7f5c54103340 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.012+0000 7f5c58c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5410d5b0 con 0x7f5c54101790 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c51e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54101790 0x7f5c54199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c51e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54101790 0x7f5c54199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:32812/0 (socket says 172.21.15.42:32812) 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c51e00640 1 -- 172.21.15.42:0/574645469 learned_addr learned my addr 172.21.15.42:0/574645469 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c49400640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c54102980 0x7f5c5419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c52800640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c54103340 0x7f5c541a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c51e00640 1 -- 172.21.15.42:0/574645469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c54103340 msgr2=0x7f5c541a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c51e00640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c54103340 0x7f5c541a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c51e00640 1 -- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c54102980 msgr2=0x7f5c5419a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c51e00640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c54102980 0x7f5c5419a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c51e00640 1 -- 172.21.15.42:0/574645469 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c541a1940 con 0x7f5c54101790 2024-04-25T00:18:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c49400640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c54102980 0x7f5c5419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:38.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.013+0000 7f5c51e00640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54101790 0x7f5c54199d30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5c44031440 tx=0x7f5c44032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.014+0000 7f5c4b400640 1 -- 172.21.15.42:0/574645469 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4403f030 con 0x7f5c54101790 2024-04-25T00:18:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.014+0000 7f5c58c00640 1 -- 172.21.15.42:0/574645469 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c541a1bd0 con 0x7f5c54101790 2024-04-25T00:18:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.014+0000 7f5c4b400640 1 -- 172.21.15.42:0/574645469 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c44003ce0 con 0x7f5c54101790 2024-04-25T00:18:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.014+0000 7f5c4b400640 1 -- 172.21.15.42:0/574645469 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c44035070 con 0x7f5c54101790 2024-04-25T00:18:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.014+0000 7f5c58c00640 1 -- 172.21.15.42:0/574645469 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c541a20c0 con 0x7f5c54101790 2024-04-25T00:18:38.016 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.015+0000 7f5c58c00640 1 -- 172.21.15.42:0/574645469 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c541a2290 con 0x7f5c54101790 2024-04-25T00:18:38.017 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.017+0000 7f5c4b400640 1 -- 172.21.15.42:0/574645469 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c44042030 con 0x7f5c54101790 2024-04-25T00:18:38.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.019+0000 7f5c4b400640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5c380a2b90 0x7f5c380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.019+0000 7f5c4b400640 1 -- 172.21.15.42:0/574645469 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5c440ef980 con 0x7f5c54101790 2024-04-25T00:18:38.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.019+0000 7f5c49400640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5c380a2b90 0x7f5c380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.020+0000 7f5c49400640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5c380a2b90 0x7f5c380a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c005fd0 tx=0x7f5c3c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:38.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.021+0000 7f5c4b400640 1 -- 172.21.15.42:0/574645469 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f5c440bc260 con 0x7f5c54101790 2024-04-25T00:18:38.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.228+0000 7f5c58c00640 1 -- 172.21.15.42:0/574645469 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5c541a2460 con 0x7f5c54101790 2024-04-25T00:18:38.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.229+0000 7f5c4b400640 1 -- 172.21.15.42:0/574645469 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) ==== 100+0+11 (secure 0 0 0) 0x7f5c440c0560 con 0x7f5c54101790 2024-04-25T00:18:38.229 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-25T00:18:38.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 -- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5c380a2b90 msgr2=0x7f5c380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5c380a2b90 0x7f5c380a5050 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c005fd0 tx=0x7f5c3c009500 comp rx=0 tx=0).stop 2024-04-25T00:18:38.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 -- 172.21.15.42:0/574645469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54101790 msgr2=0x7f5c54199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54101790 0x7f5c54199d30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5c44031440 tx=0x7f5c44032ce0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 -- 172.21.15.42:0/574645469 shutdown_connections 2024-04-25T00:18:38.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5c380a2b90 0x7f5c380a5050 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c54103340 0x7f5c541a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5c54102980 0x7f5c5419a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 --2- 172.21.15.42:0/574645469 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5c54101790 0x7f5c54199d30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 -- 172.21.15.42:0/574645469 >> 172.21.15.42:0/574645469 conn(0x7f5c540fcd20 msgr2=0x7f5c54073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:38.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 -- 172.21.15.42:0/574645469 shutdown_connections 2024-04-25T00:18:38.235 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.232+0000 7f5c58c00640 1 -- 172.21.15.42:0/574645469 wait complete. 2024-04-25T00:18:38.253 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:18:38.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:38 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2064498357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:38.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:38 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2072368552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.467+0000 7f1f19200640 1 -- 172.21.15.42:0/2577282808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f14107bc0 msgr2=0x7f1f14107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.467+0000 7f1f19200640 1 --2- 172.21.15.42:0/2577282808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f14107bc0 0x7f1f14107fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009860 tx=0x7f1f0402a340 comp rx=0 tx=0).stop 2024-04-25T00:18:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.467+0000 7f1f19200640 1 -- 172.21.15.42:0/2577282808 shutdown_connections 2024-04-25T00:18:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.467+0000 7f1f19200640 1 --2- 172.21.15.42:0/2577282808 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f14109770 0x7f1f14110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.467+0000 7f1f19200640 1 --2- 172.21.15.42:0/2577282808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f14108db0 0x7f1f14109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.467+0000 7f1f19200640 1 --2- 172.21.15.42:0/2577282808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f14107bc0 0x7f1f14107fe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.467+0000 7f1f19200640 1 -- 172.21.15.42:0/2577282808 >> 172.21.15.42:0/2577282808 conn(0x7f1f14076b80 msgr2=0x7f1f14078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.467+0000 7f1f19200640 1 -- 172.21.15.42:0/2577282808 shutdown_connections 2024-04-25T00:18:38.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.468+0000 7f1f19200640 1 -- 172.21.15.42:0/2577282808 wait complete. 2024-04-25T00:18:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.468+0000 7f1f19200640 1 Processor -- start 2024-04-25T00:18:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.469+0000 7f1f19200640 1 -- start start 2024-04-25T00:18:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.469+0000 7f1f19200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f14107bc0 0x7f1f1419dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.469+0000 7f1f19200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f14108db0 0x7f1f1419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.469+0000 7f1f19200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f14109770 0x7f1f141a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.469+0000 7f1f19200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1f14113830 con 0x7f1f14109770 2024-04-25T00:18:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.469+0000 7f1f19200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f141136b0 con 0x7f1f14108db0 2024-04-25T00:18:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.469+0000 7f1f19200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f141139b0 con 0x7f1f14107bc0 2024-04-25T00:18:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.469+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f14109770 0x7f1f141a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.469+0000 7f1f13e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f14109770 0x7f1f141a5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52136/0 (socket says 172.21.15.42:52136) 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.469+0000 7f1f13e00640 1 -- 172.21.15.42:0/2560831585 learned_addr learned my addr 172.21.15.42:0/2560831585 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f13400640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f14107bc0 0x7f1f1419dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f0be00640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f14108db0 0x7f1f1419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f13e00640 1 -- 172.21.15.42:0/2560831585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f14107bc0 msgr2=0x7f1f1419dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f13e00640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f14107bc0 0x7f1f1419dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f13e00640 1 -- 172.21.15.42:0/2560831585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f14108db0 msgr2=0x7f1f1419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f13e00640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f14108db0 0x7f1f1419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f13e00640 1 -- 172.21.15.42:0/2560831585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f141a5b00 con 0x7f1f14109770 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f13400640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f14107bc0 0x7f1f1419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f13e00640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f14109770 0x7f1f141a5490 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009fd0 tx=0x7f1f0c00c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f10c00640 1 -- 172.21.15.42:0/2560831585 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c004b20 con 0x7f1f14109770 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f10c00640 1 -- 172.21.15.42:0/2560831585 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f0c003e50 con 0x7f1f14109770 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.470+0000 7f1f10c00640 1 -- 172.21.15.42:0/2560831585 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00b630 con 0x7f1f14109770 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.471+0000 7f1f19200640 1 -- 172.21.15.42:0/2560831585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f141a5df0 con 0x7f1f14109770 2024-04-25T00:18:38.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.471+0000 7f1f19200640 1 -- 172.21.15.42:0/2560831585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f141a6260 con 0x7f1f14109770 2024-04-25T00:18:38.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.472+0000 7f1f19200640 1 -- 172.21.15.42:0/2560831585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f140688d0 con 0x7f1f14109770 2024-04-25T00:18:38.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.475+0000 7f1f10c00640 1 -- 172.21.15.42:0/2560831585 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f0c01d020 con 0x7f1f14109770 2024-04-25T00:18:38.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.476+0000 7f1f10c00640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ef40a2b90 0x7f1ef40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.476+0000 7f1f10c00640 1 -- 172.21.15.42:0/2560831585 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1f0c0cfb70 con 0x7f1f14109770 2024-04-25T00:18:38.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.476+0000 7f1f13400640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ef40a2b90 0x7f1ef40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.477+0000 7f1f13400640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ef40a2b90 0x7f1ef40a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009830 tx=0x7f1f04009560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:38.483 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.479+0000 7f1f10c00640 1 -- 172.21.15.42:0/2560831585 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1f0c024030 con 0x7f1f14109770 2024-04-25T00:18:38.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:38 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2064498357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:38.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:38 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2072368552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:38.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:38 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2064498357' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:18:38.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:38 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2072368552' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:18:38.727 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.726+0000 7f1f19200640 1 -- 172.21.15.42:0/2560831585 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f1402d170 con 0x7f1f14109770 2024-04-25T00:18:38.728 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.728+0000 7f1f10c00640 1 -- 172.21.15.42:0/2560831585 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+11868 (secure 0 0 0) 0x7f1f0c01d310 con 0x7f1f14109770 2024-04-25T00:18:38.729 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:38.729 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:16:19.771762+0000","modified":"2024-04-25T00:16:25.317872+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24568,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24538":{"gid":24538,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6839/501637536","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":501637536},{"type":"v1","addr":"172.21.15.136:6839","nonce":501637536}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24568":{"gid":24568,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6841/4068466248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4068466248},{"type":"v1","addr":"172.21.15.42:6841","nonce":4068466248}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3349564106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3349564106},{"type":"v1","addr":"172.21.15.106:6839","nonce":3349564106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24517,24568,24569]},"id":1}]} 2024-04-25T00:18:38.729 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 23 2024-04-25T00:18:38.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 -- 172.21.15.42:0/2560831585 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ef40a2b90 msgr2=0x7f1ef40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.731 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ef40a2b90 0x7f1ef40a5050 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1f04009830 tx=0x7f1f04009560 comp rx=0 tx=0).stop 2024-04-25T00:18:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 -- 172.21.15.42:0/2560831585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f14109770 msgr2=0x7f1f141a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f14109770 0x7f1f141a5490 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c009fd0 tx=0x7f1f0c00c800 comp rx=0 tx=0).stop 2024-04-25T00:18:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 -- 172.21.15.42:0/2560831585 shutdown_connections 2024-04-25T00:18:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ef40a2b90 0x7f1ef40a5050 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f14109770 0x7f1f141a5490 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f14108db0 0x7f1f1419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 --2- 172.21.15.42:0/2560831585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f14107bc0 0x7f1f1419dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 -- 172.21.15.42:0/2560831585 >> 172.21.15.42:0/2560831585 conn(0x7f1f14076b80 msgr2=0x7f1f1410c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.731+0000 7f1f19200640 1 -- 172.21.15.42:0/2560831585 shutdown_connections 2024-04-25T00:18:38.732 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.732+0000 7f1f19200640 1 -- 172.21.15.42:0/2560831585 wait complete. 2024-04-25T00:18:38.750 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:18:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.960+0000 7f8f33e00640 1 -- 172.21.15.42:0/439310071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f34107bc0 msgr2=0x7f8f34107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.960+0000 7f8f33e00640 1 --2- 172.21.15.42:0/439310071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f34107bc0 0x7f8f34107fe0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009ca0 tx=0x7f8f2402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.960+0000 7f8f33e00640 1 -- 172.21.15.42:0/439310071 shutdown_connections 2024-04-25T00:18:38.960 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.960+0000 7f8f33e00640 1 --2- 172.21.15.42:0/439310071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f34109770 0x7f8f34110000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.960+0000 7f8f33e00640 1 --2- 172.21.15.42:0/439310071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f34108db0 0x7f8f34109230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.960+0000 7f8f33e00640 1 --2- 172.21.15.42:0/439310071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f34107bc0 0x7f8f34107fe0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.960+0000 7f8f33e00640 1 -- 172.21.15.42:0/439310071 >> 172.21.15.42:0/439310071 conn(0x7f8f34076b80 msgr2=0x7f8f34078fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.960+0000 7f8f33e00640 1 -- 172.21.15.42:0/439310071 shutdown_connections 2024-04-25T00:18:38.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.960+0000 7f8f33e00640 1 -- 172.21.15.42:0/439310071 wait complete. 2024-04-25T00:18:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.961+0000 7f8f33e00640 1 Processor -- start 2024-04-25T00:18:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.961+0000 7f8f33e00640 1 -- start start 2024-04-25T00:18:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.961+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f34107bc0 0x7f8f3419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.961+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f34108db0 0x7f8f3419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.961+0000 7f8f33e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f34109770 0x7f8f341a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.961+0000 7f8f33e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34113880 con 0x7f8f34109770 2024-04-25T00:18:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.961+0000 7f8f33e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34113700 con 0x7f8f34108db0 2024-04-25T00:18:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.961+0000 7f8f33e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34113a00 con 0x7f8f34107bc0 2024-04-25T00:18:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.962+0000 7f8f33400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f34109770 0x7f8f341a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.962+0000 7f8f33400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f34109770 0x7f8f341a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52162/0 (socket says 172.21.15.42:52162) 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.962+0000 7f8f33400640 1 -- 172.21.15.42:0/212834497 learned_addr learned my addr 172.21.15.42:0/212834497 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.962+0000 7f8f32000640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f34108db0 0x7f8f3419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.962+0000 7f8f33400640 1 -- 172.21.15.42:0/212834497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f34107bc0 msgr2=0x7f8f3419def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.962+0000 7f8f33400640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f34107bc0 0x7f8f3419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.962+0000 7f8f33400640 1 -- 172.21.15.42:0/212834497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f34108db0 msgr2=0x7f8f3419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.962+0000 7f8f33400640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f34108db0 0x7f8f3419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.962+0000 7f8f33400640 1 -- 172.21.15.42:0/212834497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f341a5b10 con 0x7f8f34109770 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.962+0000 7f8f33400640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f34109770 0x7f8f341a54a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00e970 tx=0x7f8f2c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.963+0000 7f8f2b400640 1 -- 172.21.15.42:0/212834497 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c00ccc0 con 0x7f8f34109770 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.963+0000 7f8f2b400640 1 -- 172.21.15.42:0/212834497 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f2c00ce60 con 0x7f8f34109770 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.963+0000 7f8f33e00640 1 -- 172.21.15.42:0/212834497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f341a5e00 con 0x7f8f34109770 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.963+0000 7f8f2b400640 1 -- 172.21.15.42:0/212834497 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c010490 con 0x7f8f34109770 2024-04-25T00:18:38.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.963+0000 7f8f33e00640 1 -- 172.21.15.42:0/212834497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f341a6270 con 0x7f8f34109770 2024-04-25T00:18:38.965 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.964+0000 7f8f33e00640 1 -- 172.21.15.42:0/212834497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3410ef60 con 0x7f8f34109770 2024-04-25T00:18:38.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.966+0000 7f8f2b400640 1 -- 172.21.15.42:0/212834497 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8f2c005020 con 0x7f8f34109770 2024-04-25T00:18:38.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.967+0000 7f8f2b400640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8f180a2b90 0x7f8f180a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:38.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.968+0000 7f8f2b400640 1 -- 172.21.15.42:0/212834497 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f8f2c0cf5b0 con 0x7f8f34109770 2024-04-25T00:18:38.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.968+0000 7f8f32a00640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8f180a2b90 0x7f8f180a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:38.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.969+0000 7f8f32a00640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8f180a2b90 0x7f8f180a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8f24006010 tx=0x7f8f240093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:38.972 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:38.972+0000 7f8f2b400640 1 -- 172.21.15.42:0/212834497 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f8f2c09c4e0 con 0x7f8f34109770 2024-04-25T00:18:39.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.176+0000 7f8f33e00640 1 -- 172.21.15.42:0/212834497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f340623e0 con 0x7f8f34109770 2024-04-25T00:18:39.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.178+0000 7f8f2b400640 1 -- 172.21.15.42:0/212834497 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22547 (secure 0 0 0) 0x7f8f2c09bde0 con 0x7f8f34109770 2024-04-25T00:18:39.179 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:18:39.179 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":88,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:17:27.556239+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:18:39.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 -- 172.21.15.42:0/212834497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8f180a2b90 msgr2=0x7f8f180a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8f180a2b90 0x7f8f180a5050 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8f24006010 tx=0x7f8f240093f0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 -- 172.21.15.42:0/212834497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f34109770 msgr2=0x7f8f341a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f34109770 0x7f8f341a54a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00e970 tx=0x7f8f2c00ed70 comp rx=0 tx=0).stop 2024-04-25T00:18:39.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 -- 172.21.15.42:0/212834497 shutdown_connections 2024-04-25T00:18:39.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8f180a2b90 0x7f8f180a5050 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8f34109770 0x7f8f341a54a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8f34108db0 0x7f8f3419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 --2- 172.21.15.42:0/212834497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8f34107bc0 0x7f8f3419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 -- 172.21.15.42:0/212834497 >> 172.21.15.42:0/212834497 conn(0x7f8f34076b80 msgr2=0x7f8f3410c130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 -- 172.21.15.42:0/212834497 shutdown_connections 2024-04-25T00:18:39.182 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.181+0000 7f8f33e00640 1 -- 172.21.15.42:0/212834497 wait complete. 2024-04-25T00:18:39.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T00:16:19.771762+0000', 'modified': '2024-04-25T00:16:25.317872+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24569, 'mds_1': 24568, 'mds_2': 24517}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24505': {'gid': 24505, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/12913503', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 12913503}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 12913503}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24517': {'gid': 24517, 'name': 'c', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.136:6837/222488442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 222488442}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 222488442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24521': {'gid': 24521, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.106:6837/1331229112', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 1331229112}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 1331229112}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24538': {'gid': 24538, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6839/501637536', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 501637536}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 501637536}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24568': {'gid': 24568, 'name': 'a', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.42:6841/4068466248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.42:6840', 'nonce': 4068466248}, {'type': 'v1', 'addr': '172.21.15.42:6841', 'nonce': 4068466248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24569': {'gid': 24569, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6839/3349564106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6838', 'nonce': 3349564106}, {'type': 'v1', 'addr': '172.21.15.106:6839', 'nonce': 3349564106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24569, 'qdb_cluster': [24517, 24568, 24569]} 2024-04-25T00:18:39.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T00:18:39.203 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T00:18:39.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:39 smithi042 ceph-mon[28130]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:39.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:39 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/574645469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:39.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:39 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2560831585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:39.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.424+0000 7f9e5ca00640 1 -- 172.21.15.42:0/2210961606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e58103880 msgr2=0x7f9e58103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.424 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.424+0000 7f9e5ca00640 1 --2- 172.21.15.42:0/2210961606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e58103880 0x7f9e58103ca0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f9e48009be0 tx=0x7f9e4802a6c0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.424+0000 7f9e5ca00640 1 -- 172.21.15.42:0/2210961606 shutdown_connections 2024-04-25T00:18:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.424+0000 7f9e5ca00640 1 --2- 172.21.15.42:0/2210961606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e58105430 0x7f9e5810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.424+0000 7f9e5ca00640 1 --2- 172.21.15.42:0/2210961606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58104a70 0x7f9e58104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.424+0000 7f9e5ca00640 1 --2- 172.21.15.42:0/2210961606 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e58103880 0x7f9e58103ca0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.424+0000 7f9e5ca00640 1 -- 172.21.15.42:0/2210961606 >> 172.21.15.42:0/2210961606 conn(0x7f9e580fedf0 msgr2=0x7f9e58101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.424+0000 7f9e5ca00640 1 -- 172.21.15.42:0/2210961606 shutdown_connections 2024-04-25T00:18:39.425 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.424+0000 7f9e5ca00640 1 -- 172.21.15.42:0/2210961606 wait complete. 2024-04-25T00:18:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.425+0000 7f9e5ca00640 1 Processor -- start 2024-04-25T00:18:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.425+0000 7f9e5ca00640 1 -- start start 2024-04-25T00:18:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.425+0000 7f9e5ca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e58103880 0x7f9e58199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.425+0000 7f9e5ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58104a70 0x7f9e5819a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.425+0000 7f9e5ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e58105430 0x7f9e581a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.425+0000 7f9e5ca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810f4f0 con 0x7f9e58105430 2024-04-25T00:18:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.425+0000 7f9e5ca00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810f370 con 0x7f9e58103880 2024-04-25T00:18:39.427 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.425+0000 7f9e5ca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5810f670 con 0x7f9e58104a70 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e58105430 0x7f9e581a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e58105430 0x7f9e581a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52182/0 (socket says 172.21.15.42:52182) 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57e00640 1 -- 172.21.15.42:0/276695061 learned_addr learned my addr 172.21.15.42:0/276695061 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57400640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e58103880 0x7f9e58199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e56a00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58104a70 0x7f9e5819a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57e00640 1 -- 172.21.15.42:0/276695061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58104a70 msgr2=0x7f9e5819a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57e00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58104a70 0x7f9e5819a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57e00640 1 -- 172.21.15.42:0/276695061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e58103880 msgr2=0x7f9e58199ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57e00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e58103880 0x7f9e58199ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57e00640 1 -- 172.21.15.42:0/276695061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e581a17c0 con 0x7f9e58105430 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57400640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e58103880 0x7f9e58199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.426+0000 7f9e57e00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e58105430 0x7f9e581a1150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000e970 tx=0x7f9e5000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.427+0000 7f9e4fe00640 1 -- 172.21.15.42:0/276695061 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e5000ccc0 con 0x7f9e58105430 2024-04-25T00:18:39.428 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.427+0000 7f9e4fe00640 1 -- 172.21.15.42:0/276695061 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e5000ce60 con 0x7f9e58105430 2024-04-25T00:18:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.427+0000 7f9e4fe00640 1 -- 172.21.15.42:0/276695061 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e50010430 con 0x7f9e58105430 2024-04-25T00:18:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.427+0000 7f9e5ca00640 1 -- 172.21.15.42:0/276695061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e581a1ab0 con 0x7f9e58105430 2024-04-25T00:18:39.429 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.427+0000 7f9e5ca00640 1 -- 172.21.15.42:0/276695061 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e581a2000 con 0x7f9e58105430 2024-04-25T00:18:39.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.431+0000 7f9e60de5740 1 Processor -- start 2024-04-25T00:18:39.431 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.431+0000 7f9e4fe00640 1 -- 172.21.15.42:0/276695061 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e50005020 con 0x7f9e58105430 2024-04-25T00:18:39.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.432+0000 7f9e60de5740 1 -- start start 2024-04-25T00:18:39.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.432+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77e970 0x55bada77ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.432+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77f360 0x55bada783e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.432+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada786960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.432+0000 7f9e60de5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bada67eed0 con 0x55bada7844d0 2024-04-25T00:18:39.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.432+0000 7f9e60de5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bada6e1df0 con 0x55bada77f360 2024-04-25T00:18:39.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.432+0000 7f9e60de5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bada68a8d0 con 0x55bada77e970 2024-04-25T00:18:39.433 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.432+0000 7f9e56a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77e970 0x55bada77ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.432+0000 7f9e56a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77e970 0x55bada77ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:32874/0 (socket says 172.21.15.42:32874) 2024-04-25T00:18:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.432+0000 7f9e56a00640 1 -- 172.21.15.42:0/514589861 learned_addr learned my addr 172.21.15.42:0/514589861 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.433+0000 7f9e57400640 1 --2- 172.21.15.42:0/514589861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77f360 0x55bada783e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.433+0000 7f9e56a00640 1 --2- 172.21.15.42:0/514589861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada786960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.433+0000 7f9e57400640 1 -- 172.21.15.42:0/514589861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77e970 msgr2=0x55bada77ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.433+0000 7f9e57400640 1 --2- 172.21.15.42:0/514589861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77e970 0x55bada77ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.433+0000 7f9e57400640 1 -- 172.21.15.42:0/514589861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 msgr2=0x55bada786960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.433+0000 7f9e57400640 1 --2- 172.21.15.42:0/514589861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada786960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.434 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.433+0000 7f9e57400640 1 -- 172.21.15.42:0/514589861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bada3f62d0 con 0x55bada77f360 2024-04-25T00:18:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.433+0000 7f9e57400640 1 --2- 172.21.15.42:0/514589861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77f360 0x55bada783e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e48034040 tx=0x7f9e48009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d519d37a50b23bfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.433+0000 7f9e4fe00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e3c0a2c10 0x7f9e3c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.438 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.434+0000 7f9e4fe00640 1 -- 172.21.15.42:0/276695061 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f9e500cf310 con 0x7f9e58105430 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.434+0000 7f9e43e00640 1 -- 172.21.15.42:0/514589861 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48041030 con 0x55bada77f360 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.434+0000 7f9e43e00640 1 -- 172.21.15.42:0/514589861 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e48003e50 con 0x55bada77f360 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.434+0000 7f9e43e00640 1 -- 172.21.15.42:0/514589861 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48036040 con 0x55bada77f360 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.434+0000 7f9e56a00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e3c0a2c10 0x7f9e3c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 -- 172.21.15.42:0/514589861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77f360 msgr2=0x55bada783e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 --2- 172.21.15.42:0/514589861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77f360 0x55bada783e60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9e48034040 tx=0x7f9e48009330 comp rx=0 tx=0).stop 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e56a00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e3c0a2c10 0x7f9e3c0a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55bada77fbc0 tx=0x7f9e44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 -- 172.21.15.42:0/514589861 shutdown_connections 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 --2- 172.21.15.42:0/514589861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada786960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 --2- 172.21.15.42:0/514589861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77f360 0x55bada783e60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 --2- 172.21.15.42:0/514589861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77e970 0x55bada77ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 -- 172.21.15.42:0/514589861 >> 172.21.15.42:0/514589861 conn(0x55bada779fe0 msgr2=0x55bada77a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.439 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 -- 172.21.15.42:0/514589861 shutdown_connections 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 -- 172.21.15.42:0/514589861 wait complete. 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 -- start start 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.435+0000 7f9e60de5740 1 Processor -- start 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada788be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada789120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada790160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e60de5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bada69dd20 con 0x55bada7844d0 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e60de5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bada6dddb0 con 0x55bada77e970 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e60de5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bada787420 con 0x55bada77f360 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e60de5740 10 client.0 _ll_register_callbacks cb 0x55bada6d8f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e57e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada790160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e57e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada790160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52196/0 (socket says 172.21.15.42:52196) 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e57e00640 1 -- 172.21.15.42:0/2497843762 learned_addr learned my addr 172.21.15.42:0/2497843762 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.440 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e57400640 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada788be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.436+0000 7f9e56a00640 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada789120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e57e00640 1 -- 172.21.15.42:0/2497843762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 msgr2=0x55bada789120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e57e00640 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada789120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e57e00640 1 -- 172.21.15.42:0/2497843762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 msgr2=0x55bada788be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e57e00640 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada788be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e57e00640 1 -- 172.21.15.42:0/2497843762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bada4c06b0 con 0x55bada7844d0 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e57400640 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada788be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e57e00640 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada790160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e500a4280 tx=0x7f9e5009c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10328dd10d2f43ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e40c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e40c00640 1 -- 172.21.15.42:0/2497843762 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500a47b0 con 0x55bada7844d0 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e40c00640 1 -- 172.21.15.42:0/2497843762 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f9e50005310 con 0x55bada7844d0 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.437+0000 7f9e60de5740 1 -- 172.21.15.42:0/2497843762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e48003e50 con 0x55bada7844d0 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e40c00640 1 -- 172.21.15.42:0/2497843762 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9e500a4320 con 0x55bada7844d0 2024-04-25T00:18:39.441 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e60de5740 10 client.15033 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e60de5740 1 -- 172.21.15.42:0/2497843762 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55bada7935a0 con 0x55bada7844d0 2024-04-25T00:18:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e40c00640 1 -- 172.21.15.42:0/2497843762 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e48003e50 con 0x55bada7844d0 2024-04-25T00:18:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e40c00640 1 -- 172.21.15.42:0/2497843762 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9e5009af90 con 0x55bada7844d0 2024-04-25T00:18:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e60de5740 10 client.15033 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e60de5740 10 client.15033 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:18:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e60de5740 20 client.15033 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e60de5740 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55bada795c40 0x55bada7970b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e60de5740 4 client.15033 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:39.442 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.438+0000 7f9e60de5740 1 -- 172.21.15.42:0/2497843762 --> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bada6ef330 con 0x55bada795c40 2024-04-25T00:18:39.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.439+0000 7f9e57400640 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55bada795c40 0x55bada7970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.439+0000 7f9e57400640 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55bada795c40 0x55bada7970b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2ec77ef19ba0f997 server_cookie=4cbef69be766726a in_seq=0 out_seq=0 2024-04-25T00:18:39.443 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.439+0000 7f9e40c00640 10 client.15033 ms_handle_connect on v2:172.21.15.136:6836/222488442 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.440+0000 7f9e40c00640 1 -- 172.21.15.42:0/2497843762 <== mds.2 v2:172.21.15.136:6836/222488442 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bada6ef330 con 0x55bada795c40 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.440+0000 7f9e40c00640 10 client.15033 handle_command_reply: tid=0 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.440+0000 7f9e60de5740 1 client.15033 shutdown 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.440+0000 7f9e60de5740 1 -- 172.21.15.42:0/2497843762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 msgr2=0x55bada790160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e60de5740 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada790160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e500a4280 tx=0x7f9e5009c240 comp rx=0 tx=0).stop 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e60de5740 1 -- 172.21.15.42:0/2497843762 shutdown_connections 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e60de5740 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55bada795c40 0x55bada7970b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e60de5740 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada790160 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e60de5740 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada789120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e60de5740 1 --2- 172.21.15.42:0/2497843762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada788be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e40c00640 0 client.15033 ms_handle_reset on v2:172.21.15.136:6836/222488442 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e60de5740 1 -- 172.21.15.42:0/2497843762 >> 172.21.15.42:0/2497843762 conn(0x55bada779fe0 msgr2=0x55bada779120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e60de5740 1 -- 172.21.15.42:0/2497843762 shutdown_connections 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e60de5740 1 -- 172.21.15.42:0/2497843762 wait complete. 2024-04-25T00:18:39.444 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.441+0000 7f9e60de5740 20 client.15033 trim_cache size 0 max 16384 2024-04-25T00:18:39.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.461+0000 7f9e60de5740 1 Processor -- start 2024-04-25T00:18:39.461 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.461+0000 7f9e60de5740 1 -- start start 2024-04-25T00:18:39.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.461+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada7877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.461+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada790120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.461+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada77a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.461+0000 7f9e60de5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bada7953f0 con 0x55bada7844d0 2024-04-25T00:18:39.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.461+0000 7f9e60de5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bada795ac0 con 0x55bada77e970 2024-04-25T00:18:39.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.461+0000 7f9e60de5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bada796190 con 0x55bada77f360 2024-04-25T00:18:39.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.462+0000 7f9e57400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada7877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.462+0000 7f9e56a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada790120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.462+0000 7f9e57e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada77a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.462 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.462+0000 7f9e57400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada7877f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53630/0 (socket says 172.21.15.42:53630) 2024-04-25T00:18:39.463 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.462+0000 7f9e57400640 1 -- 172.21.15.42:0/2292767904 learned_addr learned my addr 172.21.15.42:0/2292767904 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.462+0000 7f9e57400640 1 -- 172.21.15.42:0/2292767904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 msgr2=0x55bada790120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.462+0000 7f9e57400640 1 --2- 172.21.15.42:0/2292767904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada790120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.462+0000 7f9e57400640 1 -- 172.21.15.42:0/2292767904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 msgr2=0x55bada77a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.462+0000 7f9e57400640 1 --2- 172.21.15.42:0/2292767904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada77a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.464 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.462+0000 7f9e57400640 1 -- 172.21.15.42:0/2292767904 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bada787590 con 0x55bada77e970 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.463+0000 7f9e57400640 1 --2- 172.21.15.42:0/2292767904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada7877f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e48030960 tx=0x7f9e48000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7e2d38039cfb50d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.463+0000 7f9e43400640 1 -- 172.21.15.42:0/2292767904 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48036040 con 0x55bada77e970 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.463+0000 7f9e43400640 1 -- 172.21.15.42:0/2292767904 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4804e8f0 con 0x55bada77e970 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.463+0000 7f9e43400640 1 -- 172.21.15.42:0/2292767904 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48004400 con 0x55bada77e970 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 -- 172.21.15.42:0/2292767904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 msgr2=0x55bada7877f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 --2- 172.21.15.42:0/2292767904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada7877f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9e48030960 tx=0x7f9e48000f40 comp rx=0 tx=0).stop 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 -- 172.21.15.42:0/2292767904 shutdown_connections 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 --2- 172.21.15.42:0/2292767904 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada77a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 --2- 172.21.15.42:0/2292767904 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada790120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 --2- 172.21.15.42:0/2292767904 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada7877f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 -- 172.21.15.42:0/2292767904 >> 172.21.15.42:0/2292767904 conn(0x55bada779fe0 msgr2=0x55bada774d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 -- 172.21.15.42:0/2292767904 shutdown_connections 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 -- 172.21.15.42:0/2292767904 wait complete. 2024-04-25T00:18:39.465 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 -- start start 2024-04-25T00:18:39.466 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.464+0000 7f9e60de5740 1 Processor -- start 2024-04-25T00:18:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada782e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada790120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e60de5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada7f9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e60de5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bada796f30 con 0x55bada7844d0 2024-04-25T00:18:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e60de5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bada796860 con 0x55bada77e970 2024-04-25T00:18:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e60de5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bada797600 con 0x55bada77f360 2024-04-25T00:18:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e60de5740 10 client.0 _ll_register_callbacks cb 0x7f9e5000ef00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:39.468 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e57e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada7f9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e57400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada782e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e57400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada782e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53638/0 (socket says 172.21.15.42:53638) 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e57400640 1 -- 172.21.15.42:0/1962149878 learned_addr learned my addr 172.21.15.42:0/1962149878 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.465+0000 7f9e56a00640 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada790120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e57e00640 1 -- 172.21.15.42:0/1962149878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 msgr2=0x55bada790120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e57e00640 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada790120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e57e00640 1 -- 172.21.15.42:0/1962149878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 msgr2=0x55bada782e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e57e00640 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada782e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e57e00640 1 -- 172.21.15.42:0/1962149878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bada784220 con 0x55bada7844d0 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e57400640 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada782e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e57e00640 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada7f9be0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9e50097fa0 tx=0x7f9e50005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f21d18d2e04181e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e40c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e40c00640 1 -- 172.21.15.42:0/1962149878 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e500a47b0 con 0x55bada7844d0 2024-04-25T00:18:39.469 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e40c00640 1 -- 172.21.15.42:0/1962149878 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f9e50005310 con 0x55bada7844d0 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.466+0000 7f9e60de5740 1 -- 172.21.15.42:0/1962149878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e4804e8f0 con 0x55bada7844d0 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e40c00640 1 -- 172.21.15.42:0/1962149878 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f9e5009ad60 con 0x55bada7844d0 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e60de5740 10 client.15039 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e60de5740 1 -- 172.21.15.42:0/1962149878 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55bada7fb190 con 0x55bada7844d0 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e40c00640 1 -- 172.21.15.42:0/1962149878 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e4804e8f0 con 0x55bada7844d0 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e40c00640 1 -- 172.21.15.42:0/1962149878 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f9e500a1080 con 0x55bada7844d0 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e60de5740 10 client.15039 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e60de5740 10 client.15039 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e60de5740 20 client.15039 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e60de5740 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55bada7ffc20 0x55bada7fe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e60de5740 4 client.15039 mds_command: new command op to 24517 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:18:39.470 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.467+0000 7f9e60de5740 1 -- 172.21.15.42:0/1962149878 --> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] -- command(tid 0: {"prefix": "status"}) -- 0x55bada6e9680 con 0x55bada7ffc20 2024-04-25T00:18:39.471 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.469+0000 7f9e57400640 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55bada7ffc20 0x55bada7fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.472+0000 7f9e57400640 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55bada7ffc20 0x55bada7fe930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a4c0414400fb5fd7 server_cookie=60808d892d97afd5 in_seq=0 out_seq=0 2024-04-25T00:18:39.472 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.472+0000 7f9e40c00640 10 client.15039 ms_handle_connect on v2:172.21.15.136:6836/222488442 2024-04-25T00:18:39.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.472+0000 7f9e40c00640 1 -- 172.21.15.42:0/1962149878 <== mds.2 v2:172.21.15.136:6836/222488442 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bada6e9680 con 0x55bada7ffc20 2024-04-25T00:18:39.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.472+0000 7f9e40c00640 10 client.15039 handle_command_reply: tid=0 2024-04-25T00:18:39.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 client.15039 shutdown 2024-04-25T00:18:39.473 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 -- 172.21.15.42:0/1962149878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 msgr2=0x55bada7f9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada7f9be0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9e50097fa0 tx=0x7f9e50005ab0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 -- 172.21.15.42:0/1962149878 shutdown_connections 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.136:6836/222488442,v1:172.21.15.136:6837/222488442] conn(0x55bada7ffc20 0x55bada7fe930 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bada7844d0 0x55bada7f9be0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bada77f360 0x55bada790120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e40c00640 0 client.15039 ms_handle_reset on v2:172.21.15.136:6836/222488442 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 --2- 172.21.15.42:0/1962149878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bada77e970 0x55bada782e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 -- 172.21.15.42:0/1962149878 >> 172.21.15.42:0/1962149878 conn(0x55bada779fe0 msgr2=0x55bada77a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 -- 172.21.15.42:0/1962149878 shutdown_connections 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 1 -- 172.21.15.42:0/1962149878 wait complete. 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.473+0000 7f9e60de5740 20 client.15039 trim_cache size 0 max 16384 2024-04-25T00:18:39.474 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "9d95543e-0297-11ef-bc93-c7b262605968", 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 2, 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24517, 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 135.16283101299999, 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 23, 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 88, 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 160.97932564000001 2024-04-25T00:18:39.475 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:18:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 -- 172.21.15.42:0/276695061 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e3c0a2c10 msgr2=0x7f9e3c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e3c0a2c10 0x7f9e3c0a50d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55bada77fbc0 tx=0x7f9e44009260 comp rx=0 tx=0).stop 2024-04-25T00:18:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 -- 172.21.15.42:0/276695061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e58105430 msgr2=0x7f9e581a1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e58105430 0x7f9e581a1150 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000e970 tx=0x7f9e5000ed70 comp rx=0 tx=0).stop 2024-04-25T00:18:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 -- 172.21.15.42:0/276695061 shutdown_connections 2024-04-25T00:18:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e3c0a2c10 0x7f9e3c0a50d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e58105430 0x7f9e581a1150 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e58104a70 0x7f9e5819a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 --2- 172.21.15.42:0/276695061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e58103880 0x7f9e58199ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 -- 172.21.15.42:0/276695061 >> 172.21.15.42:0/276695061 conn(0x7f9e580fedf0 msgr2=0x7f9e58107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 -- 172.21.15.42:0/276695061 shutdown_connections 2024-04-25T00:18:39.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.474+0000 7f9e5ca00640 1 -- 172.21.15.42:0/276695061 wait complete. 2024-04-25T00:18:39.493 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T00:18:39.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:39 smithi136 ceph-mon[32923]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:39.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:39 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/574645469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:39.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:39 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2560831585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:39.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:39 smithi106 ceph-mon[32510]: pgmap v384: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:39.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:39 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/574645469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:18:39.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:39 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2560831585' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:18:39.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.698+0000 7f1755a00640 1 -- 172.21.15.42:0/2493878085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1750076100 msgr2=0x7f1750076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.698+0000 7f1755a00640 1 --2- 172.21.15.42:0/2493878085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1750076100 0x7f1750076520 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1740009900 tx=0x7f174002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.698+0000 7f1755a00640 1 -- 172.21.15.42:0/2493878085 shutdown_connections 2024-04-25T00:18:39.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.698+0000 7f1755a00640 1 --2- 172.21.15.42:0/2493878085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1750077ce0 0x7f1750078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.698+0000 7f1755a00640 1 --2- 172.21.15.42:0/2493878085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1750077340 0x7f17500777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.698+0000 7f1755a00640 1 --2- 172.21.15.42:0/2493878085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1750076100 0x7f1750076520 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.698+0000 7f1755a00640 1 -- 172.21.15.42:0/2493878085 >> 172.21.15.42:0/2493878085 conn(0x7f17500fedf0 msgr2=0x7f1750101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.698+0000 7f1755a00640 1 -- 172.21.15.42:0/2493878085 shutdown_connections 2024-04-25T00:18:39.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.698+0000 7f1755a00640 1 -- 172.21.15.42:0/2493878085 wait complete. 2024-04-25T00:18:39.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.699+0000 7f1755a00640 1 Processor -- start 2024-04-25T00:18:39.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.699+0000 7f1755a00640 1 -- start start 2024-04-25T00:18:39.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1750076100 0x7f175019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1750077340 0x7f175019e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1750077ce0 0x7f17501a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1750113910 con 0x7f1750077ce0 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1750113790 con 0x7f1750076100 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1750113a90 con 0x7f1750077340 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1750077ce0 0x7f17501a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f174fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1750076100 0x7f175019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1750077ce0 0x7f17501a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52224/0 (socket says 172.21.15.42:52224) 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755000640 1 -- 172.21.15.42:0/3575005140 learned_addr learned my addr 172.21.15.42:0/3575005140 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f174f400640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1750077340 0x7f175019e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755000640 1 -- 172.21.15.42:0/3575005140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1750077340 msgr2=0x7f175019e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755000640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1750077340 0x7f175019e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755000640 1 -- 172.21.15.42:0/3575005140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1750076100 msgr2=0x7f175019df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755000640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1750076100 0x7f175019df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f1755000640 1 -- 172.21.15.42:0/3575005140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17501a5bb0 con 0x7f1750077ce0 2024-04-25T00:18:39.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.700+0000 7f174fe00640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1750076100 0x7f175019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:39.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.701+0000 7f1755000640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1750077ce0 0x7f17501a5540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f174800eb10 tx=0x7f174800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.701+0000 7f174cc00640 1 -- 172.21.15.42:0/3575005140 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748004b20 con 0x7f1750077ce0 2024-04-25T00:18:39.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.701+0000 7f1755a00640 1 -- 172.21.15.42:0/3575005140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17501a5ea0 con 0x7f1750077ce0 2024-04-25T00:18:39.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.701+0000 7f174cc00640 1 -- 172.21.15.42:0/3575005140 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1748004cc0 con 0x7f1750077ce0 2024-04-25T00:18:39.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.701+0000 7f174cc00640 1 -- 172.21.15.42:0/3575005140 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748010700 con 0x7f1750077ce0 2024-04-25T00:18:39.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.701+0000 7f1755a00640 1 -- 172.21.15.42:0/3575005140 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17501a6310 con 0x7f1750077ce0 2024-04-25T00:18:39.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.704+0000 7f1759d94740 1 Processor -- start 2024-04-25T00:18:39.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.705+0000 7f174cc00640 1 -- 172.21.15.42:0/3575005140 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1748005000 con 0x7f1750077ce0 2024-04-25T00:18:39.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.705+0000 7f1759d94740 1 -- start start 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.705+0000 7f1759d94740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 0x55c554b72d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.705+0000 7f1759d94740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b73360 0x55c554b77e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.705+0000 7f1759d94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b784d0 0x55c554b7a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.705+0000 7f1759d94740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c554a72ed0 con 0x55c554b73360 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.705+0000 7f1759d94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c554ad5df0 con 0x55c554b784d0 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.705+0000 7f1759d94740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c554a7e8d0 con 0x55c554b72970 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.705+0000 7f174fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b73360 0x55c554b77e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.705+0000 7f174fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b73360 0x55c554b77e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52236/0 (socket says 172.21.15.42:52236) 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174fe00640 1 -- 172.21.15.42:0/538909500 learned_addr learned my addr 172.21.15.42:0/538909500 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174f400640 1 --2- 172.21.15.42:0/538909500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 0x55c554b72d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174f400640 1 --2- 172.21.15.42:0/538909500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b784d0 0x55c554b7a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174fe00640 1 -- 172.21.15.42:0/538909500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 msgr2=0x55c554b72d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174fe00640 1 --2- 172.21.15.42:0/538909500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 0x55c554b72d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174fe00640 1 -- 172.21.15.42:0/538909500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b784d0 msgr2=0x55c554b7a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174fe00640 1 --2- 172.21.15.42:0/538909500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b784d0 0x55c554b7a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174fe00640 1 -- 172.21.15.42:0/538909500 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5547ea2d0 con 0x55c554b73360 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174f400640 1 --2- 172.21.15.42:0/538909500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b784d0 0x55c554b7a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174fe00640 1 --2- 172.21.15.42:0/538909500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b73360 0x55c554b77e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f174002a910 tx=0x7f1740009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d15f2617b602708b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.706+0000 7f174cc00640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f17340a2b90 0x7f17340a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f1745600640 1 -- 172.21.15.42:0/538909500 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740040070 con 0x55c554b73360 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f1745600640 1 -- 172.21.15.42:0/538909500 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17400319b0 con 0x55c554b73360 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f1745600640 1 -- 172.21.15.42:0/538909500 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740035040 con 0x55c554b73360 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f174cc00640 1 -- 172.21.15.42:0/3575005140 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f17480cf720 con 0x7f1750077ce0 2024-04-25T00:18:39.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f174f400640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f17340a2b90 0x7f17340a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f1759d94740 1 -- 172.21.15.42:0/538909500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b73360 msgr2=0x55c554b77e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f1759d94740 1 --2- 172.21.15.42:0/538909500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b73360 0x55c554b77e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f174002a910 tx=0x7f1740009680 comp rx=0 tx=0).stop 2024-04-25T00:18:39.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f1759d94740 1 -- 172.21.15.42:0/538909500 shutdown_connections 2024-04-25T00:18:39.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f1759d94740 1 --2- 172.21.15.42:0/538909500 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b784d0 0x55c554b7a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f1759d94740 1 --2- 172.21.15.42:0/538909500 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b73360 0x55c554b77e60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f1759d94740 1 --2- 172.21.15.42:0/538909500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 0x55c554b72d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.707+0000 7f1759d94740 1 -- 172.21.15.42:0/538909500 >> 172.21.15.42:0/538909500 conn(0x55c554b6dfe0 msgr2=0x55c554b6e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.708+0000 7f1759d94740 1 -- 172.21.15.42:0/538909500 shutdown_connections 2024-04-25T00:18:39.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.708+0000 7f174f400640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f17340a2b90 0x7f17340a5050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55c554b73bc0 tx=0x7f173c013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.708+0000 7f1759d94740 1 -- 172.21.15.42:0/538909500 wait complete. 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.708+0000 7f1759d94740 1 -- start start 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.708+0000 7f1759d94740 1 Processor -- start 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1759d94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 0x55c554b7cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1759d94740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1759d94740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b84160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1759d94740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c554a91d20 con 0x55c554b784d0 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1759d94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c554ad1db0 con 0x55c554b72970 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1759d94740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c554b7b420 con 0x55c554b73360 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1759d94740 10 client.0 _ll_register_callbacks cb 0x55c554accf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f174f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1755000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b84160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f174f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:32934/0 (socket says 172.21.15.42:32934) 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f174f400640 1 -- 172.21.15.42:0/3920040359 learned_addr learned my addr 172.21.15.42:0/3920040359 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.713 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f174fe00640 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 0x55c554b7cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1755000640 1 -- 172.21.15.42:0/3920040359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 msgr2=0x55c554b7d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1755000640 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1755000640 1 -- 172.21.15.42:0/3920040359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 msgr2=0x55c554b7cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1755000640 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 0x55c554b7cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f1755000640 1 -- 172.21.15.42:0/3920040359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5548b46b0 con 0x55c554b784d0 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f174fe00640 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 0x55c554b7cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.709+0000 7f174f400640 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.710+0000 7f1755000640 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b84160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f17480983f0 tx=0x7f174809bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2348e8048394b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.710+0000 7f173a000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.710+0000 7f173a000640 1 -- 172.21.15.42:0/3920040359 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17480badd0 con 0x55c554b784d0 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.710+0000 7f173a000640 1 -- 172.21.15.42:0/3920040359 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f17480052f0 con 0x55c554b784d0 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.710+0000 7f1759d94740 1 -- 172.21.15.42:0/3920040359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17400319b0 con 0x55c554b784d0 2024-04-25T00:18:39.714 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.710+0000 7f173a000640 1 -- 172.21.15.42:0/3920040359 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f17480bc070 con 0x55c554b784d0 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.710+0000 7f1759d94740 10 client.15051 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.710+0000 7f1759d94740 1 -- 172.21.15.42:0/3920040359 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c554b87540 con 0x55c554b784d0 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.710+0000 7f173a000640 1 -- 172.21.15.42:0/3920040359 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17400319b0 con 0x55c554b784d0 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.711+0000 7f173a000640 1 -- 172.21.15.42:0/3920040359 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f174809b470 con 0x55c554b784d0 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.711+0000 7f1759d94740 10 client.15051 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.711+0000 7f1759d94740 10 client.15051 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.711+0000 7f1759d94740 20 client.15051 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.711+0000 7f1759d94740 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55c554b89be0 0x55c554b8b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.711+0000 7f1759d94740 4 client.15051 mds_command: new command op to 24568 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.711+0000 7f1759d94740 1 -- 172.21.15.42:0/3920040359 --> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c554ae3330 con 0x55c554b89be0 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.711+0000 7f174fe00640 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55c554b89be0 0x55c554b8b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.712+0000 7f174fe00640 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55c554b89be0 0x55c554b8b050 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=894d11e92073c0bb server_cookie=56265b5887d6100c in_seq=0 out_seq=0 2024-04-25T00:18:39.715 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.712+0000 7f173a000640 10 client.15051 ms_handle_connect on v2:172.21.15.42:6840/4068466248 2024-04-25T00:18:39.716 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f173a000640 1 -- 172.21.15.42:0/3920040359 <== mds.1 v2:172.21.15.42:6840/4068466248 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c554ae3330 con 0x55c554b89be0 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f173a000640 10 client.15051 handle_command_reply: tid=0 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 client.15051 shutdown 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 -- 172.21.15.42:0/3920040359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 msgr2=0x55c554b84160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b84160 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f17480983f0 tx=0x7f174809bbc0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 -- 172.21.15.42:0/3920040359 shutdown_connections 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55c554b89be0 0x55c554b8b050 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b84160 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 --2- 172.21.15.42:0/3920040359 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 0x55c554b7cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f173a000640 0 client.15051 ms_handle_reset on v2:172.21.15.42:6840/4068466248 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 -- 172.21.15.42:0/3920040359 >> 172.21.15.42:0/3920040359 conn(0x55c554b6dfe0 msgr2=0x55c554b6d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 -- 172.21.15.42:0/3920040359 shutdown_connections 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 1 -- 172.21.15.42:0/3920040359 wait complete. 2024-04-25T00:18:39.717 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.713+0000 7f1759d94740 20 client.15051 trim_cache size 0 max 16384 2024-04-25T00:18:39.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1759d94740 1 Processor -- start 2024-04-25T00:18:39.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1759d94740 1 -- start start 2024-04-25T00:18:39.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1759d94740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 0x55c554b6e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1759d94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b73360 0x55c554b84120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1759d94740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b747c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1759d94740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c554b89390 con 0x55c554b784d0 2024-04-25T00:18:39.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1759d94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c554b89a60 con 0x55c554b73360 2024-04-25T00:18:39.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1759d94740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c554b8a130 con 0x55c554b72970 2024-04-25T00:18:39.735 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1755000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b747c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1755000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b747c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52250/0 (socket says 172.21.15.42:52250) 2024-04-25T00:18:39.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.734+0000 7f1755000640 1 -- 172.21.15.42:0/933910888 learned_addr learned my addr 172.21.15.42:0/933910888 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f174f400640 1 --2- 172.21.15.42:0/933910888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b73360 0x55c554b84120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f174fe00640 1 --2- 172.21.15.42:0/933910888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 0x55c554b6e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f1755000640 1 -- 172.21.15.42:0/933910888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 msgr2=0x55c554b6e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f1755000640 1 --2- 172.21.15.42:0/933910888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 0x55c554b6e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f1755000640 1 -- 172.21.15.42:0/933910888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b73360 msgr2=0x55c554b84120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f1755000640 1 --2- 172.21.15.42:0/933910888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b73360 0x55c554b84120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f1755000640 1 -- 172.21.15.42:0/933910888 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c554b73bf0 con 0x55c554b784d0 2024-04-25T00:18:39.736 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f174fe00640 1 --2- 172.21.15.42:0/933910888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 0x55c554b6e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f1755000640 1 --2- 172.21.15.42:0/933910888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b747c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f174809b830 tx=0x7f174809a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec62167a85def39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f1744c00640 1 -- 172.21.15.42:0/933910888 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174807e070 con 0x55c554b784d0 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f1744c00640 1 -- 172.21.15.42:0/933910888 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17480a4c00 con 0x55c554b784d0 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.735+0000 7f1744c00640 1 -- 172.21.15.42:0/933910888 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17480850c0 con 0x55c554b784d0 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.736+0000 7f1759d94740 1 -- 172.21.15.42:0/933910888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 msgr2=0x55c554b747c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.736+0000 7f1759d94740 1 --2- 172.21.15.42:0/933910888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b747c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f174809b830 tx=0x7f174809a4a0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.736+0000 7f1759d94740 1 -- 172.21.15.42:0/933910888 shutdown_connections 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.736+0000 7f1759d94740 1 --2- 172.21.15.42:0/933910888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b747c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.736+0000 7f1759d94740 1 --2- 172.21.15.42:0/933910888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b73360 0x55c554b84120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.736+0000 7f1759d94740 1 --2- 172.21.15.42:0/933910888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b72970 0x55c554b6e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.740 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.736+0000 7f1759d94740 1 -- 172.21.15.42:0/933910888 >> 172.21.15.42:0/933910888 conn(0x55c554b6dfe0 msgr2=0x55c554b68d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.736+0000 7f1759d94740 1 -- 172.21.15.42:0/933910888 shutdown_connections 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.736+0000 7f1759d94740 1 -- 172.21.15.42:0/933910888 wait complete. 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.737+0000 7f1759d94740 1 -- start start 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.737+0000 7f1759d94740 1 Processor -- start 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.737+0000 7f1759d94740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 0x55c554b7f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.737+0000 7f1759d94740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.737+0000 7f1759d94740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b82750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.737+0000 7f1759d94740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c554b8aed0 con 0x55c554b784d0 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.737+0000 7f1759d94740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c554b8a800 con 0x55c554b72970 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.737+0000 7f1759d94740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c554b8b5a0 con 0x55c554b73360 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.737+0000 7f1759d94740 10 client.0 _ll_register_callbacks cb 0x7f1748004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:32950/0 (socket says 172.21.15.42:32950) 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174f400640 1 -- 172.21.15.42:0/4282083786 learned_addr learned my addr 172.21.15.42:0/4282083786 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.741 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174fe00640 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 0x55c554b7f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f1755000640 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b82750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174fe00640 1 -- 172.21.15.42:0/4282083786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 msgr2=0x55c554b7f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174fe00640 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174fe00640 1 -- 172.21.15.42:0/4282083786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 msgr2=0x55c554b82750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174fe00640 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b82750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174fe00640 1 -- 172.21.15.42:0/4282083786 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c554b82d90 con 0x55c554b72970 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174f400640 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f174fe00640 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 0x55c554b7f390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55c554b73760 tx=0x7f1740031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d149bf5d3c6535e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.738+0000 7f173a000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f173a000640 1 -- 172.21.15.42:0/4282083786 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1740035040 con 0x55c554b72970 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f173a000640 1 -- 172.21.15.42:0/4282083786 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1740005240 con 0x55c554b72970 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f1759d94740 1 -- 172.21.15.42:0/4282083786 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17480a4c00 con 0x55c554b72970 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f173a000640 1 -- 172.21.15.42:0/4282083786 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1740040040 con 0x55c554b72970 2024-04-25T00:18:39.742 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f1759d94740 10 client.25160 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f1759d94740 1 -- 172.21.15.42:0/4282083786 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55c554b84050 con 0x55c554b72970 2024-04-25T00:18:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f173a000640 1 -- 172.21.15.42:0/4282083786 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17480a4c00 con 0x55c554b72970 2024-04-25T00:18:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f173a000640 1 -- 172.21.15.42:0/4282083786 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f174004e170 con 0x55c554b72970 2024-04-25T00:18:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f1759d94740 10 client.25160 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f1759d94740 10 client.25160 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T00:18:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.739+0000 7f1759d94740 20 client.25160 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.740+0000 7f1759d94740 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55c554bf3c50 0x55c554bf2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.740+0000 7f1759d94740 4 client.25160 mds_command: new command op to 24568 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:18:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.740+0000 7f1759d94740 1 -- 172.21.15.42:0/4282083786 --> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] -- command(tid 0: {"prefix": "status"}) -- 0x55c554add680 con 0x55c554bf3c50 2024-04-25T00:18:39.743 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.740+0000 7f174f400640 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55c554bf3c50 0x55c554bf2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.740+0000 7f174f400640 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55c554bf3c50 0x55c554bf2960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=768289ec0e516406 server_cookie=8aae2162b16d67c4 in_seq=0 out_seq=0 2024-04-25T00:18:39.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.740+0000 7f173a000640 10 client.25160 ms_handle_connect on v2:172.21.15.42:6840/4068466248 2024-04-25T00:18:39.744 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f173a000640 1 -- 172.21.15.42:0/4282083786 <== mds.1 v2:172.21.15.42:6840/4068466248 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55c554add680 con 0x55c554bf3c50 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f173a000640 10 client.25160 handle_command_reply: tid=0 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 client.25160 shutdown 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 -- 172.21.15.42:0/4282083786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 msgr2=0x55c554b7f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 0x55c554b7f390 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55c554b73760 tx=0x7f1740031a20 comp rx=0 tx=0).stop 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 -- 172.21.15.42:0/4282083786 shutdown_connections 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.42:6840/4068466248,v1:172.21.15.42:6841/4068466248] conn(0x55c554bf3c50 0x55c554bf2960 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c554b784d0 0x55c554b82750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c554b73360 0x55c554b7f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 --2- 172.21.15.42:0/4282083786 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c554b72970 0x55c554b7f390 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 -- 172.21.15.42:0/4282083786 >> 172.21.15.42:0/4282083786 conn(0x55c554b6dfe0 msgr2=0x55c554b6e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f173a000640 0 client.25160 ms_handle_reset on v2:172.21.15.42:6840/4068466248 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 -- 172.21.15.42:0/4282083786 shutdown_connections 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 1 -- 172.21.15.42:0/4282083786 wait complete. 2024-04-25T00:18:39.745 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.741+0000 7f1759d94740 20 client.25160 trim_cache size 0 max 16384 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "9d95543e-0297-11ef-bc93-c7b262605968", 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 1, 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24568, 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 136.37889837700001, 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 23, 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 88, 2024-04-25T00:18:39.746 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T00:18:39.747 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 147.59495956800001 2024-04-25T00:18:39.747 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:18:39.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 -- 172.21.15.42:0/3575005140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f17340a2b90 msgr2=0x7f17340a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.747 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f17340a2b90 0x7f17340a5050 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x55c554b73bc0 tx=0x7f173c013040 comp rx=0 tx=0).stop 2024-04-25T00:18:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 -- 172.21.15.42:0/3575005140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1750077ce0 msgr2=0x7f17501a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1750077ce0 0x7f17501a5540 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f174800eb10 tx=0x7f174800c2d0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 -- 172.21.15.42:0/3575005140 shutdown_connections 2024-04-25T00:18:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f17340a2b90 0x7f17340a5050 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1750077ce0 0x7f17501a5540 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1750077340 0x7f175019e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 --2- 172.21.15.42:0/3575005140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1750076100 0x7f175019df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 -- 172.21.15.42:0/3575005140 >> 172.21.15.42:0/3575005140 conn(0x7f17500fedf0 msgr2=0x7f1750102150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 -- 172.21.15.42:0/3575005140 shutdown_connections 2024-04-25T00:18:39.748 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.742+0000 7f1755a00640 1 -- 172.21.15.42:0/3575005140 wait complete. 2024-04-25T00:18:39.763 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T00:18:39.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.976+0000 7f4963200640 1 -- 172.21.15.42:0/258675812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4964109830 msgr2=0x7f49641100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.976+0000 7f4963200640 1 --2- 172.21.15.42:0/258675812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4964109830 0x7f49641100c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f495c009ca0 tx=0x7f495c02a8b0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.976+0000 7f4963200640 1 -- 172.21.15.42:0/258675812 shutdown_connections 2024-04-25T00:18:39.976 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.976+0000 7f4963200640 1 --2- 172.21.15.42:0/258675812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4964109830 0x7f49641100c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.976+0000 7f4963200640 1 --2- 172.21.15.42:0/258675812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4964108e70 0x7f49641092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.976+0000 7f4963200640 1 --2- 172.21.15.42:0/258675812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4964107c80 0x7f49641080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.976+0000 7f4963200640 1 -- 172.21.15.42:0/258675812 >> 172.21.15.42:0/258675812 conn(0x7f4964076c10 msgr2=0x7f4964079050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.976+0000 7f4963200640 1 -- 172.21.15.42:0/258675812 shutdown_connections 2024-04-25T00:18:39.977 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.976+0000 7f4963200640 1 -- 172.21.15.42:0/258675812 wait complete. 2024-04-25T00:18:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.977+0000 7f4963200640 1 Processor -- start 2024-04-25T00:18:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.977+0000 7f4963200640 1 -- start start 2024-04-25T00:18:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.977+0000 7f4963200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4964107c80 0x7f496419dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.977+0000 7f4963200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4964108e70 0x7f496419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.977+0000 7f4963200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4964109830 0x7f49641a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.977+0000 7f4963200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4964113930 con 0x7f4964109830 2024-04-25T00:18:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.977+0000 7f4963200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49641137b0 con 0x7f4964107c80 2024-04-25T00:18:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.977+0000 7f4963200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4964113ab0 con 0x7f4964108e70 2024-04-25T00:18:39.979 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4962800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4964109830 0x7f49641a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4962800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4964109830 0x7f49641a5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52266/0 (socket says 172.21.15.42:52266) 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4962800640 1 -- 172.21.15.42:0/2184512425 learned_addr learned my addr 172.21.15.42:0/2184512425 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4961e00640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4964107c80 0x7f496419dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4961400640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4964108e70 0x7f496419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4962800640 1 -- 172.21.15.42:0/2184512425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4964108e70 msgr2=0x7f496419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4962800640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4964108e70 0x7f496419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4962800640 1 -- 172.21.15.42:0/2184512425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4964107c80 msgr2=0x7f496419dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4962800640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4964107c80 0x7f496419dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4962800640 1 -- 172.21.15.42:0/2184512425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49641a5ae0 con 0x7f4964109830 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4961400640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4964108e70 0x7f496419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.978+0000 7f4962800640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4964109830 0x7f49641a5470 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f495c030f60 tx=0x7f495c030f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.979+0000 7f495aa00640 1 -- 172.21.15.42:0/2184512425 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c004040 con 0x7f4964109830 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.979+0000 7f495aa00640 1 -- 172.21.15.42:0/2184512425 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c040070 con 0x7f4964109830 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.979+0000 7f495aa00640 1 -- 172.21.15.42:0/2184512425 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c045c90 con 0x7f4964109830 2024-04-25T00:18:39.980 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.979+0000 7f4963200640 1 -- 172.21.15.42:0/2184512425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49641a5d70 con 0x7f4964109830 2024-04-25T00:18:39.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.979+0000 7f4963200640 1 -- 172.21.15.42:0/2184512425 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49641a6280 con 0x7f4964109830 2024-04-25T00:18:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.982+0000 7f496bf95740 1 Processor -- start 2024-04-25T00:18:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.982+0000 7f495aa00640 1 -- 172.21.15.42:0/2184512425 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f495c0041e0 con 0x7f4964109830 2024-04-25T00:18:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.982+0000 7f496bf95740 1 -- start start 2024-04-25T00:18:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f496bf95740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a65ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f496bf95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 0x55cb1a663e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f496bf95740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a6644d0 0x55cb1a666960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f496bf95740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a55eed0 con 0x55cb1a65e970 2024-04-25T00:18:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f496bf95740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a5c1df0 con 0x55cb1a65f360 2024-04-25T00:18:39.983 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f496bf95740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a56a8d0 con 0x55cb1a6644d0 2024-04-25T00:18:39.984 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f4961400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a65ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f4961400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a65ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52274/0 (socket says 172.21.15.42:52274) 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f4961e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 0x55cb1a663e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f4961400640 1 -- 172.21.15.42:0/3695711101 learned_addr learned my addr 172.21.15.42:0/3695711101 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f4961400640 1 --2- 172.21.15.42:0/3695711101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a6644d0 0x55cb1a666960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f4961e00640 1 -- 172.21.15.42:0/3695711101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a6644d0 msgr2=0x55cb1a666960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f4961e00640 1 --2- 172.21.15.42:0/3695711101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a6644d0 0x55cb1a666960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.983+0000 7f4961e00640 1 -- 172.21.15.42:0/3695711101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 msgr2=0x55cb1a65ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.984+0000 7f4961e00640 1 --2- 172.21.15.42:0/3695711101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a65ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.984+0000 7f4961e00640 1 -- 172.21.15.42:0/3695711101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb1a2d62d0 con 0x55cb1a65f360 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.984+0000 7f4961e00640 1 --2- 172.21.15.42:0/3695711101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 0x55cb1a663e60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4954009bf0 tx=0x7f4954011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90296907dcb9d3bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.984+0000 7f495aa00640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49480a2c10 0x7f49480a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.984+0000 7f495aa00640 1 -- 172.21.15.42:0/2184512425 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f495c0f0420 con 0x7f4964109830 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.984+0000 7f4953400640 1 -- 172.21.15.42:0/3695711101 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954004050 con 0x55cb1a65f360 2024-04-25T00:18:39.985 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.984+0000 7f4961400640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49480a2c10 0x7f49480a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.984+0000 7f4953400640 1 -- 172.21.15.42:0/3695711101 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4954005500 con 0x55cb1a65f360 2024-04-25T00:18:39.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.984+0000 7f4953400640 1 -- 172.21.15.42:0/3695711101 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4954004de0 con 0x55cb1a65f360 2024-04-25T00:18:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.985+0000 7f496bf95740 1 -- 172.21.15.42:0/3695711101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 msgr2=0x55cb1a663e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.985+0000 7f4961400640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49480a2c10 0x7f49480a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55cb1a65fbc0 tx=0x7f494c009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.985+0000 7f496bf95740 1 --2- 172.21.15.42:0/3695711101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 0x55cb1a663e60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4954009bf0 tx=0x7f4954011810 comp rx=0 tx=0).stop 2024-04-25T00:18:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.985+0000 7f496bf95740 1 -- 172.21.15.42:0/3695711101 shutdown_connections 2024-04-25T00:18:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.985+0000 7f496bf95740 1 --2- 172.21.15.42:0/3695711101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a6644d0 0x55cb1a666960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.985+0000 7f496bf95740 1 --2- 172.21.15.42:0/3695711101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 0x55cb1a663e60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.985+0000 7f496bf95740 1 --2- 172.21.15.42:0/3695711101 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a65ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.985+0000 7f496bf95740 1 -- 172.21.15.42:0/3695711101 >> 172.21.15.42:0/3695711101 conn(0x55cb1a659fe0 msgr2=0x55cb1a65a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.985+0000 7f496bf95740 1 -- 172.21.15.42:0/3695711101 shutdown_connections 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.985+0000 7f496bf95740 1 -- 172.21.15.42:0/3695711101 wait complete. 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.986+0000 7f496bf95740 1 -- start start 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.986+0000 7f496bf95740 1 Processor -- start 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.986+0000 7f496bf95740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65e970 0x55cb1a668c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.986+0000 7f496bf95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 0x55cb1a669160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.986+0000 7f496bf95740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a6644d0 0x55cb1a6701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.986+0000 7f496bf95740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a57dd20 con 0x55cb1a6644d0 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.986+0000 7f496bf95740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a5bddb0 con 0x55cb1a65f360 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.986+0000 7f496bf95740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a667480 con 0x55cb1a65e970 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f496bf95740 10 client.0 _ll_register_callbacks cb 0x55cb1a5b8f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4962800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a6644d0 0x55cb1a6701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4962800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a6644d0 0x55cb1a6701a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52288/0 (socket says 172.21.15.42:52288) 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4962800640 1 -- 172.21.15.42:0/3553683363 learned_addr learned my addr 172.21.15.42:0/3553683363 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:39.990 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4961400640 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 0x55cb1a669160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4961e00640 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65e970 0x55cb1a668c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4962800640 1 -- 172.21.15.42:0/3553683363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65e970 msgr2=0x55cb1a668c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4962800640 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65e970 0x55cb1a668c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4962800640 1 -- 172.21.15.42:0/3553683363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 msgr2=0x55cb1a669160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4962800640 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 0x55cb1a669160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4962800640 1 -- 172.21.15.42:0/3553683363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb1a3a06b0 con 0x55cb1a6644d0 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4962800640 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a6644d0 0x55cb1a6701a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f495c0b90b0 tx=0x7f495c0bc680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59bc6a14dac0cbb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.987+0000 7f4947e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.988+0000 7f4947e00640 1 -- 172.21.15.42:0/3553683363 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c0d2070 con 0x55cb1a6644d0 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.988+0000 7f4947e00640 1 -- 172.21.15.42:0/3553683363 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f495c045470 con 0x55cb1a6644d0 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.988+0000 7f496bf95740 1 -- 172.21.15.42:0/3553683363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4954005500 con 0x55cb1a6644d0 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.988+0000 7f4947e00640 1 -- 172.21.15.42:0/3553683363 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f495c0c1000 con 0x55cb1a6644d0 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.988+0000 7f496bf95740 10 client.15069 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:39.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.988+0000 7f496bf95740 1 -- 172.21.15.42:0/3553683363 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55cb1a673510 con 0x55cb1a6644d0 2024-04-25T00:18:39.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.988+0000 7f4947e00640 1 -- 172.21.15.42:0/3553683363 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4954005500 con 0x55cb1a6644d0 2024-04-25T00:18:39.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.988+0000 7f4947e00640 1 -- 172.21.15.42:0/3553683363 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f495c0c5430 con 0x55cb1a6644d0 2024-04-25T00:18:39.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.989+0000 7f496bf95740 10 client.15069 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:39.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.989+0000 7f496bf95740 10 client.15069 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:18:39.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.989+0000 7f496bf95740 20 client.15069 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:39.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.989+0000 7f496bf95740 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cb1a675c70 0x55cb1a6770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:39.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.989+0000 7f496bf95740 4 client.15069 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:18:39.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.989+0000 7f496bf95740 1 -- 172.21.15.42:0/3553683363 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb1a5cf330 con 0x55cb1a675c70 2024-04-25T00:18:39.992 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.989+0000 7f4961e00640 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cb1a675c70 0x55cb1a6770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.990+0000 7f4961e00640 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cb1a675c70 0x55cb1a6770e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0bb7ffe68b5b213 server_cookie=7f5fc7165491dae7 in_seq=0 out_seq=0 2024-04-25T00:18:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.990+0000 7f4947e00640 10 client.15069 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:18:39.993 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.990+0000 7f4947e00640 1 -- 172.21.15.42:0/3553683363 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb1a5cf330 con 0x55cb1a675c70 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.990+0000 7f4947e00640 10 client.15069 handle_command_reply: tid=0 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.990+0000 7f496bf95740 1 client.15069 shutdown 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 1 -- 172.21.15.42:0/3553683363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a6644d0 msgr2=0x55cb1a6701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a6644d0 0x55cb1a6701a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f495c0b90b0 tx=0x7f495c0bc680 comp rx=0 tx=0).stop 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 1 -- 172.21.15.42:0/3553683363 shutdown_connections 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cb1a675c70 0x55cb1a6770e0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a6644d0 0x55cb1a6701a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a65f360 0x55cb1a669160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 1 --2- 172.21.15.42:0/3553683363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65e970 0x55cb1a668c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 1 -- 172.21.15.42:0/3553683363 >> 172.21.15.42:0/3553683363 conn(0x55cb1a659fe0 msgr2=0x55cb1a659120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f4947e00640 0 client.15069 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 1 -- 172.21.15.42:0/3553683363 shutdown_connections 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 1 -- 172.21.15.42:0/3553683363 wait complete. 2024-04-25T00:18:39.994 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:39.991+0000 7f496bf95740 20 client.15069 trim_cache size 0 max 16384 2024-04-25T00:18:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.011+0000 7f496bf95740 1 Processor -- start 2024-04-25T00:18:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.011+0000 7f496bf95740 1 -- start start 2024-04-25T00:18:40.011 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.011+0000 7f496bf95740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65e970 0x55cb1a65fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:40.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.011+0000 7f496bf95740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65f360 0x55cb1a670160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:40.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.011+0000 7f496bf95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 0x55cb1a6668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:40.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.011+0000 7f496bf95740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a675420 con 0x55cb1a65f360 2024-04-25T00:18:40.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.011+0000 7f496bf95740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a675af0 con 0x55cb1a6644d0 2024-04-25T00:18:40.012 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.011+0000 7f496bf95740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a6761c0 con 0x55cb1a65e970 2024-04-25T00:18:40.013 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.012+0000 7f4961400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65f360 0x55cb1a670160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.012+0000 7f4961400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65f360 0x55cb1a670160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52296/0 (socket says 172.21.15.42:52296) 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.012+0000 7f4961400640 1 -- 172.21.15.42:0/835497983 learned_addr learned my addr 172.21.15.42:0/835497983 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.012+0000 7f4962800640 1 --2- 172.21.15.42:0/835497983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 0x55cb1a6668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.012+0000 7f4961e00640 1 --2- 172.21.15.42:0/835497983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65e970 0x55cb1a65fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.012+0000 7f4962800640 1 -- 172.21.15.42:0/835497983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65e970 msgr2=0x55cb1a65fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.012+0000 7f4962800640 1 --2- 172.21.15.42:0/835497983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65e970 0x55cb1a65fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.012+0000 7f4962800640 1 -- 172.21.15.42:0/835497983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65f360 msgr2=0x55cb1a670160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.012+0000 7f4962800640 1 --2- 172.21.15.42:0/835497983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65f360 0x55cb1a670160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.012+0000 7f4962800640 1 -- 172.21.15.42:0/835497983 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb1a658620 con 0x55cb1a6644d0 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.013+0000 7f4962800640 1 --2- 172.21.15.42:0/835497983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 0x55cb1a6668c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f495c0b95d0 tx=0x7f495c0bc790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1ba7d5e2e120a41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.013+0000 7f4952a00640 1 -- 172.21.15.42:0/835497983 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c0d2070 con 0x55cb1a6644d0 2024-04-25T00:18:40.014 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.013+0000 7f4952a00640 1 -- 172.21.15.42:0/835497983 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c0d5e50 con 0x55cb1a6644d0 2024-04-25T00:18:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.013+0000 7f4952a00640 1 -- 172.21.15.42:0/835497983 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495c0bc0a0 con 0x55cb1a6644d0 2024-04-25T00:18:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.013+0000 7f496bf95740 1 -- 172.21.15.42:0/835497983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 msgr2=0x55cb1a6668c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.013+0000 7f496bf95740 1 --2- 172.21.15.42:0/835497983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 0x55cb1a6668c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f495c0b95d0 tx=0x7f495c0bc790 comp rx=0 tx=0).stop 2024-04-25T00:18:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.014+0000 7f496bf95740 1 -- 172.21.15.42:0/835497983 shutdown_connections 2024-04-25T00:18:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.014+0000 7f496bf95740 1 --2- 172.21.15.42:0/835497983 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 0x55cb1a6668c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.014+0000 7f496bf95740 1 --2- 172.21.15.42:0/835497983 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65f360 0x55cb1a670160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.014+0000 7f496bf95740 1 --2- 172.21.15.42:0/835497983 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65e970 0x55cb1a65fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.014+0000 7f496bf95740 1 -- 172.21.15.42:0/835497983 >> 172.21.15.42:0/835497983 conn(0x55cb1a659fe0 msgr2=0x55cb1a65a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.014+0000 7f496bf95740 1 -- 172.21.15.42:0/835497983 shutdown_connections 2024-04-25T00:18:40.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.014+0000 7f496bf95740 1 -- 172.21.15.42:0/835497983 wait complete. 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.014+0000 7f496bf95740 1 -- start start 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.014+0000 7f496bf95740 1 Processor -- start 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f496bf95740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a66e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f496bf95740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65f360 0x55cb1a66e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f496bf95740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 0x55cb1a66ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f496bf95740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a676f60 con 0x55cb1a65e970 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f496bf95740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a676890 con 0x55cb1a6644d0 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f496bf95740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cb1a677630 con 0x55cb1a65f360 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f496bf95740 10 client.0 _ll_register_callbacks cb 0x7f495c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f4961e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a66e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f4961e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a66e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:52302/0 (socket says 172.21.15.42:52302) 2024-04-25T00:18:40.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f4961e00640 1 -- 172.21.15.42:0/2419794779 learned_addr learned my addr 172.21.15.42:0/2419794779 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f4961400640 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65f360 0x55cb1a66e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f4962800640 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 0x55cb1a66ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f4961e00640 1 -- 172.21.15.42:0/2419794779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65f360 msgr2=0x55cb1a66e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f4961e00640 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65f360 0x55cb1a66e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f4961e00640 1 -- 172.21.15.42:0/2419794779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 msgr2=0x55cb1a66ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f4961e00640 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 0x55cb1a66ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.015+0000 7f4961e00640 1 -- 172.21.15.42:0/2419794779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb1a66f5a0 con 0x55cb1a65e970 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.016+0000 7f4961e00640 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a66e2b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4954000c00 tx=0x7f4954005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=868e96595c29564a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.016+0000 7f4947e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.016+0000 7f4947e00640 1 -- 172.21.15.42:0/2419794779 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495402de60 con 0x55cb1a65e970 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.016+0000 7f4947e00640 1 -- 172.21.15.42:0/2419794779 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f495402d5e0 con 0x55cb1a65e970 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.016+0000 7f496bf95740 1 -- 172.21.15.42:0/2419794779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f495c0d5e50 con 0x55cb1a65e970 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.016+0000 7f4947e00640 1 -- 172.21.15.42:0/2419794779 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f495402d820 con 0x55cb1a65e970 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.016+0000 7f496bf95740 10 client.15075 fetch_fsmap learned FSMap version 23 2024-04-25T00:18:40.020 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.016+0000 7f496bf95740 1 -- 172.21.15.42:0/2419794779 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) -- 0x55cb1a6dbde0 con 0x55cb1a65e970 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.017+0000 7f4947e00640 1 -- 172.21.15.42:0/2419794779 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f495c0d5e50 con 0x55cb1a65e970 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.017+0000 7f4947e00640 1 -- 172.21.15.42:0/2419794779 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 23) ==== 8338+0+0 (secure 0 0 0) 0x7f4954003b70 con 0x55cb1a65e970 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.017+0000 7f496bf95740 10 client.15075 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.017+0000 7f496bf95740 10 client.15075 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.017+0000 7f496bf95740 20 client.15075 populate_metadata read hostname 'smithi042' 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.017+0000 7f496bf95740 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cb1a6e07c0 0x55cb1a6dcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.017+0000 7f496bf95740 4 client.15075 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.017+0000 7f496bf95740 1 -- 172.21.15.42:0/2419794779 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "status"}) -- 0x55cb1a5c9680 con 0x55cb1a6e07c0 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.017+0000 7f4961400640 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cb1a6e07c0 0x55cb1a6dcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f4961400640 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cb1a6e07c0 0x55cb1a6dcc00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4d28350fe0f2f15 server_cookie=e5122c6c840019f8 in_seq=0 out_seq=0 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f4947e00640 10 client.15075 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f4947e00640 1 -- 172.21.15.42:0/2419794779 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cb1a5c9680 con 0x55cb1a6e07c0 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f4947e00640 10 client.15075 handle_command_reply: tid=0 2024-04-25T00:18:40.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 client.15075 shutdown 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 -- 172.21.15.42:0/2419794779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 msgr2=0x55cb1a66e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a66e2b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4954000c00 tx=0x7f4954005ce0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 -- 172.21.15.42:0/2419794779 shutdown_connections 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cb1a6e07c0 0x55cb1a6dcc00 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cb1a6644d0 0x55cb1a66ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb1a65f360 0x55cb1a66e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 --2- 172.21.15.42:0/2419794779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cb1a65e970 0x55cb1a66e2b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f4947e00640 0 client.15075 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 -- 172.21.15.42:0/2419794779 >> 172.21.15.42:0/2419794779 conn(0x55cb1a659fe0 msgr2=0x55cb1a654d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 -- 172.21.15.42:0/2419794779 shutdown_connections 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.018+0000 7f496bf95740 1 -- 172.21.15.42:0/2419794779 wait complete. 2024-04-25T00:18:40.022 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f496bf95740 20 client.15075 trim_cache size 0 max 16384 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "cluster_fsid": "9d95543e-0297-11ef-bc93-c7b262605968", 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "whoami": 0, 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "id": 24569, 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "want_state": "up:active", 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "state": "up:active", 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "fs_name": "cephfs", 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "rank_uptime": 139.699787391, 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "mdsmap_epoch": 23, 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch": 88, 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_epoch_barrier": 86, 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout: "uptime": 145.14820178299999 2024-04-25T00:18:40.023 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:18:40.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 -- 172.21.15.42:0/2184512425 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49480a2c10 msgr2=0x7f49480a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:40.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49480a2c10 0x7f49480a50d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x55cb1a65fbc0 tx=0x7f494c009570 comp rx=0 tx=0).stop 2024-04-25T00:18:40.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 -- 172.21.15.42:0/2184512425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4964109830 msgr2=0x7f49641a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:40.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4964109830 0x7f49641a5470 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f495c030f60 tx=0x7f495c030f90 comp rx=0 tx=0).stop 2024-04-25T00:18:40.024 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 -- 172.21.15.42:0/2184512425 shutdown_connections 2024-04-25T00:18:40.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f49480a2c10 0x7f49480a50d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4964109830 0x7f49641a5470 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4964108e70 0x7f496419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 --2- 172.21.15.42:0/2184512425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4964107c80 0x7f496419dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:40.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 -- 172.21.15.42:0/2184512425 >> 172.21.15.42:0/2184512425 conn(0x7f4964076c10 msgr2=0x7f496410c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:40.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 -- 172.21.15.42:0/2184512425 shutdown_connections 2024-04-25T00:18:40.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:40.019+0000 7f4963200640 1 -- 172.21.15.42:0/2184512425 wait complete. 2024-04-25T00:18:40.037 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T00:18:40.047 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T00:18:40.047 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T00:18:40.047 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T00:18:40.047 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T00:18:40.085 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T00:18:40.085 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T00:18:40.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:40.086 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-04-25T00:18:40.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:40.148 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-04-25T00:18:40.241 INFO:teuthology.orchestra.run.smithi042.stderr:Cannot find device "ceph-brx" 2024-04-25T00:18:40.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T00:18:40.246 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-25T00:18:40.246 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T00:18:40.246 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:18:40.246 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T00:18:40.318 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:40 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/212834497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:40.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:40 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/212834497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:40.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:40 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/212834497' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:18:40.711 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:18:41.386 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:41 smithi042 ceph-mon[28130]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:18:41.414 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:41 smithi136 ceph-mon[32923]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:18:41.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:41 smithi106 ceph-mon[32510]: pgmap v385: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T00:18:43.161 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:43 smithi042 ceph-mon[28130]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:18:43.244 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.243+0000 7f369e400700 1 -- 172.21.15.42:0/4241144972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3698077080 msgr2=0x7f3698112aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.243+0000 7f369e400700 1 --2- 172.21.15.42:0/4241144972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3698077080 0x7f3698112aa0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f36840098f0 tx=0x7f3684009c00 comp rx=0 tx=0).stop 2024-04-25T00:18:43.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.243+0000 7f369e400700 1 -- 172.21.15.42:0/4241144972 shutdown_connections 2024-04-25T00:18:43.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.243+0000 7f369e400700 1 --2- 172.21.15.42:0/4241144972 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3698112fe0 0x7f36981153d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.243+0000 7f369e400700 1 --2- 172.21.15.42:0/4241144972 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3698076760 0x7f3698076b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.243+0000 7f369e400700 1 --2- 172.21.15.42:0/4241144972 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3698077080 0x7f3698112aa0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.243+0000 7f369e400700 1 -- 172.21.15.42:0/4241144972 >> 172.21.15.42:0/4241144972 conn(0x7f36980ffe10 msgr2=0x7f3698102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:43.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.243+0000 7f369e400700 1 -- 172.21.15.42:0/4241144972 shutdown_connections 2024-04-25T00:18:43.245 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.243+0000 7f369e400700 1 -- 172.21.15.42:0/4241144972 wait complete. 2024-04-25T00:18:43.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.244+0000 7f369e400700 1 Processor -- start 2024-04-25T00:18:43.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.244+0000 7f369e400700 1 -- start start 2024-04-25T00:18:43.246 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f369e400700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3698076760 0x7f369819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f369e400700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3698077080 0x7f369819f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f369e400700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3698112fe0 0x7f36981a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f369e400700 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3698118af0 con 0x7f3698112fe0 2024-04-25T00:18:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f369e400700 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3698118970 con 0x7f3698077080 2024-04-25T00:18:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f369e400700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3698118c70 con 0x7f3698076760 2024-04-25T00:18:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f369da00700 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3698112fe0 0x7f36981a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f369d000700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3698076760 0x7f369819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f3697e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3698077080 0x7f369819f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f3697e00700 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3698077080 0x7f369819f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53766/0 (socket says 172.21.15.42:53766) 2024-04-25T00:18:43.247 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.245+0000 7f3697e00700 1 -- 172.21.15.42:0/303612482 learned_addr learned my addr 172.21.15.42:0/303612482 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:43.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f3697e00700 1 -- 172.21.15.42:0/303612482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3698076760 msgr2=0x7f369819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f3697e00700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3698076760 0x7f369819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.248 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f3697e00700 1 -- 172.21.15.42:0/303612482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3698112fe0 msgr2=0x7f36981a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f3697e00700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3698112fe0 0x7f36981a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f3697e00700 1 -- 172.21.15.42:0/303612482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36981a3fa0 con 0x7f3698077080 2024-04-25T00:18:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f3697e00700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3698077080 0x7f369819f520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3684005950 tx=0x7f368400fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f3695600700 1 -- 172.21.15.42:0/303612482 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684005140 con 0x7f3698077080 2024-04-25T00:18:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f3695600700 1 -- 172.21.15.42:0/303612482 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f368400b560 con 0x7f3698077080 2024-04-25T00:18:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f369e400700 1 -- 172.21.15.42:0/303612482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36981a4230 con 0x7f3698077080 2024-04-25T00:18:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f3695600700 1 -- 172.21.15.42:0/303612482 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684003ae0 con 0x7f3698077080 2024-04-25T00:18:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.246+0000 7f369e400700 1 -- 172.21.15.42:0/303612482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36981abb10 con 0x7f3698077080 2024-04-25T00:18:43.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.248+0000 7f369e400700 1 -- 172.21.15.42:0/303612482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3698068f20 con 0x7f3698077080 2024-04-25T00:18:43.250 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.249+0000 7f3695600700 1 -- 172.21.15.42:0/303612482 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f368400ba80 con 0x7f3698077080 2024-04-25T00:18:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.251+0000 7f3695600700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36800a2ca0 0x7f36800a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.251+0000 7f3695600700 1 -- 172.21.15.42:0/303612482 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f36840cff10 con 0x7f3698077080 2024-04-25T00:18:43.252 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.251+0000 7f369d000700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36800a2ca0 0x7f36800a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:43.253 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.252+0000 7f369d000700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36800a2ca0 0x7f36800a5160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f368c009c60 tx=0x7f368c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:43.256 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.255+0000 7f3695600700 1 -- 172.21.15.42:0/303612482 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3684013030 con 0x7f3698077080 2024-04-25T00:18:43.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:43 smithi136 ceph-mon[32923]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:18:43.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.535+0000 7f369e400700 1 -- 172.21.15.42:0/303612482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f36981a02f0 con 0x7f3698077080 2024-04-25T00:18:43.537 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.536+0000 7f3695600700 1 -- 172.21.15.42:0/303612482 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) ==== 53+0+73 (secure 0 0 0) 0x7f368409ced0 con 0x7f3698077080 2024-04-25T00:18:43.538 INFO:teuthology.orchestra.run.smithi042.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T00:18:43.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.540+0000 7f369e400700 1 -- 172.21.15.42:0/303612482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36800a2ca0 msgr2=0x7f36800a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.540+0000 7f369e400700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36800a2ca0 0x7f36800a5160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f368c009c60 tx=0x7f368c009260 comp rx=0 tx=0).stop 2024-04-25T00:18:43.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.540+0000 7f369e400700 1 -- 172.21.15.42:0/303612482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3698077080 msgr2=0x7f369819f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:43.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.540+0000 7f369e400700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3698077080 0x7f369819f520 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3684005950 tx=0x7f368400fb20 comp rx=0 tx=0).stop 2024-04-25T00:18:43.542 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.541+0000 7f369e400700 1 -- 172.21.15.42:0/303612482 shutdown_connections 2024-04-25T00:18:43.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.541+0000 7f369e400700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36800a2ca0 0x7f36800a5160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.541+0000 7f369e400700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3698112fe0 0x7f36981a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.541+0000 7f369e400700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3698076760 0x7f369819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.541+0000 7f369e400700 1 --2- 172.21.15.42:0/303612482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3698077080 0x7f369819f520 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:43.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.541+0000 7f369e400700 1 -- 172.21.15.42:0/303612482 >> 172.21.15.42:0/303612482 conn(0x7f36980ffe10 msgr2=0x7f3698102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:43.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.541+0000 7f369e400700 1 -- 172.21.15.42:0/303612482 shutdown_connections 2024-04-25T00:18:43.543 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:43.541+0000 7f369e400700 1 -- 172.21.15.42:0/303612482 wait complete. 2024-04-25T00:18:43.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:43 smithi106 ceph-mon[32510]: pgmap v386: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T00:18:44.295 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:44 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/303612482' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:18:44.352 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T00:18:44.353 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T00:18:44.353 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi042.front.sepia.ceph.com 2024-04-25T00:18:44.353 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T00:18:44.353 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T00:18:44.353 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T00:18:44.353 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T00:18:44.353 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T00:18:44.353 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:18:44.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:44.353 DEBUG:teuthology.orchestra.run.smithi042:> ip addr 2024-04-25T00:18:44.407 INFO:teuthology.orchestra.run.smithi042.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T00:18:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T00:18:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T00:18:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:18:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 ::1/128 scope host 2024-04-25T00:18:44.407 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0e brd ff:ff:ff:ff:ff:ff 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f0 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:6b:fd:0f brd ff:ff:ff:ff:ff:ff 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp7s0f1 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:40 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f0 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: inet 172.21.15.42/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft 41992sec preferred_lft 41992sec 2024-04-25T00:18:44.408 INFO:teuthology.orchestra.run.smithi042.stdout: inet6 fe80::ec4:7aff:febd:1440/64 scope link noprefixroute 2024-04-25T00:18:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: valid_lft forever preferred_lft forever 2024-04-25T00:18:44.409 INFO:teuthology.orchestra.run.smithi042.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T00:18:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: link/ether 0c:c4:7a:bd:14:41 brd ff:ff:ff:ff:ff:ff 2024-04-25T00:18:44.409 INFO:teuthology.orchestra.run.smithi042.stdout: altname enp1s0f1 2024-04-25T00:18:44.409 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T00:18:44.409 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:18:44.410 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-04-25T00:18:44.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add name ceph-brx type bridge 2024-04-25T00:18:44.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr flush dev ceph-brx 2024-04-25T00:18:44.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set ceph-brx up 2024-04-25T00:18:44.410 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T00:18:44.410 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-04-25T00:18:44.508 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:18:44.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:44 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/303612482' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:18:44.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:44 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/303612482' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T00:18:44.889 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:18:44.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:44.902 DEBUG:teuthology.orchestra.run.smithi042:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T00:18:44.966 INFO:teuthology.orchestra.run.smithi042.stdout:1 2024-04-25T00:18:44.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:44.970 DEBUG:teuthology.orchestra.run.smithi042:> route 2024-04-25T00:18:45.023 INFO:teuthology.orchestra.run.smithi042.stdout:Kernel IP routing table 2024-04-25T00:18:45.024 INFO:teuthology.orchestra.run.smithi042.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T00:18:45.024 INFO:teuthology.orchestra.run.smithi042.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T00:18:45.024 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T00:18:45.024 INFO:teuthology.orchestra.run.smithi042.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T00:18:45.024 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:18:45.024 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-04-25T00:18:45.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T00:18:45.024 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T00:18:45.025 DEBUG:teuthology.orchestra.run.smithi042:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T00:18:45.025 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-04-25T00:18:45.123 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:18:45.403 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:45 smithi042 ceph-mon[28130]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:45 smithi136 ceph-mon[32923]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:45 smithi106 ceph-mon[32510]: pgmap v387: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T00:18:45.802 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:18:45.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:45.812 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-04-25T00:18:45.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:45.865 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list-id 2024-04-25T00:18:45.918 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:18:45.918 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-04-25T00:18:45.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:18:45.918 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T00:18:45.918 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-04-25T00:18:46.019 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:18:46.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:18:46.104 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T00:18:46.104 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:18:46.104 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-04-25T00:18:46.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T00:18:46.104 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T00:18:46.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T00:18:46.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T00:18:46.105 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T00:18:46.105 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-04-25T00:18:46.216 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:18:46.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:18:46.425 DEBUG:teuthology.orchestra.run.smithi042:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T00:18:46.425 DEBUG:teuthology.orchestra.run.smithi042:> set -e 2024-04-25T00:18:46.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set brx.0 up 2024-04-25T00:18:46.425 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T00:18:46.425 DEBUG:teuthology.orchestra.run.smithi042:> ') 2024-04-25T00:18:46.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:18:46.631 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:18:46.641 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:18:46.642 DEBUG:teuthology.orchestra.run.smithi042:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T00:18:46.691 INFO:teuthology.orchestra.run.smithi042.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T00:18:46.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T00:18:46.691 DEBUG:teuthology.orchestra.run.smithi042:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:18:46.740 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T00:18:46.740 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T00:18:46.741 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/66188292-0dcc-4827-bc9f-81c849b8381e 2024-04-25T00:18:46.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:46.741 DEBUG:teuthology.orchestra.run.smithi042:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/66188292-0dcc-4827-bc9f-81c849b8381e /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-25T00:18:46.878 INFO:teuthology.orchestra.run.smithi042.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-25T00:18:46.878 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-04-25T00:18:46.878 INFO:teuthology.orchestra.run.smithi042.stdout:invalid new device string format 2024-04-25T00:18:47.352 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:47 smithi042 ceph-mon[28130]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:47.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:47 smithi136 ceph-mon[32923]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:47.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:47 smithi106 ceph-mon[32510]: pgmap v388: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T00:18:47.620 INFO:teuthology.orchestra.run.smithi042.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-04-25T00:18:47.620 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-04-25T00:18:47.621 INFO:teuthology.orchestra.run.smithi042.stdout:invalid new device string format 2024-04-25T00:18:47.621 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: resolved to: "172.21.15.42:6789,172.21.15.106:6789,172.21.15.136:6789" 2024-04-25T00:18:47.621 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.42:6789,172.21.15.106:6789,172.21.15.136:6789:/volumes/qa/sv_1/66188292-0dcc-4827-bc9f-81c849b8381e 2024-04-25T00:18:47.621 INFO:teuthology.orchestra.run.smithi042.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=9d95543e-0297-11ef-bc93-c7b262605968" will pass to kernel 2024-04-25T00:18:47.624 INFO:teuthology.orchestra.run.smithi042.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T00:18:47.624 INFO:teuthology.orchestra.run.smithi042.stdout: You just mounted a file system that supports labels which does not 2024-04-25T00:18:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T00:18:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T00:18:47.625 INFO:teuthology.orchestra.run.smithi042.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T00:18:47.628 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T00:18:47.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:18:47.628 DEBUG:teuthology.orchestra.run.smithi042:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T00:18:47.717 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:18:47.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:18:47.837 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:18:47.846 INFO:teuthology.orchestra.run.smithi042.stdout:client25189 2024-04-25T00:18:47.846 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:18:47.846 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:18:47.903 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:18:48.015 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:18:48.021 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:18:48.031 INFO:teuthology.orchestra.run.smithi042.stdout:client25189 2024-04-25T00:18:48.032 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T00:18:48.144 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T00:18:48.150 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:18:48.159 INFO:teuthology.orchestra.run.smithi042.stdout:client25189 2024-04-25T00:18:48.160 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/9d95543e-0297-11ef-bc93-c7b262605968.client25189/status') 2024-04-25T00:18:48.259 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T00:18:48.290 INFO:teuthology.orchestra.run.smithi042.stdout:instance: client.25189 (3)172.21.15.42:0/306101293 2024-04-25T00:18:48.290 INFO:teuthology.orchestra.run.smithi042.stdout:blocklisted: false 2024-04-25T00:18:48.290 INFO:teuthology.orchestra.run.smithi042.stderr:0+1 records in 2024-04-25T00:18:48.290 INFO:teuthology.orchestra.run.smithi042.stderr:0+1 records out 2024-04-25T00:18:48.290 INFO:teuthology.orchestra.run.smithi042.stderr:70 bytes copied, 0.000106916 s, 655 kB/s 2024-04-25T00:18:48.293 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T00:18:48.303 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T00:18:48.314 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T00:18:48.314 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-04-25T00:18:48.314 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-25T00:18:48.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.524+0000 7f3cba000640 1 -- 172.21.15.42:0/3954175613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cb4103800 msgr2=0x7f3cb4103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:48.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.524+0000 7f3cba000640 1 --2- 172.21.15.42:0/3954175613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cb4103800 0x7f3cb4103c20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8009900 tx=0x7f3ca802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:18:48.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.524+0000 7f3cba000640 1 -- 172.21.15.42:0/3954175613 shutdown_connections 2024-04-25T00:18:48.525 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.524+0000 7f3cba000640 1 --2- 172.21.15.42:0/3954175613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb41053b0 0x7f3cb410bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.524+0000 7f3cba000640 1 --2- 172.21.15.42:0/3954175613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb41049f0 0x7f3cb4104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.524+0000 7f3cba000640 1 --2- 172.21.15.42:0/3954175613 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cb4103800 0x7f3cb4103c20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.524+0000 7f3cba000640 1 -- 172.21.15.42:0/3954175613 >> 172.21.15.42:0/3954175613 conn(0x7f3cb40fedb0 msgr2=0x7f3cb41011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.524+0000 7f3cba000640 1 -- 172.21.15.42:0/3954175613 shutdown_connections 2024-04-25T00:18:48.526 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.524+0000 7f3cba000640 1 -- 172.21.15.42:0/3954175613 wait complete. 2024-04-25T00:18:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.525+0000 7f3cba000640 1 Processor -- start 2024-04-25T00:18:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.525+0000 7f3cba000640 1 -- start start 2024-04-25T00:18:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.525+0000 7f3cba000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb4103800 0x7f3cb4199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.525+0000 7f3cba000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cb41049f0 0x7f3cb419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.525+0000 7f3cba000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb41053b0 0x7f3cb41a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.525+0000 7f3cba000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb410f4c0 con 0x7f3cb41049f0 2024-04-25T00:18:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.525+0000 7f3cba000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb410f340 con 0x7f3cb41053b0 2024-04-25T00:18:48.527 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.525+0000 7f3cba000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb410f640 con 0x7f3cb4103800 2024-04-25T00:18:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cb41049f0 0x7f3cb419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb41053b0 0x7f3cb41a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb8c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb4103800 0x7f3cb4199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cb41049f0 0x7f3cb419a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43826/0 (socket says 172.21.15.42:43826) 2024-04-25T00:18:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb3e00640 1 -- 172.21.15.42:0/1069733568 learned_addr learned my addr 172.21.15.42:0/1069733568 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:18:48.528 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb41053b0 0x7f3cb41a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60440/0 (socket says 172.21.15.42:60440) 2024-04-25T00:18:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb3e00640 1 -- 172.21.15.42:0/1069733568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb4103800 msgr2=0x7f3cb4199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb3e00640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb4103800 0x7f3cb4199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb3e00640 1 -- 172.21.15.42:0/1069733568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb41053b0 msgr2=0x7f3cb41a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb3e00640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb41053b0 0x7f3cb41a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb3e00640 1 -- 172.21.15.42:0/1069733568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb41a1720 con 0x7f3cb41049f0 2024-04-25T00:18:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.526+0000 7f3cb9600640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb41053b0 0x7f3cb41a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:18:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.527+0000 7f3cb3e00640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cb41049f0 0x7f3cb419a040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000e9f0 tx=0x7f3ca000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:48.529 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.527+0000 7f3cb1600640 1 -- 172.21.15.42:0/1069733568 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000cd80 con 0x7f3cb41049f0 2024-04-25T00:18:48.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.527+0000 7f3cb1600640 1 -- 172.21.15.42:0/1069733568 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3ca0005500 con 0x7f3cb41049f0 2024-04-25T00:18:48.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.527+0000 7f3cba000640 1 -- 172.21.15.42:0/1069733568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb41a1a10 con 0x7f3cb41049f0 2024-04-25T00:18:48.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.527+0000 7f3cb1600640 1 -- 172.21.15.42:0/1069733568 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0010690 con 0x7f3cb41049f0 2024-04-25T00:18:48.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.527+0000 7f3cba000640 1 -- 172.21.15.42:0/1069733568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb41a1f60 con 0x7f3cb41049f0 2024-04-25T00:18:48.530 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.528+0000 7f3cba000640 1 -- 172.21.15.42:0/1069733568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb410ab20 con 0x7f3cb41049f0 2024-04-25T00:18:48.532 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.531+0000 7f3cb1600640 1 -- 172.21.15.42:0/1069733568 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ca00046c0 con 0x7f3cb41049f0 2024-04-25T00:18:48.534 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.533+0000 7f3cb1600640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3c9c0a2c00 0x7f3c9c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:18:48.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.534+0000 7f3cb1600640 1 -- 172.21.15.42:0/1069733568 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f3ca00cf3f0 con 0x7f3cb41049f0 2024-04-25T00:18:48.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.534+0000 7f3cb8c00640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3c9c0a2c00 0x7f3c9c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:18:48.535 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.534+0000 7f3cb8c00640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3c9c0a2c00 0x7f3c9c0a50c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ca802a9e0 tx=0x7f3ca80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:18:48.539 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.538+0000 7f3cb1600640 1 -- 172.21.15.42:0/1069733568 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f3ca009c320 con 0x7f3cb41049f0 2024-04-25T00:18:48.791 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:48.789+0000 7f3cba000640 1 -- 172.21.15.42:0/1069733568 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f3cb4062340 con 0x7f3cb41049f0 2024-04-25T00:18:49.162 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.161+0000 7f3cb1600640 1 -- 172.21.15.42:0/1069733568 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v24) ==== 117+0+0 (secure 0 0 0) 0x7f3ca0086070 con 0x7f3cb41049f0 2024-04-25T00:18:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 -- 172.21.15.42:0/1069733568 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3c9c0a2c00 msgr2=0x7f3c9c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3c9c0a2c00 0x7f3c9c0a50c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3ca802a9e0 tx=0x7f3ca80315b0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 -- 172.21.15.42:0/1069733568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cb41049f0 msgr2=0x7f3cb419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:18:49.166 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cb41049f0 0x7f3cb419a040 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000e9f0 tx=0x7f3ca000edf0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 -- 172.21.15.42:0/1069733568 shutdown_connections 2024-04-25T00:18:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f3c9c0a2c00 0x7f3c9c0a50c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb41053b0 0x7f3cb41a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3cb41049f0 0x7f3cb419a040 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 --2- 172.21.15.42:0/1069733568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3cb4103800 0x7f3cb4199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:18:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 -- 172.21.15.42:0/1069733568 >> 172.21.15.42:0/1069733568 conn(0x7f3cb40fedb0 msgr2=0x7f3cb4107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:18:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 -- 172.21.15.42:0/1069733568 shutdown_connections 2024-04-25T00:18:49.167 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:18:49.165+0000 7f3cba000640 1 -- 172.21.15.42:0/1069733568 wait complete. 2024-04-25T00:18:49.188 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T00:18:49.199 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T00:18:49.207 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi042.front.sepia.ceph.com 2024-04-25T00:18:49.207 DEBUG:teuthology.orchestra.run.smithi042:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T00:18:49.213 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T00:18:49.224 INFO:tasks.workunit:Pulling workunits from ref d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:18:49.225 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T00:18:49.225 INFO:tasks.workunit:timeout=3h 2024-04-25T00:18:49.225 INFO:tasks.workunit:cleanup=True 2024-04-25T00:18:49.225 DEBUG:teuthology.orchestra.run.smithi042:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T00:18:49.275 INFO:teuthology.orchestra.run.smithi042.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T00:18:49.275 INFO:teuthology.orchestra.run.smithi042.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi042.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi042.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi042.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi042.stdout:Access: 2024-04-25 00:16:27.325450604 +0000 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi042.stdout:Modify: 2024-04-25 00:16:27.325450604 +0000 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi042.stdout:Change: 2024-04-25 00:18:47.708956781 +0000 2024-04-25T00:18:49.276 INFO:teuthology.orchestra.run.smithi042.stdout: Birth: 2024-04-25 00:16:27.325450604 +0000 2024-04-25T00:18:49.276 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T00:18:49.276 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T00:18:49.368 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-25T00:18:49.423 INFO:tasks.workunit.client.0.smithi042.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T00:18:49.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:49 smithi136 ceph-mon[32923]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:49.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:49 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1069733568' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T00:18:49.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:49 smithi106 ceph-mon[32510]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:49.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:49 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1069733568' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T00:18:49.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:49 smithi042 ceph-mon[28130]: pgmap v389: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:18:49.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:49 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1069733568' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-25T00:18:50.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:50 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1069733568' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T00:18:50.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:50 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:18:50.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:50 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1069733568' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T00:18:50.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:50 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:18:50.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:50 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1069733568' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-25T00:18:50.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:50 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:18:51.507 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:51 smithi136 ceph-mon[32923]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:18:51.507 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:51 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:51.537 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:51 smithi106 ceph-mon[32510]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:18:51.537 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:51 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:51.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:51 smithi042 ceph-mon[28130]: pgmap v390: 97 pgs: 97 active+clean; 641 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:18:51.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:51 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:18:53.010 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.011 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.011 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.011 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.011 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.011 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.116 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.116 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.116 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.116 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:53.116 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:53 smithi136 ceph-mon[32923]: pgmap v391: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T00:18:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:53 smithi106 ceph-mon[32510]: pgmap v391: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T00:18:54.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:53 smithi042 ceph-mon[28130]: pgmap v391: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T00:18:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:55 smithi042 ceph-mon[28130]: pgmap v392: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T00:18:55.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:55.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:55 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:55 smithi136 ceph-mon[32923]: pgmap v392: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T00:18:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:55 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:55 smithi106 ceph-mon[32510]: pgmap v392: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-04-25T00:18:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:55 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:57.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:57.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:57.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:57.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:18:57.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:18:57.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:18:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:57 smithi042 ceph-mon[28130]: pgmap v393: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-25T00:18:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:57 smithi136 ceph-mon[32923]: pgmap v393: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-25T00:18:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:57 smithi106 ceph-mon[32510]: pgmap v393: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-25T00:18:59.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:18:59 smithi042 ceph-mon[28130]: pgmap v394: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-25T00:19:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:18:59 smithi136 ceph-mon[32923]: pgmap v394: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-25T00:19:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:18:59 smithi106 ceph-mon[32510]: pgmap v394: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.4 KiB/s wr, 7 op/s 2024-04-25T00:19:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:01 smithi136 ceph-mon[32923]: pgmap v395: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-25T00:19:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:01 smithi106 ceph-mon[32510]: pgmap v395: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-25T00:19:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:01 smithi042 ceph-mon[28130]: pgmap v395: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-04-25T00:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.463+0000 7f03b1400640 1 -- 172.21.15.42:0/2699811793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03ac107300 msgr2=0x7f03ac1096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.463+0000 7f03b1400640 1 --2- 172.21.15.42:0/2699811793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03ac107300 0x7f03ac1096f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009ca0 tx=0x7f03a402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.463+0000 7f03b1400640 1 -- 172.21.15.42:0/2699811793 shutdown_connections 2024-04-25T00:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.463+0000 7f03b1400640 1 --2- 172.21.15.42:0/2699811793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03ac107300 0x7f03ac1096f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.463+0000 7f03b1400640 1 --2- 172.21.15.42:0/2699811793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03ac1049d0 0x7f03ac106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.463+0000 7f03b1400640 1 --2- 172.21.15.42:0/2699811793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03ac1020a0 0x7f03ac104490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.463+0000 7f03b1400640 1 -- 172.21.15.42:0/2699811793 >> 172.21.15.42:0/2699811793 conn(0x7f03ac0fba30 msgr2=0x7f03ac0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.463+0000 7f03b1400640 1 -- 172.21.15.42:0/2699811793 shutdown_connections 2024-04-25T00:19:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.463+0000 7f03b1400640 1 -- 172.21.15.42:0/2699811793 wait complete. 2024-04-25T00:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.464+0000 7f03b1400640 1 Processor -- start 2024-04-25T00:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03b1400640 1 -- start start 2024-04-25T00:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03b1400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03ac1020a0 0x7f03ac199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03b1400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03ac1049d0 0x7f03ac19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03b1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03ac107300 0x7f03ac1a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03b1400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10d250 con 0x7f03ac107300 2024-04-25T00:19:02.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03b1400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10d0d0 con 0x7f03ac1049d0 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03b1400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac10d3d0 con 0x7f03ac1020a0 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03b0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03ac107300 0x7f03ac1a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03abe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03ac1020a0 0x7f03ac199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03b0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03ac107300 0x7f03ac1a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45202/0 (socket says 172.21.15.42:45202) 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03abe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03ac1020a0 0x7f03ac199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:37986/0 (socket says 172.21.15.42:37986) 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03b0a00640 1 -- 172.21.15.42:0/3111487745 learned_addr learned my addr 172.21.15.42:0/3111487745 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.465+0000 7f03ab400640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03ac1049d0 0x7f03ac19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.466+0000 7f03b0a00640 1 -- 172.21.15.42:0/3111487745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03ac1020a0 msgr2=0x7f03ac199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.466+0000 7f03b0a00640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03ac1020a0 0x7f03ac199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.466+0000 7f03b0a00640 1 -- 172.21.15.42:0/3111487745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03ac1049d0 msgr2=0x7f03ac19a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.466+0000 7f03b0a00640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03ac1049d0 0x7f03ac19a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.466+0000 7f03b0a00640 1 -- 172.21.15.42:0/3111487745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03ac1a16e0 con 0x7f03ac107300 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.466+0000 7f03ab400640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03ac1049d0 0x7f03ac19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.466+0000 7f03b0a00640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03ac107300 0x7f03ac1a1070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f03a4031ae0 tx=0x7f03a4031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.466+0000 7f03a8c00640 1 -- 172.21.15.42:0/3111487745 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4003b70 con 0x7f03ac107300 2024-04-25T00:19:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.467+0000 7f03a8c00640 1 -- 172.21.15.42:0/3111487745 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f03a4003d10 con 0x7f03ac107300 2024-04-25T00:19:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.467+0000 7f03b1400640 1 -- 172.21.15.42:0/3111487745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03ac1a1970 con 0x7f03ac107300 2024-04-25T00:19:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.467+0000 7f03a8c00640 1 -- 172.21.15.42:0/3111487745 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a402fc40 con 0x7f03ac107300 2024-04-25T00:19:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.467+0000 7f03b1400640 1 -- 172.21.15.42:0/3111487745 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03ac1a1ec0 con 0x7f03ac107300 2024-04-25T00:19:02.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.471+0000 7f03b53fd740 1 Processor -- start 2024-04-25T00:19:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.471+0000 7f03a8c00640 1 -- 172.21.15.42:0/3111487745 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f03a4004050 con 0x7f03ac107300 2024-04-25T00:19:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.471+0000 7f03b53fd740 1 -- start start 2024-04-25T00:19:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 0x55f3ee3df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 0x55f3ee3e4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3e49b0 0x55f3ee3e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03b53fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f3ee341ab0 con 0x55f3ee3e49b0 2024-04-25T00:19:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03b53fd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f3ee2ea640 con 0x55f3ee3df840 2024-04-25T00:19:02.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03b53fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f3ee33da70 con 0x55f3ee3dee50 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03ab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3e49b0 0x55f3ee3e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03abe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 0x55f3ee3e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03abe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 0x55f3ee3e4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53056/0 (socket says 172.21.15.42:53056) 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03abe00640 1 -- 172.21.15.42:0/822667642 learned_addr learned my addr 172.21.15.42:0/822667642 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03ab400640 1 --2- 172.21.15.42:0/822667642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3e49b0 0x55f3ee3e6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45208/0 (socket says 172.21.15.42:45208) 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03ab400640 1 --2- 172.21.15.42:0/822667642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 0x55f3ee3df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03abe00640 1 -- 172.21.15.42:0/822667642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 msgr2=0x55f3ee3df270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.472+0000 7f03abe00640 1 --2- 172.21.15.42:0/822667642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 0x55f3ee3df270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.473+0000 7f03abe00640 1 -- 172.21.15.42:0/822667642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3e49b0 msgr2=0x55f3ee3e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.473+0000 7f03abe00640 1 --2- 172.21.15.42:0/822667642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3e49b0 0x55f3ee3e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.473+0000 7f03abe00640 1 -- 172.21.15.42:0/822667642 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ee056210 con 0x55f3ee3df840 2024-04-25T00:19:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.473+0000 7f03abe00640 1 --2- 172.21.15.42:0/822667642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 0x55f3ee3e4340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f039c009cf0 tx=0x7f039c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=155a3e0059d08533 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.473+0000 7f03a1200640 1 -- 172.21.15.42:0/822667642 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c01e070 con 0x55f3ee3df840 2024-04-25T00:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.473+0000 7f03a1200640 1 -- 172.21.15.42:0/822667642 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c009de0 con 0x55f3ee3df840 2024-04-25T00:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.473+0000 7f03a1200640 1 -- 172.21.15.42:0/822667642 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c0196b0 con 0x55f3ee3df840 2024-04-25T00:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.473+0000 7f03a8c00640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f03900a2b80 0x7f03900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03a8c00640 1 -- 172.21.15.42:0/3111487745 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f03a40f00f0 con 0x7f03ac107300 2024-04-25T00:19:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03ab400640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f03900a2b80 0x7f03900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03b53fd740 1 -- 172.21.15.42:0/822667642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 msgr2=0x55f3ee3e4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03b53fd740 1 --2- 172.21.15.42:0/822667642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 0x55f3ee3e4340 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f039c009cf0 tx=0x7f039c011810 comp rx=0 tx=0).stop 2024-04-25T00:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03b53fd740 1 -- 172.21.15.42:0/822667642 shutdown_connections 2024-04-25T00:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03b53fd740 1 --2- 172.21.15.42:0/822667642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3e49b0 0x55f3ee3e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03b53fd740 1 --2- 172.21.15.42:0/822667642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 0x55f3ee3e4340 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03b53fd740 1 --2- 172.21.15.42:0/822667642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 0x55f3ee3df270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03b53fd740 1 -- 172.21.15.42:0/822667642 >> 172.21.15.42:0/822667642 conn(0x55f3ee3da4c0 msgr2=0x55f3ee3da8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03b53fd740 1 -- 172.21.15.42:0/822667642 shutdown_connections 2024-04-25T00:19:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03b53fd740 1 -- 172.21.15.42:0/822667642 wait complete. 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.474+0000 7f03ab400640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f03900a2b80 0x7f03900a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55f3ee3e00a0 tx=0x7f0398005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.475+0000 7f03b53fd740 1 -- start start 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.475+0000 7f03b53fd740 1 Processor -- start 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.475+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3dee50 0x55f3ee3e9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.475+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 0x55f3ee3e9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.475+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3e49b0 0x55f3ee3f0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03b53fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f3ee3e7960 con 0x55f3ee3dee50 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03b53fd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f3ee2fd990 con 0x55f3ee3df840 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03b53fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f3ee3e7ae0 con 0x55f3ee3e49b0 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03b53fd740 10 client.0 _ll_register_callbacks cb 0x55f3ee2ff950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3dee50 0x55f3ee3e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3dee50 0x55f3ee3e9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45214/0 (socket says 172.21.15.42:45214) 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03abe00640 1 -- 172.21.15.42:0/818577186 learned_addr learned my addr 172.21.15.42:0/818577186 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03ab400640 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 0x55f3ee3e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03b0a00640 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3e49b0 0x55f3ee3f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03abe00640 1 -- 172.21.15.42:0/818577186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3e49b0 msgr2=0x55f3ee3f0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03abe00640 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3e49b0 0x55f3ee3f0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03abe00640 1 -- 172.21.15.42:0/818577186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 msgr2=0x55f3ee3e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03abe00640 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 0x55f3ee3e9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.476+0000 7f03abe00640 1 -- 172.21.15.42:0/818577186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ee122620 con 0x55f3ee3dee50 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.477+0000 7f03abe00640 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3dee50 0x55f3ee3e9120 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f039c01b000 tx=0x7f039c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771b64c0f27ba535 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.477+0000 7f0395600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.477+0000 7f0395600640 1 -- 172.21.15.42:0/818577186 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c026020 con 0x55f3ee3dee50 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.477+0000 7f0395600640 1 -- 172.21.15.42:0/818577186 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f039c004260 con 0x55f3ee3dee50 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.477+0000 7f03b53fd740 1 -- 172.21.15.42:0/818577186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f039c009de0 con 0x55f3ee3dee50 2024-04-25T00:19:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.477+0000 7f0395600640 1 -- 172.21.15.42:0/818577186 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f039c01e040 con 0x55f3ee3dee50 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.477+0000 7f03b53fd740 10 client.15099 fetch_fsmap learned FSMap version 24 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.477+0000 7f03b53fd740 1 -- 172.21.15.42:0/818577186 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f3ee3f3610 con 0x55f3ee3dee50 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.477+0000 7f0395600640 1 -- 172.21.15.42:0/818577186 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c009de0 con 0x55f3ee3dee50 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.478+0000 7f0395600640 1 -- 172.21.15.42:0/818577186 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f039c021030 con 0x55f3ee3dee50 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.478+0000 7f03b53fd740 10 client.15099 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.478+0000 7f03b53fd740 10 client.15099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.478+0000 7f03b53fd740 20 client.15099 populate_metadata read hostname 'smithi042' 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.478+0000 7f03b53fd740 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3ee3f4a30 0x55f3ee3f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.478+0000 7f03b53fd740 4 client.15099 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.478+0000 7f03b53fd740 1 -- 172.21.15.42:0/818577186 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3ee34eff0 con 0x55f3ee3f4a30 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.478+0000 7f03ab400640 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3ee3f4a30 0x55f3ee3f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.479+0000 7f03ab400640 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3ee3f4a30 0x55f3ee3f6e60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b056314d9427aaf8 server_cookie=35225af8f07c70c7 in_seq=0 out_seq=0 2024-04-25T00:19:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.479+0000 7f0395600640 10 client.15099 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f0395600640 1 -- 172.21.15.42:0/818577186 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3ee34eff0 con 0x55f3ee3f4a30 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f0395600640 10 client.15099 handle_command_reply: tid=0 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 client.15099 shutdown 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 -- 172.21.15.42:0/818577186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3dee50 msgr2=0x55f3ee3e9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3dee50 0x55f3ee3e9120 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f039c01b000 tx=0x7f039c004040 comp rx=0 tx=0).stop 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 -- 172.21.15.42:0/818577186 shutdown_connections 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3ee3f4a30 0x55f3ee3f6e60 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3e49b0 0x55f3ee3f0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3df840 0x55f3ee3e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 --2- 172.21.15.42:0/818577186 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3dee50 0x55f3ee3e9120 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f0395600640 0 client.15099 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 -- 172.21.15.42:0/818577186 >> 172.21.15.42:0/818577186 conn(0x55f3ee3da4c0 msgr2=0x55f3ee3d9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 -- 172.21.15.42:0/818577186 shutdown_connections 2024-04-25T00:19:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 1 -- 172.21.15.42:0/818577186 wait complete. 2024-04-25T00:19:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.480+0000 7f03b53fd740 20 client.15099 trim_cache size 0 max 16384 2024-04-25T00:19:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.497+0000 7f03b53fd740 1 Processor -- start 2024-04-25T00:19:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.497+0000 7f03b53fd740 1 -- start start 2024-04-25T00:19:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.497+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3d94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.497+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3dee50 0x55f3ee3d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.497+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3df840 0x55f3ee3fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.497+0000 7f03b53fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1ad500 con 0x55f3ee3da4c0 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.497+0000 7f03b53fd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1aadd0 con 0x55f3ee3dee50 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.497+0000 7f03b53fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1a8c20 con 0x55f3ee3df840 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.497+0000 7f03abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3d94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3d94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45220/0 (socket says 172.21.15.42:45220) 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03abe00640 1 -- 172.21.15.42:0/893718662 learned_addr learned my addr 172.21.15.42:0/893718662 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03ab400640 1 --2- 172.21.15.42:0/893718662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3dee50 0x55f3ee3d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03b0a00640 1 --2- 172.21.15.42:0/893718662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3df840 0x55f3ee3fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03abe00640 1 -- 172.21.15.42:0/893718662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3df840 msgr2=0x55f3ee3fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03abe00640 1 --2- 172.21.15.42:0/893718662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3df840 0x55f3ee3fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03abe00640 1 -- 172.21.15.42:0/893718662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3dee50 msgr2=0x55f3ee3d99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03abe00640 1 --2- 172.21.15.42:0/893718662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3dee50 0x55f3ee3d99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03abe00640 1 -- 172.21.15.42:0/893718662 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ee3f74e0 con 0x55f3ee3da4c0 2024-04-25T00:19:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03b0a00640 1 --2- 172.21.15.42:0/893718662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3df840 0x55f3ee3fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f03abe00640 1 --2- 172.21.15.42:0/893718662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3d94a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f039c004980 tx=0x7f039c009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39de92a29f608ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f0397e00640 1 -- 172.21.15.42:0/893718662 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c026040 con 0x55f3ee3da4c0 2024-04-25T00:19:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f0397e00640 1 -- 172.21.15.42:0/893718662 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c004d70 con 0x55f3ee3da4c0 2024-04-25T00:19:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.498+0000 7f0397e00640 1 -- 172.21.15.42:0/893718662 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f039c005230 con 0x55f3ee3da4c0 2024-04-25T00:19:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.499+0000 7f03b53fd740 1 -- 172.21.15.42:0/893718662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3da4c0 msgr2=0x55f3ee3d94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.499+0000 7f03b53fd740 1 --2- 172.21.15.42:0/893718662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3d94a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f039c004980 tx=0x7f039c009640 comp rx=0 tx=0).stop 2024-04-25T00:19:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.499+0000 7f03b53fd740 1 -- 172.21.15.42:0/893718662 shutdown_connections 2024-04-25T00:19:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.499+0000 7f03b53fd740 1 --2- 172.21.15.42:0/893718662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3df840 0x55f3ee3fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.499+0000 7f03b53fd740 1 --2- 172.21.15.42:0/893718662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3dee50 0x55f3ee3d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.499+0000 7f03b53fd740 1 --2- 172.21.15.42:0/893718662 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3d94a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.499+0000 7f03b53fd740 1 -- 172.21.15.42:0/893718662 >> 172.21.15.42:0/893718662 conn(0x55f3ee3e4480 msgr2=0x55f3ee3d8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.499+0000 7f03b53fd740 1 -- 172.21.15.42:0/893718662 shutdown_connections 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.499+0000 7f03b53fd740 1 -- 172.21.15.42:0/893718662 wait complete. 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.500+0000 7f03b53fd740 1 -- start start 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.500+0000 7f03b53fd740 1 Processor -- start 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.500+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3edd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.500+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 0x55f3ee3ee260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.500+0000 7f03b53fd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3df840 0x55f3ee3ef270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.500+0000 7f03b53fd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1a9290 con 0x55f3ee3df840 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.500+0000 7f03b53fd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1aa700 con 0x55f3ee3da4c0 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.500+0000 7f03b53fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03ac1a9960 con 0x55f3ee3dee50 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.500+0000 7f03b53fd740 10 client.0 _ll_register_callbacks cb 0x7f039c01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03b0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3df840 0x55f3ee3ef270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03ab400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 0x55f3ee3ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03abe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3edd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03ab400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 0x55f3ee3ee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38010/0 (socket says 172.21.15.42:38010) 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03ab400640 1 -- 172.21.15.42:0/3713761178 learned_addr learned my addr 172.21.15.42:0/3713761178 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03b0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3df840 0x55f3ee3ef270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45236/0 (socket says 172.21.15.42:45236) 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03ab400640 1 -- 172.21.15.42:0/3713761178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3da4c0 msgr2=0x55f3ee3edd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03ab400640 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3edd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03ab400640 1 -- 172.21.15.42:0/3713761178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3df840 msgr2=0x55f3ee3ef270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03ab400640 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3df840 0x55f3ee3ef270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03ab400640 1 -- 172.21.15.42:0/3713761178 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ee3da2f0 con 0x55f3ee3dee50 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03abe00640 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03b0a00640 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3df840 0x55f3ee3ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f03ab400640 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 0x55f3ee3ee260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f03ac19afd0 tx=0x7f0398004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a96cc4c1a22f303 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.501+0000 7f0395600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.502+0000 7f0395600640 1 -- 172.21.15.42:0/3713761178 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0398003e20 con 0x55f3ee3dee50 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.502+0000 7f0395600640 1 -- 172.21.15.42:0/3713761178 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f039801a070 con 0x55f3ee3dee50 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.502+0000 7f03b53fd740 1 -- 172.21.15.42:0/3713761178 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f039c004d70 con 0x55f3ee3dee50 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.502+0000 7f0395600640 1 -- 172.21.15.42:0/3713761178 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f039803c070 con 0x55f3ee3dee50 2024-04-25T00:19:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.502+0000 7f03b53fd740 10 client.25213 fetch_fsmap learned FSMap version 24 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.502+0000 7f03b53fd740 1 -- 172.21.15.42:0/3713761178 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55f3ee45b1a0 con 0x55f3ee3dee50 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.502+0000 7f0395600640 1 -- 172.21.15.42:0/3713761178 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f039c004d70 con 0x55f3ee3dee50 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.503+0000 7f0395600640 1 -- 172.21.15.42:0/3713761178 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f039801d070 con 0x55f3ee3dee50 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.503+0000 7f03b53fd740 10 client.25213 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.503+0000 7f03b53fd740 10 client.25213 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.503+0000 7f03b53fd740 20 client.25213 populate_metadata read hostname 'smithi042' 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.503+0000 7f03b53fd740 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3ee3e49b0 0x55f3ee45dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.503+0000 7f03b53fd740 4 client.25213 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.503+0000 7f03b53fd740 1 -- 172.21.15.42:0/3713761178 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3ee349340 con 0x55f3ee3e49b0 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.503+0000 7f03abe00640 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3ee3e49b0 0x55f3ee45dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.504+0000 7f03abe00640 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3ee3e49b0 0x55f3ee45dbc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29fa59f43e668ae3 server_cookie=101b190385a257af in_seq=0 out_seq=0 2024-04-25T00:19:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.504+0000 7f0395600640 10 client.25213 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f0395600640 1 -- 172.21.15.42:0/3713761178 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3ee349340 con 0x55f3ee3e49b0 2024-04-25T00:19:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f0395600640 10 client.25213 handle_command_reply: tid=0 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 client.25213 shutdown 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 -- 172.21.15.42:0/3713761178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 msgr2=0x55f3ee3ee260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 0x55f3ee3ee260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f03ac19afd0 tx=0x7f0398004730 comp rx=0 tx=0).stop 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 -- 172.21.15.42:0/3713761178 shutdown_connections 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3ee3e49b0 0x55f3ee45dbc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3ee3df840 0x55f3ee3ef270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3ee3dee50 0x55f3ee3ee260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 --2- 172.21.15.42:0/3713761178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3ee3da4c0 0x55f3ee3edd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f0395600640 0 client.25213 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 -- 172.21.15.42:0/3713761178 >> 172.21.15.42:0/3713761178 conn(0x55f3ee3e4480 msgr2=0x55f3ee3d8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 -- 172.21.15.42:0/3713761178 shutdown_connections 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.515+0000 7f03b53fd740 1 -- 172.21.15.42:0/3713761178 wait complete. 2024-04-25T00:19:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.516+0000 7f03b53fd740 20 client.25213 trim_cache size 0 max 16384 2024-04-25T00:19:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:19:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:19:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:19:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.516+0000 7f03b1400640 1 -- 172.21.15.42:0/3111487745 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f03900a2b80 msgr2=0x7f03900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.516+0000 7f03b1400640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f03900a2b80 0x7f03900a5040 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x55f3ee3e00a0 tx=0x7f0398005850 comp rx=0 tx=0).stop 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.516+0000 7f03b1400640 1 -- 172.21.15.42:0/3111487745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03ac107300 msgr2=0x7f03ac1a1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.516+0000 7f03b1400640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03ac107300 0x7f03ac1a1070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f03a4031ae0 tx=0x7f03a4031ee0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.517+0000 7f03b1400640 1 -- 172.21.15.42:0/3111487745 shutdown_connections 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.517+0000 7f03b1400640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f03900a2b80 0x7f03900a5040 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.517+0000 7f03b1400640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f03ac107300 0x7f03ac1a1070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.517+0000 7f03b1400640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f03ac1049d0 0x7f03ac19a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.517+0000 7f03b1400640 1 --2- 172.21.15.42:0/3111487745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03ac1020a0 0x7f03ac199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.517+0000 7f03b1400640 1 -- 172.21.15.42:0/3111487745 >> 172.21.15.42:0/3111487745 conn(0x7f03ac0fba30 msgr2=0x7f03ac101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.517+0000 7f03b1400640 1 -- 172.21.15.42:0/3111487745 shutdown_connections 2024-04-25T00:19:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:02.517+0000 7f03b1400640 1 -- 172.21.15.42:0/3111487745 wait complete. 2024-04-25T00:19:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:03 smithi136 ceph-mon[32923]: pgmap v396: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T00:19:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:03 smithi106 ceph-mon[32510]: pgmap v396: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T00:19:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:03 smithi042 ceph-mon[28130]: pgmap v396: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-04-25T00:19:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:05 smithi136 ceph-mon[32923]: pgmap v397: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:05 smithi106 ceph-mon[32510]: pgmap v397: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:05 smithi042 ceph-mon[28130]: pgmap v397: 97 pgs: 97 active+clean; 652 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:07 smithi136 ceph-mon[32923]: pgmap v398: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T00:19:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:07 smithi106 ceph-mon[32510]: pgmap v398: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T00:19:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:07 smithi042 ceph-mon[28130]: pgmap v398: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 9 op/s 2024-04-25T00:19:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:09 smithi136 ceph-mon[32923]: pgmap v399: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-25T00:19:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:09 smithi106 ceph-mon[32510]: pgmap v399: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-25T00:19:10.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:09 smithi042 ceph-mon[28130]: pgmap v399: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 1.4 KiB/s wr, 10 op/s 2024-04-25T00:19:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:11 smithi136 ceph-mon[32923]: pgmap v400: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-25T00:19:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:11 smithi106 ceph-mon[32510]: pgmap v400: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-25T00:19:12.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:11 smithi042 ceph-mon[28130]: pgmap v400: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-25T00:19:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:13 smithi136 ceph-mon[32923]: pgmap v401: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-25T00:19:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:13 smithi106 ceph-mon[32510]: pgmap v401: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-25T00:19:14.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:13 smithi042 ceph-mon[28130]: pgmap v401: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 11 op/s 2024-04-25T00:19:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.662+0000 7f1a4ac00640 1 -- 172.21.15.42:0/130152435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a44075720 msgr2=0x7f1a44073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.662+0000 7f1a4ac00640 1 --2- 172.21.15.42:0/130152435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a44075720 0x7f1a44073b70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009ca0 tx=0x7f1a3802a940 comp rx=0 tx=0).stop 2024-04-25T00:19:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.662+0000 7f1a4ac00640 1 -- 172.21.15.42:0/130152435 shutdown_connections 2024-04-25T00:19:15.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.662+0000 7f1a4ac00640 1 --2- 172.21.15.42:0/130152435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a44109450 0x7f1a4410b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.662+0000 7f1a4ac00640 1 --2- 172.21.15.42:0/130152435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a44074140 0x7f1a440745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.662+0000 7f1a4ac00640 1 --2- 172.21.15.42:0/130152435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a44075720 0x7f1a44073b70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.662+0000 7f1a4ac00640 1 -- 172.21.15.42:0/130152435 >> 172.21.15.42:0/130152435 conn(0x7f1a440fcb50 msgr2=0x7f1a440fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.662+0000 7f1a4ac00640 1 -- 172.21.15.42:0/130152435 shutdown_connections 2024-04-25T00:19:15.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.662+0000 7f1a4ac00640 1 -- 172.21.15.42:0/130152435 wait complete. 2024-04-25T00:19:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.664+0000 7f1a4ac00640 1 Processor -- start 2024-04-25T00:19:15.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.664+0000 7f1a4ac00640 1 -- start start 2024-04-25T00:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.664+0000 7f1a4ac00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a44074140 0x7f1a44199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a4ac00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a44075720 0x7f1a4419a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a4ac00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a44109450 0x7f1a441a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a4ac00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4410f430 con 0x7f1a44074140 2024-04-25T00:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a4ac00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4410f2b0 con 0x7f1a44109450 2024-04-25T00:19:15.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a4ac00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a4410f5b0 con 0x7f1a44075720 2024-04-25T00:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a49800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a44074140 0x7f1a44199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a48e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a44075720 0x7f1a4419a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a49800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a44074140 0x7f1a44199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34038/0 (socket says 172.21.15.42:34038) 2024-04-25T00:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a49800640 1 -- 172.21.15.42:0/1953661555 learned_addr learned my addr 172.21.15.42:0/1953661555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a4a200640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a44109450 0x7f1a441a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a48e00640 1 -- 172.21.15.42:0/1953661555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a44109450 msgr2=0x7f1a441a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a48e00640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a44109450 0x7f1a441a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a48e00640 1 -- 172.21.15.42:0/1953661555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a44074140 msgr2=0x7f1a44199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a48e00640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a44074140 0x7f1a44199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a48e00640 1 -- 172.21.15.42:0/1953661555 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a441a1720 con 0x7f1a44075720 2024-04-25T00:19:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.665+0000 7f1a49800640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a44074140 0x7f1a44199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:15.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.666+0000 7f1a48e00640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a44075720 0x7f1a4419a040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000e9f0 tx=0x7f1a3000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.667+0000 7f1a42000640 1 -- 172.21.15.42:0/1953661555 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3000ccc0 con 0x7f1a44075720 2024-04-25T00:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.668+0000 7f1a42000640 1 -- 172.21.15.42:0/1953661555 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a30005020 con 0x7f1a44075720 2024-04-25T00:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.668+0000 7f1a42000640 1 -- 172.21.15.42:0/1953661555 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a30004a00 con 0x7f1a44075720 2024-04-25T00:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.668+0000 7f1a4ac00640 1 -- 172.21.15.42:0/1953661555 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a441a1a10 con 0x7f1a44075720 2024-04-25T00:19:15.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.668+0000 7f1a4ac00640 1 -- 172.21.15.42:0/1953661555 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a441a1f60 con 0x7f1a44075720 2024-04-25T00:19:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.671+0000 7f1a4eff5740 1 Processor -- start 2024-04-25T00:19:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.671+0000 7f1a4eff5740 1 -- start start 2024-04-25T00:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf2d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf2d840 0x55657bf32340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf329b0 0x55657bf34e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55657be8fab0 con 0x55657bf2d840 2024-04-25T00:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55657be38640 con 0x55657bf2ce50 2024-04-25T00:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55657be8ba70 con 0x55657bf329b0 2024-04-25T00:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a42000640 1 -- 172.21.15.42:0/1953661555 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a300051d0 con 0x7f1a44075720 2024-04-25T00:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4a200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf2d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4a200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf2d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:47060/0 (socket says 172.21.15.42:47060) 2024-04-25T00:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a49800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf2d840 0x55657bf32340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4a200640 1 -- 172.21.15.42:0/1097523227 learned_addr learned my addr 172.21.15.42:0/1097523227 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a48e00640 1 --2- 172.21.15.42:0/1097523227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf329b0 0x55657bf34e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4a200640 1 -- 172.21.15.42:0/1097523227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf329b0 msgr2=0x55657bf34e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4a200640 1 --2- 172.21.15.42:0/1097523227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf329b0 0x55657bf34e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4a200640 1 -- 172.21.15.42:0/1097523227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf2d840 msgr2=0x55657bf32340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4a200640 1 --2- 172.21.15.42:0/1097523227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf2d840 0x55657bf32340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a4a200640 1 -- 172.21.15.42:0/1097523227 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55657bba4210 con 0x55657bf2ce50 2024-04-25T00:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.672+0000 7f1a48e00640 1 --2- 172.21.15.42:0/1097523227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf329b0 0x55657bf34e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:15.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.673+0000 7f1a4a200640 1 --2- 172.21.15.42:0/1097523227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf2d270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c90 tx=0x7f1a3c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b3f07484106e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.673+0000 7f1a36a00640 1 -- 172.21.15.42:0/1097523227 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c004230 con 0x55657bf2ce50 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.673+0000 7f1a36a00640 1 -- 172.21.15.42:0/1097523227 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a3c009e40 con 0x55657bf2ce50 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.673+0000 7f1a36a00640 1 -- 172.21.15.42:0/1097523227 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c019760 con 0x55657bf2ce50 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.673+0000 7f1a42000640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1a2c0a2c00 0x7f1a2c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a42000640 1 -- 172.21.15.42:0/1953661555 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1a300cef30 con 0x7f1a44075720 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a49800640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1a2c0a2c00 0x7f1a2c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a4eff5740 1 -- 172.21.15.42:0/1097523227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 msgr2=0x55657bf2d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/1097523227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf2d270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c90 tx=0x7f1a3c011810 comp rx=0 tx=0).stop 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a4eff5740 1 -- 172.21.15.42:0/1097523227 shutdown_connections 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/1097523227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf329b0 0x55657bf34e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/1097523227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf2d840 0x55657bf32340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/1097523227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf2d270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a4eff5740 1 -- 172.21.15.42:0/1097523227 >> 172.21.15.42:0/1097523227 conn(0x55657bf284c0 msgr2=0x55657bf288c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a4eff5740 1 -- 172.21.15.42:0/1097523227 shutdown_connections 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.674+0000 7f1a49800640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1a2c0a2c00 0x7f1a2c0a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55657bf2dc40 tx=0x7f1a3802f930 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.675+0000 7f1a4eff5740 1 -- 172.21.15.42:0/1097523227 wait complete. 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.675+0000 7f1a4eff5740 1 -- start start 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.675+0000 7f1a4eff5740 1 Processor -- start 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf37040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 0x55657bf37580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55657bf35960 con 0x55657bf329b0 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55657be4b990 con 0x55657bf2ce50 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55657bf35ae0 con 0x55657bf2d840 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a4eff5740 10 client.0 _ll_register_callbacks cb 0x55657be4d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a49800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf37040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a49800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf37040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:47072/0 (socket says 172.21.15.42:47072) 2024-04-25T00:19:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a49800640 1 -- 172.21.15.42:0/2781055955 learned_addr learned my addr 172.21.15.42:0/2781055955 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a48e00640 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 0x55657bf37580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a4a200640 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a49800640 1 -- 172.21.15.42:0/2781055955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 msgr2=0x55657bf37580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a49800640 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 0x55657bf37580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a49800640 1 -- 172.21.15.42:0/2781055955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 msgr2=0x55657bf3e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a49800640 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a49800640 1 -- 172.21.15.42:0/2781055955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55657bc70620 con 0x55657bf2ce50 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a48e00640 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 0x55657bf37580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.676+0000 7f1a4a200640 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.677+0000 7f1a49800640 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf37040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009dc0 tx=0x7f1a380095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d7f8fc1ed718ca4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.677+0000 7f1a2b400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.677+0000 7f1a2b400640 1 -- 172.21.15.42:0/2781055955 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38034040 con 0x55657bf2ce50 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.677+0000 7f1a2b400640 1 -- 172.21.15.42:0/2781055955 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1a380041c0 con 0x55657bf2ce50 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.677+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2781055955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55657be9cff0 con 0x55657bf2ce50 2024-04-25T00:19:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.677+0000 7f1a2b400640 1 -- 172.21.15.42:0/2781055955 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1a38004400 con 0x55657bf2ce50 2024-04-25T00:19:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.677+0000 7f1a4eff5740 10 client.25220 fetch_fsmap learned FSMap version 24 2024-04-25T00:19:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.677+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2781055955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55657bf41660 con 0x55657bf2ce50 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.678+0000 7f1a2b400640 1 -- 172.21.15.42:0/2781055955 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55657be9cff0 con 0x55657bf2ce50 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.678+0000 7f1a2b400640 1 -- 172.21.15.42:0/2781055955 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1a38050070 con 0x55657bf2ce50 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.678+0000 7f1a4eff5740 10 client.25220 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.678+0000 7f1a4eff5740 10 client.25220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.678+0000 7f1a4eff5740 20 client.25220 populate_metadata read hostname 'smithi042' 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.678+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55657bf42cf0 0x55657bf450e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.678+0000 7f1a4eff5740 4 client.25220 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.678+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2781055955 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55657be97340 con 0x55657bf42cf0 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.679+0000 7f1a48e00640 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55657bf42cf0 0x55657bf450e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.679+0000 7f1a48e00640 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55657bf42cf0 0x55657bf450e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f58c112ea74b90f2 server_cookie=2efd36c7a83a9faa in_seq=0 out_seq=0 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.679+0000 7f1a2b400640 10 client.25220 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a2b400640 1 -- 172.21.15.42:0/2781055955 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55657be97340 con 0x55657bf42cf0 2024-04-25T00:19:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a2b400640 10 client.25220 handle_command_reply: tid=0 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 client.25220 shutdown 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2781055955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 msgr2=0x55657bf37040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf37040 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009dc0 tx=0x7f1a380095c0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2781055955 shutdown_connections 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55657bf42cf0 0x55657bf450e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 0x55657bf37580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/2781055955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf37040 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a2b400640 0 client.25220 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2781055955 >> 172.21.15.42:0/2781055955 conn(0x55657bf284c0 msgr2=0x55657bf27620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2781055955 shutdown_connections 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.680+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2781055955 wait complete. 2024-04-25T00:19:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.681+0000 7f1a4eff5740 20 client.25220 trim_cache size 0 max 16384 2024-04-25T00:19:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a4eff5740 1 Processor -- start 2024-04-25T00:19:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a4eff5740 1 -- start start 2024-04-25T00:19:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf27f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 0x55657bf374e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441ad430 con 0x55657bf329b0 2024-04-25T00:19:15.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441aad00 con 0x55657bf2ce50 2024-04-25T00:19:15.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441a8b50 con 0x55657bf2d840 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a4a200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a4a200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34068/0 (socket says 172.21.15.42:34068) 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.697+0000 7f1a48e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 0x55657bf374e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.698+0000 7f1a4a200640 1 -- 172.21.15.42:0/2977612562 learned_addr learned my addr 172.21.15.42:0/2977612562 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.698+0000 7f1a49800640 1 --2- 172.21.15.42:0/2977612562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf27f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.698+0000 7f1a4a200640 1 -- 172.21.15.42:0/2977612562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 msgr2=0x55657bf374e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.698+0000 7f1a4a200640 1 --2- 172.21.15.42:0/2977612562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 0x55657bf374e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.698+0000 7f1a4a200640 1 -- 172.21.15.42:0/2977612562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 msgr2=0x55657bf27f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.698+0000 7f1a4a200640 1 --2- 172.21.15.42:0/2977612562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf27f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.698+0000 7f1a4a200640 1 -- 172.21.15.42:0/2977612562 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55657bf31cf0 con 0x55657bf329b0 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.698+0000 7f1a48e00640 1 --2- 172.21.15.42:0/2977612562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 0x55657bf374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.698+0000 7f1a4a200640 1 --2- 172.21.15.42:0/2977612562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c30 tx=0x7f1a3c005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c069c6c7c07920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.699+0000 7f1a36000640 1 -- 172.21.15.42:0/2977612562 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c0043b0 con 0x55657bf329b0 2024-04-25T00:19:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.699+0000 7f1a36000640 1 -- 172.21.15.42:0/2977612562 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a3c003ad0 con 0x55657bf329b0 2024-04-25T00:19:15.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.699+0000 7f1a36000640 1 -- 172.21.15.42:0/2977612562 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c025b40 con 0x55657bf329b0 2024-04-25T00:19:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.699+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2977612562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 msgr2=0x55657bf3e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.699+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/2977612562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e550 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009c30 tx=0x7f1a3c005740 comp rx=0 tx=0).stop 2024-04-25T00:19:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.699+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2977612562 shutdown_connections 2024-04-25T00:19:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.699+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/2977612562 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3e550 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.699+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/2977612562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2d840 0x55657bf374e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.699+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/2977612562 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2ce50 0x55657bf27f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.699+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2977612562 >> 172.21.15.42:0/2977612562 conn(0x55657bf284c0 msgr2=0x55657bf32070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.700+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2977612562 shutdown_connections 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.700+0000 7f1a4eff5740 1 -- 172.21.15.42:0/2977612562 wait complete. 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.700+0000 7f1a4eff5740 1 -- start start 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.700+0000 7f1a4eff5740 1 Processor -- start 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.700+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2ce50 0x55657bf3bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2d840 0x55657bf3c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a4eff5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441a91c0 con 0x55657bf329b0 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441aa630 con 0x55657bf2d840 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a4eff5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1a441a9890 con 0x55657bf2ce50 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a4eff5740 10 client.0 _ll_register_callbacks cb 0x7f1a38009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a49800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2ce50 0x55657bf3bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a49800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2ce50 0x55657bf3bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:37198/0 (socket says 172.21.15.42:37198) 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a49800640 1 -- 172.21.15.42:0/886565162 learned_addr learned my addr 172.21.15.42:0/886565162 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a4a200640 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a48e00640 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2d840 0x55657bf3c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a49800640 1 -- 172.21.15.42:0/886565162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2d840 msgr2=0x55657bf3c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a49800640 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2d840 0x55657bf3c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a49800640 1 -- 172.21.15.42:0/886565162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 msgr2=0x55657bf3d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a49800640 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.701+0000 7f1a49800640 1 -- 172.21.15.42:0/886565162 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55657bf3d780 con 0x55657bf2ce50 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.702+0000 7f1a49800640 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2ce50 0x55657bf3bc40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1a380524e0 tx=0x7f1a38042000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=227b322a8248e674 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.702+0000 7f1a2b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.702+0000 7f1a2b400640 1 -- 172.21.15.42:0/886565162 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38034070 con 0x55657bf2ce50 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.702+0000 7f1a4eff5740 1 -- 172.21.15.42:0/886565162 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a3c003ad0 con 0x55657bf2ce50 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.702+0000 7f1a2b400640 1 -- 172.21.15.42:0/886565162 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f1a38047880 con 0x55657bf2ce50 2024-04-25T00:19:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.702+0000 7f1a2b400640 1 -- 172.21.15.42:0/886565162 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f1a3806d050 con 0x55657bf2ce50 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.702+0000 7f1a4eff5740 10 client.25231 fetch_fsmap learned FSMap version 24 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.702+0000 7f1a4eff5740 1 -- 172.21.15.42:0/886565162 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55657bfb1a70 con 0x55657bf2ce50 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.702+0000 7f1a2b400640 1 -- 172.21.15.42:0/886565162 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a3c003ad0 con 0x55657bf2ce50 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.703+0000 7f1a2b400640 1 -- 172.21.15.42:0/886565162 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f1a38050050 con 0x55657bf2ce50 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.703+0000 7f1a4eff5740 10 client.25231 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.703+0000 7f1a4eff5740 10 client.25231 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.703+0000 7f1a4eff5740 20 client.25231 populate_metadata read hostname 'smithi042' 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.703+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55657bfb2440 0x55657bfb4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.703+0000 7f1a4eff5740 4 client.25231 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.703+0000 7f1a4eff5740 1 -- 172.21.15.42:0/886565162 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55657be157a0 con 0x55657bfb2440 2024-04-25T00:19:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.703+0000 7f1a48e00640 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55657bfb2440 0x55657bfb4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.704+0000 7f1a48e00640 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55657bfb2440 0x55657bfb4860 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df9aabb8c8b05c0 server_cookie=76286ab4d64b38c4 in_seq=0 out_seq=0 2024-04-25T00:19:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.704+0000 7f1a2b400640 10 client.25231 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.707+0000 7f1a2b400640 1 -- 172.21.15.42:0/886565162 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55657be157a0 con 0x55657bfb2440 2024-04-25T00:19:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.707+0000 7f1a2b400640 10 client.25231 handle_command_reply: tid=0 2024-04-25T00:19:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 client.25231 shutdown 2024-04-25T00:19:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 -- 172.21.15.42:0/886565162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2ce50 msgr2=0x55657bf3bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2ce50 0x55657bf3bc40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1a380524e0 tx=0x7f1a38042000 comp rx=0 tx=0).stop 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 -- 172.21.15.42:0/886565162 shutdown_connections 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55657bfb2440 0x55657bfb4860 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55657bf329b0 0x55657bf3d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55657bf2d840 0x55657bf3c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 --2- 172.21.15.42:0/886565162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55657bf2ce50 0x55657bf3bc40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a2b400640 0 client.25231 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 -- 172.21.15.42:0/886565162 >> 172.21.15.42:0/886565162 conn(0x55657bf284c0 msgr2=0x55657bf2b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 -- 172.21.15.42:0/886565162 shutdown_connections 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 1 -- 172.21.15.42:0/886565162 wait complete. 2024-04-25T00:19:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.708+0000 7f1a4eff5740 20 client.25231 trim_cache size 0 max 16384 2024-04-25T00:19:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:19:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:19:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:19:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 -- 172.21.15.42:0/1953661555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1a2c0a2c00 msgr2=0x7f1a2c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1a2c0a2c00 0x7f1a2c0a50c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55657bf2dc40 tx=0x7f1a3802f930 comp rx=0 tx=0).stop 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 -- 172.21.15.42:0/1953661555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a44075720 msgr2=0x7f1a4419a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a44075720 0x7f1a4419a040 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1a3000e9f0 tx=0x7f1a3000edf0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 -- 172.21.15.42:0/1953661555 shutdown_connections 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1a2c0a2c00 0x7f1a2c0a50c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a44109450 0x7f1a441a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1a44075720 0x7f1a4419a040 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 --2- 172.21.15.42:0/1953661555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1a44074140 0x7f1a44199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 -- 172.21.15.42:0/1953661555 >> 172.21.15.42:0/1953661555 conn(0x7f1a440fcb50 msgr2=0x7f1a441053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 -- 172.21.15.42:0/1953661555 shutdown_connections 2024-04-25T00:19:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:15.709+0000 7f1a4ac00640 1 -- 172.21.15.42:0/1953661555 wait complete. 2024-04-25T00:19:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:15 smithi136 ceph-mon[32923]: pgmap v402: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-25T00:19:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:15 smithi106 ceph-mon[32510]: pgmap v402: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-25T00:19:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:15 smithi042 ceph-mon[28130]: pgmap v402: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-25T00:19:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:17 smithi136 ceph-mon[32923]: pgmap v403: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T00:19:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:17 smithi106 ceph-mon[32510]: pgmap v403: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T00:19:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:17 smithi042 ceph-mon[28130]: pgmap v403: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-25T00:19:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:19 smithi136 ceph-mon[32923]: pgmap v404: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:19:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:19 smithi106 ceph-mon[32510]: pgmap v404: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:19:20.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:19 smithi042 ceph-mon[28130]: pgmap v404: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 170 B/s wr, 7 op/s 2024-04-25T00:19:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:21 smithi136 ceph-mon[32923]: pgmap v405: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:19:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:21 smithi106 ceph-mon[32510]: pgmap v405: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:19:22.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:21 smithi042 ceph-mon[28130]: pgmap v405: 97 pgs: 97 active+clean; 653 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:19:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:23 smithi136 ceph-mon[32923]: pgmap v406: 97 pgs: 97 active+clean; 654 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:19:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:23 smithi106 ceph-mon[32510]: pgmap v406: 97 pgs: 97 active+clean; 654 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:19:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:23 smithi042 ceph-mon[28130]: pgmap v406: 97 pgs: 97 active+clean; 654 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:19:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:25 smithi136 ceph-mon[32923]: pgmap v407: 97 pgs: 97 active+clean; 654 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:25 smithi106 ceph-mon[32510]: pgmap v407: 97 pgs: 97 active+clean; 654 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:25 smithi042 ceph-mon[28130]: pgmap v407: 97 pgs: 97 active+clean; 654 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:27 smithi136 ceph-mon[32923]: pgmap v408: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T00:19:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:27 smithi106 ceph-mon[32510]: pgmap v408: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T00:19:28.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:27 smithi042 ceph-mon[28130]: pgmap v408: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T00:19:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.851+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2457831994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe8102850 msgr2=0x7fcfe8102cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.851+0000 7fcfe7e00640 1 --2- 172.21.15.42:0/2457831994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe8102850 0x7fcfe8102cd0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009900 tx=0x7fcfd402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.851+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2457831994 shutdown_connections 2024-04-25T00:19:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.851+0000 7fcfe7e00640 1 --2- 172.21.15.42:0/2457831994 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe8103210 0x7fcfe8109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.851+0000 7fcfe7e00640 1 --2- 172.21.15.42:0/2457831994 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe8102850 0x7fcfe8102cd0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.851+0000 7fcfe7e00640 1 --2- 172.21.15.42:0/2457831994 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfe8101660 0x7fcfe8101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.851+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2457831994 >> 172.21.15.42:0/2457831994 conn(0x7fcfe80fcbd0 msgr2=0x7fcfe80ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.851+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2457831994 shutdown_connections 2024-04-25T00:19:28.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.851+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2457831994 wait complete. 2024-04-25T00:19:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.852+0000 7fcfe7e00640 1 Processor -- start 2024-04-25T00:19:28.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe7e00640 1 -- start start 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe8101660 0x7fcfe8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfe8102850 0x7fcfe819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe8103210 0x7fcfe81a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe7e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe810d2b0 con 0x7fcfe8101660 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe7e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe810d130 con 0x7fcfe8103210 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe7e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe810d430 con 0x7fcfe8102850 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe8101660 0x7fcfe8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe6a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe8101660 0x7fcfe8199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41012/0 (socket says 172.21.15.42:41012) 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe6a00640 1 -- 172.21.15.42:0/2515944077 learned_addr learned my addr 172.21.15.42:0/2515944077 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe6000640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfe8102850 0x7fcfe819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.853+0000 7fcfe7400640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe8103210 0x7fcfe81a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfe6a00640 1 -- 172.21.15.42:0/2515944077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfe8102850 msgr2=0x7fcfe819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:28.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfe6a00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfe8102850 0x7fcfe819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfe6a00640 1 -- 172.21.15.42:0/2515944077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe8103210 msgr2=0x7fcfe81a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfe6a00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe8103210 0x7fcfe81a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfe6a00640 1 -- 172.21.15.42:0/2515944077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe81a1760 con 0x7fcfe8101660 2024-04-25T00:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfe6a00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe8101660 0x7fcfe8199b40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800e9c0 tx=0x7fcfd800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfdf400640 1 -- 172.21.15.42:0/2515944077 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800ccc0 con 0x7fcfe8101660 2024-04-25T00:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2515944077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe81a1a50 con 0x7fcfe8101660 2024-04-25T00:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfdf400640 1 -- 172.21.15.42:0/2515944077 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd80035f0 con 0x7fcfe8101660 2024-04-25T00:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfdf400640 1 -- 172.21.15.42:0/2515944077 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd800ce60 con 0x7fcfe8101660 2024-04-25T00:19:28.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.854+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2515944077 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe81a1fc0 con 0x7fcfe8101660 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.858+0000 7fcff057a740 1 Processor -- start 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.858+0000 7fcfdf400640 1 -- 172.21.15.42:0/2515944077 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcfd80037a0 con 0x7fcfe8101660 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.858+0000 7fcff057a740 1 -- start start 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.858+0000 7fcff057a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 0x5591dc210270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.858+0000 7fcff057a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc210840 0x5591dc215340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.858+0000 7fcff057a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 0x5591dc217e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.858+0000 7fcff057a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5591dc172ab0 con 0x5591dc2159b0 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.858+0000 7fcff057a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5591dc11b640 con 0x5591dc210840 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.858+0000 7fcff057a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5591dc16ea70 con 0x5591dc20fe50 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 0x5591dc210270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc210840 0x5591dc215340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 0x5591dc210270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39184/0 (socket says 172.21.15.42:39184) 2024-04-25T00:19:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe7400640 1 -- 172.21.15.42:0/2983692214 learned_addr learned my addr 172.21.15.42:0/2983692214 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe6000640 1 --2- 172.21.15.42:0/2983692214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 0x5591dc217e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe7400640 1 -- 172.21.15.42:0/2983692214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc210840 msgr2=0x5591dc215340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe7400640 1 --2- 172.21.15.42:0/2983692214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc210840 0x5591dc215340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe7400640 1 -- 172.21.15.42:0/2983692214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 msgr2=0x5591dc217e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe7400640 1 --2- 172.21.15.42:0/2983692214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 0x5591dc217e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe7400640 1 -- 172.21.15.42:0/2983692214 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591dbe87210 con 0x5591dc20fe50 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.859+0000 7fcfe7400640 1 --2- 172.21.15.42:0/2983692214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 0x5591dc210270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0011440 tx=0x7fcfe0011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ffeb9b1f8a91c1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.860+0000 7fcfdf400640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfcc0a2b80 0x7fcfcc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.860+0000 7fcfd3e00640 1 -- 172.21.15.42:0/2983692214 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0003c30 con 0x5591dc20fe50 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.860+0000 7fcfd3e00640 1 -- 172.21.15.42:0/2983692214 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe0003dd0 con 0x5591dc20fe50 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.860+0000 7fcfd3e00640 1 -- 172.21.15.42:0/2983692214 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe000fb50 con 0x5591dc20fe50 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.860+0000 7fcfdf400640 1 -- 172.21.15.42:0/2515944077 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fcfd80cf460 con 0x7fcfe8101660 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.860+0000 7fcfe6a00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfcc0a2b80 0x7fcfcc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcff057a740 1 -- 172.21.15.42:0/2983692214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 msgr2=0x5591dc210270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcff057a740 1 --2- 172.21.15.42:0/2983692214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 0x5591dc210270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0011440 tx=0x7fcfe0011900 comp rx=0 tx=0).stop 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcfe6a00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfcc0a2b80 0x7fcfcc0a5040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5591dc210c40 tx=0x7fcfd80d2ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcff057a740 1 -- 172.21.15.42:0/2983692214 shutdown_connections 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcff057a740 1 --2- 172.21.15.42:0/2983692214 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 0x5591dc217e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcff057a740 1 --2- 172.21.15.42:0/2983692214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc210840 0x5591dc215340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcff057a740 1 --2- 172.21.15.42:0/2983692214 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 0x5591dc210270 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcff057a740 1 -- 172.21.15.42:0/2983692214 >> 172.21.15.42:0/2983692214 conn(0x5591dc20b4c0 msgr2=0x5591dc20b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcff057a740 1 -- 172.21.15.42:0/2983692214 shutdown_connections 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcff057a740 1 -- 172.21.15.42:0/2983692214 wait complete. 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.861+0000 7fcff057a740 1 -- start start 2024-04-25T00:19:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.862+0000 7fcff057a740 1 Processor -- start 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.862+0000 7fcff057a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc20fe50 0x5591dc21a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.862+0000 7fcff057a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.862+0000 7fcff057a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 0x5591dc221640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.862+0000 7fcff057a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5591dc218960 con 0x5591dc2159b0 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.862+0000 7fcff057a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5591dc12e990 con 0x5591dc20fe50 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.862+0000 7fcff057a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5591dc218ae0 con 0x5591dc210840 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.862+0000 7fcff057a740 10 client.0 _ll_register_callbacks cb 0x5591dc130950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 0x5591dc221640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.862+0000 7fcfe6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39190/0 (socket says 172.21.15.42:39190) 2024-04-25T00:19:28.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 0x5591dc221640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41042/0 (socket says 172.21.15.42:41042) 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe6000640 1 -- 172.21.15.42:0/3485404978 learned_addr learned my addr 172.21.15.42:0/3485404978 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe7400640 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc20fe50 0x5591dc21a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe6000640 1 -- 172.21.15.42:0/3485404978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc20fe50 msgr2=0x5591dc21a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe6000640 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc20fe50 0x5591dc21a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe6000640 1 -- 172.21.15.42:0/3485404978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 msgr2=0x5591dc221640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe6000640 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 0x5591dc221640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe6000640 1 -- 172.21.15.42:0/3485404978 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591dbf53620 con 0x5591dc210840 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfe6000640 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21a600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4000c00 tx=0x7fcfd4004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c067c96cba25b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.863+0000 7fcfd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.864+0000 7fcfd0c00640 1 -- 172.21.15.42:0/3485404978 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4035040 con 0x5591dc210840 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.864+0000 7fcff057a740 1 -- 172.21.15.42:0/3485404978 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfe0003dd0 con 0x5591dc210840 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.864+0000 7fcfd0c00640 1 -- 172.21.15.42:0/3485404978 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fcfd4005450 con 0x5591dc210840 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.864+0000 7fcfd0c00640 1 -- 172.21.15.42:0/3485404978 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcfd404c070 con 0x5591dc210840 2024-04-25T00:19:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.864+0000 7fcff057a740 10 client.25243 fetch_fsmap learned FSMap version 24 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.864+0000 7fcff057a740 1 -- 172.21.15.42:0/3485404978 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5591dc2246e0 con 0x5591dc210840 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.864+0000 7fcfd0c00640 1 -- 172.21.15.42:0/3485404978 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfe0003dd0 con 0x5591dc210840 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.865+0000 7fcfd0c00640 1 -- 172.21.15.42:0/3485404978 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcfd40318f0 con 0x5591dc210840 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.865+0000 7fcff057a740 10 client.25243 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.865+0000 7fcff057a740 10 client.25243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.865+0000 7fcff057a740 20 client.25243 populate_metadata read hostname 'smithi042' 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.865+0000 7fcff057a740 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5591dc225dd0 0x5591dc2281e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.865+0000 7fcff057a740 4 client.25243 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.865+0000 7fcff057a740 1 -- 172.21.15.42:0/3485404978 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591dc17fff0 con 0x5591dc225dd0 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.865+0000 7fcfe7400640 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5591dc225dd0 0x5591dc2281e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.866+0000 7fcfe7400640 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5591dc225dd0 0x5591dc2281e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f95dd4f519416b8 server_cookie=826a1ce87f79a606 in_seq=0 out_seq=0 2024-04-25T00:19:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.866+0000 7fcfd0c00640 10 client.25243 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcfd0c00640 1 -- 172.21.15.42:0/3485404978 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591dc17fff0 con 0x5591dc225dd0 2024-04-25T00:19:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcfd0c00640 10 client.25243 handle_command_reply: tid=0 2024-04-25T00:19:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 client.25243 shutdown 2024-04-25T00:19:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 -- 172.21.15.42:0/3485404978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 msgr2=0x5591dc21a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21a600 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4000c00 tx=0x7fcfd4004270 comp rx=0 tx=0).stop 2024-04-25T00:19:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 -- 172.21.15.42:0/3485404978 shutdown_connections 2024-04-25T00:19:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5591dc225dd0 0x5591dc2281e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc2159b0 0x5591dc221640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21a600 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 --2- 172.21.15.42:0/3485404978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc20fe50 0x5591dc21a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 -- 172.21.15.42:0/3485404978 >> 172.21.15.42:0/3485404978 conn(0x5591dc20b4c0 msgr2=0x5591dc20a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcfd0c00640 0 client.25243 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 -- 172.21.15.42:0/3485404978 shutdown_connections 2024-04-25T00:19:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 1 -- 172.21.15.42:0/3485404978 wait complete. 2024-04-25T00:19:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.867+0000 7fcff057a740 20 client.25243 trim_cache size 0 max 16384 2024-04-25T00:19:28.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.885+0000 7fcff057a740 1 Processor -- start 2024-04-25T00:19:28.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.885+0000 7fcff057a740 1 -- start start 2024-04-25T00:19:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcff057a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 0x5591dc215080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcff057a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc210840 0x5591dc221600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcff057a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc2159b0 0x5591dc211a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcff057a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81ad570 con 0x5591dc210840 2024-04-25T00:19:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcff057a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81aae40 con 0x5591dc2159b0 2024-04-25T00:19:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcff057a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81a8c90 con 0x5591dc20fe50 2024-04-25T00:19:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcfe6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc210840 0x5591dc221600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcfe6000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc210840 0x5591dc221600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41054/0 (socket says 172.21.15.42:41054) 2024-04-25T00:19:28.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcfe6000640 1 -- 172.21.15.42:0/3872746655 learned_addr learned my addr 172.21.15.42:0/3872746655 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcfe7400640 1 --2- 172.21.15.42:0/3872746655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 0x5591dc215080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.886+0000 7fcfe7400640 1 --2- 172.21.15.42:0/3872746655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc2159b0 0x5591dc211a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.887+0000 7fcfe6000640 1 -- 172.21.15.42:0/3872746655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 msgr2=0x5591dc215080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.887+0000 7fcfe6000640 1 --2- 172.21.15.42:0/3872746655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 0x5591dc215080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.887+0000 7fcfe6000640 1 -- 172.21.15.42:0/3872746655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc2159b0 msgr2=0x5591dc211a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.887+0000 7fcfe6000640 1 --2- 172.21.15.42:0/3872746655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc2159b0 0x5591dc211a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.887+0000 7fcfe6000640 1 -- 172.21.15.42:0/3872746655 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591dc20b2f0 con 0x5591dc210840 2024-04-25T00:19:28.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.887+0000 7fcfe6000640 1 --2- 172.21.15.42:0/3872746655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc210840 0x5591dc221600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4031d90 tx=0x7fcfd402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1563b1a432b8dea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.887+0000 7fcfd3400640 1 -- 172.21.15.42:0/3872746655 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4000e60 con 0x5591dc210840 2024-04-25T00:19:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.887+0000 7fcfd3400640 1 -- 172.21.15.42:0/3872746655 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd4055070 con 0x5591dc210840 2024-04-25T00:19:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.887+0000 7fcfd3400640 1 -- 172.21.15.42:0/3872746655 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4035070 con 0x5591dc210840 2024-04-25T00:19:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.888+0000 7fcff057a740 1 -- 172.21.15.42:0/3872746655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc210840 msgr2=0x5591dc221600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.888+0000 7fcff057a740 1 --2- 172.21.15.42:0/3872746655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc210840 0x5591dc221600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4031d90 tx=0x7fcfd402a410 comp rx=0 tx=0).stop 2024-04-25T00:19:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.888+0000 7fcff057a740 1 -- 172.21.15.42:0/3872746655 shutdown_connections 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.888+0000 7fcff057a740 1 --2- 172.21.15.42:0/3872746655 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc2159b0 0x5591dc211a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.888+0000 7fcff057a740 1 --2- 172.21.15.42:0/3872746655 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc210840 0x5591dc221600 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.888+0000 7fcff057a740 1 --2- 172.21.15.42:0/3872746655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc20fe50 0x5591dc215080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.888+0000 7fcff057a740 1 -- 172.21.15.42:0/3872746655 >> 172.21.15.42:0/3872746655 conn(0x5591dc20b4c0 msgr2=0x5591dc20b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.888+0000 7fcff057a740 1 -- 172.21.15.42:0/3872746655 shutdown_connections 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.888+0000 7fcff057a740 1 -- 172.21.15.42:0/3872746655 wait complete. 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.889+0000 7fcff057a740 1 -- start start 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.889+0000 7fcff057a740 1 Processor -- start 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.889+0000 7fcff057a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc20fe50 0x5591dc21ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.889+0000 7fcff057a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.889+0000 7fcff057a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc2159b0 0x5591dc21feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.889+0000 7fcff057a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81a9300 con 0x5591dc20fe50 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.889+0000 7fcff057a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81aa770 con 0x5591dc2159b0 2024-04-25T00:19:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.889+0000 7fcff057a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81a99d0 con 0x5591dc210840 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.889+0000 7fcff057a740 10 client.0 _ll_register_callbacks cb 0x7fcfd8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc20fe50 0x5591dc21ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc20fe50 0x5591dc21ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:41060/0 (socket says 172.21.15.42:41060) 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe7400640 1 -- 172.21.15.42:0/2021484412 learned_addr learned my addr 172.21.15.42:0/2021484412 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe6000640 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe7400640 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc2159b0 0x5591dc21feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe7400640 1 -- 172.21.15.42:0/2021484412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 msgr2=0x5591dc21ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe7400640 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe7400640 1 -- 172.21.15.42:0/2021484412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc2159b0 msgr2=0x5591dc21feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe7400640 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc2159b0 0x5591dc21feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe7400640 1 -- 172.21.15.42:0/2021484412 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591dc2204a0 con 0x5591dc20fe50 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe6000640 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfe7400640 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc20fe50 0x5591dc21ea20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcfe001b040 tx=0x7fcfd4005d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472bcbc4fbf0058c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.890+0000 7fcfd0c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:19:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.891+0000 7fcfd0c00640 1 -- 172.21.15.42:0/2021484412 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0004af0 con 0x5591dc20fe50 2024-04-25T00:19:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.891+0000 7fcff057a740 1 -- 172.21.15.42:0/2021484412 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd4055070 con 0x5591dc20fe50 2024-04-25T00:19:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.891+0000 7fcfd0c00640 1 -- 172.21.15.42:0/2021484412 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fcfe0004c90 con 0x5591dc20fe50 2024-04-25T00:19:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.891+0000 7fcfd0c00640 1 -- 172.21.15.42:0/2021484412 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcfe0004000 con 0x5591dc20fe50 2024-04-25T00:19:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.891+0000 7fcff057a740 10 client.15141 fetch_fsmap learned FSMap version 24 2024-04-25T00:19:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.891+0000 7fcff057a740 1 -- 172.21.15.42:0/2021484412 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5591dc221800 con 0x5591dc20fe50 2024-04-25T00:19:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.891+0000 7fcfd0c00640 1 -- 172.21.15.42:0/2021484412 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd4055070 con 0x5591dc20fe50 2024-04-25T00:19:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.891+0000 7fcfd0c00640 1 -- 172.21.15.42:0/2021484412 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fcfe0025040 con 0x5591dc20fe50 2024-04-25T00:19:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.892+0000 7fcff057a740 10 client.15141 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:19:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.892+0000 7fcff057a740 10 client.15141 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.892+0000 7fcff057a740 20 client.15141 populate_metadata read hostname 'smithi042' 2024-04-25T00:19:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.892+0000 7fcff057a740 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5591dc28c870 0x5591dc28ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.892+0000 7fcff057a740 4 client.15141 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:19:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.892+0000 7fcff057a740 1 -- 172.21.15.42:0/2021484412 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591dc17a340 con 0x5591dc28c870 2024-04-25T00:19:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.892+0000 7fcfe6000640 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5591dc28c870 0x5591dc28ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.893+0000 7fcfe6000640 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5591dc28c870 0x5591dc28ec90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f8e782c06aff1a server_cookie=669fcd8b70617e56 in_seq=0 out_seq=0 2024-04-25T00:19:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.893+0000 7fcfd0c00640 10 client.15141 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcfd0c00640 1 -- 172.21.15.42:0/2021484412 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5591dc17a340 con 0x5591dc28c870 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcfd0c00640 10 client.15141 handle_command_reply: tid=0 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcff057a740 1 client.15141 shutdown 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcff057a740 1 -- 172.21.15.42:0/2021484412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc20fe50 msgr2=0x5591dc21ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcff057a740 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc20fe50 0x5591dc21ea20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcfe001b040 tx=0x7fcfd4005d70 comp rx=0 tx=0).stop 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcff057a740 1 -- 172.21.15.42:0/2021484412 shutdown_connections 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcff057a740 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5591dc28c870 0x5591dc28ec90 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcff057a740 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5591dc2159b0 0x5591dc21feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcfd0c00640 0 client.15141 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcff057a740 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5591dc210840 0x5591dc21ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcff057a740 1 --2- 172.21.15.42:0/2021484412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5591dc20fe50 0x5591dc21ea20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcff057a740 1 -- 172.21.15.42:0/2021484412 >> 172.21.15.42:0/2021484412 conn(0x5591dc20b4c0 msgr2=0x5591dc21a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.900+0000 7fcff057a740 1 -- 172.21.15.42:0/2021484412 shutdown_connections 2024-04-25T00:19:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.901+0000 7fcff057a740 1 -- 172.21.15.42:0/2021484412 wait complete. 2024-04-25T00:19:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.901+0000 7fcff057a740 20 client.15141 trim_cache size 0 max 16384 2024-04-25T00:19:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:19:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:19:28.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:19:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:19:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.901+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2515944077 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfcc0a2b80 msgr2=0x7fcfcc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.901+0000 7fcfe7e00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfcc0a2b80 0x7fcfcc0a5040 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x5591dc210c40 tx=0x7fcfd80d2ea0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.901+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2515944077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe8101660 msgr2=0x7fcfe8199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.901+0000 7fcfe7e00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe8101660 0x7fcfe8199b40 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fcfd800e9c0 tx=0x7fcfd800edc0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.901+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2515944077 shutdown_connections 2024-04-25T00:19:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.901+0000 7fcfe7e00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfcc0a2b80 0x7fcfcc0a5040 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.901+0000 7fcfe7e00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfe8103210 0x7fcfe81a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.902+0000 7fcfe7e00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfe8102850 0x7fcfe819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.902+0000 7fcfe7e00640 1 --2- 172.21.15.42:0/2515944077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfe8101660 0x7fcfe8199b40 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.902+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2515944077 >> 172.21.15.42:0/2515944077 conn(0x7fcfe80fcbd0 msgr2=0x7fcfe8073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.902+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2515944077 shutdown_connections 2024-04-25T00:19:28.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:28.902+0000 7fcfe7e00640 1 -- 172.21.15.42:0/2515944077 wait complete. 2024-04-25T00:19:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:29 smithi136 ceph-mon[32923]: pgmap v409: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T00:19:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:29 smithi106 ceph-mon[32510]: pgmap v409: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T00:19:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:29 smithi042 ceph-mon[28130]: pgmap v409: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T00:19:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:31 smithi136 ceph-mon[32923]: pgmap v410: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-25T00:19:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:31 smithi106 ceph-mon[32510]: pgmap v410: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-25T00:19:32.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:31 smithi042 ceph-mon[28130]: pgmap v410: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.4 KiB/s wr, 6 op/s 2024-04-25T00:19:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:33 smithi136 ceph-mon[32923]: pgmap v411: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-25T00:19:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:33 smithi106 ceph-mon[32510]: pgmap v411: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-25T00:19:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:33 smithi042 ceph-mon[28130]: pgmap v411: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.8 KiB/s wr, 9 op/s 2024-04-25T00:19:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:35 smithi136 ceph-mon[32923]: pgmap v412: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T00:19:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:35 smithi106 ceph-mon[32510]: pgmap v412: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T00:19:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:35 smithi042 ceph-mon[28130]: pgmap v412: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-04-25T00:19:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:37 smithi136 ceph-mon[32923]: pgmap v413: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T00:19:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:37 smithi106 ceph-mon[32510]: pgmap v413: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T00:19:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:37 smithi042 ceph-mon[28130]: pgmap v413: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-25T00:19:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:38 smithi136 ceph-mon[32923]: pgmap v414: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T00:19:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:38 smithi106 ceph-mon[32510]: pgmap v414: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T00:19:39.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:38 smithi042 ceph-mon[28130]: pgmap v414: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 7 op/s 2024-04-25T00:19:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:41 smithi106 ceph-mon[32510]: pgmap v415: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T00:19:41.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:41 smithi042 ceph-mon[28130]: pgmap v415: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T00:19:41.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:41 smithi136 ceph-mon[32923]: pgmap v415: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T00:19:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.051+0000 7f20afe00640 1 -- 172.21.15.42:0/3789583381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b0101600 msgr2=0x7f20b0101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.052+0000 7f20afe00640 1 --2- 172.21.15.42:0/3789583381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b0101600 0x7f20b0101a20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009900 tx=0x7f20a002a410 comp rx=0 tx=0).stop 2024-04-25T00:19:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.052+0000 7f20afe00640 1 -- 172.21.15.42:0/3789583381 shutdown_connections 2024-04-25T00:19:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.052+0000 7f20afe00640 1 --2- 172.21.15.42:0/3789583381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b01031b0 0x7f20b0109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.052+0000 7f20afe00640 1 --2- 172.21.15.42:0/3789583381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b01027f0 0x7f20b0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.052+0000 7f20afe00640 1 --2- 172.21.15.42:0/3789583381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b0101600 0x7f20b0101a20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.052+0000 7f20afe00640 1 -- 172.21.15.42:0/3789583381 >> 172.21.15.42:0/3789583381 conn(0x7f20b00fcb70 msgr2=0x7f20b00fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.052+0000 7f20afe00640 1 -- 172.21.15.42:0/3789583381 shutdown_connections 2024-04-25T00:19:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.052+0000 7f20afe00640 1 -- 172.21.15.42:0/3789583381 wait complete. 2024-04-25T00:19:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.053+0000 7f20afe00640 1 Processor -- start 2024-04-25T00:19:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.053+0000 7f20afe00640 1 -- start start 2024-04-25T00:19:42.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.054+0000 7f20afe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b0101600 0x7f20b00754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.054+0000 7f20afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b01027f0 0x7f20b0073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.054+0000 7f20afe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b01031b0 0x7f20b0074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.054+0000 7f20afe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f20b010d2c0 con 0x7f20b01027f0 2024-04-25T00:19:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.054+0000 7f20afe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20b010d140 con 0x7f20b0101600 2024-04-25T00:19:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.054+0000 7f20afe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20b010d440 con 0x7f20b01031b0 2024-04-25T00:19:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.054+0000 7f20aea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b0101600 0x7f20b00754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.054+0000 7f20aea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b0101600 0x7f20b00754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35562/0 (socket says 172.21.15.42:35562) 2024-04-25T00:19:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.054+0000 7f20aea00640 1 -- 172.21.15.42:0/3308425723 learned_addr learned my addr 172.21.15.42:0/3308425723 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.054+0000 7f20ae000640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b01027f0 0x7f20b0073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.055+0000 7f20af400640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b01031b0 0x7f20b0074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.055+0000 7f20aea00640 1 -- 172.21.15.42:0/3308425723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b01031b0 msgr2=0x7f20b0074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.055+0000 7f20aea00640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b01031b0 0x7f20b0074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.055+0000 7f20aea00640 1 -- 172.21.15.42:0/3308425723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b01027f0 msgr2=0x7f20b0073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.055+0000 7f20aea00640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b01027f0 0x7f20b0073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.055+0000 7f20aea00640 1 -- 172.21.15.42:0/3308425723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b0074900 con 0x7f20b0101600 2024-04-25T00:19:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.055+0000 7f20af400640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b01031b0 0x7f20b0074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.055+0000 7f20aea00640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b0101600 0x7f20b00754f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20a0031440 tx=0x7f20a0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.056+0000 7f20a7400640 1 -- 172.21.15.42:0/3308425723 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a003f030 con 0x7f20b0101600 2024-04-25T00:19:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.056+0000 7f20afe00640 1 -- 172.21.15.42:0/3308425723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b01a19d0 con 0x7f20b0101600 2024-04-25T00:19:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.056+0000 7f20a7400640 1 -- 172.21.15.42:0/3308425723 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a0003ce0 con 0x7f20b0101600 2024-04-25T00:19:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.056+0000 7f20a7400640 1 -- 172.21.15.42:0/3308425723 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0035070 con 0x7f20b0101600 2024-04-25T00:19:42.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.056+0000 7f20afe00640 1 -- 172.21.15.42:0/3308425723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b01a1f70 con 0x7f20b0101600 2024-04-25T00:19:42.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.060+0000 7f20b8b7c740 1 Processor -- start 2024-04-25T00:19:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20b8b7c740 1 -- start start 2024-04-25T00:19:42.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20a7400640 1 -- 172.21.15.42:0/3308425723 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f20a0042030 con 0x7f20b0101600 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 0x563eb0cf9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0cfe340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 0x563eb0d00e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563eb0c5bab0 con 0x563eb0cf8e50 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563eb0c04640 con 0x563eb0cfe9b0 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563eb0c57a70 con 0x563eb0cf9840 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20ae000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 0x563eb0d00e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20af400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 0x563eb0cf9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20ae000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 0x563eb0d00e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35568/0 (socket says 172.21.15.42:35568) 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20aea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0cfe340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.061+0000 7f20ae000640 1 -- 172.21.15.42:0/194548857 learned_addr learned my addr 172.21.15.42:0/194548857 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.062+0000 7f20aea00640 1 -- 172.21.15.42:0/194548857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 msgr2=0x563eb0d00e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.062+0000 7f20aea00640 1 --2- 172.21.15.42:0/194548857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 0x563eb0d00e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.062+0000 7f20aea00640 1 -- 172.21.15.42:0/194548857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 msgr2=0x563eb0cf9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.062+0000 7f20aea00640 1 --2- 172.21.15.42:0/194548857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 0x563eb0cf9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.062+0000 7f20aea00640 1 -- 172.21.15.42:0/194548857 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eb0970210 con 0x563eb0cf9840 2024-04-25T00:19:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.062+0000 7f20aea00640 1 --2- 172.21.15.42:0/194548857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0cfe340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f20a00ed0f0 tx=0x7f20a0106850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc2cb21c641373a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.063+0000 7f20a7400640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20940a2c00 0x7f20940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.063+0000 7f20a7400640 1 -- 172.21.15.42:0/3308425723 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f20a00ef570 con 0x7f20b0101600 2024-04-25T00:19:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.063+0000 7f209be00640 1 -- 172.21.15.42:0/194548857 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0113070 con 0x563eb0cf9840 2024-04-25T00:19:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.063+0000 7f209be00640 1 -- 172.21.15.42:0/194548857 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a00bc500 con 0x563eb0cf9840 2024-04-25T00:19:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.063+0000 7f209be00640 1 -- 172.21.15.42:0/194548857 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a00b80a0 con 0x563eb0cf9840 2024-04-25T00:19:42.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.063+0000 7f20af400640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20940a2c00 0x7f20940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.064+0000 7f20b8b7c740 1 -- 172.21.15.42:0/194548857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 msgr2=0x563eb0cfe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.064+0000 7f20af400640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20940a2c00 0x7f20940a50c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x563eb0cfa0a0 tx=0x7f20a8009800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:42.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.064+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/194548857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0cfe340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f20a00ed0f0 tx=0x7f20a0106850 comp rx=0 tx=0).stop 2024-04-25T00:19:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.064+0000 7f20b8b7c740 1 -- 172.21.15.42:0/194548857 shutdown_connections 2024-04-25T00:19:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.064+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/194548857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 0x563eb0d00e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.064+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/194548857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0cfe340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.064+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/194548857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 0x563eb0cf9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.064+0000 7f20b8b7c740 1 -- 172.21.15.42:0/194548857 >> 172.21.15.42:0/194548857 conn(0x563eb0cf44c0 msgr2=0x563eb0cf48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.064+0000 7f20b8b7c740 1 -- 172.21.15.42:0/194548857 shutdown_connections 2024-04-25T00:19:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.064+0000 7f20b8b7c740 1 -- 172.21.15.42:0/194548857 wait complete. 2024-04-25T00:19:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.065+0000 7f20b8b7c740 1 -- start start 2024-04-25T00:19:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.065+0000 7f20b8b7c740 1 Processor -- start 2024-04-25T00:19:42.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.065+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d02d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.065+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf9840 0x563eb0d032a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.065+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cfe9b0 0x563eb0d0a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.065+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563eb0d01720 con 0x563eb0cf9840 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563eb0c17990 con 0x563eb0cf8e50 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563eb0d018a0 con 0x563eb0cfe9b0 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20b8b7c740 10 client.0 _ll_register_callbacks cb 0x563eb0c19950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20aea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d02d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20aea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d02d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35570/0 (socket says 172.21.15.42:35570) 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20aea00640 1 -- 172.21.15.42:0/424527322 learned_addr learned my addr 172.21.15.42:0/424527322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20ae000640 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf9840 0x563eb0d032a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20af400640 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cfe9b0 0x563eb0d0a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20aea00640 1 -- 172.21.15.42:0/424527322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cfe9b0 msgr2=0x563eb0d0a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:42.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20aea00640 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cfe9b0 0x563eb0d0a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20aea00640 1 -- 172.21.15.42:0/424527322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf9840 msgr2=0x563eb0d032a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20aea00640 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf9840 0x563eb0d032a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20aea00640 1 -- 172.21.15.42:0/424527322 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eb0a3c620 con 0x563eb0cf8e50 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.066+0000 7f20ae000640 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf9840 0x563eb0d032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.067+0000 7f20aea00640 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d02d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f20a00ed0f0 tx=0x7f20a0032f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5619673463291197 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.067+0000 7f2098c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.067+0000 7f2098c00640 1 -- 172.21.15.42:0/424527322 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0049070 con 0x563eb0cf8e50 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.067+0000 7f20b8b7c740 1 -- 172.21.15.42:0/424527322 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a00bc500 con 0x563eb0cf8e50 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.067+0000 7f2098c00640 1 -- 172.21.15.42:0/424527322 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f20a00f2ce0 con 0x563eb0cf8e50 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.067+0000 7f2098c00640 1 -- 172.21.15.42:0/424527322 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f20a00bb760 con 0x563eb0cf8e50 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.067+0000 7f20b8b7c740 10 client.25253 fetch_fsmap learned FSMap version 24 2024-04-25T00:19:42.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.067+0000 7f20b8b7c740 1 -- 172.21.15.42:0/424527322 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563eb0d01720 con 0x563eb0cf8e50 2024-04-25T00:19:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.067+0000 7f2098c00640 1 -- 172.21.15.42:0/424527322 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a00bc500 con 0x563eb0cf8e50 2024-04-25T00:19:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.068+0000 7f2098c00640 1 -- 172.21.15.42:0/424527322 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f20a0003800 con 0x563eb0cf8e50 2024-04-25T00:19:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.068+0000 7f20b8b7c740 10 client.25253 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:19:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.068+0000 7f20b8b7c740 10 client.25253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.068+0000 7f20b8b7c740 20 client.25253 populate_metadata read hostname 'smithi042' 2024-04-25T00:19:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.068+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563eb0d0e6c0 0x563eb0d10ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.068+0000 7f20b8b7c740 4 client.25253 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.068+0000 7f20b8b7c740 1 -- 172.21.15.42:0/424527322 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eb0c68ff0 con 0x563eb0d0e6c0 2024-04-25T00:19:42.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.068+0000 7f20ae000640 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563eb0d0e6c0 0x563eb0d10ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.069+0000 7f2098c00640 10 client.25253 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.069+0000 7f20ae000640 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563eb0d0e6c0 0x563eb0d10ad0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b4594499d19d4e server_cookie=599dc292a22cda04 in_seq=0 out_seq=0 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f2098c00640 1 -- 172.21.15.42:0/424527322 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563eb0c68ff0 con 0x563eb0d0e6c0 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f2098c00640 10 client.25253 handle_command_reply: tid=0 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 client.25253 shutdown 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 -- 172.21.15.42:0/424527322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 msgr2=0x563eb0d02d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d02d60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f20a00ed0f0 tx=0x7f20a0032f90 comp rx=0 tx=0).stop 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 -- 172.21.15.42:0/424527322 shutdown_connections 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563eb0d0e6c0 0x563eb0d10ad0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cfe9b0 0x563eb0d0a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf9840 0x563eb0d032a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/424527322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d02d60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 -- 172.21.15.42:0/424527322 >> 172.21.15.42:0/424527322 conn(0x563eb0cf44c0 msgr2=0x563eb0cf3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f2098c00640 0 client.25253 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 -- 172.21.15.42:0/424527322 shutdown_connections 2024-04-25T00:19:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.070+0000 7f20b8b7c740 1 -- 172.21.15.42:0/424527322 wait complete. 2024-04-25T00:19:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.071+0000 7f20b8b7c740 20 client.25253 trim_cache size 0 max 16384 2024-04-25T00:19:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.101+0000 7f20b8b7c740 1 Processor -- start 2024-04-25T00:19:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.101+0000 7f20b8b7c740 1 -- start start 2024-04-25T00:19:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.101+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 0x563eb0cf73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.101+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0d0a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.101+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 0x563eb0cf7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.101+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f20b01a4d20 con 0x563eb0cf8e50 2024-04-25T00:19:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.101+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20b01a25f0 con 0x563eb0cfe9b0 2024-04-25T00:19:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0071c60 con 0x563eb0cf9840 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20aea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 0x563eb0cf73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20af400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 0x563eb0cf7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20aea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 0x563eb0cf73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56402/0 (socket says 172.21.15.42:56402) 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20af400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 0x563eb0cf7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35582/0 (socket says 172.21.15.42:35582) 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20aea00640 1 -- 172.21.15.42:0/2929922314 learned_addr learned my addr 172.21.15.42:0/2929922314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20ae000640 1 --2- 172.21.15.42:0/2929922314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0d0a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20aea00640 1 -- 172.21.15.42:0/2929922314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 msgr2=0x563eb0d0a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20aea00640 1 --2- 172.21.15.42:0/2929922314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0d0a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20aea00640 1 -- 172.21.15.42:0/2929922314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 msgr2=0x563eb0cf7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.102+0000 7f20aea00640 1 --2- 172.21.15.42:0/2929922314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 0x563eb0cf7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.103+0000 7f20aea00640 1 -- 172.21.15.42:0/2929922314 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eb0cf42f0 con 0x563eb0cf8e50 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.103+0000 7f20aea00640 1 --2- 172.21.15.42:0/2929922314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 0x563eb0cf73d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f20a0005d00 tx=0x7f20a003b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19de3681b020a3c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:42.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.103+0000 7f209b400640 1 -- 172.21.15.42:0/2929922314 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0049040 con 0x563eb0cf8e50 2024-04-25T00:19:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.103+0000 7f209b400640 1 -- 172.21.15.42:0/2929922314 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a0106e90 con 0x563eb0cf8e50 2024-04-25T00:19:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.103+0000 7f209b400640 1 -- 172.21.15.42:0/2929922314 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a00bb940 con 0x563eb0cf8e50 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.104+0000 7f20b8b7c740 1 -- 172.21.15.42:0/2929922314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 msgr2=0x563eb0cf73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.104+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/2929922314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 0x563eb0cf73d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f20a0005d00 tx=0x7f20a003b800 comp rx=0 tx=0).stop 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.104+0000 7f20b8b7c740 1 -- 172.21.15.42:0/2929922314 shutdown_connections 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.104+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/2929922314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cfe9b0 0x563eb0cf7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.104+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/2929922314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0d0a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.104+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/2929922314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cf8e50 0x563eb0cf73d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.104+0000 7f20b8b7c740 1 -- 172.21.15.42:0/2929922314 >> 172.21.15.42:0/2929922314 conn(0x563eb0cf44c0 msgr2=0x563eb0cfe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.104+0000 7f20b8b7c740 1 -- 172.21.15.42:0/2929922314 shutdown_connections 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.104+0000 7f20b8b7c740 1 -- 172.21.15.42:0/2929922314 wait complete. 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.105+0000 7f20b8b7c740 1 -- start start 2024-04-25T00:19:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.105+0000 7f20b8b7c740 1 Processor -- start 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d07910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0d07e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20b8b7c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cfe9b0 0x563eb0d08e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f20b00722d0 con 0x563eb0cfe9b0 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20b0073740 con 0x563eb0cf8e50 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20b8b7c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20b00729a0 con 0x563eb0cf9840 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20b8b7c740 10 client.0 _ll_register_callbacks cb 0x7f20a0111070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20aea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d07910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20aea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d07910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35584/0 (socket says 172.21.15.42:35584) 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20aea00640 1 -- 172.21.15.42:0/1738541236 learned_addr learned my addr 172.21.15.42:0/1738541236 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20af400640 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cfe9b0 0x563eb0d08e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.106+0000 7f20ae000640 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0d07e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f20aea00640 1 -- 172.21.15.42:0/1738541236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 msgr2=0x563eb0d07e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f20aea00640 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0d07e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f20aea00640 1 -- 172.21.15.42:0/1738541236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cfe9b0 msgr2=0x563eb0d08e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f20aea00640 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cfe9b0 0x563eb0d08e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f20aea00640 1 -- 172.21.15.42:0/1738541236 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eb0d09450 con 0x563eb0cf8e50 2024-04-25T00:19:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f20aea00640 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d07910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20a00bbd90 tx=0x7f20a0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82beb57de202006e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f2098c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:19:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f2098c00640 1 -- 172.21.15.42:0/1738541236 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a0049050 con 0x563eb0cf8e50 2024-04-25T00:19:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f2098c00640 1 -- 172.21.15.42:0/1738541236 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f20a00c47e0 con 0x563eb0cf8e50 2024-04-25T00:19:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f20b8b7c740 1 -- 172.21.15.42:0/1738541236 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a0106e90 con 0x563eb0cf8e50 2024-04-25T00:19:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f2098c00640 1 -- 172.21.15.42:0/1738541236 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f20a00a1050 con 0x563eb0cf8e50 2024-04-25T00:19:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f20b8b7c740 10 client.25259 fetch_fsmap learned FSMap version 24 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.107+0000 7f20b8b7c740 1 -- 172.21.15.42:0/1738541236 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x563eb0d09450 con 0x563eb0cf8e50 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.108+0000 7f2098c00640 1 -- 172.21.15.42:0/1738541236 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a0106e90 con 0x563eb0cf8e50 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.108+0000 7f2098c00640 1 -- 172.21.15.42:0/1738541236 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f20a00bc060 con 0x563eb0cf8e50 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.108+0000 7f20b8b7c740 10 client.25259 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.108+0000 7f20b8b7c740 10 client.25259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.108+0000 7f20b8b7c740 20 client.25259 populate_metadata read hostname 'smithi042' 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.108+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563eb0d75000 0x563eb0d773f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.108+0000 7f20b8b7c740 4 client.25259 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.108+0000 7f20b8b7c740 1 -- 172.21.15.42:0/1738541236 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563eb0d021a0 con 0x563eb0d75000 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.108+0000 7f20ae000640 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563eb0d75000 0x563eb0d773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.109+0000 7f20ae000640 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563eb0d75000 0x563eb0d773f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a52ca755188b9430 server_cookie=b1aff6c7e8287144 in_seq=0 out_seq=0 2024-04-25T00:19:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.109+0000 7f2098c00640 10 client.25259 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f2098c00640 1 -- 172.21.15.42:0/1738541236 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563eb0d021a0 con 0x563eb0d75000 2024-04-25T00:19:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f2098c00640 10 client.25259 handle_command_reply: tid=0 2024-04-25T00:19:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 client.25259 shutdown 2024-04-25T00:19:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 -- 172.21.15.42:0/1738541236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 msgr2=0x563eb0d07910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d07910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20a00bbd90 tx=0x7f20a0009260 comp rx=0 tx=0).stop 2024-04-25T00:19:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 -- 172.21.15.42:0/1738541236 shutdown_connections 2024-04-25T00:19:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563eb0d75000 0x563eb0d773f0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563eb0cfe9b0 0x563eb0d08e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563eb0cf9840 0x563eb0d07e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 --2- 172.21.15.42:0/1738541236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563eb0cf8e50 0x563eb0d07910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 -- 172.21.15.42:0/1738541236 >> 172.21.15.42:0/1738541236 conn(0x563eb0cf44c0 msgr2=0x563eb0cf73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f2098c00640 0 client.25259 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 -- 172.21.15.42:0/1738541236 shutdown_connections 2024-04-25T00:19:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.113+0000 7f20b8b7c740 1 -- 172.21.15.42:0/1738541236 wait complete. 2024-04-25T00:19:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.114+0000 7f20b8b7c740 20 client.25259 trim_cache size 0 max 16384 2024-04-25T00:19:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:19:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:19:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:19:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:19:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.114+0000 7f20afe00640 1 -- 172.21.15.42:0/3308425723 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20940a2c00 msgr2=0x7f20940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.114+0000 7f20afe00640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20940a2c00 0x7f20940a50c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x563eb0cfa0a0 tx=0x7f20a8009800 comp rx=0 tx=0).stop 2024-04-25T00:19:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.114+0000 7f20afe00640 1 -- 172.21.15.42:0/3308425723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b0101600 msgr2=0x7f20b00754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.114+0000 7f20afe00640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b0101600 0x7f20b00754f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f20a0031440 tx=0x7f20a0032ce0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.115+0000 7f20afe00640 1 -- 172.21.15.42:0/3308425723 shutdown_connections 2024-04-25T00:19:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.115+0000 7f20afe00640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20940a2c00 0x7f20940a50c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.115+0000 7f20afe00640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20b01031b0 0x7f20b0074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.115+0000 7f20afe00640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20b01027f0 0x7f20b0073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.115+0000 7f20afe00640 1 --2- 172.21.15.42:0/3308425723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20b0101600 0x7f20b00754f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.115+0000 7f20afe00640 1 -- 172.21.15.42:0/3308425723 >> 172.21.15.42:0/3308425723 conn(0x7f20b00fcb70 msgr2=0x7f20b0105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.115+0000 7f20afe00640 1 -- 172.21.15.42:0/3308425723 shutdown_connections 2024-04-25T00:19:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:42.115+0000 7f20afe00640 1 -- 172.21.15.42:0/3308425723 wait complete. 2024-04-25T00:19:43.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:43 smithi106 ceph-mon[32510]: pgmap v416: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T00:19:43.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:43 smithi042 ceph-mon[28130]: pgmap v416: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T00:19:43.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:43 smithi136 ceph-mon[32923]: pgmap v416: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T00:19:45.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:45 smithi106 ceph-mon[32510]: pgmap v417: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:19:45.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:45 smithi042 ceph-mon[28130]: pgmap v417: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:19:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:45 smithi136 ceph-mon[32923]: pgmap v417: 97 pgs: 97 active+clean; 664 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T00:19:47.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:47 smithi106 ceph-mon[32510]: pgmap v418: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:19:47.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:47 smithi042 ceph-mon[28130]: pgmap v418: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:19:47.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:47 smithi136 ceph-mon[32923]: pgmap v418: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:19:49.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:49 smithi106 ceph-mon[32510]: pgmap v419: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:19:49.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:49 smithi042 ceph-mon[28130]: pgmap v419: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:19:49.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:49 smithi136 ceph-mon[32923]: pgmap v419: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:19:51.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:51 smithi106 ceph-mon[32510]: pgmap v420: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:51.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:51 smithi042 ceph-mon[28130]: pgmap v420: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:51.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:51 smithi136 ceph-mon[32923]: pgmap v420: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:53.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:53 smithi106 ceph-mon[32510]: pgmap v421: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:19:53.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:53 smithi042 ceph-mon[28130]: pgmap v421: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:19:53.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:53 smithi136 ceph-mon[32923]: pgmap v421: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:19:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.304+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1846055159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e64100790 msgr2=0x7f5e64100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.304+0000 7f5e6ae00640 1 --2- 172.21.15.42:0/1846055159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e64100790 0x7f5e64100bf0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e50009900 tx=0x7f5e5002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.305+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1846055159 shutdown_connections 2024-04-25T00:19:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.305+0000 7f5e6ae00640 1 --2- 172.21.15.42:0/1846055159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e64101130 0x7f5e641015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.305+0000 7f5e6ae00640 1 --2- 172.21.15.42:0/1846055159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e64100790 0x7f5e64100bf0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.305+0000 7f5e6ae00640 1 --2- 172.21.15.42:0/1846055159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e640ffe30 0x7f5e64100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.305+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1846055159 >> 172.21.15.42:0/1846055159 conn(0x7f5e640fba30 msgr2=0x7f5e640fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.305+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1846055159 shutdown_connections 2024-04-25T00:19:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.305+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1846055159 wait complete. 2024-04-25T00:19:55.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.306+0000 7f5e6ae00640 1 Processor -- start 2024-04-25T00:19:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.307+0000 7f5e6ae00640 1 -- start start 2024-04-25T00:19:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.307+0000 7f5e6ae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e640ffe30 0x7f5e64199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.307+0000 7f5e6ae00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e64100790 0x7f5e6419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.307+0000 7f5e6ae00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e64101130 0x7f5e641a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.307+0000 7f5e6ae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6410f480 con 0x7f5e640ffe30 2024-04-25T00:19:55.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.307+0000 7f5e6ae00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6410f300 con 0x7f5e64101130 2024-04-25T00:19:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.307+0000 7f5e6ae00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6410f600 con 0x7f5e64100790 2024-04-25T00:19:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.307+0000 7f5e69a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e640ffe30 0x7f5e64199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.307+0000 7f5e69a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e640ffe30 0x7f5e64199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45590/0 (socket says 172.21.15.42:45590) 2024-04-25T00:19:55.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.307+0000 7f5e69a00640 1 -- 172.21.15.42:0/1395746418 learned_addr learned my addr 172.21.15.42:0/1395746418 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.308+0000 7f5e6a400640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e64101130 0x7f5e641a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.308+0000 7f5e69000640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e64100790 0x7f5e6419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.308+0000 7f5e69a00640 1 -- 172.21.15.42:0/1395746418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e64100790 msgr2=0x7f5e6419a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.308+0000 7f5e69a00640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e64100790 0x7f5e6419a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.308+0000 7f5e69a00640 1 -- 172.21.15.42:0/1395746418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e64101130 msgr2=0x7f5e641a1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.308+0000 7f5e69a00640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e64101130 0x7f5e641a1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.308+0000 7f5e69a00640 1 -- 172.21.15.42:0/1395746418 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e641a16e0 con 0x7f5e640ffe30 2024-04-25T00:19:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.308+0000 7f5e69a00640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e640ffe30 0x7f5e64199ac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800ea60 tx=0x7f5e5800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.309+0000 7f5e62000640 1 -- 172.21.15.42:0/1395746418 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5800cde0 con 0x7f5e640ffe30 2024-04-25T00:19:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.309+0000 7f5e62000640 1 -- 172.21.15.42:0/1395746418 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e580035f0 con 0x7f5e640ffe30 2024-04-25T00:19:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.309+0000 7f5e62000640 1 -- 172.21.15.42:0/1395746418 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58010670 con 0x7f5e640ffe30 2024-04-25T00:19:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.310+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1395746418 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e641a19d0 con 0x7f5e640ffe30 2024-04-25T00:19:55.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.310+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1395746418 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e641a1ea0 con 0x7f5e640ffe30 2024-04-25T00:19:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.314+0000 7f5e6f236740 1 Processor -- start 2024-04-25T00:19:55.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.314+0000 7f5e6f236740 1 -- start start 2024-04-25T00:19:55.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.314+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 0x55696c61b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.314+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61b840 0x55696c620340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 0x55696c622e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e6f236740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55696c57dab0 con 0x55696c61ae50 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e6f236740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55696c526640 con 0x55696c6209b0 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e6f236740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55696c579a70 con 0x55696c61b840 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e69000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 0x55696c622e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e6a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 0x55696c61b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e69000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 0x55696c622e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45658/0 (socket says 172.21.15.42:45658) 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e69000640 1 -- 172.21.15.42:0/1984855416 learned_addr learned my addr 172.21.15.42:0/1984855416 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e62000640 1 -- 172.21.15.42:0/1395746418 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e580037a0 con 0x7f5e640ffe30 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e69000640 1 -- 172.21.15.42:0/1984855416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61b840 msgr2=0x55696c620340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:19:55.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e69000640 1 --2- 172.21.15.42:0/1984855416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61b840 0x55696c620340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e69000640 1 -- 172.21.15.42:0/1984855416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 msgr2=0x55696c61b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e69000640 1 --2- 172.21.15.42:0/1984855416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 0x55696c61b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.315+0000 7f5e69000640 1 -- 172.21.15.42:0/1984855416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696c292210 con 0x55696c6209b0 2024-04-25T00:19:55.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.317+0000 7f5e69000640 1 --2- 172.21.15.42:0/1984855416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 0x55696c622e40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e50000c00 tx=0x7f5e50004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5da8e1defe67ce21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.318+0000 7f5e62000640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5e4c0a2c00 0x7f5e4c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.318+0000 7f5e62000640 1 -- 172.21.15.42:0/1395746418 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5e580cf580 con 0x7f5e640ffe30 2024-04-25T00:19:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.318+0000 7f5e56a00640 1 -- 172.21.15.42:0/1984855416 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50035040 con 0x55696c6209b0 2024-04-25T00:19:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.318+0000 7f5e6a400640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5e4c0a2c00 0x7f5e4c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.318+0000 7f5e56a00640 1 -- 172.21.15.42:0/1984855416 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e50005520 con 0x55696c6209b0 2024-04-25T00:19:55.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.318+0000 7f5e56a00640 1 -- 172.21.15.42:0/1984855416 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e500316b0 con 0x55696c6209b0 2024-04-25T00:19:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.319+0000 7f5e6f236740 1 -- 172.21.15.42:0/1984855416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 msgr2=0x55696c622e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.319+0000 7f5e6f236740 1 --2- 172.21.15.42:0/1984855416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 0x55696c622e40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5e50000c00 tx=0x7f5e50004040 comp rx=0 tx=0).stop 2024-04-25T00:19:55.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.319+0000 7f5e6a400640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5e4c0a2c00 0x7f5e4c0a50c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55696c61c0a0 tx=0x7f5e5c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.319+0000 7f5e6f236740 1 -- 172.21.15.42:0/1984855416 shutdown_connections 2024-04-25T00:19:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.319+0000 7f5e6f236740 1 --2- 172.21.15.42:0/1984855416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 0x55696c622e40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.319+0000 7f5e6f236740 1 --2- 172.21.15.42:0/1984855416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61b840 0x55696c620340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.319+0000 7f5e6f236740 1 --2- 172.21.15.42:0/1984855416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 0x55696c61b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.319+0000 7f5e6f236740 1 -- 172.21.15.42:0/1984855416 >> 172.21.15.42:0/1984855416 conn(0x55696c6164c0 msgr2=0x55696c6168c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.319+0000 7f5e6f236740 1 -- 172.21.15.42:0/1984855416 shutdown_connections 2024-04-25T00:19:55.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.319+0000 7f5e6f236740 1 -- 172.21.15.42:0/1984855416 wait complete. 2024-04-25T00:19:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.320+0000 7f5e6f236740 1 -- start start 2024-04-25T00:19:55.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.320+0000 7f5e6f236740 1 Processor -- start 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61ae50 0x55696c62d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 0x55696c62de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 0x55696c634e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e6f236740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55696c623900 con 0x55696c61b840 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e6f236740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55696c539990 con 0x55696c6209b0 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e6f236740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55696c623a80 con 0x55696c61ae50 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e6f236740 10 client.0 _ll_register_callbacks cb 0x55696c53b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e69a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61ae50 0x55696c62d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e69a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61ae50 0x55696c62d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44382/0 (socket says 172.21.15.42:44382) 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e69a00640 1 -- 172.21.15.42:0/1301431075 learned_addr learned my addr 172.21.15.42:0/1301431075 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e6a400640 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 0x55696c634e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.321+0000 7f5e69000640 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 0x55696c62de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e69a00640 1 -- 172.21.15.42:0/1301431075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 msgr2=0x55696c634e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e69a00640 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 0x55696c634e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e69a00640 1 -- 172.21.15.42:0/1301431075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 msgr2=0x55696c62de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e69a00640 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 0x55696c62de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e69a00640 1 -- 172.21.15.42:0/1301431075 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696c35e620 con 0x55696c61ae50 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e69000640 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 0x55696c62de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e69a00640 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61ae50 0x55696c62d900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e580a44f0 tx=0x7f5e5809b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abdc5f92f29e96f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e4b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e4b400640 1 -- 172.21.15.42:0/1301431075 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e580a4a20 con 0x55696c61ae50 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e4b400640 1 -- 172.21.15.42:0/1301431075 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5e580cf0e0 con 0x55696c61ae50 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.322+0000 7f5e6f236740 1 -- 172.21.15.42:0/1301431075 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e50005520 con 0x55696c61ae50 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e4b400640 1 -- 172.21.15.42:0/1301431075 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5e580cf2e0 con 0x55696c61ae50 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e6f236740 10 client.25276 fetch_fsmap learned FSMap version 24 2024-04-25T00:19:55.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e6f236740 1 -- 172.21.15.42:0/1301431075 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55696c637e60 con 0x55696c61ae50 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e4b400640 1 -- 172.21.15.42:0/1301431075 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e50005520 con 0x55696c61ae50 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e4b400640 1 -- 172.21.15.42:0/1301431075 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5e58020070 con 0x55696c61ae50 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e6f236740 10 client.25276 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e6f236740 10 client.25276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e6f236740 20 client.25276 populate_metadata read hostname 'smithi042' 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e6f236740 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55696c639460 0x55696c63b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e6f236740 4 client.25276 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.323+0000 7f5e6f236740 1 -- 172.21.15.42:0/1301431075 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55696c58aff0 con 0x55696c639460 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.324+0000 7f5e69000640 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55696c639460 0x55696c63b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.324+0000 7f5e4b400640 10 client.25276 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.325+0000 7f5e69000640 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55696c639460 0x55696c63b8b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba86b309589069c4 server_cookie=a10358a341563dd in_seq=0 out_seq=0 2024-04-25T00:19:55.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e4b400640 1 -- 172.21.15.42:0/1301431075 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55696c58aff0 con 0x55696c639460 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e4b400640 10 client.25276 handle_command_reply: tid=0 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 client.25276 shutdown 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 -- 172.21.15.42:0/1301431075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61ae50 msgr2=0x55696c62d900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61ae50 0x55696c62d900 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e580a44f0 tx=0x7f5e5809b800 comp rx=0 tx=0).stop 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 -- 172.21.15.42:0/1301431075 shutdown_connections 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55696c639460 0x55696c63b8b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c6209b0 0x55696c634e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 0x55696c62de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 --2- 172.21.15.42:0/1301431075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c61ae50 0x55696c62d900 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 -- 172.21.15.42:0/1301431075 >> 172.21.15.42:0/1301431075 conn(0x55696c6164c0 msgr2=0x55696c615620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e4b400640 0 client.25276 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 -- 172.21.15.42:0/1301431075 shutdown_connections 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.326+0000 7f5e6f236740 1 -- 172.21.15.42:0/1301431075 wait complete. 2024-04-25T00:19:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.327+0000 7f5e6f236740 20 client.25276 trim_cache size 0 max 16384 2024-04-25T00:19:55.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:55 smithi106 ceph-mon[32510]: pgmap v422: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.359+0000 7f5e6f236740 1 Processor -- start 2024-04-25T00:19:55.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.359+0000 7f5e6f236740 1 -- start start 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.359+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c62dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.359+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 0x55696c634dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.359+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c61b840 0x55696c6401b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.359+0000 7f5e6f236740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641a8ff0 con 0x55696c61ae50 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.359+0000 7f5e6f236740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641a68c0 con 0x55696c61b840 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.359+0000 7f5e6f236740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641a4710 con 0x55696c6164c0 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e69000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 0x55696c634dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e69000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 0x55696c634dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45612/0 (socket says 172.21.15.42:45612) 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e69000640 1 -- 172.21.15.42:0/3067347336 learned_addr learned my addr 172.21.15.42:0/3067347336 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e69a00640 1 --2- 172.21.15.42:0/3067347336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c62dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e6a400640 1 --2- 172.21.15.42:0/3067347336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c61b840 0x55696c6401b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e69000640 1 -- 172.21.15.42:0/3067347336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 msgr2=0x55696c62dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e69000640 1 --2- 172.21.15.42:0/3067347336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c62dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e69000640 1 -- 172.21.15.42:0/3067347336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c61b840 msgr2=0x55696c6401b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e69000640 1 --2- 172.21.15.42:0/3067347336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c61b840 0x55696c6401b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e69000640 1 -- 172.21.15.42:0/3067347336 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55696c6162f0 con 0x55696c61ae50 2024-04-25T00:19:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.360+0000 7f5e69a00640 1 --2- 172.21.15.42:0/3067347336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c62dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:19:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.361+0000 7f5e69000640 1 --2- 172.21.15.42:0/3067347336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 0x55696c634dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5e50005ec0 tx=0x7f5e5002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a4e4b890b7dced server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.361+0000 7f5e56000640 1 -- 172.21.15.42:0/3067347336 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50053020 con 0x55696c61ae50 2024-04-25T00:19:55.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.361+0000 7f5e56000640 1 -- 172.21.15.42:0/3067347336 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e500538a0 con 0x55696c61ae50 2024-04-25T00:19:55.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.361+0000 7f5e56000640 1 -- 172.21.15.42:0/3067347336 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e50035040 con 0x55696c61ae50 2024-04-25T00:19:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.362+0000 7f5e6f236740 1 -- 172.21.15.42:0/3067347336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 msgr2=0x55696c634dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.362+0000 7f5e6f236740 1 --2- 172.21.15.42:0/3067347336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 0x55696c634dd0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5e50005ec0 tx=0x7f5e5002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.362+0000 7f5e6f236740 1 -- 172.21.15.42:0/3067347336 shutdown_connections 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.362+0000 7f5e6f236740 1 --2- 172.21.15.42:0/3067347336 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c61b840 0x55696c6401b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.362+0000 7f5e6f236740 1 --2- 172.21.15.42:0/3067347336 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61ae50 0x55696c634dd0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.362+0000 7f5e6f236740 1 --2- 172.21.15.42:0/3067347336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c62dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.362+0000 7f5e6f236740 1 -- 172.21.15.42:0/3067347336 >> 172.21.15.42:0/3067347336 conn(0x55696c620480 msgr2=0x55696c62d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.362+0000 7f5e6f236740 1 -- 172.21.15.42:0/3067347336 shutdown_connections 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.362+0000 7f5e6f236740 1 -- 172.21.15.42:0/3067347336 wait complete. 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.363+0000 7f5e6f236740 1 -- start start 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.363+0000 7f5e6f236740 1 Processor -- start 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.363+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c634770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.363+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c61ae50 0x55696c634cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.363+0000 7f5e6f236740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 0x55696c632d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.363+0000 7f5e6f236740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641a4d80 con 0x55696c61b840 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.363+0000 7f5e6f236740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641a61f0 con 0x55696c61ae50 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.363+0000 7f5e6f236740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5e641a5450 con 0x55696c6164c0 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.363+0000 7f5e6f236740 10 client.0 _ll_register_callbacks cb 0x7f5e58003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e69a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c634770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e69a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c634770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44394/0 (socket says 172.21.15.42:44394) 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e69a00640 1 -- 172.21.15.42:0/3941794027 learned_addr learned my addr 172.21.15.42:0/3941794027 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e69000640 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c61ae50 0x55696c634cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e69a00640 1 -- 172.21.15.42:0/3941794027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c61ae50 msgr2=0x55696c634cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e6a400640 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 0x55696c632d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e69a00640 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c61ae50 0x55696c634cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e69a00640 1 -- 172.21.15.42:0/3941794027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 msgr2=0x55696c632d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e69a00640 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 0x55696c632d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e69a00640 1 -- 172.21.15.42:0/3941794027 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55696c633490 con 0x55696c6164c0 2024-04-25T00:19:55.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.364+0000 7f5e6a400640 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 0x55696c632d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:19:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.365+0000 7f5e69a00640 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c634770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e5809bab0 tx=0x7f5e580095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c37d1cfc0802addb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:19:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.365+0000 7f5e4b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:19:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.365+0000 7f5e4b400640 1 -- 172.21.15.42:0/3941794027 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e58081060 con 0x55696c6164c0 2024-04-25T00:19:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.365+0000 7f5e4b400640 1 -- 172.21.15.42:0/3941794027 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f5e580cf0d0 con 0x55696c6164c0 2024-04-25T00:19:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.365+0000 7f5e6f236740 1 -- 172.21.15.42:0/3941794027 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e500538a0 con 0x55696c6164c0 2024-04-25T00:19:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.365+0000 7f5e4b400640 1 -- 172.21.15.42:0/3941794027 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f5e580a4a20 con 0x55696c6164c0 2024-04-25T00:19:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.365+0000 7f5e6f236740 10 client.25282 fetch_fsmap learned FSMap version 24 2024-04-25T00:19:55.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.365+0000 7f5e6f236740 1 -- 172.21.15.42:0/3941794027 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55696c6a04b0 con 0x55696c6164c0 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.366+0000 7f5e4b400640 1 -- 172.21.15.42:0/3941794027 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e500538a0 con 0x55696c6164c0 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.366+0000 7f5e4b400640 1 -- 172.21.15.42:0/3941794027 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f5e58020070 con 0x55696c6164c0 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.366+0000 7f5e6f236740 10 client.25282 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.366+0000 7f5e6f236740 10 client.25282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.366+0000 7f5e6f236740 20 client.25282 populate_metadata read hostname 'smithi042' 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.366+0000 7f5e6f236740 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55696c6209b0 0x55696c6a2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.366+0000 7f5e6f236740 4 client.25282 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.366+0000 7f5e6f236740 1 -- 172.21.15.42:0/3941794027 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55696c585340 con 0x55696c6209b0 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.366+0000 7f5e69000640 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55696c6209b0 0x55696c6a2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.367+0000 7f5e69000640 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55696c6209b0 0x55696c6a2dc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba7bd20e444af02b server_cookie=6952894eae209078 in_seq=0 out_seq=0 2024-04-25T00:19:55.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.367+0000 7f5e4b400640 10 client.25282 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e4b400640 1 -- 172.21.15.42:0/3941794027 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55696c585340 con 0x55696c6209b0 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e4b400640 10 client.25282 handle_command_reply: tid=0 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e6f236740 1 client.25282 shutdown 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e6f236740 1 -- 172.21.15.42:0/3941794027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 msgr2=0x55696c634770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e6f236740 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c634770 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5e5809bab0 tx=0x7f5e580095a0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e6f236740 1 -- 172.21.15.42:0/3941794027 shutdown_connections 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e6f236740 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55696c6209b0 0x55696c6a2dc0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e6f236740 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55696c61b840 0x55696c632d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e6f236740 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55696c61ae50 0x55696c634cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e6f236740 1 --2- 172.21.15.42:0/3941794027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55696c6164c0 0x55696c634770 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e6f236740 1 -- 172.21.15.42:0/3941794027 >> 172.21.15.42:0/3941794027 conn(0x55696c620480 msgr2=0x55696c614d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.371+0000 7f5e4b400640 0 client.25282 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:19:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.372+0000 7f5e6f236740 1 -- 172.21.15.42:0/3941794027 shutdown_connections 2024-04-25T00:19:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.372+0000 7f5e6f236740 1 -- 172.21.15.42:0/3941794027 wait complete. 2024-04-25T00:19:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.372+0000 7f5e6f236740 20 client.25282 trim_cache size 0 max 16384 2024-04-25T00:19:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:19:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:19:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:19:55.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:19:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.372+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1395746418 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5e4c0a2c00 msgr2=0x7f5e4c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.372+0000 7f5e6ae00640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5e4c0a2c00 0x7f5e4c0a50c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x55696c61c0a0 tx=0x7f5e5c009500 comp rx=0 tx=0).stop 2024-04-25T00:19:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.372+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1395746418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e640ffe30 msgr2=0x7f5e64199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:19:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.372+0000 7f5e6ae00640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e640ffe30 0x7f5e64199ac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5e5800ea60 tx=0x7f5e5800ef20 comp rx=0 tx=0).stop 2024-04-25T00:19:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.373+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1395746418 shutdown_connections 2024-04-25T00:19:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.373+0000 7f5e6ae00640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5e4c0a2c00 0x7f5e4c0a50c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.373+0000 7f5e6ae00640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e64101130 0x7f5e641a1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.373+0000 7f5e6ae00640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5e64100790 0x7f5e6419a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.373+0000 7f5e6ae00640 1 --2- 172.21.15.42:0/1395746418 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5e640ffe30 0x7f5e64199ac0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:19:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.373+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1395746418 >> 172.21.15.42:0/1395746418 conn(0x7f5e640fba30 msgr2=0x7f5e640fd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:19:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.373+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1395746418 shutdown_connections 2024-04-25T00:19:55.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:19:55.373+0000 7f5e6ae00640 1 -- 172.21.15.42:0/1395746418 wait complete. 2024-04-25T00:19:55.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:55 smithi042 ceph-mon[28130]: pgmap v422: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:55.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:55 smithi136 ceph-mon[32923]: pgmap v422: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:19:57.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:57 smithi106 ceph-mon[32510]: pgmap v423: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:19:57.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:57.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:57 smithi042 ceph-mon[28130]: pgmap v423: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:19:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:57 smithi136 ceph-mon[32923]: pgmap v423: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:19:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:19:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:58.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:58.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:58.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:58.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:58.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:58.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:58.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:19:58.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:19:58.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:19:59.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:19:59 smithi042 ceph-mon[28130]: pgmap v424: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:20:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:19:59 smithi136 ceph-mon[32923]: pgmap v424: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:20:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:19:59 smithi106 ceph-mon[32510]: pgmap v424: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-04-25T00:20:01.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:00 smithi136 ceph-mon[32923]: overall HEALTH_OK 2024-04-25T00:20:01.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:00 smithi106 ceph-mon[32510]: overall HEALTH_OK 2024-04-25T00:20:01.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:00 smithi042 ceph-mon[28130]: overall HEALTH_OK 2024-04-25T00:20:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:01 smithi136 ceph-mon[32923]: pgmap v425: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:20:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:01 smithi106 ceph-mon[32510]: pgmap v425: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:20:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:01 smithi042 ceph-mon[28130]: pgmap v425: 97 pgs: 97 active+clean; 665 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-25T00:20:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:03 smithi136 ceph-mon[32923]: pgmap v426: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:20:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:03 smithi106 ceph-mon[32510]: pgmap v426: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:20:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:03 smithi042 ceph-mon[28130]: pgmap v426: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T00:20:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:05 smithi136 ceph-mon[32923]: pgmap v427: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:20:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:05 smithi106 ceph-mon[32510]: pgmap v427: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:20:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:05 smithi042 ceph-mon[28130]: pgmap v427: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:20:07.910 INFO:tasks.workunit.client.0.smithi042.stderr:Updating files: 60% (7555/12492) Updating files: 61% (7621/12492) Updating files: 62% (7746/12492) Updating files: 63% (7870/12492) Updating files: 64% (7995/12492) Updating files: 65% (8120/12492) Updating files: 66% (8245/12492) Updating files: 67% (8370/12492) Updating files: 68% (8495/12492) Updating files: 69% (8620/12492) Updating files: 70% (8745/12492) Updating files: 71% (8870/12492) Updating files: 72% (8995/12492) Updating files: 73% (9120/12492) Updating files: 74% (9245/12492) Updating files: 75% (9369/12492) Updating files: 76% (9494/12492) Updating files: 77% (9619/12492) Updating files: 78% (9744/12492) Updating files: 79% (9869/12492) Updating files: 80% (9994/12492) Updating files: 81% (10119/12492) Updating files: 82% (10244/12492) Updating files: 83% (10369/12492) Updating files: 84% (10494/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 96% (12082/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-25T00:20:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:07 smithi136 ceph-mon[32923]: pgmap v428: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:20:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:07 smithi106 ceph-mon[32510]: pgmap v428: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:20:08.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:07 smithi042 ceph-mon[28130]: pgmap v428: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:20:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.507+0000 7f95abe00640 1 -- 172.21.15.42:0/3340674520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1004c0 msgr2=0x7f95ac1008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.507+0000 7f95abe00640 1 --2- 172.21.15.42:0/3340674520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1004c0 0x7f95ac1008e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f959c009ca0 tx=0x7f959c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.507+0000 7f95abe00640 1 -- 172.21.15.42:0/3340674520 shutdown_connections 2024-04-25T00:20:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.507+0000 7f95abe00640 1 --2- 172.21.15.42:0/3340674520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95ac102070 0x7f95ac108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.507+0000 7f95abe00640 1 --2- 172.21.15.42:0/3340674520 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ac1016b0 0x7f95ac101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.507+0000 7f95abe00640 1 --2- 172.21.15.42:0/3340674520 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1004c0 0x7f95ac1008e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.507+0000 7f95abe00640 1 -- 172.21.15.42:0/3340674520 >> 172.21.15.42:0/3340674520 conn(0x7f95ac0fba30 msgr2=0x7f95ac0fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.508+0000 7f95abe00640 1 -- 172.21.15.42:0/3340674520 shutdown_connections 2024-04-25T00:20:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.508+0000 7f95abe00640 1 -- 172.21.15.42:0/3340674520 wait complete. 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.508+0000 7f95abe00640 1 Processor -- start 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95abe00640 1 -- start start 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95abe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1004c0 0x7f95ac1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95abe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95ac1016b0 0x7f95ac197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95abe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ac102070 0x7f95ac19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95abe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac10c1a0 con 0x7f95ac1004c0 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95abe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac10c020 con 0x7f95ac1016b0 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95abe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac10c320 con 0x7f95ac102070 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1004c0 0x7f95ac1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1004c0 0x7f95ac1979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51054/0 (socket says 172.21.15.42:51054) 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95aaa00640 1 -- 172.21.15.42:0/1597944136 learned_addr learned my addr 172.21.15.42:0/1597944136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.509+0000 7f95aa000640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95ac1016b0 0x7f95ac197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.510+0000 7f95ab400640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ac102070 0x7f95ac19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.510+0000 7f95aa000640 1 -- 172.21.15.42:0/1597944136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ac102070 msgr2=0x7f95ac19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.510+0000 7f95aa000640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ac102070 0x7f95ac19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.510+0000 7f95aa000640 1 -- 172.21.15.42:0/1597944136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1004c0 msgr2=0x7f95ac1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.510+0000 7f95aa000640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1004c0 0x7f95ac1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.510+0000 7f95aa000640 1 -- 172.21.15.42:0/1597944136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95ac19f580 con 0x7f95ac1016b0 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.510+0000 7f95aa000640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95ac1016b0 0x7f95ac197ef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f959400e9f0 tx=0x7f959400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.510+0000 7f95b0a00640 1 -- 172.21.15.42:0/1597944136 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959400ccc0 con 0x7f95ac1016b0 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.510+0000 7f95abe00640 1 -- 172.21.15.42:0/1597944136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95ac19f870 con 0x7f95ac1016b0 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.511+0000 7f95b0a00640 1 -- 172.21.15.42:0/1597944136 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9594005020 con 0x7f95ac1016b0 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.511+0000 7f95b0a00640 1 -- 172.21.15.42:0/1597944136 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940049e0 con 0x7f95ac1016b0 2024-04-25T00:20:08.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.511+0000 7f95abe00640 1 -- 172.21.15.42:0/1597944136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95ac19fdc0 con 0x7f95ac1016b0 2024-04-25T00:20:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95b4b69740 1 Processor -- start 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95b4b69740 1 -- start start 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c96270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c9b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975c9de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95b4b69740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b975bf8ab0 con 0x55b975c96840 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95b4b69740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b975ba1640 con 0x55b975c95e50 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95b4b69740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b975bf4a70 con 0x55b975c9b9b0 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95b0a00640 1 -- 172.21.15.42:0/1597944136 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9594005560 con 0x7f95ac1016b0 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c9b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95ab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c96270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95aaa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c9b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51060/0 (socket says 172.21.15.42:51060) 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.514+0000 7f95aaa00640 1 -- 172.21.15.42:0/17183509 learned_addr learned my addr 172.21.15.42:0/17183509 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.515+0000 7f95aa000640 1 --2- 172.21.15.42:0/17183509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975c9de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.515+0000 7f95aaa00640 1 -- 172.21.15.42:0/17183509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 msgr2=0x55b975c9de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.515+0000 7f95aaa00640 1 --2- 172.21.15.42:0/17183509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975c9de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.515+0000 7f95aaa00640 1 -- 172.21.15.42:0/17183509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 msgr2=0x55b975c96270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.515+0000 7f95aaa00640 1 --2- 172.21.15.42:0/17183509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c96270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.515+0000 7f95aaa00640 1 -- 172.21.15.42:0/17183509 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b97590d210 con 0x55b975c96840 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.515+0000 7f95ab400640 1 --2- 172.21.15.42:0/17183509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c96270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.515+0000 7f95aaa00640 1 --2- 172.21.15.42:0/17183509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c9b340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f959c02ade0 tx=0x7f959c005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789f26064e78155a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.516+0000 7f959be00640 1 -- 172.21.15.42:0/17183509 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c031650 con 0x55b975c96840 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.516+0000 7f959be00640 1 -- 172.21.15.42:0/17183509 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f959c036030 con 0x55b975c96840 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.516+0000 7f959be00640 1 -- 172.21.15.42:0/17183509 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c004730 con 0x55b975c96840 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.516+0000 7f95b0a00640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95900a2b80 0x7f95900a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.516+0000 7f95b0a00640 1 -- 172.21.15.42:0/1597944136 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f95940cef90 con 0x7f95ac1016b0 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.516+0000 7f95ab400640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95900a2b80 0x7f95900a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.516+0000 7f95b4b69740 1 -- 172.21.15.42:0/17183509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 msgr2=0x55b975c9b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.516+0000 7f95b4b69740 1 --2- 172.21.15.42:0/17183509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c9b340 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f959c02ade0 tx=0x7f959c005f60 comp rx=0 tx=0).stop 2024-04-25T00:20:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.517+0000 7f95b4b69740 1 -- 172.21.15.42:0/17183509 shutdown_connections 2024-04-25T00:20:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.517+0000 7f95b4b69740 1 --2- 172.21.15.42:0/17183509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975c9de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.517+0000 7f95b4b69740 1 --2- 172.21.15.42:0/17183509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c9b340 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.517+0000 7f95b4b69740 1 --2- 172.21.15.42:0/17183509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c96270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.517+0000 7f95b4b69740 1 -- 172.21.15.42:0/17183509 >> 172.21.15.42:0/17183509 conn(0x55b975c914c0 msgr2=0x55b975c918c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.517+0000 7f95b4b69740 1 -- 172.21.15.42:0/17183509 shutdown_connections 2024-04-25T00:20:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.517+0000 7f95b4b69740 1 -- 172.21.15.42:0/17183509 wait complete. 2024-04-25T00:20:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.517+0000 7f95ab400640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95900a2b80 0x7f95900a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55b975c970a0 tx=0x7f95a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.517+0000 7f95b4b69740 1 -- start start 2024-04-25T00:20:08.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95b4b69740 1 Processor -- start 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c9fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975ca0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975ca7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95b4b69740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b975c9e780 con 0x55b975c96840 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95b4b69740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b975bb4990 con 0x55b975c95e50 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95b4b69740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b975c9e900 con 0x55b975c9b9b0 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95b4b69740 10 client.0 _ll_register_callbacks cb 0x55b975bb6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95aaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c9fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95aa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975ca0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95aaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c9fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44310/0 (socket says 172.21.15.42:44310) 2024-04-25T00:20:08.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95aaa00640 1 -- 172.21.15.42:0/4198633242 learned_addr learned my addr 172.21.15.42:0/4198633242 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.518+0000 7f95ab400640 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975ca7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.519+0000 7f95aaa00640 1 -- 172.21.15.42:0/4198633242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 msgr2=0x55b975ca7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.519+0000 7f95aaa00640 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975ca7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.519+0000 7f95aaa00640 1 -- 172.21.15.42:0/4198633242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 msgr2=0x55b975ca0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.519+0000 7f95aaa00640 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975ca0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.519+0000 7f95aaa00640 1 -- 172.21.15.42:0/4198633242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9759d9620 con 0x55b975c95e50 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.519+0000 7f95aaa00640 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c9fec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f959c000f80 tx=0x7f959c031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd58a6f39912f48a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.519+0000 7f9598c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.519+0000 7f9598c00640 1 -- 172.21.15.42:0/4198633242 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c004860 con 0x55b975c95e50 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.520+0000 7f9598c00640 1 -- 172.21.15.42:0/4198633242 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f959c003d80 con 0x55b975c95e50 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.520+0000 7f95b4b69740 1 -- 172.21.15.42:0/4198633242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f959c036030 con 0x55b975c95e50 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.520+0000 7f9598c00640 1 -- 172.21.15.42:0/4198633242 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f959c003f80 con 0x55b975c95e50 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.520+0000 7f95b4b69740 10 client.25289 fetch_fsmap learned FSMap version 24 2024-04-25T00:20:08.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.520+0000 7f95b4b69740 1 -- 172.21.15.42:0/4198633242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b975c9e780 con 0x55b975c95e50 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.520+0000 7f9598c00640 1 -- 172.21.15.42:0/4198633242 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f959c036030 con 0x55b975c95e50 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.520+0000 7f9598c00640 1 -- 172.21.15.42:0/4198633242 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f959c0305c0 con 0x55b975c95e50 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.521+0000 7f95b4b69740 10 client.25289 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.521+0000 7f95b4b69740 10 client.25289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.521+0000 7f95b4b69740 20 client.25289 populate_metadata read hostname 'smithi042' 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.521+0000 7f95b4b69740 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b975cab820 0x55b975cadc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.521+0000 7f95b4b69740 4 client.25289 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.521+0000 7f95b4b69740 1 -- 172.21.15.42:0/4198633242 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b975c05ff0 con 0x55b975cab820 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.521+0000 7f95aa000640 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b975cab820 0x55b975cadc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.521+0000 7f95aa000640 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b975cab820 0x55b975cadc30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5152d9b4675ba5b1 server_cookie=13b4dadf3e47f8f9 in_seq=0 out_seq=0 2024-04-25T00:20:08.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.521+0000 7f9598c00640 10 client.25289 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.522+0000 7f9598c00640 1 -- 172.21.15.42:0/4198633242 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b975c05ff0 con 0x55b975cab820 2024-04-25T00:20:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.522+0000 7f9598c00640 10 client.25289 handle_command_reply: tid=0 2024-04-25T00:20:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.522+0000 7f95b4b69740 1 client.25289 shutdown 2024-04-25T00:20:08.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 1 -- 172.21.15.42:0/4198633242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 msgr2=0x55b975c9fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c9fec0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f959c000f80 tx=0x7f959c031700 comp rx=0 tx=0).stop 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 1 -- 172.21.15.42:0/4198633242 shutdown_connections 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b975cab820 0x55b975cadc30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975ca7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975ca0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f9598c00640 0 client.25289 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 1 --2- 172.21.15.42:0/4198633242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975c9fec0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 1 -- 172.21.15.42:0/4198633242 >> 172.21.15.42:0/4198633242 conn(0x55b975c914c0 msgr2=0x55b975c90620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 1 -- 172.21.15.42:0/4198633242 shutdown_connections 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 1 -- 172.21.15.42:0/4198633242 wait complete. 2024-04-25T00:20:08.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.523+0000 7f95b4b69740 20 client.25289 trim_cache size 0 max 16384 2024-04-25T00:20:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.539+0000 7f95b4b69740 1 Processor -- start 2024-04-25T00:20:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.539+0000 7f95b4b69740 1 -- start start 2024-04-25T00:20:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.539+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975cadc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c94480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975c949c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95b4b69740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1ab190 con 0x55b975c96840 2024-04-25T00:20:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95b4b69740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1a8a60 con 0x55b975c95e50 2024-04-25T00:20:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95b4b69740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1a68b0 con 0x55b975c9b9b0 2024-04-25T00:20:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c94480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975cadc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975cadc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44320/0 (socket says 172.21.15.42:44320) 2024-04-25T00:20:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c94480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51084/0 (socket says 172.21.15.42:51084) 2024-04-25T00:20:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aaa00640 1 -- 172.21.15.42:0/249474898 learned_addr learned my addr 172.21.15.42:0/249474898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95ab400640 1 --2- 172.21.15.42:0/249474898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975c949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aa000640 1 -- 172.21.15.42:0/249474898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 msgr2=0x55b975c949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aa000640 1 --2- 172.21.15.42:0/249474898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975c949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aa000640 1 -- 172.21.15.42:0/249474898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 msgr2=0x55b975cadc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aa000640 1 --2- 172.21.15.42:0/249474898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975cadc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aa000640 1 -- 172.21.15.42:0/249474898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b975c9ad20 con 0x55b975c96840 2024-04-25T00:20:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.540+0000 7f95aaa00640 1 --2- 172.21.15.42:0/249474898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975cadc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.541+0000 7f95aa000640 1 --2- 172.21.15.42:0/249474898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c94480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f95940ccb10 tx=0x7f9594086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7834fa12db49c60a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.541+0000 7f959b400640 1 -- 172.21.15.42:0/249474898 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959401c040 con 0x55b975c96840 2024-04-25T00:20:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.541+0000 7f959b400640 1 -- 172.21.15.42:0/249474898 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95940051d0 con 0x55b975c96840 2024-04-25T00:20:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.541+0000 7f959b400640 1 -- 172.21.15.42:0/249474898 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959409fbd0 con 0x55b975c96840 2024-04-25T00:20:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.542+0000 7f95b4b69740 1 -- 172.21.15.42:0/249474898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 msgr2=0x55b975c94480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.542+0000 7f95b4b69740 1 --2- 172.21.15.42:0/249474898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c94480 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f95940ccb10 tx=0x7f9594086040 comp rx=0 tx=0).stop 2024-04-25T00:20:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.542+0000 7f95b4b69740 1 -- 172.21.15.42:0/249474898 shutdown_connections 2024-04-25T00:20:08.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.542+0000 7f95b4b69740 1 --2- 172.21.15.42:0/249474898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c9b9b0 0x55b975c949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.542+0000 7f95b4b69740 1 --2- 172.21.15.42:0/249474898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975c94480 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.542+0000 7f95b4b69740 1 --2- 172.21.15.42:0/249474898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c95e50 0x55b975cadc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.542+0000 7f95b4b69740 1 -- 172.21.15.42:0/249474898 >> 172.21.15.42:0/249474898 conn(0x55b975c914c0 msgr2=0x55b975c918a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.542+0000 7f95b4b69740 1 -- 172.21.15.42:0/249474898 shutdown_connections 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.542+0000 7f95b4b69740 1 -- 172.21.15.42:0/249474898 wait complete. 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.542+0000 7f95b4b69740 1 -- start start 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95b4b69740 1 Processor -- start 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c95e50 0x55b975d0d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975d0d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95b4b69740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c9b9b0 0x55b975d14820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95b4b69740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1a6f20 con 0x55b975c96840 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95b4b69740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1a8390 con 0x55b975c9b9b0 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95b4b69740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f95ac1a75f0 con 0x55b975c95e50 2024-04-25T00:20:08.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95b4b69740 10 client.0 _ll_register_callbacks cb 0x7f95a4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95aa000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975d0d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95ab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c9b9b0 0x55b975d14820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95ab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c9b9b0 0x55b975d14820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44326/0 (socket says 172.21.15.42:44326) 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95ab400640 1 -- 172.21.15.42:0/749006266 learned_addr learned my addr 172.21.15.42:0/749006266 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.543+0000 7f95aaa00640 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c95e50 0x55b975d0d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f95aa000640 1 -- 172.21.15.42:0/749006266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c95e50 msgr2=0x55b975d0d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f95aa000640 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c95e50 0x55b975d0d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f95aa000640 1 -- 172.21.15.42:0/749006266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c9b9b0 msgr2=0x55b975d14820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f95aa000640 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c9b9b0 0x55b975d14820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f95aa000640 1 -- 172.21.15.42:0/749006266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b975d14e10 con 0x55b975c96840 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f95ab400640 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c9b9b0 0x55b975d14820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f95aa000640 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975d0d8c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f959409b760 tx=0x7f959409b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a078d6ddb8e42652 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f9598c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f9598c00640 1 -- 172.21.15.42:0/749006266 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95940b3070 con 0x55b975c96840 2024-04-25T00:20:08.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f9598c00640 1 -- 172.21.15.42:0/749006266 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f95940ba1c0 con 0x55b975c96840 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.544+0000 7f95b4b69740 1 -- 172.21.15.42:0/749006266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95940051d0 con 0x55b975c96840 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f9598c00640 1 -- 172.21.15.42:0/749006266 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f959401c040 con 0x55b975c96840 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f95b4b69740 10 client.15192 fetch_fsmap learned FSMap version 24 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f95b4b69740 1 -- 172.21.15.42:0/749006266 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55b975d16190 con 0x55b975c96840 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f9598c00640 1 -- 172.21.15.42:0/749006266 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f95940051d0 con 0x55b975c96840 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f9598c00640 1 -- 172.21.15.42:0/749006266 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f95940860b0 con 0x55b975c96840 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f95b4b69740 10 client.15192 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f95b4b69740 10 client.15192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f95b4b69740 20 client.15192 populate_metadata read hostname 'smithi042' 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f95b4b69740 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b975d16ad0 0x55b975d18ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f95b4b69740 4 client.15192 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f95b4b69740 1 -- 172.21.15.42:0/749006266 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b975c00340 con 0x55b975d16ad0 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.545+0000 7f95aaa00640 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b975d16ad0 0x55b975d18ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.546+0000 7f95aaa00640 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b975d16ad0 0x55b975d18ef0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50c8e5d327aec174 server_cookie=3c5ff3340d30aa42 in_seq=0 out_seq=0 2024-04-25T00:20:08.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.546+0000 7f9598c00640 10 client.15192 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.551+0000 7f9598c00640 1 -- 172.21.15.42:0/749006266 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b975c00340 con 0x55b975d16ad0 2024-04-25T00:20:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.551+0000 7f9598c00640 10 client.15192 handle_command_reply: tid=0 2024-04-25T00:20:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.551+0000 7f95b4b69740 1 client.15192 shutdown 2024-04-25T00:20:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 1 -- 172.21.15.42:0/749006266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 msgr2=0x55b975d0d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975d0d8c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f959409b760 tx=0x7f959409b5f0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 1 -- 172.21.15.42:0/749006266 shutdown_connections 2024-04-25T00:20:08.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b975d16ad0 0x55b975d18ef0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b975c9b9b0 0x55b975d14820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f9598c00640 0 client.15192 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b975c96840 0x55b975d0d8c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 1 --2- 172.21.15.42:0/749006266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b975c95e50 0x55b975d0d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 1 -- 172.21.15.42:0/749006266 >> 172.21.15.42:0/749006266 conn(0x55b975c914c0 msgr2=0x55b975c94480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 1 -- 172.21.15.42:0/749006266 shutdown_connections 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 1 -- 172.21.15.42:0/749006266 wait complete. 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.552+0000 7f95b4b69740 20 client.15192 trim_cache size 0 max 16384 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:20:08.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:20:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:20:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 -- 172.21.15.42:0/1597944136 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95900a2b80 msgr2=0x7f95900a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95900a2b80 0x7f95900a5040 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55b975c970a0 tx=0x7f95a4005f90 comp rx=0 tx=0).stop 2024-04-25T00:20:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 -- 172.21.15.42:0/1597944136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95ac1016b0 msgr2=0x7f95ac197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95ac1016b0 0x7f95ac197ef0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f959400e9f0 tx=0x7f959400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 -- 172.21.15.42:0/1597944136 shutdown_connections 2024-04-25T00:20:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f95900a2b80 0x7f95900a5040 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f95ac102070 0x7f95ac19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95ac1016b0 0x7f95ac197ef0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 --2- 172.21.15.42:0/1597944136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f95ac1004c0 0x7f95ac1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 -- 172.21.15.42:0/1597944136 >> 172.21.15.42:0/1597944136 conn(0x7f95ac0fba30 msgr2=0x7f95ac068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 -- 172.21.15.42:0/1597944136 shutdown_connections 2024-04-25T00:20:08.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:08.553+0000 7f95abe00640 1 -- 172.21.15.42:0/1597944136 wait complete. 2024-04-25T00:20:08.740 INFO:tasks.workunit.client.0.smithi042.stderr:Note: switching to 'd2c45d977142e024a59b68a8f1dce8f0b9840551'. 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr: git switch -c 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr:Or undo this operation with: 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr: git switch - 2024-04-25T00:20:08.741 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-25T00:20:08.742 INFO:tasks.workunit.client.0.smithi042.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T00:20:08.742 INFO:tasks.workunit.client.0.smithi042.stderr: 2024-04-25T00:20:08.742 INFO:tasks.workunit.client.0.smithi042.stderr:HEAD is now at d2c45d97714 Merge pull request #56528 from adk3798/squid-mypy-bump 2024-04-25T00:20:08.750 DEBUG:teuthology.orchestra.run.smithi042:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T00:20:08.851 INFO:tasks.workunit.client.0.smithi042.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T00:20:08.859 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T00:20:08.859 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T00:20:09.644 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T00:20:09.749 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T00:20:09.838 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T00:20:09.843 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T00:20:09.843 INFO:tasks.workunit.client.0.smithi042.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T00:20:09.935 INFO:tasks.workunit.client.0.smithi042.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T00:20:09.945 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:20:09.946 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T00:20:10.003 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-04-25T00:20:10.005 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-04-25T00:20:10.005 DEBUG:teuthology.orchestra.run.smithi042:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=d2c45d977142e024a59b68a8f1dce8f0b9840551 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-04-25T00:20:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:09 smithi136 ceph-mon[32923]: pgmap v429: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:20:10.082 INFO:tasks.workunit.client.0.smithi042.stderr:+ dbench 1 2024-04-25T00:20:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:09 smithi106 ceph-mon[32510]: pgmap v429: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:20:10.098 INFO:tasks.workunit.client.0.smithi042.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-25T00:20:10.098 INFO:tasks.workunit.client.0.smithi042.stdout: 2024-04-25T00:20:10.099 INFO:tasks.workunit.client.0.smithi042.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-25T00:20:10.099 INFO:tasks.workunit.client.0.smithi042.stdout:0 of 1 processes prepared for launch 0 sec 2024-04-25T00:20:10.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:09 smithi042 ceph-mon[28130]: pgmap v429: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 255 B/s wr, 6 op/s 2024-04-25T00:20:10.200 INFO:tasks.workunit.client.0.smithi042.stdout:1 of 1 processes prepared for launch 0 sec 2024-04-25T00:20:10.200 INFO:tasks.workunit.client.0.smithi042.stdout:releasing clients 2024-04-25T00:20:11.200 INFO:tasks.workunit.client.0.smithi042.stdout: 1 1686 26.22 MB/sec warmup 1 sec latency 11.524 ms 2024-04-25T00:20:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:11 smithi136 ceph-mon[32923]: pgmap v430: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:20:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:11 smithi106 ceph-mon[32510]: pgmap v430: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:20:12.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:11 smithi042 ceph-mon[28130]: pgmap v430: 97 pgs: 97 active+clean; 666 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-04-25T00:20:12.200 INFO:tasks.workunit.client.0.smithi042.stdout: 1 3979 20.03 MB/sec warmup 2 sec latency 11.924 ms 2024-04-25T00:20:13.200 INFO:tasks.workunit.client.0.smithi042.stdout: 1 6013 15.74 MB/sec warmup 3 sec latency 12.897 ms 2024-04-25T00:20:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:13 smithi136 ceph-mon[32923]: pgmap v431: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 46 KiB/s wr, 9 op/s 2024-04-25T00:20:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:13 smithi106 ceph-mon[32510]: pgmap v431: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 46 KiB/s wr, 9 op/s 2024-04-25T00:20:14.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:13 smithi042 ceph-mon[28130]: pgmap v431: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 46 KiB/s wr, 9 op/s 2024-04-25T00:20:14.200 INFO:tasks.workunit.client.0.smithi042.stdout: 1 7818 14.86 MB/sec warmup 4 sec latency 13.489 ms 2024-04-25T00:20:15.201 INFO:tasks.workunit.client.0.smithi042.stdout: 1 10106 14.35 MB/sec warmup 5 sec latency 10.809 ms 2024-04-25T00:20:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:15 smithi136 ceph-mon[32923]: pgmap v432: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 46 KiB/s wr, 8 op/s 2024-04-25T00:20:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:15 smithi106 ceph-mon[32510]: pgmap v432: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 46 KiB/s wr, 8 op/s 2024-04-25T00:20:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:15 smithi042 ceph-mon[28130]: pgmap v432: 97 pgs: 97 active+clean; 1.1 MiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 46 KiB/s wr, 8 op/s 2024-04-25T00:20:16.202 INFO:tasks.workunit.client.0.smithi042.stdout: 1 11585 13.19 MB/sec warmup 6 sec latency 13.352 ms 2024-04-25T00:20:17.201 INFO:tasks.workunit.client.0.smithi042.stdout: 1 14047 13.26 MB/sec warmup 7 sec latency 21.089 ms 2024-04-25T00:20:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:17 smithi136 ceph-mon[32923]: pgmap v433: 97 pgs: 97 active+clean; 5.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 446 KiB/s wr, 62 op/s 2024-04-25T00:20:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:17 smithi106 ceph-mon[32510]: pgmap v433: 97 pgs: 97 active+clean; 5.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 446 KiB/s wr, 62 op/s 2024-04-25T00:20:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:17 smithi042 ceph-mon[28130]: pgmap v433: 97 pgs: 97 active+clean; 5.0 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 446 KiB/s wr, 62 op/s 2024-04-25T00:20:18.201 INFO:tasks.workunit.client.0.smithi042.stdout: 1 15383 12.44 MB/sec warmup 8 sec latency 13.652 ms 2024-04-25T00:20:19.201 INFO:tasks.workunit.client.0.smithi042.stdout: 1 17898 12.81 MB/sec warmup 9 sec latency 10.049 ms 2024-04-25T00:20:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:19 smithi136 ceph-mon[32923]: pgmap v434: 97 pgs: 97 active+clean; 5.5 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 512 KiB/s wr, 66 op/s 2024-04-25T00:20:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:19 smithi106 ceph-mon[32510]: pgmap v434: 97 pgs: 97 active+clean; 5.5 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 512 KiB/s wr, 66 op/s 2024-04-25T00:20:20.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:19 smithi042 ceph-mon[28130]: pgmap v434: 97 pgs: 97 active+clean; 5.5 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 512 KiB/s wr, 66 op/s 2024-04-25T00:20:20.201 INFO:tasks.workunit.client.0.smithi042.stdout: 1 19430 12.18 MB/sec warmup 10 sec latency 12.408 ms 2024-04-25T00:20:21.201 INFO:tasks.workunit.client.0.smithi042.stdout: 1 21649 12.31 MB/sec warmup 11 sec latency 8.750 ms 2024-04-25T00:20:21.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.699+0000 7f207e800640 1 -- 172.21.15.42:0/2600291971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2078103800 msgr2=0x7f2078103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.699+0000 7f207e800640 1 --2- 172.21.15.42:0/2600291971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2078103800 0x7f2078103c20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f206c009900 tx=0x7f206c02a410 comp rx=0 tx=0).stop 2024-04-25T00:20:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.699+0000 7f207e800640 1 -- 172.21.15.42:0/2600291971 shutdown_connections 2024-04-25T00:20:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.699+0000 7f207e800640 1 --2- 172.21.15.42:0/2600291971 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f20781053b0 0x7f207810bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.699+0000 7f207e800640 1 --2- 172.21.15.42:0/2600291971 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20781049f0 0x7f2078104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.699+0000 7f207e800640 1 --2- 172.21.15.42:0/2600291971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2078103800 0x7f2078103c20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.699+0000 7f207e800640 1 -- 172.21.15.42:0/2600291971 >> 172.21.15.42:0/2600291971 conn(0x7f20780fedb0 msgr2=0x7f20781011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.699+0000 7f207e800640 1 -- 172.21.15.42:0/2600291971 shutdown_connections 2024-04-25T00:20:21.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.699+0000 7f207e800640 1 -- 172.21.15.42:0/2600291971 wait complete. 2024-04-25T00:20:21.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.700+0000 7f207e800640 1 Processor -- start 2024-04-25T00:20:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.700+0000 7f207e800640 1 -- start start 2024-04-25T00:20:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2078103800 0x7f2078199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20781049f0 0x7f207819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20781053b0 0x7f20781a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207e800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f207810f4b0 con 0x7f2078103800 2024-04-25T00:20:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207e800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f207810f330 con 0x7f20781053b0 2024-04-25T00:20:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207e800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f207810f630 con 0x7f20781049f0 2024-04-25T00:20:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2078103800 0x7f2078199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20781053b0 0x7f20781a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2078103800 0x7f2078199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46318/0 (socket says 172.21.15.42:46318) 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207d400640 1 -- 172.21.15.42:0/460706463 learned_addr learned my addr 172.21.15.42:0/460706463 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207ca00640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20781049f0 0x7f207819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207d400640 1 -- 172.21.15.42:0/460706463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20781049f0 msgr2=0x7f207819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207d400640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20781049f0 0x7f207819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207d400640 1 -- 172.21.15.42:0/460706463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20781053b0 msgr2=0x7f20781a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207d400640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20781053b0 0x7f20781a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207d400640 1 -- 172.21.15.42:0/460706463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20781a1720 con 0x7f2078103800 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.701+0000 7f207de00640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20781053b0 0x7f20781a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.702+0000 7f207d400640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2078103800 0x7f2078199b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f206c031440 tx=0x7f206c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.702+0000 7f2076000640 1 -- 172.21.15.42:0/460706463 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c03f030 con 0x7f2078103800 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.702+0000 7f2076000640 1 -- 172.21.15.42:0/460706463 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206c003ce0 con 0x7f2078103800 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.702+0000 7f207e800640 1 -- 172.21.15.42:0/460706463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20781a19b0 con 0x7f2078103800 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.702+0000 7f2076000640 1 -- 172.21.15.42:0/460706463 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c035070 con 0x7f2078103800 2024-04-25T00:20:21.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.702+0000 7f207e800640 1 -- 172.21.15.42:0/460706463 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20781a1ea0 con 0x7f2078103800 2024-04-25T00:20:21.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.706+0000 7f2082de1740 1 Processor -- start 2024-04-25T00:20:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.706+0000 7f2082de1740 1 -- start start 2024-04-25T00:20:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.706+0000 7f2076000640 1 -- 172.21.15.42:0/460706463 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f206c042030 con 0x7f2078103800 2024-04-25T00:20:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.706+0000 7f2082de1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1641e50 0x5575e1642270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.706+0000 7f2082de1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e1642840 0x5575e1647340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f2082de1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e16479b0 0x5575e1649e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f2082de1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5575e15a4ab0 con 0x5575e1642840 2024-04-25T00:20:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f2082de1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5575e154d640 con 0x5575e16479b0 2024-04-25T00:20:21.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f2082de1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5575e15a0a70 con 0x5575e1641e50 2024-04-25T00:20:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f207ca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e16479b0 0x5575e1649e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f207ca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e16479b0 0x5575e1649e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58300/0 (socket says 172.21.15.42:58300) 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f207d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e1642840 0x5575e1647340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f207ca00640 1 -- 172.21.15.42:0/211415810 learned_addr learned my addr 172.21.15.42:0/211415810 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f207de00640 1 --2- 172.21.15.42:0/211415810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1641e50 0x5575e1642270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f207ca00640 1 -- 172.21.15.42:0/211415810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1641e50 msgr2=0x5575e1642270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f207ca00640 1 --2- 172.21.15.42:0/211415810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1641e50 0x5575e1642270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f207ca00640 1 -- 172.21.15.42:0/211415810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e1642840 msgr2=0x5575e1647340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f207ca00640 1 --2- 172.21.15.42:0/211415810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e1642840 0x5575e1647340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.707+0000 7f207ca00640 1 -- 172.21.15.42:0/211415810 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e12b9210 con 0x5575e16479b0 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.708+0000 7f207ca00640 1 --2- 172.21.15.42:0/211415810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e16479b0 0x5575e1649e40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2064009ca0 tx=0x7f2064011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6f78c3e5055f28f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.708+0000 7f206a600640 1 -- 172.21.15.42:0/211415810 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064004230 con 0x5575e16479b0 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.708+0000 7f206a600640 1 -- 172.21.15.42:0/211415810 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20640043d0 con 0x5575e16479b0 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.708+0000 7f206a600640 1 -- 172.21.15.42:0/211415810 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064004e60 con 0x5575e16479b0 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.708+0000 7f2076000640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20600a2b80 0x7f20600a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f2076000640 1 -- 172.21.15.42:0/460706463 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f206c0ef580 con 0x7f2078103800 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f207de00640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20600a2b80 0x7f20600a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f2082de1740 1 -- 172.21.15.42:0/211415810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e16479b0 msgr2=0x5575e1649e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f2082de1740 1 --2- 172.21.15.42:0/211415810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e16479b0 0x5575e1649e40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2064009ca0 tx=0x7f2064011810 comp rx=0 tx=0).stop 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f2082de1740 1 -- 172.21.15.42:0/211415810 shutdown_connections 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f2082de1740 1 --2- 172.21.15.42:0/211415810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e16479b0 0x5575e1649e40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f2082de1740 1 --2- 172.21.15.42:0/211415810 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e1642840 0x5575e1647340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f2082de1740 1 --2- 172.21.15.42:0/211415810 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1641e50 0x5575e1642270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f2082de1740 1 -- 172.21.15.42:0/211415810 >> 172.21.15.42:0/211415810 conn(0x5575e163d4c0 msgr2=0x5575e163d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f2082de1740 1 -- 172.21.15.42:0/211415810 shutdown_connections 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.709+0000 7f2082de1740 1 -- 172.21.15.42:0/211415810 wait complete. 2024-04-25T00:20:21.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.710+0000 7f207de00640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20600a2b80 0x7f20600a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5575e16430a0 tx=0x7f20700096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.710+0000 7f2082de1740 1 -- start start 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.710+0000 7f2082de1740 1 Processor -- start 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f2082de1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e1641e50 0x5575e164c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f2082de1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1642840 0x5575e164c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f2082de1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e16479b0 0x5575e1653670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f2082de1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5575e164a960 con 0x5575e1641e50 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f2082de1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5575e1560990 con 0x5575e1642840 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f2082de1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5575e164aae0 con 0x5575e16479b0 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f2082de1740 10 client.0 _ll_register_callbacks cb 0x5575e1562950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f207de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e16479b0 0x5575e1653670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f207de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e16479b0 0x5575e1653670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:50300/0 (socket says 172.21.15.42:50300) 2024-04-25T00:20:21.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f207de00640 1 -- 172.21.15.42:0/975273941 learned_addr learned my addr 172.21.15.42:0/975273941 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f207d400640 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e1641e50 0x5575e164c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f207ca00640 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1642840 0x5575e164c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f207de00640 1 -- 172.21.15.42:0/975273941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1642840 msgr2=0x5575e164c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f207de00640 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1642840 0x5575e164c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.711+0000 7f207de00640 1 -- 172.21.15.42:0/975273941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e1641e50 msgr2=0x5575e164c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.712+0000 7f207de00640 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e1641e50 0x5575e164c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.712+0000 7f207de00640 1 -- 172.21.15.42:0/975273941 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e1385620 con 0x5575e16479b0 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.712+0000 7f207ca00640 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1642840 0x5575e164c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.712+0000 7f207de00640 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e16479b0 0x5575e1653670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2070012cf0 tx=0x7f2070004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f258e1788915a10e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.712+0000 7f205ea00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.712+0000 7f205ea00640 1 -- 172.21.15.42:0/975273941 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f207001dd30 con 0x5575e16479b0 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.712+0000 7f205ea00640 1 -- 172.21.15.42:0/975273941 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f207001e410 con 0x5575e16479b0 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.712+0000 7f2082de1740 1 -- 172.21.15.42:0/975273941 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20640043d0 con 0x5575e16479b0 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f205ea00640 1 -- 172.21.15.42:0/975273941 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f207001e610 con 0x5575e16479b0 2024-04-25T00:20:21.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f2082de1740 10 client.25312 fetch_fsmap learned FSMap version 24 2024-04-25T00:20:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f2082de1740 1 -- 172.21.15.42:0/975273941 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5575e1656720 con 0x5575e16479b0 2024-04-25T00:20:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f205ea00640 1 -- 172.21.15.42:0/975273941 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20640043d0 con 0x5575e16479b0 2024-04-25T00:20:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f205ea00640 1 -- 172.21.15.42:0/975273941 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f207001b8f0 con 0x5575e16479b0 2024-04-25T00:20:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f2082de1740 10 client.25312 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:20:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f2082de1740 10 client.25312 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f2082de1740 20 client.25312 populate_metadata read hostname 'smithi042' 2024-04-25T00:20:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f2082de1740 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5575e1657e60 0x5575e165a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f2082de1740 4 client.25312 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.713+0000 7f2082de1740 1 -- 172.21.15.42:0/975273941 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575e15b1ff0 con 0x5575e1657e60 2024-04-25T00:20:21.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.714+0000 7f207d400640 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5575e1657e60 0x5575e165a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.714+0000 7f207d400640 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5575e1657e60 0x5575e165a250 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=655bc278410b4ee6 server_cookie=eabcb936ed5871d1 in_seq=0 out_seq=0 2024-04-25T00:20:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.714+0000 7f205ea00640 10 client.25312 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:21.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.715+0000 7f205ea00640 1 -- 172.21.15.42:0/975273941 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5575e15b1ff0 con 0x5575e1657e60 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.715+0000 7f205ea00640 10 client.25312 handle_command_reply: tid=0 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.715+0000 7f2082de1740 1 client.25312 shutdown 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.715+0000 7f2082de1740 1 -- 172.21.15.42:0/975273941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e16479b0 msgr2=0x5575e1653670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.715+0000 7f2082de1740 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e16479b0 0x5575e1653670 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2070012cf0 tx=0x7f2070004a40 comp rx=0 tx=0).stop 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.716+0000 7f2082de1740 1 -- 172.21.15.42:0/975273941 shutdown_connections 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.716+0000 7f2082de1740 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5575e1657e60 0x5575e165a250 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.716+0000 7f2082de1740 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e16479b0 0x5575e1653670 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.716+0000 7f2082de1740 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1642840 0x5575e164c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.716+0000 7f2082de1740 1 --2- 172.21.15.42:0/975273941 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e1641e50 0x5575e164c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.716+0000 7f205ea00640 0 client.25312 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.716+0000 7f2082de1740 1 -- 172.21.15.42:0/975273941 >> 172.21.15.42:0/975273941 conn(0x5575e163d4c0 msgr2=0x5575e163c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.716+0000 7f2082de1740 1 -- 172.21.15.42:0/975273941 shutdown_connections 2024-04-25T00:20:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.716+0000 7f2082de1740 1 -- 172.21.15.42:0/975273941 wait complete. 2024-04-25T00:20:21.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.716+0000 7f2082de1740 20 client.25312 trim_cache size 0 max 16384 2024-04-25T00:20:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.736+0000 7f2082de1740 1 Processor -- start 2024-04-25T00:20:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.736+0000 7f2082de1740 1 -- start start 2024-04-25T00:20:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.736+0000 7f2082de1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1640ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.736+0000 7f2082de1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1642840 0x5575e1653630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.736+0000 7f2082de1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e16479b0 0x5575e165eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.736+0000 7f2082de1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f20781ad470 con 0x5575e16479b0 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.736+0000 7f2082de1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20781aad40 con 0x5575e1641e50 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.736+0000 7f2082de1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20781a8b90 con 0x5575e1642840 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f207d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1640ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f207d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1640ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58324/0 (socket says 172.21.15.42:58324) 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f207d400640 1 -- 172.21.15.42:0/1373741458 learned_addr learned my addr 172.21.15.42:0/1373741458 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f207de00640 1 --2- 172.21.15.42:0/1373741458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e16479b0 0x5575e165eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f207d400640 1 -- 172.21.15.42:0/1373741458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1642840 msgr2=0x5575e1653630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f207d400640 1 --2- 172.21.15.42:0/1373741458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1642840 0x5575e1653630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f207d400640 1 -- 172.21.15.42:0/1373741458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e16479b0 msgr2=0x5575e165eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f207d400640 1 --2- 172.21.15.42:0/1373741458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e16479b0 0x5575e165eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f207d400640 1 -- 172.21.15.42:0/1373741458 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575e163d2f0 con 0x5575e1641e50 2024-04-25T00:20:21.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f207d400640 1 --2- 172.21.15.42:0/1373741458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1640ae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f206c0b82c0 tx=0x7f206c0ecf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae44c8c412a1149a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.737+0000 7f2069c00640 1 -- 172.21.15.42:0/1373741458 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c049040 con 0x5575e1641e50 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.738+0000 7f2069c00640 1 -- 172.21.15.42:0/1373741458 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206c0bf070 con 0x5575e1641e50 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.738+0000 7f2069c00640 1 -- 172.21.15.42:0/1373741458 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c0c4a20 con 0x5575e1641e50 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.738+0000 7f2082de1740 1 -- 172.21.15.42:0/1373741458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 msgr2=0x5575e1640ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.738+0000 7f2082de1740 1 --2- 172.21.15.42:0/1373741458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1640ae0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f206c0b82c0 tx=0x7f206c0ecf20 comp rx=0 tx=0).stop 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.738+0000 7f2082de1740 1 -- 172.21.15.42:0/1373741458 shutdown_connections 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.738+0000 7f2082de1740 1 --2- 172.21.15.42:0/1373741458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e16479b0 0x5575e165eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.738+0000 7f2082de1740 1 --2- 172.21.15.42:0/1373741458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1642840 0x5575e1653630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.738+0000 7f2082de1740 1 --2- 172.21.15.42:0/1373741458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1640ae0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.738+0000 7f2082de1740 1 -- 172.21.15.42:0/1373741458 >> 172.21.15.42:0/1373741458 conn(0x5575e163d4c0 msgr2=0x5575e16403d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.738+0000 7f2082de1740 1 -- 172.21.15.42:0/1373741458 shutdown_connections 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.739+0000 7f2082de1740 1 -- 172.21.15.42:0/1373741458 wait complete. 2024-04-25T00:20:21.742 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.739+0000 7f2082de1740 1 -- start start 2024-04-25T00:20:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.739+0000 7f2082de1740 1 Processor -- start 2024-04-25T00:20:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f2082de1740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1651290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f2082de1740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1642840 0x5575e16517d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f2082de1740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e16479b0 0x5575e16527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f2082de1740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f20781a9200 con 0x5575e16479b0 2024-04-25T00:20:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f2082de1740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f20781aa670 con 0x5575e1641e50 2024-04-25T00:20:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f2082de1740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20781a98d0 con 0x5575e1642840 2024-04-25T00:20:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f2082de1740 10 client.0 _ll_register_callbacks cb 0x7f2070009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f207d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1651290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f207ca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1642840 0x5575e16517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f207d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1651290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58328/0 (socket says 172.21.15.42:58328) 2024-04-25T00:20:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f207d400640 1 -- 172.21.15.42:0/1016949618 learned_addr learned my addr 172.21.15.42:0/1016949618 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f207de00640 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e16479b0 0x5575e16527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f207d400640 1 -- 172.21.15.42:0/1016949618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1642840 msgr2=0x5575e16517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f207d400640 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1642840 0x5575e16517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f207d400640 1 -- 172.21.15.42:0/1016949618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e16479b0 msgr2=0x5575e16527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f207d400640 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e16479b0 0x5575e16527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.740+0000 7f207d400640 1 -- 172.21.15.42:0/1016949618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575e1652dd0 con 0x5575e1641e50 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f207d400640 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1651290 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f206c0bbb60 tx=0x7f206c0095d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=738f8d3f5cc811a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f205ea00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f205ea00640 1 -- 172.21.15.42:0/1016949618 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c0d9060 con 0x5575e1641e50 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f2082de1740 1 -- 172.21.15.42:0/1016949618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f206c0bf070 con 0x5575e1641e50 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f205ea00640 1 -- 172.21.15.42:0/1016949618 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f206c03bd10 con 0x5575e1641e50 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f205ea00640 1 -- 172.21.15.42:0/1016949618 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f206c049030 con 0x5575e1641e50 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f2082de1740 10 client.25313 fetch_fsmap learned FSMap version 24 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f2082de1740 1 -- 172.21.15.42:0/1016949618 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5575e16be9a0 con 0x5575e1641e50 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f205ea00640 1 -- 172.21.15.42:0/1016949618 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f206c0bf070 con 0x5575e1641e50 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f205ea00640 1 -- 172.21.15.42:0/1016949618 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f206c0b84d0 con 0x5575e1641e50 2024-04-25T00:20:21.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f2082de1740 10 client.25313 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T00:20:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f2082de1740 10 client.25313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f2082de1740 20 client.25313 populate_metadata read hostname 'smithi042' 2024-04-25T00:20:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f2082de1740 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5575e16bf2a0 0x5575e16c16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f2082de1740 4 client.25313 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:20:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.741+0000 7f2082de1740 1 -- 172.21.15.42:0/1016949618 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5575e15ac340 con 0x5575e16bf2a0 2024-04-25T00:20:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.742+0000 7f207ca00640 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5575e16bf2a0 0x5575e16c16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.742+0000 7f207ca00640 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5575e16bf2a0 0x5575e16c16c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597db970343d8e6a server_cookie=1235640f15a2edaa in_seq=0 out_seq=0 2024-04-25T00:20:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.742+0000 7f205ea00640 10 client.25313 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.840+0000 7f205ea00640 1 -- 172.21.15.42:0/1016949618 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5575e15ac340 con 0x5575e16bf2a0 2024-04-25T00:20:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.840+0000 7f205ea00640 10 client.25313 handle_command_reply: tid=0 2024-04-25T00:20:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.840+0000 7f2082de1740 1 client.25313 shutdown 2024-04-25T00:20:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.840+0000 7f2082de1740 1 -- 172.21.15.42:0/1016949618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 msgr2=0x5575e1651290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.840+0000 7f2082de1740 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1651290 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f206c0bbb60 tx=0x7f206c0095d0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.841+0000 7f2082de1740 1 -- 172.21.15.42:0/1016949618 shutdown_connections 2024-04-25T00:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.841+0000 7f2082de1740 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5575e16bf2a0 0x5575e16c16c0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.841+0000 7f2082de1740 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5575e16479b0 0x5575e16527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.841+0000 7f205ea00640 0 client.25313 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.841+0000 7f2082de1740 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5575e1642840 0x5575e16517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.841+0000 7f2082de1740 1 --2- 172.21.15.42:0/1016949618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5575e1641e50 0x5575e1651290 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.841+0000 7f2082de1740 1 -- 172.21.15.42:0/1016949618 >> 172.21.15.42:0/1016949618 conn(0x5575e163d4c0 msgr2=0x5575e16403d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.841+0000 7f2082de1740 1 -- 172.21.15.42:0/1016949618 shutdown_connections 2024-04-25T00:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.841+0000 7f2082de1740 1 -- 172.21.15.42:0/1016949618 wait complete. 2024-04-25T00:20:21.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.841+0000 7f2082de1740 20 client.25313 trim_cache size 0 max 16384 2024-04-25T00:20:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:20:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:20:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:20:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:20:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 -- 172.21.15.42:0/460706463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20600a2b80 msgr2=0x7f20600a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20600a2b80 0x7f20600a5040 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5575e16430a0 tx=0x7f20700096a0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 -- 172.21.15.42:0/460706463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2078103800 msgr2=0x7f2078199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2078103800 0x7f2078199b50 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f206c031440 tx=0x7f206c032ce0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 -- 172.21.15.42:0/460706463 shutdown_connections 2024-04-25T00:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f20600a2b80 0x7f20600a5040 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20781053b0 0x7f20781a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20781049f0 0x7f207819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 --2- 172.21.15.42:0/460706463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2078103800 0x7f2078199b50 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 -- 172.21.15.42:0/460706463 >> 172.21.15.42:0/460706463 conn(0x7f20780fedb0 msgr2=0x7f2078107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 -- 172.21.15.42:0/460706463 shutdown_connections 2024-04-25T00:20:21.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:21.842+0000 7f207e800640 1 -- 172.21.15.42:0/460706463 wait complete. 2024-04-25T00:20:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:21 smithi136 ceph-mon[32923]: pgmap v435: 97 pgs: 97 active+clean; 6.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 571 KiB/s wr, 69 op/s 2024-04-25T00:20:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:21 smithi106 ceph-mon[32510]: pgmap v435: 97 pgs: 97 active+clean; 6.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 571 KiB/s wr, 69 op/s 2024-04-25T00:20:22.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:21 smithi042 ceph-mon[28130]: pgmap v435: 97 pgs: 97 active+clean; 6.0 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 571 KiB/s wr, 69 op/s 2024-04-25T00:20:22.201 INFO:tasks.workunit.client.0.smithi042.stdout: 1 23150 11.76 MB/sec warmup 12 sec latency 36.610 ms 2024-04-25T00:20:23.202 INFO:tasks.workunit.client.0.smithi042.stdout: 1 25254 11.91 MB/sec warmup 13 sec latency 9.299 ms 2024-04-25T00:20:23.932 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:23 smithi042 ceph-mon[28130]: pgmap v436: 97 pgs: 97 active+clean; 10 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-25T00:20:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:23 smithi136 ceph-mon[32923]: pgmap v436: 97 pgs: 97 active+clean; 10 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-25T00:20:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:23 smithi106 ceph-mon[32510]: pgmap v436: 97 pgs: 97 active+clean; 10 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.0 MiB/s wr, 132 op/s 2024-04-25T00:20:24.202 INFO:tasks.workunit.client.0.smithi042.stdout: 1 27221 11.56 MB/sec warmup 14 sec latency 12.670 ms 2024-04-25T00:20:25.202 INFO:tasks.workunit.client.0.smithi042.stdout: 1 28899 11.60 MB/sec warmup 15 sec latency 10.349 ms 2024-04-25T00:20:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:25 smithi136 ceph-mon[32923]: pgmap v437: 97 pgs: 97 active+clean; 11 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 999 KiB/s wr, 130 op/s 2024-04-25T00:20:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:25 smithi106 ceph-mon[32510]: pgmap v437: 97 pgs: 97 active+clean; 11 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 999 KiB/s wr, 130 op/s 2024-04-25T00:20:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:25 smithi042 ceph-mon[28130]: pgmap v437: 97 pgs: 97 active+clean; 11 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 999 KiB/s wr, 130 op/s 2024-04-25T00:20:26.202 INFO:tasks.workunit.client.0.smithi042.stdout: 1 30434 11.22 MB/sec warmup 16 sec latency 40.670 ms 2024-04-25T00:20:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:26 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:20:27.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:26 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:20:27.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:26 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:20:27.202 INFO:tasks.workunit.client.0.smithi042.stdout: 1 32207 11.34 MB/sec warmup 17 sec latency 96.454 ms 2024-04-25T00:20:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:27 smithi136 ceph-mon[32923]: pgmap v438: 97 pgs: 97 active+clean; 7.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-25T00:20:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:27 smithi106 ceph-mon[32510]: pgmap v438: 97 pgs: 97 active+clean; 7.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-25T00:20:28.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:27 smithi042 ceph-mon[28130]: pgmap v438: 97 pgs: 97 active+clean; 7.9 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 1.2 MiB/s wr, 204 op/s 2024-04-25T00:20:28.202 INFO:tasks.workunit.client.0.smithi042.stdout: 1 32207 10.71 MB/sec warmup 18 sec latency 1096.607 ms 2024-04-25T00:20:29.202 INFO:tasks.workunit.client.0.smithi042.stdout: 1 32745 10.18 MB/sec warmup 19 sec latency 1412.946 ms 2024-04-25T00:20:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:29 smithi136 ceph-mon[32923]: pgmap v439: 97 pgs: 97 active+clean; 11 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.1 MiB/s wr, 214 op/s 2024-04-25T00:20:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:29 smithi106 ceph-mon[32510]: pgmap v439: 97 pgs: 97 active+clean; 11 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.1 MiB/s wr, 214 op/s 2024-04-25T00:20:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:29 smithi042 ceph-mon[28130]: pgmap v439: 97 pgs: 97 active+clean; 11 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.1 MiB/s wr, 214 op/s 2024-04-25T00:20:30.202 INFO:tasks.workunit.client.0.smithi042.stdout: 1 34339 10.00 MB/sec warmup 20 sec latency 31.734 ms 2024-04-25T00:20:31.203 INFO:tasks.workunit.client.0.smithi042.stdout: 1 35899 10.10 MB/sec warmup 21 sec latency 13.177 ms 2024-04-25T00:20:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:31 smithi136 ceph-mon[32923]: pgmap v440: 97 pgs: 97 active+clean; 11 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.0 MiB/s wr, 213 op/s 2024-04-25T00:20:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:31 smithi106 ceph-mon[32510]: pgmap v440: 97 pgs: 97 active+clean; 11 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.0 MiB/s wr, 213 op/s 2024-04-25T00:20:32.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:31 smithi042 ceph-mon[28130]: pgmap v440: 97 pgs: 97 active+clean; 11 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 1.0 MiB/s wr, 213 op/s 2024-04-25T00:20:32.203 INFO:tasks.workunit.client.0.smithi042.stdout: 1 37066 9.88 MB/sec warmup 22 sec latency 22.654 ms 2024-04-25T00:20:33.203 INFO:tasks.workunit.client.0.smithi042.stdout: 1 39114 9.99 MB/sec warmup 23 sec latency 20.018 ms 2024-04-25T00:20:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:33 smithi136 ceph-mon[32923]: pgmap v441: 97 pgs: 97 active+clean; 12 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-04-25T00:20:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:33 smithi106 ceph-mon[32510]: pgmap v441: 97 pgs: 97 active+clean; 12 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-04-25T00:20:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:33 smithi042 ceph-mon[28130]: pgmap v441: 97 pgs: 97 active+clean; 12 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 588 KiB/s rd, 1.1 MiB/s wr, 241 op/s 2024-04-25T00:20:34.203 INFO:tasks.workunit.client.0.smithi042.stdout: 1 39882 9.66 MB/sec warmup 24 sec latency 18.731 ms 2024-04-25T00:20:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.011+0000 7f75cb200640 1 -- 172.21.15.42:0/2076243791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75cc103180 msgr2=0x7f75cc109a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.011+0000 7f75cb200640 1 --2- 172.21.15.42:0/2076243791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75cc103180 0x7f75cc109a10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009860 tx=0x7f75c402a340 comp rx=0 tx=0).stop 2024-04-25T00:20:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.012+0000 7f75cb200640 1 -- 172.21.15.42:0/2076243791 shutdown_connections 2024-04-25T00:20:35.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.012+0000 7f75cb200640 1 --2- 172.21.15.42:0/2076243791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75cc103180 0x7f75cc109a10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.012+0000 7f75cb200640 1 --2- 172.21.15.42:0/2076243791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75cc1027c0 0x7f75cc102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.012+0000 7f75cb200640 1 --2- 172.21.15.42:0/2076243791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75cc1015d0 0x7f75cc1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.012+0000 7f75cb200640 1 -- 172.21.15.42:0/2076243791 >> 172.21.15.42:0/2076243791 conn(0x7f75cc0fcb40 msgr2=0x7f75cc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.012+0000 7f75cb200640 1 -- 172.21.15.42:0/2076243791 shutdown_connections 2024-04-25T00:20:35.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.012+0000 7f75cb200640 1 -- 172.21.15.42:0/2076243791 wait complete. 2024-04-25T00:20:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.013+0000 7f75cb200640 1 Processor -- start 2024-04-25T00:20:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.013+0000 7f75cb200640 1 -- start start 2024-04-25T00:20:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.013+0000 7f75cb200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75cc1015d0 0x7f75cc0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75cb200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75cc1027c0 0x7f75cc073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75cb200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75cc103180 0x7f75cc074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75cb200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10d230 con 0x7f75cc103180 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75cb200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10d0b0 con 0x7f75cc1027c0 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75cb200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc10d3b0 con 0x7f75cc1015d0 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75c9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75cc1027c0 0x7f75cc073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75cc1015d0 0x7f75cc0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75c9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75cc1027c0 0x7f75cc073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42812/0 (socket says 172.21.15.42:42812) 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75c9400640 1 -- 172.21.15.42:0/783305355 learned_addr learned my addr 172.21.15.42:0/783305355 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75ca800640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75cc103180 0x7f75cc074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75c9400640 1 -- 172.21.15.42:0/783305355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75cc1015d0 msgr2=0x7f75cc0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75c9400640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75cc1015d0 0x7f75cc0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75c9400640 1 -- 172.21.15.42:0/783305355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75cc103180 msgr2=0x7f75cc074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75c9400640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75cc103180 0x7f75cc074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.014+0000 7f75c9400640 1 -- 172.21.15.42:0/783305355 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75cc074900 con 0x7f75cc1027c0 2024-04-25T00:20:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.015+0000 7f75c9400640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75cc1027c0 0x7f75cc073b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f75b400e9f0 tx=0x7f75b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.015+0000 7f75c2a00640 1 -- 172.21.15.42:0/783305355 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b400cd80 con 0x7f75cc1027c0 2024-04-25T00:20:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.015+0000 7f75c2a00640 1 -- 172.21.15.42:0/783305355 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b4010910 con 0x7f75cc1027c0 2024-04-25T00:20:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.015+0000 7f75c2a00640 1 -- 172.21.15.42:0/783305355 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b4018990 con 0x7f75cc1027c0 2024-04-25T00:20:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.015+0000 7f75cb200640 1 -- 172.21.15.42:0/783305355 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75cc1a1970 con 0x7f75cc1027c0 2024-04-25T00:20:35.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.015+0000 7f75cb200640 1 -- 172.21.15.42:0/783305355 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75cc1a1e70 con 0x7f75cc1027c0 2024-04-25T00:20:35.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.018+0000 7f75c2a00640 1 -- 172.21.15.42:0/783305355 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f75b4010ac0 con 0x7f75cc1027c0 2024-04-25T00:20:35.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.019+0000 7f75d41c2740 1 Processor -- start 2024-04-25T00:20:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.019+0000 7f75d41c2740 1 -- start start 2024-04-25T00:20:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.019+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x5642756fe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.019+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642756fe840 0x564275703340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.019+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642757039b0 0x564275705e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.019+0000 7f75d41c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564275660ab0 con 0x5642756fe840 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.019+0000 7f75d41c2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564275609640 con 0x5642756fde50 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.019+0000 7f75d41c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56427565ca70 con 0x5642757039b0 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75c9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642757039b0 0x564275705e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.019+0000 7f75ca800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x5642756fe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642756fe840 0x564275703340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75ca800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x5642756fe270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42816/0 (socket says 172.21.15.42:42816) 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75ca800640 1 -- 172.21.15.42:0/1326826905 learned_addr learned my addr 172.21.15.42:0/1326826905 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75c9400640 1 -- 172.21.15.42:0/1326826905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 msgr2=0x5642756fe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75c9400640 1 --2- 172.21.15.42:0/1326826905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x5642756fe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75c9400640 1 -- 172.21.15.42:0/1326826905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642756fe840 msgr2=0x564275703340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75c9400640 1 --2- 172.21.15.42:0/1326826905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642756fe840 0x564275703340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75c9400640 1 -- 172.21.15.42:0/1326826905 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564275375210 con 0x5642757039b0 2024-04-25T00:20:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75c9400640 1 --2- 172.21.15.42:0/1326826905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642757039b0 0x564275705e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f75b40ccd60 tx=0x7f75b40d7910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b50defda77c7b09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:35.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.020+0000 7f75c2a00640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f75b00a2c00 0x7f75b00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.021+0000 7f75bb400640 1 -- 172.21.15.42:0/1326826905 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b40e2070 con 0x5642757039b0 2024-04-25T00:20:35.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.021+0000 7f75c2a00640 1 -- 172.21.15.42:0/783305355 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f75b40cf2f0 con 0x7f75cc1027c0 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.021+0000 7f75bb400640 1 -- 172.21.15.42:0/1326826905 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b409bbd0 con 0x5642757039b0 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.021+0000 7f75bb400640 1 -- 172.21.15.42:0/1326826905 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b409b530 con 0x5642757039b0 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.021+0000 7f75ca800640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f75b00a2c00 0x7f75b00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75d41c2740 1 -- 172.21.15.42:0/1326826905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642757039b0 msgr2=0x564275705e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75d41c2740 1 --2- 172.21.15.42:0/1326826905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642757039b0 0x564275705e40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f75b40ccd60 tx=0x7f75b40d7910 comp rx=0 tx=0).stop 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75ca800640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f75b00a2c00 0x7f75b00a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f75c40057c0 tx=0x7f75c4030340 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75d41c2740 1 -- 172.21.15.42:0/1326826905 shutdown_connections 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75d41c2740 1 --2- 172.21.15.42:0/1326826905 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642757039b0 0x564275705e40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75d41c2740 1 --2- 172.21.15.42:0/1326826905 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642756fe840 0x564275703340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75d41c2740 1 --2- 172.21.15.42:0/1326826905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x5642756fe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75d41c2740 1 -- 172.21.15.42:0/1326826905 >> 172.21.15.42:0/1326826905 conn(0x5642756f94c0 msgr2=0x5642756f98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75d41c2740 1 -- 172.21.15.42:0/1326826905 shutdown_connections 2024-04-25T00:20:35.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75d41c2740 1 -- 172.21.15.42:0/1326826905 wait complete. 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.022+0000 7f75d41c2740 1 -- start start 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.023+0000 7f75d41c2740 1 Processor -- start 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.023+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x564275708090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.023+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642756fe840 0x5642757085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.023+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642757039b0 0x56427570f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.023+0000 7f75d41c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564275706960 con 0x5642756fe840 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.023+0000 7f75d41c2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56427561c990 con 0x5642757039b0 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.023+0000 7f75d41c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564275706ae0 con 0x5642756fde50 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.023+0000 7f75d41c2740 10 client.0 _ll_register_callbacks cb 0x56427561e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x564275708090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75c9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642756fe840 0x5642757085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x564275708090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60448/0 (socket says 172.21.15.42:60448) 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75c9e00640 1 -- 172.21.15.42:0/2634306510 learned_addr learned my addr 172.21.15.42:0/2634306510 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:35.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75ca800640 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642757039b0 0x56427570f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75c9e00640 1 -- 172.21.15.42:0/2634306510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642757039b0 msgr2=0x56427570f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75c9e00640 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642757039b0 0x56427570f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75c9e00640 1 -- 172.21.15.42:0/2634306510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642756fe840 msgr2=0x5642757085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75c9e00640 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642756fe840 0x5642757085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75c9e00640 1 -- 172.21.15.42:0/2634306510 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564275441620 con 0x5642756fde50 2024-04-25T00:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75ca800640 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642757039b0 0x56427570f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.024+0000 7f75c9e00640 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x564275708090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00ee90 tx=0x7f75bc00d7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f79c6f74204463a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:35.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.025+0000 7f75afe00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:20:35.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.025+0000 7f75afe00640 1 -- 172.21.15.42:0/2634306510 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc01c030 con 0x5642756fde50 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.025+0000 7f75afe00640 1 -- 172.21.15.42:0/2634306510 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f75bc003970 con 0x5642756fde50 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.025+0000 7f75d41c2740 1 -- 172.21.15.42:0/2634306510 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75b409bbd0 con 0x5642756fde50 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.025+0000 7f75afe00640 1 -- 172.21.15.42:0/2634306510 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75bc003b70 con 0x5642756fde50 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.025+0000 7f75d41c2740 10 client.25327 fetch_fsmap learned FSMap version 25 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.025+0000 7f75d41c2740 1 -- 172.21.15.42:0/2634306510 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5642757126c0 con 0x5642756fde50 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.025+0000 7f75afe00640 1 -- 172.21.15.42:0/2634306510 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b409bbd0 con 0x5642756fde50 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.026+0000 7f75afe00640 1 -- 172.21.15.42:0/2634306510 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f75bc00b040 con 0x5642756fde50 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.026+0000 7f75d41c2740 10 client.25327 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.026+0000 7f75d41c2740 10 client.25327 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.026+0000 7f75d41c2740 20 client.25327 populate_metadata read hostname 'smithi042' 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.026+0000 7f75d41c2740 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564275713e70 0x564275716300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.026+0000 7f75d41c2740 4 client.25327 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:35.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.026+0000 7f75d41c2740 1 -- 172.21.15.42:0/2634306510 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56427566dff0 con 0x564275713e70 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.026+0000 7f75c9400640 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564275713e70 0x564275716300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.027+0000 7f75c9400640 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564275713e70 0x564275716300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=621ab9b208c17054 server_cookie=b352e715423991d9 in_seq=0 out_seq=0 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.027+0000 7f75afe00640 10 client.25327 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.027+0000 7f75afe00640 1 -- 172.21.15.42:0/2634306510 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56427566dff0 con 0x564275713e70 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.027+0000 7f75afe00640 10 client.25327 handle_command_reply: tid=0 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.027+0000 7f75d41c2740 1 client.25327 shutdown 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 1 -- 172.21.15.42:0/2634306510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 msgr2=0x564275708090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x564275708090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00ee90 tx=0x7f75bc00d7b0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 1 -- 172.21.15.42:0/2634306510 shutdown_connections 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564275713e70 0x564275716300 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642757039b0 0x56427570f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642756fe840 0x5642757085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 1 --2- 172.21.15.42:0/2634306510 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x564275708090 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75afe00640 0 client.25327 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 1 -- 172.21.15.42:0/2634306510 >> 172.21.15.42:0/2634306510 conn(0x5642756f94c0 msgr2=0x5642756f8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 1 -- 172.21.15.42:0/2634306510 shutdown_connections 2024-04-25T00:20:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 1 -- 172.21.15.42:0/2634306510 wait complete. 2024-04-25T00:20:35.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.028+0000 7f75d41c2740 20 client.25327 trim_cache size 0 max 16384 2024-04-25T00:20:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.045+0000 7f75d41c2740 1 Processor -- start 2024-04-25T00:20:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75d41c2740 1 -- start start 2024-04-25T00:20:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x5642757076b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fe840 0x56427570f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642757039b0 0x564275707bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75d41c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc1ad530 con 0x5642757039b0 2024-04-25T00:20:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75d41c2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc1aae00 con 0x5642756fe840 2024-04-25T00:20:35.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75d41c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc1a8c50 con 0x5642756fde50 2024-04-25T00:20:35.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75c9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fe840 0x56427570f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x5642757076b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75ca800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642757039b0 0x564275707bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75c9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fe840 0x56427570f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42834/0 (socket says 172.21.15.42:42834) 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x5642757076b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60460/0 (socket says 172.21.15.42:60460) 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.046+0000 7f75c9400640 1 -- 172.21.15.42:0/796397775 learned_addr learned my addr 172.21.15.42:0/796397775 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.047+0000 7f75c9400640 1 -- 172.21.15.42:0/796397775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 msgr2=0x5642757076b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.047+0000 7f75c9400640 1 --2- 172.21.15.42:0/796397775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x5642757076b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.047+0000 7f75c9400640 1 -- 172.21.15.42:0/796397775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642757039b0 msgr2=0x564275707bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.047+0000 7f75c9400640 1 --2- 172.21.15.42:0/796397775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642757039b0 0x564275707bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.047+0000 7f75c9400640 1 -- 172.21.15.42:0/796397775 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642756f3730 con 0x5642756fe840 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.047+0000 7f75c9400640 1 --2- 172.21.15.42:0/796397775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fe840 0x56427570f5d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75b4098030 tx=0x7f75b409b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27c53b796c7309b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.047+0000 7f75baa00640 1 -- 172.21.15.42:0/796397775 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b409b530 con 0x5642756fe840 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.047+0000 7f75baa00640 1 -- 172.21.15.42:0/796397775 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b409ce50 con 0x5642756fe840 2024-04-25T00:20:35.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.047+0000 7f75baa00640 1 -- 172.21.15.42:0/796397775 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75b409b710 con 0x5642756fe840 2024-04-25T00:20:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.048+0000 7f75d41c2740 1 -- 172.21.15.42:0/796397775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fe840 msgr2=0x56427570f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.048+0000 7f75d41c2740 1 --2- 172.21.15.42:0/796397775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fe840 0x56427570f5d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f75b4098030 tx=0x7f75b409b9a0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.048+0000 7f75d41c2740 1 -- 172.21.15.42:0/796397775 shutdown_connections 2024-04-25T00:20:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.048+0000 7f75d41c2740 1 --2- 172.21.15.42:0/796397775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642757039b0 0x564275707bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.048+0000 7f75d41c2740 1 --2- 172.21.15.42:0/796397775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fe840 0x56427570f5d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.048+0000 7f75d41c2740 1 --2- 172.21.15.42:0/796397775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fde50 0x5642757076b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.048+0000 7f75d41c2740 1 -- 172.21.15.42:0/796397775 >> 172.21.15.42:0/796397775 conn(0x5642756f94c0 msgr2=0x564275703060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.048+0000 7f75d41c2740 1 -- 172.21.15.42:0/796397775 shutdown_connections 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.048+0000 7f75d41c2740 1 -- 172.21.15.42:0/796397775 wait complete. 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.049+0000 7f75d41c2740 1 -- start start 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.049+0000 7f75d41c2740 1 Processor -- start 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.049+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x56427570cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.049+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fe840 0x56427570d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.049+0000 7f75d41c2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642757039b0 0x56427570e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.049+0000 7f75d41c2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc1a92c0 con 0x5642757039b0 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.049+0000 7f75d41c2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc1aa730 con 0x5642756fde50 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.049+0000 7f75d41c2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75cc1a9990 con 0x5642756fe840 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.049+0000 7f75d41c2740 10 client.0 _ll_register_callbacks cb 0x7f75bc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:35.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x56427570cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75c9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fe840 0x56427570d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75c9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x56427570cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42836/0 (socket says 172.21.15.42:42836) 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75c9e00640 1 -- 172.21.15.42:0/1184151246 learned_addr learned my addr 172.21.15.42:0/1184151246 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75ca800640 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642757039b0 0x56427570e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75c9e00640 1 -- 172.21.15.42:0/1184151246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fe840 msgr2=0x56427570d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75c9e00640 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fe840 0x56427570d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75c9e00640 1 -- 172.21.15.42:0/1184151246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642757039b0 msgr2=0x56427570e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75c9e00640 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642757039b0 0x56427570e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75c9e00640 1 -- 172.21.15.42:0/1184151246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642756f92f0 con 0x5642756fde50 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75c9e00640 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x56427570cf30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75bc021970 tx=0x7f75bc0092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f9e96168ad207c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75afe00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.050+0000 7f75afe00640 1 -- 172.21.15.42:0/1184151246 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75bc01c040 con 0x5642756fde50 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.051+0000 7f75afe00640 1 -- 172.21.15.42:0/1184151246 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f75bc02d470 con 0x5642756fde50 2024-04-25T00:20:35.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.051+0000 7f75d41c2740 1 -- 172.21.15.42:0/1184151246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75b409ce50 con 0x5642756fde50 2024-04-25T00:20:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.051+0000 7f75afe00640 1 -- 172.21.15.42:0/1184151246 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75bc0095a0 con 0x5642756fde50 2024-04-25T00:20:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.051+0000 7f75d41c2740 10 client.25331 fetch_fsmap learned FSMap version 25 2024-04-25T00:20:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.051+0000 7f75d41c2740 1 -- 172.21.15.42:0/1184151246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56427577a480 con 0x5642756fde50 2024-04-25T00:20:35.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.051+0000 7f75afe00640 1 -- 172.21.15.42:0/1184151246 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f75b409ce50 con 0x5642756fde50 2024-04-25T00:20:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.051+0000 7f75afe00640 1 -- 172.21.15.42:0/1184151246 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f75bc00b070 con 0x5642756fde50 2024-04-25T00:20:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.051+0000 7f75d41c2740 10 client.25331 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:20:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.051+0000 7f75d41c2740 10 client.25331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.051+0000 7f75d41c2740 20 client.25331 populate_metadata read hostname 'smithi042' 2024-04-25T00:20:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.052+0000 7f75d41c2740 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56427577ad40 0x56427577d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.052+0000 7f75d41c2740 4 client.25331 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:20:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.052+0000 7f75d41c2740 1 -- 172.21.15.42:0/1184151246 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564275668340 con 0x56427577ad40 2024-04-25T00:20:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.052+0000 7f75c9400640 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56427577ad40 0x56427577d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.052+0000 7f75afe00640 10 client.25331 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:35.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.052+0000 7f75c9400640 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56427577ad40 0x56427577d160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da510ee91612fac7 server_cookie=34b7c70ac7fae38e in_seq=0 out_seq=0 2024-04-25T00:20:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75afe00640 1 -- 172.21.15.42:0/1184151246 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564275668340 con 0x56427577ad40 2024-04-25T00:20:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75afe00640 10 client.25331 handle_command_reply: tid=0 2024-04-25T00:20:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75d41c2740 1 client.25331 shutdown 2024-04-25T00:20:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75d41c2740 1 -- 172.21.15.42:0/1184151246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 msgr2=0x56427570cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75d41c2740 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x56427570cf30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f75bc021970 tx=0x7f75bc0092c0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75d41c2740 1 -- 172.21.15.42:0/1184151246 shutdown_connections 2024-04-25T00:20:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75d41c2740 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56427577ad40 0x56427577d160 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75d41c2740 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5642757039b0 0x56427570e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75d41c2740 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5642756fe840 0x56427570d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75d41c2740 1 --2- 172.21.15.42:0/1184151246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5642756fde50 0x56427570cf30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75d41c2740 1 -- 172.21.15.42:0/1184151246 >> 172.21.15.42:0/1184151246 conn(0x5642756f94c0 msgr2=0x5642756f7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75afe00640 0 client.25331 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.113+0000 7f75d41c2740 1 -- 172.21.15.42:0/1184151246 shutdown_connections 2024-04-25T00:20:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.114+0000 7f75d41c2740 1 -- 172.21.15.42:0/1184151246 wait complete. 2024-04-25T00:20:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.114+0000 7f75d41c2740 20 client.25331 trim_cache size 0 max 16384 2024-04-25T00:20:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:20:35.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:20:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:20:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:20:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 -- 172.21.15.42:0/783305355 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f75b00a2c00 msgr2=0x7f75b00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f75b00a2c00 0x7f75b00a50c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f75c40057c0 tx=0x7f75c4030340 comp rx=0 tx=0).stop 2024-04-25T00:20:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 -- 172.21.15.42:0/783305355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75cc1027c0 msgr2=0x7f75cc073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75cc1027c0 0x7f75cc073b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f75b400e9f0 tx=0x7f75b400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 -- 172.21.15.42:0/783305355 shutdown_connections 2024-04-25T00:20:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f75b00a2c00 0x7f75b00a50c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f75cc103180 0x7f75cc074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75cc1027c0 0x7f75cc073b30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 --2- 172.21.15.42:0/783305355 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75cc1015d0 0x7f75cc0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 -- 172.21.15.42:0/783305355 >> 172.21.15.42:0/783305355 conn(0x7f75cc0fcb40 msgr2=0x7f75cc105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 -- 172.21.15.42:0/783305355 shutdown_connections 2024-04-25T00:20:35.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:35.115+0000 7f75cb200640 1 -- 172.21.15.42:0/783305355 wait complete. 2024-04-25T00:20:35.203 INFO:tasks.workunit.client.0.smithi042.stdout: 1 41555 9.56 MB/sec warmup 25 sec latency 28.016 ms 2024-04-25T00:20:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:35 smithi136 ceph-mon[32923]: pgmap v442: 97 pgs: 97 active+clean; 13 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 759 KiB/s wr, 192 op/s 2024-04-25T00:20:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:35 smithi106 ceph-mon[32510]: pgmap v442: 97 pgs: 97 active+clean; 13 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 759 KiB/s wr, 192 op/s 2024-04-25T00:20:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:35 smithi042 ceph-mon[28130]: pgmap v442: 97 pgs: 97 active+clean; 13 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 364 KiB/s rd, 759 KiB/s wr, 192 op/s 2024-04-25T00:20:36.203 INFO:tasks.workunit.client.0.smithi042.stdout: 1 42993 9.63 MB/sec warmup 26 sec latency 57.261 ms 2024-04-25T00:20:37.203 INFO:tasks.workunit.client.0.smithi042.stdout: 1 44023 9.46 MB/sec warmup 27 sec latency 24.846 ms 2024-04-25T00:20:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:37 smithi136 ceph-mon[32923]: pgmap v443: 97 pgs: 97 active+clean; 15 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 997 KiB/s wr, 249 op/s 2024-04-25T00:20:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:37 smithi106 ceph-mon[32510]: pgmap v443: 97 pgs: 97 active+clean; 15 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 997 KiB/s wr, 249 op/s 2024-04-25T00:20:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:37 smithi042 ceph-mon[28130]: pgmap v443: 97 pgs: 97 active+clean; 15 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 472 KiB/s rd, 997 KiB/s wr, 249 op/s 2024-04-25T00:20:38.203 INFO:tasks.workunit.client.0.smithi042.stdout: 1 45792 9.42 MB/sec warmup 28 sec latency 16.813 ms 2024-04-25T00:20:39.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:38 smithi136 ceph-mon[32923]: pgmap v444: 97 pgs: 97 active+clean; 17 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-04-25T00:20:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:38 smithi106 ceph-mon[32510]: pgmap v444: 97 pgs: 97 active+clean; 17 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-04-25T00:20:39.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:38 smithi042 ceph-mon[28130]: pgmap v444: 97 pgs: 97 active+clean; 17 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 547 KiB/s rd, 1.3 MiB/s wr, 211 op/s 2024-04-25T00:20:39.203 INFO:tasks.workunit.client.0.smithi042.stdout: 1 46721 9.30 MB/sec warmup 29 sec latency 15.055 ms 2024-04-25T00:20:40.204 INFO:tasks.workunit.client.0.smithi042.stdout: 1 47911 9.16 MB/sec warmup 30 sec latency 23.762 ms 2024-04-25T00:20:41.204 INFO:tasks.workunit.client.0.smithi042.stdout: 1 49827 9.27 MB/sec warmup 31 sec latency 37.094 ms 2024-04-25T00:20:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:41 smithi106 ceph-mon[32510]: pgmap v445: 97 pgs: 97 active+clean; 23 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-04-25T00:20:41.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:41 smithi042 ceph-mon[28130]: pgmap v445: 97 pgs: 97 active+clean; 23 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-04-25T00:20:41.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:41 smithi136 ceph-mon[32923]: pgmap v445: 97 pgs: 97 active+clean; 23 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 1.6 MiB/s wr, 162 op/s 2024-04-25T00:20:42.204 INFO:tasks.workunit.client.0.smithi042.stdout: 1 50138 9.02 MB/sec warmup 32 sec latency 181.588 ms 2024-04-25T00:20:43.204 INFO:tasks.workunit.client.0.smithi042.stdout: 1 51248 8.90 MB/sec warmup 33 sec latency 36.283 ms 2024-04-25T00:20:43.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:43 smithi106 ceph-mon[32510]: pgmap v446: 97 pgs: 97 active+clean; 38 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 220 op/s 2024-04-25T00:20:43.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:43 smithi042 ceph-mon[28130]: pgmap v446: 97 pgs: 97 active+clean; 38 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 220 op/s 2024-04-25T00:20:43.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:43 smithi136 ceph-mon[32923]: pgmap v446: 97 pgs: 97 active+clean; 38 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 220 op/s 2024-04-25T00:20:44.204 INFO:tasks.workunit.client.0.smithi042.stdout: 1 52967 8.91 MB/sec warmup 34 sec latency 23.730 ms 2024-04-25T00:20:45.204 INFO:tasks.workunit.client.0.smithi042.stdout: 1 53706 8.79 MB/sec warmup 35 sec latency 50.147 ms 2024-04-25T00:20:45.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:45 smithi106 ceph-mon[32510]: pgmap v447: 97 pgs: 97 active+clean; 38 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-04-25T00:20:45.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:45 smithi042 ceph-mon[28130]: pgmap v447: 97 pgs: 97 active+clean; 38 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-04-25T00:20:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:45 smithi136 ceph-mon[32923]: pgmap v447: 97 pgs: 97 active+clean; 38 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-04-25T00:20:46.204 INFO:tasks.workunit.client.0.smithi042.stdout: 1 54805 8.69 MB/sec warmup 36 sec latency 23.771 ms 2024-04-25T00:20:47.205 INFO:tasks.workunit.client.0.smithi042.stdout: 1 56444 8.68 MB/sec warmup 37 sec latency 23.410 ms 2024-04-25T00:20:47.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:47 smithi106 ceph-mon[32510]: pgmap v448: 97 pgs: 97 active+clean; 36 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 285 op/s 2024-04-25T00:20:47.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:47 smithi042 ceph-mon[28130]: pgmap v448: 97 pgs: 97 active+clean; 36 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 285 op/s 2024-04-25T00:20:47.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:47 smithi136 ceph-mon[32923]: pgmap v448: 97 pgs: 97 active+clean; 36 MiB data, 545 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 285 op/s 2024-04-25T00:20:48.205 INFO:tasks.workunit.client.0.smithi042.stdout: 1 57266 8.60 MB/sec warmup 38 sec latency 16.156 ms 2024-04-25T00:20:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.263+0000 7fabd5600640 1 -- 172.21.15.42:0/918772429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabd0102070 msgr2=0x7fabd0108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.263+0000 7fabd5600640 1 --2- 172.21.15.42:0/918772429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabd0102070 0x7fabd0108900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009ca0 tx=0x7fabc802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.263+0000 7fabd5600640 1 -- 172.21.15.42:0/918772429 shutdown_connections 2024-04-25T00:20:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.263+0000 7fabd5600640 1 --2- 172.21.15.42:0/918772429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabd0102070 0x7fabd0108900 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.263+0000 7fabd5600640 1 --2- 172.21.15.42:0/918772429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabd01016b0 0x7fabd0101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.263+0000 7fabd5600640 1 --2- 172.21.15.42:0/918772429 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd01004c0 0x7fabd01008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.263+0000 7fabd5600640 1 -- 172.21.15.42:0/918772429 >> 172.21.15.42:0/918772429 conn(0x7fabd00fba30 msgr2=0x7fabd00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.263+0000 7fabd5600640 1 -- 172.21.15.42:0/918772429 shutdown_connections 2024-04-25T00:20:48.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.263+0000 7fabd5600640 1 -- 172.21.15.42:0/918772429 wait complete. 2024-04-25T00:20:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.264+0000 7fabd5600640 1 Processor -- start 2024-04-25T00:20:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.264+0000 7fabd5600640 1 -- start start 2024-04-25T00:20:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd01004c0 0x7fabd0199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd5600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabd01016b0 0x7fabd019a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd5600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabd0102070 0x7fabd01a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd5600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabd010c170 con 0x7fabd01004c0 2024-04-25T00:20:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd5600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fabd010bff0 con 0x7fabd0102070 2024-04-25T00:20:48.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd5600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fabd010c2f0 con 0x7fabd01016b0 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabd0102070 0x7fabd01a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd01004c0 0x7fabd0199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabcf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabd01016b0 0x7fabd019a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabd0102070 0x7fabd01a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49890/0 (socket says 172.21.15.42:49890) 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd4c00640 1 -- 172.21.15.42:0/3709236075 learned_addr learned my addr 172.21.15.42:0/3709236075 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd4c00640 1 -- 172.21.15.42:0/3709236075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabd01016b0 msgr2=0x7fabd019a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd4c00640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabd01016b0 0x7fabd019a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd4c00640 1 -- 172.21.15.42:0/3709236075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd01004c0 msgr2=0x7fabd0199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd4c00640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd01004c0 0x7fabd0199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.265+0000 7fabd4c00640 1 -- 172.21.15.42:0/3709236075 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd01a16e0 con 0x7fabd0102070 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.266+0000 7fabd4c00640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabd0102070 0x7fabd01a1070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fabc802f540 tx=0x7fabc8031750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.266+0000 7fabccc00640 1 -- 172.21.15.42:0/3709236075 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc8003b70 con 0x7fabd0102070 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.266+0000 7fabccc00640 1 -- 172.21.15.42:0/3709236075 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc8003d10 con 0x7fabd0102070 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.266+0000 7fabccc00640 1 -- 172.21.15.42:0/3709236075 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc802fb50 con 0x7fabd0102070 2024-04-25T00:20:48.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.266+0000 7fabd5600640 1 -- 172.21.15.42:0/3709236075 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd01a1970 con 0x7fabd0102070 2024-04-25T00:20:48.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.266+0000 7fabd5600640 1 -- 172.21.15.42:0/3709236075 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd01a1e70 con 0x7fabd0102070 2024-04-25T00:20:48.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.269+0000 7fabccc00640 1 -- 172.21.15.42:0/3709236075 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fabc802fcf0 con 0x7fabd0102070 2024-04-25T00:20:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.270+0000 7fabd9722740 1 Processor -- start 2024-04-25T00:20:48.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.270+0000 7fabd9722740 1 -- start start 2024-04-25T00:20:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.270+0000 7fabd9722740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc9099e50 0x564cc909a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.270+0000 7fabd9722740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909a840 0x564cc909f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.270+0000 7fabd9722740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.270+0000 7fabd9722740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564cc8ffcab0 con 0x564cc909f9b0 2024-04-25T00:20:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.270+0000 7fabd9722740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564cc8fa5640 con 0x564cc909a840 2024-04-25T00:20:48.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.270+0000 7fabd9722740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564cc8ff8a70 con 0x564cc9099e50 2024-04-25T00:20:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.270+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909a840 0x564cc909f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.271+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909a840 0x564cc909f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49906/0 (socket says 172.21.15.42:49906) 2024-04-25T00:20:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.271+0000 7fabcfe00640 1 -- 172.21.15.42:0/276651080 learned_addr learned my addr 172.21.15.42:0/276651080 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.271+0000 7fabcf400640 1 --2- 172.21.15.42:0/276651080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc9099e50 0x564cc909a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.271+0000 7fabcf400640 1 --2- 172.21.15.42:0/276651080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.271+0000 7fabcfe00640 1 -- 172.21.15.42:0/276651080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc9099e50 msgr2=0x564cc909a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.271+0000 7fabcfe00640 1 --2- 172.21.15.42:0/276651080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc9099e50 0x564cc909a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.271+0000 7fabcfe00640 1 -- 172.21.15.42:0/276651080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 msgr2=0x564cc90a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.271+0000 7fabcfe00640 1 --2- 172.21.15.42:0/276651080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.271+0000 7fabcfe00640 1 -- 172.21.15.42:0/276651080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cc8d11210 con 0x564cc909a840 2024-04-25T00:20:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.271+0000 7fabcfe00640 1 --2- 172.21.15.42:0/276651080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909a840 0x564cc909f340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009ca0 tx=0x7fabc0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212a55ee4036461d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.272+0000 7fabccc00640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fabb40a2c00 0x7fabb40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.272+0000 7fabc5200640 1 -- 172.21.15.42:0/276651080 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0004220 con 0x564cc909a840 2024-04-25T00:20:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.272+0000 7fabccc00640 1 -- 172.21.15.42:0/3709236075 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fabc80f03d0 con 0x7fabd0102070 2024-04-25T00:20:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.272+0000 7fabc5200640 1 -- 172.21.15.42:0/276651080 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc00043c0 con 0x564cc909a840 2024-04-25T00:20:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.272+0000 7fabcf400640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fabb40a2c00 0x7fabb40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.273+0000 7fabd9722740 1 -- 172.21.15.42:0/276651080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909a840 msgr2=0x564cc909f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.273+0000 7fabcf400640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fabb40a2c00 0x7fabb40a50c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x564cc909b0a0 tx=0x7fabbc013040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.273+0000 7fabd9722740 1 --2- 172.21.15.42:0/276651080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909a840 0x564cc909f340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009ca0 tx=0x7fabc0011810 comp rx=0 tx=0).stop 2024-04-25T00:20:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.272+0000 7fabc5200640 1 -- 172.21.15.42:0/276651080 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 0 0 0) 0x7fabc0019930 con 0x564cc909a840 2024-04-25T00:20:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.273+0000 7fabd9722740 1 -- 172.21.15.42:0/276651080 shutdown_connections 2024-04-25T00:20:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.273+0000 7fabd9722740 1 --2- 172.21.15.42:0/276651080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.273+0000 7fabd9722740 1 --2- 172.21.15.42:0/276651080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909a840 0x564cc909f340 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.273+0000 7fabd9722740 1 --2- 172.21.15.42:0/276651080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc9099e50 0x564cc909a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.273+0000 7fabd9722740 1 -- 172.21.15.42:0/276651080 >> 172.21.15.42:0/276651080 conn(0x564cc90954c0 msgr2=0x564cc90958c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.273+0000 7fabd9722740 1 -- 172.21.15.42:0/276651080 shutdown_connections 2024-04-25T00:20:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.273+0000 7fabd9722740 1 -- 172.21.15.42:0/276651080 wait complete. 2024-04-25T00:20:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.274+0000 7fabd9722740 1 -- start start 2024-04-25T00:20:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.274+0000 7fabd9722740 1 Processor -- start 2024-04-25T00:20:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.274+0000 7fabd9722740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc90a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.274+0000 7fabd9722740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.274+0000 7fabd9722740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90ab660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.274+0000 7fabd9722740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564cc90a29b0 con 0x564cc909f9b0 2024-04-25T00:20:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.274+0000 7fabd9722740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564cc8fb8990 con 0x564cc9099e50 2024-04-25T00:20:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.274+0000 7fabd9722740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564cc90a2b30 con 0x564cc909a840 2024-04-25T00:20:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc90a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc90a40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49912/0 (socket says 172.21.15.42:49912) 2024-04-25T00:20:48.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabcfe00640 1 -- 172.21.15.42:0/4091755179 learned_addr learned my addr 172.21.15.42:0/4091755179 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabcf400640 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabd9722740 10 client.0 _ll_register_callbacks cb 0x564cc8fba950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabd4c00640 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabcfe00640 1 -- 172.21.15.42:0/4091755179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 msgr2=0x564cc90a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabcfe00640 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabcfe00640 1 -- 172.21.15.42:0/4091755179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 msgr2=0x564cc90ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabcfe00640 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabcfe00640 1 -- 172.21.15.42:0/4091755179 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cc8ddd620 con 0x564cc9099e50 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabcfe00640 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc90a40e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabc0011d10 tx=0x7fabc0003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a51db22aecc437fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.275+0000 7fabb9600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.276+0000 7fabb9600640 1 -- 172.21.15.42:0/4091755179 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0028070 con 0x564cc9099e50 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.276+0000 7fabd9722740 1 -- 172.21.15.42:0/4091755179 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc00043c0 con 0x564cc9099e50 2024-04-25T00:20:48.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.276+0000 7fabb9600640 1 -- 172.21.15.42:0/4091755179 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fabc0005040 con 0x564cc9099e50 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.276+0000 7fabb9600640 1 -- 172.21.15.42:0/4091755179 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fabc0034050 con 0x564cc9099e50 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.276+0000 7fabd9722740 10 client.25343 fetch_fsmap learned FSMap version 25 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.276+0000 7fabd9722740 1 -- 172.21.15.42:0/4091755179 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564cc90ae690 con 0x564cc9099e50 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.276+0000 7fabb9600640 1 -- 172.21.15.42:0/4091755179 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc00043c0 con 0x564cc9099e50 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.277+0000 7fabb9600640 1 -- 172.21.15.42:0/4091755179 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fabc0004a80 con 0x564cc9099e50 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.277+0000 7fabd9722740 10 client.25343 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.277+0000 7fabd9722740 10 client.25343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.277+0000 7fabd9722740 20 client.25343 populate_metadata read hostname 'smithi042' 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.277+0000 7fabd9722740 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564cc90afc20 0x564cc90b2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.277+0000 7fabd9722740 4 client.25343 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.277+0000 7fabd9722740 1 -- 172.21.15.42:0/4091755179 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cc9009ff0 con 0x564cc90afc20 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.277+0000 7fabcf400640 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564cc90afc20 0x564cc90b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.278+0000 7fabcf400640 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564cc90afc20 0x564cc90b2030 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c776883dcffe375d server_cookie=daed445d32950c95 in_seq=0 out_seq=0 2024-04-25T00:20:48.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.278+0000 7fabb9600640 10 client.25343 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:48.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.278+0000 7fabb9600640 1 -- 172.21.15.42:0/4091755179 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564cc9009ff0 con 0x564cc90afc20 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.278+0000 7fabb9600640 10 client.25343 handle_command_reply: tid=0 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.278+0000 7fabd9722740 1 client.25343 shutdown 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 1 -- 172.21.15.42:0/4091755179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 msgr2=0x564cc90a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc90a40e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabc0011d10 tx=0x7fabc0003820 comp rx=0 tx=0).stop 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 1 -- 172.21.15.42:0/4091755179 shutdown_connections 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564cc90afc20 0x564cc90b2030 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 1 --2- 172.21.15.42:0/4091755179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc90a40e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabb9600640 0 client.25343 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 1 -- 172.21.15.42:0/4091755179 >> 172.21.15.42:0/4091755179 conn(0x564cc90954c0 msgr2=0x564cc9094620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 1 -- 172.21.15.42:0/4091755179 shutdown_connections 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 1 -- 172.21.15.42:0/4091755179 wait complete. 2024-04-25T00:20:48.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.279+0000 7fabd9722740 20 client.25343 trim_cache size 0 max 16384 2024-04-25T00:20:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.296+0000 7fabd9722740 1 Processor -- start 2024-04-25T00:20:48.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabd9722740 1 -- start start 2024-04-25T00:20:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabd9722740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc9094620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabd9722740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90ab620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabd9722740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90a3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabd9722740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01ad470 con 0x564cc909f9b0 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabd9722740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01aad40 con 0x564cc9099e50 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabd9722740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01a8b90 con 0x564cc909a840 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc9094620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc9094620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49922/0 (socket says 172.21.15.42:49922) 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabcfe00640 1 -- 172.21.15.42:0/128488912 learned_addr learned my addr 172.21.15.42:0/128488912 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.297+0000 7fabcf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.298+0000 7fabd4c00640 1 --2- 172.21.15.42:0/128488912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.298+0000 7fabcfe00640 1 -- 172.21.15.42:0/128488912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 msgr2=0x564cc90ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.298+0000 7fabcfe00640 1 --2- 172.21.15.42:0/128488912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.298+0000 7fabcfe00640 1 -- 172.21.15.42:0/128488912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 msgr2=0x564cc90a3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.298+0000 7fabcfe00640 1 --2- 172.21.15.42:0/128488912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90a3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.298+0000 7fabcfe00640 1 -- 172.21.15.42:0/128488912 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cc90b26b0 con 0x564cc9099e50 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.298+0000 7fabcfe00640 1 --2- 172.21.15.42:0/128488912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc9094620 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009ca0 tx=0x7fabc0005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e30914dd465ada3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.298+0000 7fabbbe00640 1 -- 172.21.15.42:0/128488912 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0028040 con 0x564cc9099e50 2024-04-25T00:20:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.298+0000 7fabbbe00640 1 -- 172.21.15.42:0/128488912 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc0019430 con 0x564cc9099e50 2024-04-25T00:20:48.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.298+0000 7fabbbe00640 1 -- 172.21.15.42:0/128488912 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0005440 con 0x564cc9099e50 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.299+0000 7fabd9722740 1 -- 172.21.15.42:0/128488912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 msgr2=0x564cc9094620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.299+0000 7fabd9722740 1 --2- 172.21.15.42:0/128488912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc9094620 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fabc0009ca0 tx=0x7fabc0005f90 comp rx=0 tx=0).stop 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.299+0000 7fabd9722740 1 -- 172.21.15.42:0/128488912 shutdown_connections 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.299+0000 7fabd9722740 1 --2- 172.21.15.42:0/128488912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc909f9b0 0x564cc90a3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.299+0000 7fabd9722740 1 --2- 172.21.15.42:0/128488912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.299+0000 7fabd9722740 1 --2- 172.21.15.42:0/128488912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc9099e50 0x564cc9094620 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.299+0000 7fabd9722740 1 -- 172.21.15.42:0/128488912 >> 172.21.15.42:0/128488912 conn(0x564cc90954c0 msgr2=0x564cc909eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.299+0000 7fabd9722740 1 -- 172.21.15.42:0/128488912 shutdown_connections 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.299+0000 7fabd9722740 1 -- 172.21.15.42:0/128488912 wait complete. 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.300+0000 7fabd9722740 1 -- start start 2024-04-25T00:20:48.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.300+0000 7fabd9722740 1 Processor -- start 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.300+0000 7fabd9722740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc9099e50 0x564cc90a8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.300+0000 7fabd9722740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90a9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.300+0000 7fabd9722740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909f9b0 0x564cc90aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.300+0000 7fabd9722740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01a9200 con 0x564cc9099e50 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.300+0000 7fabd9722740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01aa670 con 0x564cc909f9b0 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.300+0000 7fabd9722740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fabd01a98d0 con 0x564cc909a840 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.300+0000 7fabd9722740 10 client.0 _ll_register_callbacks cb 0x7fabbc009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabd4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909f9b0 0x564cc90aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabcf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabd4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909f9b0 0x564cc90aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49928/0 (socket says 172.21.15.42:49928) 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabcfe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc9099e50 0x564cc90a8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabd4c00640 1 -- 172.21.15.42:0/513331556 learned_addr learned my addr 172.21.15.42:0/513331556 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:20:48.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabd4c00640 1 -- 172.21.15.42:0/513331556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 msgr2=0x564cc90a9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabd4c00640 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90a9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabd4c00640 1 -- 172.21.15.42:0/513331556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc9099e50 msgr2=0x564cc90a8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabd4c00640 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc9099e50 0x564cc90a8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabd4c00640 1 -- 172.21.15.42:0/513331556 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cc90aa770 con 0x564cc909f9b0 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabcf400640 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabd4c00640 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909f9b0 0x564cc90aa180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006010 tx=0x7fabc80bc810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e586bf6c4ceb56ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabb9600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabb9600640 1 -- 172.21.15.42:0/513331556 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc80c0fb0 con 0x564cc909f9b0 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabb9600640 1 -- 172.21.15.42:0/513331556 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7fabc80d5050 con 0x564cc909f9b0 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.301+0000 7fabd9722740 1 -- 172.21.15.42:0/513331556 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc0019430 con 0x564cc909f9b0 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabb9600640 1 -- 172.21.15.42:0/513331556 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fabc80c5870 con 0x564cc909f9b0 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabd9722740 10 client.25349 fetch_fsmap learned FSMap version 25 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabd9722740 1 -- 172.21.15.42:0/513331556 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564cc91161b0 con 0x564cc909f9b0 2024-04-25T00:20:48.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabb9600640 1 -- 172.21.15.42:0/513331556 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fabc0019430 con 0x564cc909f9b0 2024-04-25T00:20:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabb9600640 1 -- 172.21.15.42:0/513331556 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fabc80bcef0 con 0x564cc909f9b0 2024-04-25T00:20:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabd9722740 10 client.25349 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:20:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabd9722740 10 client.25349 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:20:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabd9722740 20 client.25349 populate_metadata read hostname 'smithi042' 2024-04-25T00:20:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabd9722740 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564cc9116b40 0x564cc9118f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:20:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabd9722740 4 client.25349 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:20:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabd9722740 1 -- 172.21.15.42:0/513331556 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564cc9004340 con 0x564cc9116b40 2024-04-25T00:20:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.302+0000 7fabcfe00640 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564cc9116b40 0x564cc9118f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:20:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.303+0000 7fabcfe00640 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564cc9116b40 0x564cc9118f60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=409dc829eb0a615f server_cookie=34e6bfa1dec40464 in_seq=0 out_seq=0 2024-04-25T00:20:48.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.303+0000 7fabb9600640 10 client.25349 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.358+0000 7fabb9600640 1 -- 172.21.15.42:0/513331556 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564cc9004340 con 0x564cc9116b40 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.358+0000 7fabb9600640 10 client.25349 handle_command_reply: tid=0 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.358+0000 7fabd9722740 1 client.25349 shutdown 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.358+0000 7fabd9722740 1 -- 172.21.15.42:0/513331556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909f9b0 msgr2=0x564cc90aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.358+0000 7fabd9722740 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909f9b0 0x564cc90aa180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fabc8006010 tx=0x7fabc80bc810 comp rx=0 tx=0).stop 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.359+0000 7fabd9722740 1 -- 172.21.15.42:0/513331556 shutdown_connections 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.359+0000 7fabd9722740 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564cc9116b40 0x564cc9118f60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.359+0000 7fabd9722740 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564cc909f9b0 0x564cc90aa180 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.359+0000 7fabb9600640 0 client.25349 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.359+0000 7fabd9722740 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564cc909a840 0x564cc90a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.359+0000 7fabd9722740 1 --2- 172.21.15.42:0/513331556 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564cc9099e50 0x564cc90a8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.359+0000 7fabd9722740 1 -- 172.21.15.42:0/513331556 >> 172.21.15.42:0/513331556 conn(0x564cc90954c0 msgr2=0x564cc9094620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.359+0000 7fabd9722740 1 -- 172.21.15.42:0/513331556 shutdown_connections 2024-04-25T00:20:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.359+0000 7fabd9722740 1 -- 172.21.15.42:0/513331556 wait complete. 2024-04-25T00:20:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.359+0000 7fabd9722740 20 client.25349 trim_cache size 0 max 16384 2024-04-25T00:20:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:20:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:20:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:20:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 -- 172.21.15.42:0/3709236075 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fabb40a2c00 msgr2=0x7fabb40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fabb40a2c00 0x7fabb40a50c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x564cc909b0a0 tx=0x7fabbc013040 comp rx=0 tx=0).stop 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 -- 172.21.15.42:0/3709236075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabd0102070 msgr2=0x7fabd01a1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabd0102070 0x7fabd01a1070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fabc802f540 tx=0x7fabc8031750 comp rx=0 tx=0).stop 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 -- 172.21.15.42:0/3709236075 shutdown_connections 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fabb40a2c00 0x7fabb40a50c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fabd0102070 0x7fabd01a1070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabd01016b0 0x7fabd019a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 --2- 172.21.15.42:0/3709236075 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fabd01004c0 0x7fabd0199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 -- 172.21.15.42:0/3709236075 >> 172.21.15.42:0/3709236075 conn(0x7fabd00fba30 msgr2=0x7fabd00fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 -- 172.21.15.42:0/3709236075 shutdown_connections 2024-04-25T00:20:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:20:48.360+0000 7fabd5600640 1 -- 172.21.15.42:0/3709236075 wait complete. 2024-04-25T00:20:49.205 INFO:tasks.workunit.client.0.smithi042.stdout: 1 58411 8.52 MB/sec warmup 39 sec latency 29.955 ms 2024-04-25T00:20:49.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:49 smithi042 ceph-mon[28130]: pgmap v449: 97 pgs: 97 active+clean; 32 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 267 op/s 2024-04-25T00:20:49.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:49 smithi136 ceph-mon[32923]: pgmap v449: 97 pgs: 97 active+clean; 32 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 267 op/s 2024-04-25T00:20:49.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:49 smithi106 ceph-mon[32510]: pgmap v449: 97 pgs: 97 active+clean; 32 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 267 op/s 2024-04-25T00:20:50.205 INFO:tasks.workunit.client.0.smithi042.stdout: 1 60128 8.53 MB/sec warmup 40 sec latency 21.094 ms 2024-04-25T00:20:51.205 INFO:tasks.workunit.client.0.smithi042.stdout: 1 60883 8.44 MB/sec warmup 41 sec latency 16.913 ms 2024-04-25T00:20:51.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:51 smithi042 ceph-mon[28130]: pgmap v450: 97 pgs: 97 active+clean; 33 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 266 op/s 2024-04-25T00:20:51.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:51 smithi136 ceph-mon[32923]: pgmap v450: 97 pgs: 97 active+clean; 33 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 266 op/s 2024-04-25T00:20:51.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:51 smithi106 ceph-mon[32510]: pgmap v450: 97 pgs: 97 active+clean; 33 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.8 MiB/s wr, 266 op/s 2024-04-25T00:20:52.205 INFO:tasks.workunit.client.0.smithi042.stdout: 1 61978 8.36 MB/sec warmup 42 sec latency 25.325 ms 2024-04-25T00:20:53.205 INFO:tasks.workunit.client.0.smithi042.stdout: 1 63594 8.38 MB/sec warmup 43 sec latency 32.714 ms 2024-04-25T00:20:53.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:53 smithi042 ceph-mon[28130]: pgmap v451: 97 pgs: 97 active+clean; 30 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 284 op/s 2024-04-25T00:20:53.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:53 smithi136 ceph-mon[32923]: pgmap v451: 97 pgs: 97 active+clean; 30 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 284 op/s 2024-04-25T00:20:53.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:53 smithi106 ceph-mon[32510]: pgmap v451: 97 pgs: 97 active+clean; 30 MiB data, 524 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 284 op/s 2024-04-25T00:20:54.205 INFO:tasks.workunit.client.0.smithi042.stdout: 1 64353 8.30 MB/sec warmup 44 sec latency 17.569 ms 2024-04-25T00:20:55.206 INFO:tasks.workunit.client.0.smithi042.stdout: 1 65399 8.23 MB/sec warmup 45 sec latency 24.883 ms 2024-04-25T00:20:55.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:55 smithi042 ceph-mon[28130]: pgmap v452: 97 pgs: 97 active+clean; 31 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 239 op/s 2024-04-25T00:20:55.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:55 smithi136 ceph-mon[32923]: pgmap v452: 97 pgs: 97 active+clean; 31 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 239 op/s 2024-04-25T00:20:55.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:55 smithi106 ceph-mon[32510]: pgmap v452: 97 pgs: 97 active+clean; 31 MiB data, 528 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.1 MiB/s wr, 239 op/s 2024-04-25T00:20:56.205 INFO:tasks.workunit.client.0.smithi042.stdout: 1 67080 8.23 MB/sec warmup 46 sec latency 21.114 ms 2024-04-25T00:20:57.206 INFO:tasks.workunit.client.0.smithi042.stdout: 1 67903 8.18 MB/sec warmup 47 sec latency 15.104 ms 2024-04-25T00:20:57.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:57 smithi042 ceph-mon[28130]: pgmap v453: 97 pgs: 97 active+clean; 32 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-04-25T00:20:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:57 smithi136 ceph-mon[32923]: pgmap v453: 97 pgs: 97 active+clean; 32 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-04-25T00:20:57.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:57 smithi106 ceph-mon[32510]: pgmap v453: 97 pgs: 97 active+clean; 32 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 261 op/s 2024-04-25T00:20:58.206 INFO:tasks.workunit.client.0.smithi042.stdout: 1 69124 8.12 MB/sec warmup 48 sec latency 28.875 ms 2024-04-25T00:20:58.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:58.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:58.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:20:59.206 INFO:tasks.workunit.client.0.smithi042.stdout: 1 71095 8.21 MB/sec warmup 49 sec latency 16.370 ms 2024-04-25T00:20:59.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:20:59 smithi042 ceph-mon[28130]: pgmap v454: 97 pgs: 97 active+clean; 34 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-04-25T00:20:59.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:20:59 smithi136 ceph-mon[32923]: pgmap v454: 97 pgs: 97 active+clean; 34 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-04-25T00:20:59.590 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:20:59 smithi106 ceph-mon[32510]: pgmap v454: 97 pgs: 97 active+clean; 34 MiB data, 525 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.0 MiB/s wr, 190 op/s 2024-04-25T00:21:00.206 INFO:tasks.workunit.client.0.smithi042.stdout: 1 71673 8.07 MB/sec warmup 50 sec latency 27.846 ms 2024-04-25T00:21:00.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:00.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:00.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:21:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:00.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:21:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:21:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:21:00.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:21:01.206 INFO:tasks.workunit.client.0.smithi042.stdout: 1 72739 8.01 MB/sec warmup 51 sec latency 35.485 ms 2024-04-25T00:21:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.509+0000 7ff198200640 1 -- 172.21.15.42:0/1887011009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1901018b0 msgr2=0x7ff190101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.509+0000 7ff198200640 1 --2- 172.21.15.42:0/1887011009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1901018b0 0x7ff190101d30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009980 tx=0x7ff17c02e820 comp rx=0 tx=0).stop 2024-04-25T00:21:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.510+0000 7ff198200640 1 -- 172.21.15.42:0/1887011009 shutdown_connections 2024-04-25T00:21:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.510+0000 7ff198200640 1 --2- 172.21.15.42:0/1887011009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff190102270 0x7ff190108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.510+0000 7ff198200640 1 --2- 172.21.15.42:0/1887011009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1901018b0 0x7ff190101d30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.510+0000 7ff198200640 1 --2- 172.21.15.42:0/1887011009 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1901006c0 0x7ff190100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.510+0000 7ff198200640 1 -- 172.21.15.42:0/1887011009 >> 172.21.15.42:0/1887011009 conn(0x7ff1900fbc30 msgr2=0x7ff1900fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.510+0000 7ff198200640 1 -- 172.21.15.42:0/1887011009 shutdown_connections 2024-04-25T00:21:01.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.510+0000 7ff198200640 1 -- 172.21.15.42:0/1887011009 wait complete. 2024-04-25T00:21:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff198200640 1 Processor -- start 2024-04-25T00:21:01.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff198200640 1 -- start start 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff198200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1901006c0 0x7ff1901957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff198200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1901018b0 0x7ff190195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff198200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff190102270 0x7ff19019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff198200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff19010b290 con 0x7ff1901018b0 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff198200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff19010b110 con 0x7ff1901006c0 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff198200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff19010b410 con 0x7ff190102270 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff195800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1901006c0 0x7ff1901957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff195800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1901006c0 0x7ff1901957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53698/0 (socket says 172.21.15.42:53698) 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.511+0000 7ff195800640 1 -- 172.21.15.42:0/2521583414 learned_addr learned my addr 172.21.15.42:0/2521583414 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.512+0000 7ff194e00640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1901018b0 0x7ff190195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.512+0000 7ff196200640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff190102270 0x7ff19019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.512+0000 7ff195800640 1 -- 172.21.15.42:0/2521583414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff190102270 msgr2=0x7ff19019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.512+0000 7ff195800640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff190102270 0x7ff19019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.512+0000 7ff195800640 1 -- 172.21.15.42:0/2521583414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1901018b0 msgr2=0x7ff190195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.512+0000 7ff195800640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1901018b0 0x7ff190195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.512+0000 7ff195800640 1 -- 172.21.15.42:0/2521583414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19019d3f0 con 0x7ff1901006c0 2024-04-25T00:21:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.512+0000 7ff195800640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1901006c0 0x7ff1901957d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff18000e9c0 tx=0x7ff18000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.513+0000 7ff18e000640 1 -- 172.21.15.42:0/2521583414 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18000cd80 con 0x7ff1901006c0 2024-04-25T00:21:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.513+0000 7ff18e000640 1 -- 172.21.15.42:0/2521583414 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1800035f0 con 0x7ff1901006c0 2024-04-25T00:21:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.513+0000 7ff18e000640 1 -- 172.21.15.42:0/2521583414 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff180010690 con 0x7ff1901006c0 2024-04-25T00:21:01.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.513+0000 7ff198200640 1 -- 172.21.15.42:0/2521583414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19019d6e0 con 0x7ff1901006c0 2024-04-25T00:21:01.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.513+0000 7ff198200640 1 -- 172.21.15.42:0/2521583414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19019dc50 con 0x7ff1901006c0 2024-04-25T00:21:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.516+0000 7ff19b042740 1 Processor -- start 2024-04-25T00:21:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff18e000640 1 -- 172.21.15.42:0/2521583414 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1800037a0 con 0x7ff1901006c0 2024-04-25T00:21:01.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff19b042740 1 -- start start 2024-04-25T00:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff19b042740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcf10c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff19b042740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf1690 0x55b8dfcf6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff19b042740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf6800 0x55b8dfcf8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff19b042740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b8dfc53ab0 con 0x55b8dfcf1690 2024-04-25T00:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff19b042740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8dfbfc640 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff19b042740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8dfc4fa70 con 0x55b8dfcf6800 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff196200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcf10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff195800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf1690 0x55b8dfcf6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff196200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcf10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53710/0 (socket says 172.21.15.42:53710) 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.517+0000 7ff196200640 1 -- 172.21.15.42:0/1279979377 learned_addr learned my addr 172.21.15.42:0/1279979377 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.518+0000 7ff194e00640 1 --2- 172.21.15.42:0/1279979377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf6800 0x55b8dfcf8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.518+0000 7ff196200640 1 -- 172.21.15.42:0/1279979377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf6800 msgr2=0x55b8dfcf8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.518+0000 7ff196200640 1 --2- 172.21.15.42:0/1279979377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf6800 0x55b8dfcf8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.518+0000 7ff196200640 1 -- 172.21.15.42:0/1279979377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf1690 msgr2=0x55b8dfcf6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.518+0000 7ff196200640 1 --2- 172.21.15.42:0/1279979377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf1690 0x55b8dfcf6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.518+0000 7ff196200640 1 -- 172.21.15.42:0/1279979377 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8df968210 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.518+0000 7ff194e00640 1 --2- 172.21.15.42:0/1279979377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf6800 0x55b8dfcf8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.518+0000 7ff196200640 1 --2- 172.21.15.42:0/1279979377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcf10c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff18800bf50 tx=0x7ff18800a740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f00d67c63ffe5b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.518+0000 7ff18e000640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff1740a2cb0 0x7ff1740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.518+0000 7ff186a00640 1 -- 172.21.15.42:0/1279979377 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188014760 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.519+0000 7ff186a00640 1 -- 172.21.15.42:0/1279979377 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff18801a070 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.519+0000 7ff186a00640 1 -- 172.21.15.42:0/1279979377 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188015d20 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.519+0000 7ff18e000640 1 -- 172.21.15.42:0/2521583414 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff1800cf420 con 0x7ff1901006c0 2024-04-25T00:21:01.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.519+0000 7ff195800640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff1740a2cb0 0x7ff1740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.519+0000 7ff19b042740 1 -- 172.21.15.42:0/1279979377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 msgr2=0x55b8dfcf10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.519+0000 7ff19b042740 1 --2- 172.21.15.42:0/1279979377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcf10c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff18800bf50 tx=0x7ff18800a740 comp rx=0 tx=0).stop 2024-04-25T00:21:01.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.520+0000 7ff19b042740 1 -- 172.21.15.42:0/1279979377 shutdown_connections 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.520+0000 7ff19b042740 1 --2- 172.21.15.42:0/1279979377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf6800 0x55b8dfcf8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.520+0000 7ff19b042740 1 --2- 172.21.15.42:0/1279979377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf1690 0x55b8dfcf6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.520+0000 7ff19b042740 1 --2- 172.21.15.42:0/1279979377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcf10c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.520+0000 7ff19b042740 1 -- 172.21.15.42:0/1279979377 >> 172.21.15.42:0/1279979377 conn(0x55b8dfcec310 msgr2=0x55b8dfcec710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.520+0000 7ff19b042740 1 -- 172.21.15.42:0/1279979377 shutdown_connections 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.520+0000 7ff19b042740 1 -- 172.21.15.42:0/1279979377 wait complete. 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.520+0000 7ff195800640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff1740a2cb0 0x7ff1740a5170 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff1800cd2c0 tx=0x7ff1800d2f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.520+0000 7ff19b042740 1 -- start start 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.520+0000 7ff19b042740 1 Processor -- start 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff19b042740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcfaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff19b042740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf1690 0x55b8dfcfb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff19b042740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf6800 0x55b8dfd02470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff19b042740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b8dfcf97b0 con 0x55b8dfcf1690 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff19b042740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8dfc0f990 con 0x55b8dfcf6800 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff19b042740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8dfcf9930 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff19b042740 10 client.0 _ll_register_callbacks cb 0x55b8dfc11950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff196200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcfaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff194e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf1690 0x55b8dfcfb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff196200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcfaef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:34906/0 (socket says 172.21.15.42:34906) 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff196200640 1 -- 172.21.15.42:0/3877233591 learned_addr learned my addr 172.21.15.42:0/3877233591 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.521+0000 7ff196200640 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf6800 0x55b8dfd02470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.522+0000 7ff196200640 1 -- 172.21.15.42:0/3877233591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf6800 msgr2=0x55b8dfd02470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.522+0000 7ff196200640 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf6800 0x55b8dfd02470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.522+0000 7ff196200640 1 -- 172.21.15.42:0/3877233591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf1690 msgr2=0x55b8dfcfb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.522+0000 7ff196200640 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf1690 0x55b8dfcfb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.522+0000 7ff196200640 1 -- 172.21.15.42:0/3877233591 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8dfa34620 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.522+0000 7ff196200640 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcfaef0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff18800afd0 tx=0x7ff18800d040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb45b2304c582766 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.522+0000 7ff17b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.522+0000 7ff17b400640 1 -- 172.21.15.42:0/3877233591 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188024030 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.522+0000 7ff17b400640 1 -- 172.21.15.42:0/3877233591 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff188003b70 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.522+0000 7ff19b042740 1 -- 172.21.15.42:0/3877233591 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff18801a070 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff17b400640 1 -- 172.21.15.42:0/3877233591 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff188014770 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff19b042740 10 client.25363 fetch_fsmap learned FSMap version 25 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff19b042740 1 -- 172.21.15.42:0/3877233591 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b8dfd05450 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff17b400640 1 -- 172.21.15.42:0/3877233591 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff18801a070 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff17b400640 1 -- 172.21.15.42:0/3877233591 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff188020930 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff19b042740 10 client.25363 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff19b042740 10 client.25363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff19b042740 20 client.25363 populate_metadata read hostname 'smithi042' 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff19b042740 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8dfd06860 0x55b8dfd08c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff19b042740 4 client.25363 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:01.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.523+0000 7ff19b042740 1 -- 172.21.15.42:0/3877233591 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8dfc60ff0 con 0x55b8dfd06860 2024-04-25T00:21:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.524+0000 7ff194e00640 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8dfd06860 0x55b8dfd08c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.524+0000 7ff194e00640 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8dfd06860 0x55b8dfd08c50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85673f816559b2ab server_cookie=268cf7c0c787c9fc in_seq=0 out_seq=0 2024-04-25T00:21:01.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.524+0000 7ff17b400640 10 client.25363 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff17b400640 1 -- 172.21.15.42:0/3877233591 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8dfc60ff0 con 0x55b8dfd06860 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff17b400640 10 client.25363 handle_command_reply: tid=0 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff19b042740 1 client.25363 shutdown 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff19b042740 1 -- 172.21.15.42:0/3877233591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf0ca0 msgr2=0x55b8dfcfaef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff19b042740 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcfaef0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff18800afd0 tx=0x7ff18800d040 comp rx=0 tx=0).stop 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff19b042740 1 -- 172.21.15.42:0/3877233591 shutdown_connections 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff19b042740 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8dfd06860 0x55b8dfd08c50 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff19b042740 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf6800 0x55b8dfd02470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff19b042740 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf1690 0x55b8dfcfb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff19b042740 1 --2- 172.21.15.42:0/3877233591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcfaef0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff19b042740 1 -- 172.21.15.42:0/3877233591 >> 172.21.15.42:0/3877233591 conn(0x55b8dfcec310 msgr2=0x55b8dfceb480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff17b400640 0 client.25363 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.525+0000 7ff19b042740 1 -- 172.21.15.42:0/3877233591 shutdown_connections 2024-04-25T00:21:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.526+0000 7ff19b042740 1 -- 172.21.15.42:0/3877233591 wait complete. 2024-04-25T00:21:01.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.526+0000 7ff19b042740 20 client.25363 trim_cache size 0 max 16384 2024-04-25T00:21:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.544+0000 7ff19b042740 1 Processor -- start 2024-04-25T00:21:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.544+0000 7ff19b042740 1 -- start start 2024-04-25T00:21:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.544+0000 7ff19b042740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcef880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.544+0000 7ff19b042740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.544+0000 7ff19b042740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 0x55b8dfd0d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.544+0000 7ff19b042740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901a9290 con 0x55b8dfcf6800 2024-04-25T00:21:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.544+0000 7ff19b042740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901a6b60 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.544+0000 7ff19b042740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901a49b0 con 0x55b8dfcf1690 2024-04-25T00:21:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.544+0000 7ff196200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcef880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.544+0000 7ff194e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff196200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcef880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53722/0 (socket says 172.21.15.42:53722) 2024-04-25T00:21:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff196200640 1 -- 172.21.15.42:0/1472522707 learned_addr learned my addr 172.21.15.42:0/1472522707 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff196200640 1 --2- 172.21.15.42:0/1472522707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 0x55b8dfd0d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff196200640 1 -- 172.21.15.42:0/1472522707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 msgr2=0x55b8dfd02430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:01.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff196200640 1 --2- 172.21.15.42:0/1472522707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff196200640 1 -- 172.21.15.42:0/1472522707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 msgr2=0x55b8dfd0d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff196200640 1 --2- 172.21.15.42:0/1472522707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 0x55b8dfd0d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff196200640 1 -- 172.21.15.42:0/1472522707 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8dfd08fc0 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff194e00640 1 --2- 172.21.15.42:0/1472522707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff196200640 1 --2- 172.21.15.42:0/1472522707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcef880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff18800b480 tx=0x7ff18800b850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a32e9cc205da454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.545+0000 7ff186000640 1 -- 172.21.15.42:0/1472522707 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188024020 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.546+0000 7ff186000640 1 -- 172.21.15.42:0/1472522707 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff18801a040 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.546+0000 7ff186000640 1 -- 172.21.15.42:0/1472522707 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18801f870 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.546+0000 7ff19b042740 1 -- 172.21.15.42:0/1472522707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 msgr2=0x55b8dfcef880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.546+0000 7ff19b042740 1 --2- 172.21.15.42:0/1472522707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcef880 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff18800b480 tx=0x7ff18800b850 comp rx=0 tx=0).stop 2024-04-25T00:21:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.546+0000 7ff19b042740 1 -- 172.21.15.42:0/1472522707 shutdown_connections 2024-04-25T00:21:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.546+0000 7ff19b042740 1 --2- 172.21.15.42:0/1472522707 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 0x55b8dfd0d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.546+0000 7ff19b042740 1 --2- 172.21.15.42:0/1472522707 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.546+0000 7ff19b042740 1 --2- 172.21.15.42:0/1472522707 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfcef880 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.546+0000 7ff19b042740 1 -- 172.21.15.42:0/1472522707 >> 172.21.15.42:0/1472522707 conn(0x55b8dfcec310 msgr2=0x55b8dfcfb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.547+0000 7ff19b042740 1 -- 172.21.15.42:0/1472522707 shutdown_connections 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.547+0000 7ff19b042740 1 -- 172.21.15.42:0/1472522707 wait complete. 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.547+0000 7ff19b042740 1 -- start start 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.547+0000 7ff19b042740 1 Processor -- start 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff19b042740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfd01c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff19b042740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff19b042740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 0x55b8dfd002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff19b042740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901a5020 con 0x55b8dfcf6800 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff19b042740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901a6490 con 0x55b8dfcf0ca0 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff19b042740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1901a56f0 con 0x55b8dfcf1690 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff19b042740 10 client.0 _ll_register_callbacks cb 0x7ff1800d2dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff196200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfd01c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff196200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 0x55b8dfd002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff194e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff196200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 0x55b8dfd002b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60674/0 (socket says 172.21.15.42:60674) 2024-04-25T00:21:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff196200640 1 -- 172.21.15.42:0/1804259850 learned_addr learned my addr 172.21.15.42:0/1804259850 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.548+0000 7ff194e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:34932/0 (socket says 172.21.15.42:34932) 2024-04-25T00:21:01.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff194e00640 1 -- 172.21.15.42:0/1804259850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 msgr2=0x55b8dfd01c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff194e00640 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfd01c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff194e00640 1 -- 172.21.15.42:0/1804259850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 msgr2=0x55b8dfd002b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff194e00640 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 0x55b8dfd002b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff194e00640 1 -- 172.21.15.42:0/1804259850 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8dfd008f0 con 0x55b8dfcf1690 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff194e00640 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009230 tx=0x7ff17c0367b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7134518e7f120eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff17b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff17b400640 1 -- 172.21.15.42:0/1804259850 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c031040 con 0x55b8dfcf1690 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff17b400640 1 -- 172.21.15.42:0/1804259850 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff17c033040 con 0x55b8dfcf1690 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff19b042740 1 -- 172.21.15.42:0/1804259850 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff18801a040 con 0x55b8dfcf1690 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff17b400640 1 -- 172.21.15.42:0/1804259850 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff17c004050 con 0x55b8dfcf1690 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff19b042740 10 client.25369 fetch_fsmap learned FSMap version 25 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.549+0000 7ff19b042740 1 -- 172.21.15.42:0/1804259850 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b8dfd75f40 con 0x55b8dfcf1690 2024-04-25T00:21:01.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.550+0000 7ff17b400640 1 -- 172.21.15.42:0/1804259850 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff18801a040 con 0x55b8dfcf1690 2024-04-25T00:21:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.550+0000 7ff17b400640 1 -- 172.21.15.42:0/1804259850 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff17c0037b0 con 0x55b8dfcf1690 2024-04-25T00:21:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.550+0000 7ff19b042740 10 client.25369 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:21:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.550+0000 7ff19b042740 10 client.25369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.550+0000 7ff19b042740 20 client.25369 populate_metadata read hostname 'smithi042' 2024-04-25T00:21:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.550+0000 7ff19b042740 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8dfd76950 0x55b8dfd78d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.550+0000 7ff19b042740 4 client.25369 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:21:01.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.550+0000 7ff19b042740 1 -- 172.21.15.42:0/1804259850 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8dfc5b340 con 0x55b8dfd76950 2024-04-25T00:21:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.550+0000 7ff196200640 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8dfd76950 0x55b8dfd78d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.551+0000 7ff196200640 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8dfd76950 0x55b8dfd78d70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78fac6b34c2d0eba server_cookie=3f447f9bd71a7b4a in_seq=0 out_seq=0 2024-04-25T00:21:01.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.551+0000 7ff17b400640 10 client.25369 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:01.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:01 smithi106 ceph-mon[32510]: pgmap v455: 97 pgs: 97 active+clean; 35 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 726 KiB/s wr, 166 op/s 2024-04-25T00:21:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.598+0000 7ff17b400640 1 -- 172.21.15.42:0/1804259850 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b8dfc5b340 con 0x55b8dfd76950 2024-04-25T00:21:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.598+0000 7ff17b400640 10 client.25369 handle_command_reply: tid=0 2024-04-25T00:21:01.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.598+0000 7ff19b042740 1 client.25369 shutdown 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 1 -- 172.21.15.42:0/1804259850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 msgr2=0x55b8dfd02190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02190 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009230 tx=0x7ff17c0367b0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 1 -- 172.21.15.42:0/1804259850 shutdown_connections 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8dfd76950 0x55b8dfd78d70 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8dfcf6800 0x55b8dfd002b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff17b400640 0 client.25369 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8dfcf1690 0x55b8dfd02190 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 1 --2- 172.21.15.42:0/1804259850 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8dfcf0ca0 0x55b8dfd01c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 1 -- 172.21.15.42:0/1804259850 >> 172.21.15.42:0/1804259850 conn(0x55b8dfcec310 msgr2=0x55b8dfcf5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 1 -- 172.21.15.42:0/1804259850 shutdown_connections 2024-04-25T00:21:01.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 1 -- 172.21.15.42:0/1804259850 wait complete. 2024-04-25T00:21:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.599+0000 7ff19b042740 20 client.25369 trim_cache size 0 max 16384 2024-04-25T00:21:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:21:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:21:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:21:01.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:21:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 -- 172.21.15.42:0/2521583414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff1740a2cb0 msgr2=0x7ff1740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff1740a2cb0 0x7ff1740a5170 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff1800cd2c0 tx=0x7ff1800d2f00 comp rx=0 tx=0).stop 2024-04-25T00:21:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 -- 172.21.15.42:0/2521583414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1901006c0 msgr2=0x7ff1901957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1901006c0 0x7ff1901957d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff18000e9c0 tx=0x7ff18000ee80 comp rx=0 tx=0).stop 2024-04-25T00:21:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 -- 172.21.15.42:0/2521583414 shutdown_connections 2024-04-25T00:21:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff1740a2cb0 0x7ff1740a5170 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff190102270 0x7ff19019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1901018b0 0x7ff190195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 --2- 172.21.15.42:0/2521583414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff1901006c0 0x7ff1901957d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 -- 172.21.15.42:0/2521583414 >> 172.21.15.42:0/2521583414 conn(0x7ff1900fbc30 msgr2=0x7ff190104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 -- 172.21.15.42:0/2521583414 shutdown_connections 2024-04-25T00:21:01.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:01.600+0000 7ff198200640 1 -- 172.21.15.42:0/2521583414 wait complete. 2024-04-25T00:21:01.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:01 smithi042 ceph-mon[28130]: pgmap v455: 97 pgs: 97 active+clean; 35 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 726 KiB/s wr, 166 op/s 2024-04-25T00:21:01.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:01 smithi136 ceph-mon[32923]: pgmap v455: 97 pgs: 97 active+clean; 35 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 726 KiB/s wr, 166 op/s 2024-04-25T00:21:02.206 INFO:tasks.workunit.client.0.smithi042.stdout: 1 74648 8.10 MB/sec warmup 52 sec latency 22.748 ms 2024-04-25T00:21:03.206 INFO:tasks.workunit.client.0.smithi042.stdout: 1 75142 7.97 MB/sec warmup 53 sec latency 17.709 ms 2024-04-25T00:21:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:03 smithi106 ceph-mon[32510]: pgmap v456: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 787 KiB/s wr, 169 op/s 2024-04-25T00:21:03.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:03 smithi042 ceph-mon[28130]: pgmap v456: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 787 KiB/s wr, 169 op/s 2024-04-25T00:21:03.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:03 smithi136 ceph-mon[32923]: pgmap v456: 97 pgs: 97 active+clean; 36 MiB data, 542 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 787 KiB/s wr, 169 op/s 2024-04-25T00:21:04.206 INFO:tasks.workunit.client.0.smithi042.stdout: 1 76152 7.92 MB/sec warmup 54 sec latency 24.108 ms 2024-04-25T00:21:05.206 INFO:tasks.workunit.client.0.smithi042.stdout: 1 77811 7.94 MB/sec warmup 55 sec latency 26.672 ms 2024-04-25T00:21:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:05 smithi106 ceph-mon[32510]: pgmap v457: 97 pgs: 97 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 698 KiB/s wr, 142 op/s 2024-04-25T00:21:05.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:05 smithi042 ceph-mon[28130]: pgmap v457: 97 pgs: 97 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 698 KiB/s wr, 142 op/s 2024-04-25T00:21:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:05 smithi136 ceph-mon[32923]: pgmap v457: 97 pgs: 97 active+clean; 37 MiB data, 543 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 698 KiB/s wr, 142 op/s 2024-04-25T00:21:06.207 INFO:tasks.workunit.client.0.smithi042.stdout: 1 78521 7.89 MB/sec warmup 56 sec latency 22.132 ms 2024-04-25T00:21:07.207 INFO:tasks.workunit.client.0.smithi042.stdout: 1 79579 7.84 MB/sec warmup 57 sec latency 22.757 ms 2024-04-25T00:21:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:07 smithi106 ceph-mon[32510]: pgmap v458: 97 pgs: 97 active+clean; 34 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 812 KiB/s wr, 178 op/s 2024-04-25T00:21:07.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:07 smithi042 ceph-mon[28130]: pgmap v458: 97 pgs: 97 active+clean; 34 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 812 KiB/s wr, 178 op/s 2024-04-25T00:21:07.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:07 smithi136 ceph-mon[32923]: pgmap v458: 97 pgs: 97 active+clean; 34 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 812 KiB/s wr, 178 op/s 2024-04-25T00:21:08.207 INFO:tasks.workunit.client.0.smithi042.stdout: 1 81098 7.83 MB/sec warmup 58 sec latency 28.368 ms 2024-04-25T00:21:09.207 INFO:tasks.workunit.client.0.smithi042.stdout: 1 82043 7.81 MB/sec warmup 59 sec latency 17.611 ms 2024-04-25T00:21:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:09 smithi106 ceph-mon[32510]: pgmap v459: 97 pgs: 97 active+clean; 35 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 751 KiB/s wr, 166 op/s 2024-04-25T00:21:09.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:09 smithi042 ceph-mon[28130]: pgmap v459: 97 pgs: 97 active+clean; 35 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 751 KiB/s wr, 166 op/s 2024-04-25T00:21:09.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:09 smithi136 ceph-mon[32923]: pgmap v459: 97 pgs: 97 active+clean; 35 MiB data, 561 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 751 KiB/s wr, 166 op/s 2024-04-25T00:21:10.207 INFO:tasks.workunit.client.0.smithi042.stdout: 1 82881 7.72 MB/sec warmup 60 sec latency 29.642 ms 2024-04-25T00:21:11.207 INFO:tasks.workunit.client.0.smithi042.stdout: 1 84126 7.68 MB/sec warmup 61 sec latency 23.808 ms 2024-04-25T00:21:11.565 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:11 smithi136 ceph-mon[32923]: pgmap v460: 97 pgs: 97 active+clean; 36 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 666 KiB/s wr, 144 op/s 2024-04-25T00:21:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:11 smithi106 ceph-mon[32510]: pgmap v460: 97 pgs: 97 active+clean; 36 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 666 KiB/s wr, 144 op/s 2024-04-25T00:21:11.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:11 smithi042 ceph-mon[28130]: pgmap v460: 97 pgs: 97 active+clean; 36 MiB data, 558 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 666 KiB/s wr, 144 op/s 2024-04-25T00:21:12.207 INFO:tasks.workunit.client.0.smithi042.stdout: 1 85465 7.74 MB/sec warmup 62 sec latency 16.174 ms 2024-04-25T00:21:13.207 INFO:tasks.workunit.client.0.smithi042.stdout: 1 85984 7.63 MB/sec warmup 63 sec latency 55.507 ms 2024-04-25T00:21:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:13 smithi106 ceph-mon[32510]: pgmap v461: 97 pgs: 97 active+clean; 38 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 914 KiB/s wr, 184 op/s 2024-04-25T00:21:13.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:13 smithi042 ceph-mon[28130]: pgmap v461: 97 pgs: 97 active+clean; 38 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 914 KiB/s wr, 184 op/s 2024-04-25T00:21:13.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:13 smithi136 ceph-mon[32923]: pgmap v461: 97 pgs: 97 active+clean; 38 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 914 KiB/s wr, 184 op/s 2024-04-25T00:21:14.207 INFO:tasks.workunit.client.0.smithi042.stdout: 1 87224 7.59 MB/sec warmup 64 sec latency 19.825 ms 2024-04-25T00:21:14.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.754+0000 7f599de00640 1 -- 172.21.15.42:0/2811910791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5998100750 msgr2=0x7f5998100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.754+0000 7f599de00640 1 --2- 172.21.15.42:0/2811910791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5998100750 0x7f5998100b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5988009900 tx=0x7f598802a410 comp rx=0 tx=0).stop 2024-04-25T00:21:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.754+0000 7f599de00640 1 -- 172.21.15.42:0/2811910791 shutdown_connections 2024-04-25T00:21:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.754+0000 7f599de00640 1 --2- 172.21.15.42:0/2811910791 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998102300 0x7f5998108b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.754+0000 7f599de00640 1 --2- 172.21.15.42:0/2811910791 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998101940 0x7f5998101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.754+0000 7f599de00640 1 --2- 172.21.15.42:0/2811910791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5998100750 0x7f5998100b70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.754+0000 7f599de00640 1 -- 172.21.15.42:0/2811910791 >> 172.21.15.42:0/2811910791 conn(0x7f59980fbcc0 msgr2=0x7f59980fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.754+0000 7f599de00640 1 -- 172.21.15.42:0/2811910791 shutdown_connections 2024-04-25T00:21:14.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.754+0000 7f599de00640 1 -- 172.21.15.42:0/2811910791 wait complete. 2024-04-25T00:21:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.755+0000 7f599de00640 1 Processor -- start 2024-04-25T00:21:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.755+0000 7f599de00640 1 -- start start 2024-04-25T00:21:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.755+0000 7f599de00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998100750 0x7f5998199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.755+0000 7f599de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998101940 0x7f599819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.755+0000 7f599de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5998102300 0x7f59981a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.755+0000 7f599de00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f599810d4c0 con 0x7f5998100750 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.755+0000 7f599de00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f599810d340 con 0x7f5998102300 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.755+0000 7f599de00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f599810d640 con 0x7f5998101940 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.756+0000 7f5996000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998101940 0x7f599819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.756+0000 7f5997400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5998102300 0x7f59981a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.756+0000 7f5996a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998100750 0x7f5998199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.756+0000 7f5996000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998101940 0x7f599819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48598/0 (socket says 172.21.15.42:48598) 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.756+0000 7f5996000640 1 -- 172.21.15.42:0/4092217348 learned_addr learned my addr 172.21.15.42:0/4092217348 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.756+0000 7f5997400640 1 -- 172.21.15.42:0/4092217348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998101940 msgr2=0x7f599819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.756+0000 7f5997400640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998101940 0x7f599819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.756+0000 7f5997400640 1 -- 172.21.15.42:0/4092217348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998100750 msgr2=0x7f5998199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.756+0000 7f5997400640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998100750 0x7f5998199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.756+0000 7f5997400640 1 -- 172.21.15.42:0/4092217348 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59981a1850 con 0x7f5998102300 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.757+0000 7f5997400640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5998102300 0x7f59981a11b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5990009fd0 tx=0x7f599000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:14.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.757+0000 7f599ca00640 1 -- 172.21.15.42:0/4092217348 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599000ce20 con 0x7f5998102300 2024-04-25T00:21:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.757+0000 7f599ca00640 1 -- 172.21.15.42:0/4092217348 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5990005020 con 0x7f5998102300 2024-04-25T00:21:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.757+0000 7f599de00640 1 -- 172.21.15.42:0/4092217348 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59981a1b40 con 0x7f5998102300 2024-04-25T00:21:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.757+0000 7f599ca00640 1 -- 172.21.15.42:0/4092217348 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5990010720 con 0x7f5998102300 2024-04-25T00:21:14.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.757+0000 7f599de00640 1 -- 172.21.15.42:0/4092217348 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59981a2060 con 0x7f5998102300 2024-04-25T00:21:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.760+0000 7f59a0bdd740 1 Processor -- start 2024-04-25T00:21:14.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.760+0000 7f599ca00640 1 -- 172.21.15.42:0/4092217348 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5990005540 con 0x7f5998102300 2024-04-25T00:21:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f59a0bdd740 1 -- start start 2024-04-25T00:21:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 0x5619374320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937432690 0x561937437190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 0x561937439c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561937394ab0 con 0x561937437800 2024-04-25T00:21:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56193733d640 con 0x561937432690 2024-04-25T00:21:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561937390a70 con 0x561937431ca0 2024-04-25T00:21:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f5996a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937432690 0x561937437190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f5996000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 0x5619374320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f5996a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937432690 0x561937437190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45774/0 (socket says 172.21.15.42:45774) 2024-04-25T00:21:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f5996a00640 1 -- 172.21.15.42:0/2667310780 learned_addr learned my addr 172.21.15.42:0/2667310780 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:14.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.761+0000 7f5996000640 1 --2- 172.21.15.42:0/2667310780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 0x561937439c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.762+0000 7f5996a00640 1 -- 172.21.15.42:0/2667310780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 msgr2=0x5619374320c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.762+0000 7f5996a00640 1 --2- 172.21.15.42:0/2667310780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 0x5619374320c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.762+0000 7f5996a00640 1 -- 172.21.15.42:0/2667310780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 msgr2=0x561937439c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.762+0000 7f5996a00640 1 --2- 172.21.15.42:0/2667310780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 0x561937439c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.762+0000 7f5996a00640 1 -- 172.21.15.42:0/2667310780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619370a9210 con 0x561937432690 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.762+0000 7f599ca00640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f597c0a2cb0 0x7f597c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.762+0000 7f5996a00640 1 --2- 172.21.15.42:0/2667310780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937432690 0x561937437190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5988031440 tx=0x7f5988004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d68fec1aab6f7d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.762+0000 7f599ca00640 1 -- 172.21.15.42:0/4092217348 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f59900cf780 con 0x7f5998102300 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.762+0000 7f5996000640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f597c0a2cb0 0x7f597c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.762+0000 7f5983e00640 1 -- 172.21.15.42:0/2667310780 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5988035040 con 0x561937432690 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f5983e00640 1 -- 172.21.15.42:0/2667310780 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5988004430 con 0x561937432690 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f5983e00640 1 -- 172.21.15.42:0/2667310780 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59880316b0 con 0x561937432690 2024-04-25T00:21:14.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2667310780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937432690 msgr2=0x561937437190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/2667310780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937432690 0x561937437190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5988031440 tx=0x7f5988004040 comp rx=0 tx=0).stop 2024-04-25T00:21:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f5996000640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f597c0a2cb0 0x7f597c0a5170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x561937432ef0 tx=0x7f5984005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2667310780 shutdown_connections 2024-04-25T00:21:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/2667310780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 0x561937439c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/2667310780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937432690 0x561937437190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/2667310780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 0x5619374320c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2667310780 >> 172.21.15.42:0/2667310780 conn(0x56193742d310 msgr2=0x56193742d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.763+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2667310780 shutdown_connections 2024-04-25T00:21:14.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.764+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2667310780 wait complete. 2024-04-25T00:21:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.764+0000 7f59a0bdd740 1 -- start start 2024-04-25T00:21:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.764+0000 7f59a0bdd740 1 Processor -- start 2024-04-25T00:21:14.768 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937431ca0 0x56193743c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x56193743c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619374416c0 0x561937443ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56193743a750 con 0x561937431ca0 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561937350990 con 0x561937437800 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56193743a8d0 con 0x5619374416c0 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f59a0bdd740 10 client.0 _ll_register_callbacks cb 0x561937352950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f5996000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x56193743c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f5996a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937431ca0 0x56193743c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f5996000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x56193743c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45782/0 (socket says 172.21.15.42:45782) 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f5996000640 1 -- 172.21.15.42:0/4256305201 learned_addr learned my addr 172.21.15.42:0/4256305201 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.765+0000 7f5997400640 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619374416c0 0x561937443ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.766+0000 7f5996000640 1 -- 172.21.15.42:0/4256305201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619374416c0 msgr2=0x561937443ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.766+0000 7f5996000640 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619374416c0 0x561937443ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.766+0000 7f5996000640 1 -- 172.21.15.42:0/4256305201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937431ca0 msgr2=0x56193743c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.766+0000 7f5996000640 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937431ca0 0x56193743c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.766+0000 7f5996000640 1 -- 172.21.15.42:0/4256305201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561937175620 con 0x561937437800 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.766+0000 7f5996000640 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x56193743c690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5984005bc0 tx=0x7f5984004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4f4cf1514be61e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.766+0000 7f5980c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.766+0000 7f5980c00640 1 -- 172.21.15.42:0/4256305201 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984030060 con 0x561937437800 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.766+0000 7f5980c00640 1 -- 172.21.15.42:0/4256305201 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f598401a050 con 0x561937437800 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.766+0000 7f59a0bdd740 1 -- 172.21.15.42:0/4256305201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5988004430 con 0x561937437800 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f5980c00640 1 -- 172.21.15.42:0/4256305201 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5984011e20 con 0x561937437800 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f59a0bdd740 10 client.25379 fetch_fsmap learned FSMap version 25 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f59a0bdd740 1 -- 172.21.15.42:0/4256305201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561937446b50 con 0x561937437800 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f5980c00640 1 -- 172.21.15.42:0/4256305201 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5988004430 con 0x561937437800 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f5980c00640 1 -- 172.21.15.42:0/4256305201 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5984020aa0 con 0x561937437800 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f59a0bdd740 10 client.25379 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f59a0bdd740 10 client.25379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:14.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f59a0bdd740 20 client.25379 populate_metadata read hostname 'smithi042' 2024-04-25T00:21:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561937448230 0x56193744a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f59a0bdd740 4 client.25379 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:14.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.767+0000 7f59a0bdd740 1 -- 172.21.15.42:0/4256305201 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619373a1ff0 con 0x561937448230 2024-04-25T00:21:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.768+0000 7f5996a00640 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561937448230 0x56193744a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.768+0000 7f5996a00640 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561937448230 0x56193744a6d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c6208a11ec1e46c server_cookie=afd6d12341206052 in_seq=0 out_seq=0 2024-04-25T00:21:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.768+0000 7f5980c00640 10 client.25379 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.769+0000 7f5980c00640 1 -- 172.21.15.42:0/4256305201 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5619373a1ff0 con 0x561937448230 2024-04-25T00:21:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.769+0000 7f5980c00640 10 client.25379 handle_command_reply: tid=0 2024-04-25T00:21:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.769+0000 7f59a0bdd740 1 client.25379 shutdown 2024-04-25T00:21:14.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.769+0000 7f59a0bdd740 1 -- 172.21.15.42:0/4256305201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 msgr2=0x56193743c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.769+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x56193743c690 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5984005bc0 tx=0x7f5984004730 comp rx=0 tx=0).stop 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.770+0000 7f59a0bdd740 1 -- 172.21.15.42:0/4256305201 shutdown_connections 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.770+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561937448230 0x56193744a6d0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.770+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619374416c0 0x561937443ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.770+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x56193743c690 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.770+0000 7f5980c00640 0 client.25379 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.770+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/4256305201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937431ca0 0x56193743c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.770+0000 7f59a0bdd740 1 -- 172.21.15.42:0/4256305201 >> 172.21.15.42:0/4256305201 conn(0x56193742d310 msgr2=0x56193742c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.770+0000 7f59a0bdd740 1 -- 172.21.15.42:0/4256305201 shutdown_connections 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.770+0000 7f59a0bdd740 1 -- 172.21.15.42:0/4256305201 wait complete. 2024-04-25T00:21:14.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.770+0000 7f59a0bdd740 20 client.25379 trim_cache size 0 max 16384 2024-04-25T00:21:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.788+0000 7f59a0bdd740 1 Processor -- start 2024-04-25T00:21:14.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.788+0000 7f59a0bdd740 1 -- start start 2024-04-25T00:21:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.788+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 0x561937433b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.788+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 0x5619374340d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.788+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619374416c0 0x56193742bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.788+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f59981ad500 con 0x561937437800 2024-04-25T00:21:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f59981aadd0 con 0x5619374416c0 2024-04-25T00:21:14.790 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f59981a8c20 con 0x561937431ca0 2024-04-25T00:21:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5996000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 0x5619374340d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5997400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619374416c0 0x56193742bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5996a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 0x561937433b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5996000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 0x5619374340d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37448/0 (socket says 172.21.15.42:37448) 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5996000640 1 -- 172.21.15.42:0/3066436640 learned_addr learned my addr 172.21.15.42:0/3066436640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5997400640 1 -- 172.21.15.42:0/3066436640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 msgr2=0x561937433b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5997400640 1 --2- 172.21.15.42:0/3066436640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 0x561937433b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5997400640 1 -- 172.21.15.42:0/3066436640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 msgr2=0x5619374340d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5997400640 1 --2- 172.21.15.42:0/3066436640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 0x5619374340d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5997400640 1 -- 172.21.15.42:0/3066436640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561937437550 con 0x5619374416c0 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5996a00640 1 --2- 172.21.15.42:0/3066436640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 0x561937433b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.789+0000 7f5997400640 1 --2- 172.21.15.42:0/3066436640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619374416c0 0x56193742bbf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f599009bcb0 tx=0x7f5990005940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd4161c529f122e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.790+0000 7f5983400640 1 -- 172.21.15.42:0/3066436640 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900b4070 con 0x5619374416c0 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.790+0000 7f5983400640 1 -- 172.21.15.42:0/3066436640 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59900a0360 con 0x5619374416c0 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.790+0000 7f5983400640 1 -- 172.21.15.42:0/3066436640 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59900a49d0 con 0x5619374416c0 2024-04-25T00:21:14.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.790+0000 7f59a0bdd740 1 -- 172.21.15.42:0/3066436640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619374416c0 msgr2=0x56193742bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.790+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/3066436640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619374416c0 0x56193742bbf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f599009bcb0 tx=0x7f5990005940 comp rx=0 tx=0).stop 2024-04-25T00:21:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.791+0000 7f59a0bdd740 1 -- 172.21.15.42:0/3066436640 shutdown_connections 2024-04-25T00:21:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.791+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/3066436640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619374416c0 0x56193742bbf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.791+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/3066436640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937437800 0x5619374340d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.791+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/3066436640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561937431ca0 0x561937433b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.791+0000 7f59a0bdd740 1 -- 172.21.15.42:0/3066436640 >> 172.21.15.42:0/3066436640 conn(0x56193742d310 msgr2=0x561937427fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.791+0000 7f59a0bdd740 1 -- 172.21.15.42:0/3066436640 shutdown_connections 2024-04-25T00:21:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.791+0000 7f59a0bdd740 1 -- 172.21.15.42:0/3066436640 wait complete. 2024-04-25T00:21:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.791+0000 7f59a0bdd740 1 -- start start 2024-04-25T00:21:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.791+0000 7f59a0bdd740 1 Processor -- start 2024-04-25T00:21:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937431ca0 0x561937459740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x561937459c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f59a0bdd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619374416c0 0x56193745ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f59981a9290 con 0x561937431ca0 2024-04-25T00:21:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f59981aa700 con 0x561937437800 2024-04-25T00:21:14.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f59a0bdd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f59981a9960 con 0x5619374416c0 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f59a0bdd740 10 client.0 _ll_register_callbacks cb 0x7f59840115f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f5996000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x561937459c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f5996000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x561937459c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45808/0 (socket says 172.21.15.42:45808) 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f5996000640 1 -- 172.21.15.42:0/2771523615 learned_addr learned my addr 172.21.15.42:0/2771523615 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f5997400640 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619374416c0 0x56193745ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.792+0000 7f5996a00640 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937431ca0 0x561937459740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f5996000640 1 -- 172.21.15.42:0/2771523615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619374416c0 msgr2=0x56193745ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f5996000640 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619374416c0 0x56193745ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f5996000640 1 -- 172.21.15.42:0/2771523615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937431ca0 msgr2=0x561937459740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f5996000640 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937431ca0 0x561937459740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f5996000640 1 -- 172.21.15.42:0/2771523615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56193745b280 con 0x561937437800 2024-04-25T00:21:14.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f5996000640 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x561937459c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5984020e10 tx=0x7f5984022f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bf11425b100892 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f5980c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f5980c00640 1 -- 172.21.15.42:0/2771523615 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598403b070 con 0x561937437800 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f5980c00640 1 -- 172.21.15.42:0/2771523615 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f598401a040 con 0x561937437800 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2771523615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59900a0360 con 0x561937437800 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f5980c00640 1 -- 172.21.15.42:0/2771523615 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5984030040 con 0x561937437800 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f59a0bdd740 10 client.25385 fetch_fsmap learned FSMap version 25 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.793+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2771523615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619374ae8c0 con 0x561937437800 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.794+0000 7f5980c00640 1 -- 172.21.15.42:0/2771523615 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59900a0360 con 0x561937437800 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.794+0000 7f5980c00640 1 -- 172.21.15.42:0/2771523615 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f59840365f0 con 0x561937437800 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.794+0000 7f59a0bdd740 10 client.25385 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.794+0000 7f59a0bdd740 10 client.25385 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.794+0000 7f59a0bdd740 20 client.25385 populate_metadata read hostname 'smithi042' 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.794+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619374af290 0x5619374b16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:14.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.794+0000 7f59a0bdd740 4 client.25385 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:21:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.794+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2771523615 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56193739c340 con 0x5619374af290 2024-04-25T00:21:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.794+0000 7f5996a00640 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619374af290 0x5619374b16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.795+0000 7f5996a00640 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619374af290 0x5619374b16b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba668502d7b5c842 server_cookie=b9b7c3e638605fd2 in_seq=0 out_seq=0 2024-04-25T00:21:14.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.795+0000 7f5980c00640 10 client.25385 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.845+0000 7f5980c00640 1 -- 172.21.15.42:0/2771523615 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56193739c340 con 0x5619374af290 2024-04-25T00:21:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.845+0000 7f5980c00640 10 client.25385 handle_command_reply: tid=0 2024-04-25T00:21:14.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.845+0000 7f59a0bdd740 1 client.25385 shutdown 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2771523615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 msgr2=0x561937459c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x561937459c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f5984020e10 tx=0x7f5984022f90 comp rx=0 tx=0).stop 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2771523615 shutdown_connections 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619374af290 0x5619374b16b0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619374416c0 0x56193745ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561937437800 0x561937459c80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 1 --2- 172.21.15.42:0/2771523615 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561937431ca0 0x561937459740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2771523615 >> 172.21.15.42:0/2771523615 conn(0x56193742d310 msgr2=0x56193743c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f5980c00640 0 client.25385 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2771523615 shutdown_connections 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 1 -- 172.21.15.42:0/2771523615 wait complete. 2024-04-25T00:21:14.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.846+0000 7f59a0bdd740 20 client.25385 trim_cache size 0 max 16384 2024-04-25T00:21:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:21:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:21:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:21:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:21:14.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 -- 172.21.15.42:0/4092217348 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f597c0a2cb0 msgr2=0x7f597c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f597c0a2cb0 0x7f597c0a5170 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x561937432ef0 tx=0x7f5984005f90 comp rx=0 tx=0).stop 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 -- 172.21.15.42:0/4092217348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5998102300 msgr2=0x7f59981a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5998102300 0x7f59981a11b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5990009fd0 tx=0x7f599000ef10 comp rx=0 tx=0).stop 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 -- 172.21.15.42:0/4092217348 shutdown_connections 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f597c0a2cb0 0x7f597c0a5170 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5998102300 0x7f59981a11b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5998101940 0x7f599819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 --2- 172.21.15.42:0/4092217348 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5998100750 0x7f5998199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 -- 172.21.15.42:0/4092217348 >> 172.21.15.42:0/4092217348 conn(0x7f59980fbcc0 msgr2=0x7f5998109550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 -- 172.21.15.42:0/4092217348 shutdown_connections 2024-04-25T00:21:14.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:14.847+0000 7f599de00640 1 -- 172.21.15.42:0/4092217348 wait complete. 2024-04-25T00:21:15.208 INFO:tasks.workunit.client.0.smithi042.stdout: 1 88939 7.67 MB/sec warmup 65 sec latency 30.495 ms 2024-04-25T00:21:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:15 smithi106 ceph-mon[32510]: pgmap v462: 97 pgs: 97 active+clean; 38 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 717 KiB/s wr, 145 op/s 2024-04-25T00:21:15.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:15 smithi042 ceph-mon[28130]: pgmap v462: 97 pgs: 97 active+clean; 38 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 717 KiB/s wr, 145 op/s 2024-04-25T00:21:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:15 smithi136 ceph-mon[32923]: pgmap v462: 97 pgs: 97 active+clean; 38 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 717 KiB/s wr, 145 op/s 2024-04-25T00:21:16.208 INFO:tasks.workunit.client.0.smithi042.stdout: 1 89404 7.56 MB/sec warmup 66 sec latency 17.727 ms 2024-04-25T00:21:17.208 INFO:tasks.workunit.client.0.smithi042.stdout: 1 90412 7.53 MB/sec warmup 67 sec latency 26.552 ms 2024-04-25T00:21:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:17 smithi106 ceph-mon[32510]: pgmap v463: 97 pgs: 97 active+clean; 39 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-04-25T00:21:17.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:17 smithi042 ceph-mon[28130]: pgmap v463: 97 pgs: 97 active+clean; 39 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-04-25T00:21:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:17 smithi136 ceph-mon[32923]: pgmap v463: 97 pgs: 97 active+clean; 39 MiB data, 585 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.1 MiB/s wr, 198 op/s 2024-04-25T00:21:18.208 INFO:tasks.workunit.client.0.smithi042.stdout: 1 92213 7.59 MB/sec warmup 68 sec latency 35.139 ms 2024-04-25T00:21:19.208 INFO:tasks.workunit.client.0.smithi042.stdout: 1 92721 7.51 MB/sec warmup 69 sec latency 22.962 ms 2024-04-25T00:21:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:19 smithi106 ceph-mon[32510]: pgmap v464: 97 pgs: 97 active+clean; 40 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2024-04-25T00:21:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:19 smithi042 ceph-mon[28130]: pgmap v464: 97 pgs: 97 active+clean; 40 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2024-04-25T00:21:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:19 smithi136 ceph-mon[32923]: pgmap v464: 97 pgs: 97 active+clean; 40 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 184 op/s 2024-04-25T00:21:20.208 INFO:tasks.workunit.client.0.smithi042.stdout: 1 93679 7.47 MB/sec warmup 70 sec latency 26.736 ms 2024-04-25T00:21:21.208 INFO:tasks.workunit.client.0.smithi042.stdout: 1 95020 7.45 MB/sec warmup 71 sec latency 20.803 ms 2024-04-25T00:21:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:21 smithi106 ceph-mon[32510]: pgmap v465: 97 pgs: 97 active+clean; 40 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-04-25T00:21:21.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:21 smithi042 ceph-mon[28130]: pgmap v465: 97 pgs: 97 active+clean; 40 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-04-25T00:21:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:21 smithi136 ceph-mon[32923]: pgmap v465: 97 pgs: 97 active+clean; 40 MiB data, 601 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-04-25T00:21:22.208 INFO:tasks.workunit.client.0.smithi042.stdout: 1 96109 7.46 MB/sec warmup 72 sec latency 19.299 ms 2024-04-25T00:21:23.209 INFO:tasks.workunit.client.0.smithi042.stdout: 1 96730 7.37 MB/sec warmup 73 sec latency 23.448 ms 2024-04-25T00:21:23.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:23 smithi042 ceph-mon[28130]: pgmap v466: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 236 op/s 2024-04-25T00:21:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:23 smithi136 ceph-mon[32923]: pgmap v466: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 236 op/s 2024-04-25T00:21:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:23 smithi106 ceph-mon[32510]: pgmap v466: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.8 MiB/s wr, 236 op/s 2024-04-25T00:21:24.209 INFO:tasks.workunit.client.0.smithi042.stdout: 1 98000 7.34 MB/sec warmup 74 sec latency 29.384 ms 2024-04-25T00:21:25.209 INFO:tasks.workunit.client.0.smithi042.stdout: 1 99573 7.41 MB/sec warmup 75 sec latency 17.155 ms 2024-04-25T00:21:25.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:25 smithi042 ceph-mon[28130]: pgmap v467: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-25T00:21:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:25 smithi136 ceph-mon[32923]: pgmap v467: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-25T00:21:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:25 smithi106 ceph-mon[32510]: pgmap v467: 97 pgs: 97 active+clean; 41 MiB data, 622 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-25T00:21:26.209 INFO:tasks.workunit.client.0.smithi042.stdout: 1 100038 7.32 MB/sec warmup 76 sec latency 22.372 ms 2024-04-25T00:21:27.209 INFO:tasks.workunit.client.0.smithi042.stdout: 1 101010 7.29 MB/sec warmup 77 sec latency 25.011 ms 2024-04-25T00:21:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:27 smithi042 ceph-mon[28130]: pgmap v468: 97 pgs: 97 active+clean; 41 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-04-25T00:21:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:27 smithi136 ceph-mon[32923]: pgmap v468: 97 pgs: 97 active+clean; 41 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-04-25T00:21:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:27 smithi106 ceph-mon[32510]: pgmap v468: 97 pgs: 97 active+clean; 41 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 200 op/s 2024-04-25T00:21:28.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.001+0000 7effbdc00640 1 -- 172.21.15.42:0/3960622315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effb8102280 msgr2=0x7effb8104670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.001+0000 7effbdc00640 1 --2- 172.21.15.42:0/3960622315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effb8102280 0x7effb8104670 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7effa8009be0 tx=0x7effa802a6c0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.001+0000 7effbdc00640 1 -- 172.21.15.42:0/3960622315 shutdown_connections 2024-04-25T00:21:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.001+0000 7effbdc00640 1 --2- 172.21.15.42:0/3960622315 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effb81074e0 0x7effb81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.001+0000 7effbdc00640 1 --2- 172.21.15.42:0/3960622315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb8104bb0 0x7effb8106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.001+0000 7effbdc00640 1 --2- 172.21.15.42:0/3960622315 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effb8102280 0x7effb8104670 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.001+0000 7effbdc00640 1 -- 172.21.15.42:0/3960622315 >> 172.21.15.42:0/3960622315 conn(0x7effb80fbc10 msgr2=0x7effb80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.001+0000 7effbdc00640 1 -- 172.21.15.42:0/3960622315 shutdown_connections 2024-04-25T00:21:28.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.001+0000 7effbdc00640 1 -- 172.21.15.42:0/3960622315 wait complete. 2024-04-25T00:21:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.002+0000 7effbdc00640 1 Processor -- start 2024-04-25T00:21:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.002+0000 7effbdc00640 1 -- start start 2024-04-25T00:21:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effbdc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effb8102280 0x7effb8199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effbdc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb8104bb0 0x7effb819a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effbdc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effb81074e0 0x7effb81a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effbdc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7effb810d410 con 0x7effb8102280 2024-04-25T00:21:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effbdc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7effb810d290 con 0x7effb81074e0 2024-04-25T00:21:28.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effbdc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7effb810d590 con 0x7effb8104bb0 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effb7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effb81074e0 0x7effb81a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effb6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb8104bb0 0x7effb819a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effb7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effb81074e0 0x7effb81a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44036/0 (socket says 172.21.15.42:44036) 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effb7400640 1 -- 172.21.15.42:0/229254282 learned_addr learned my addr 172.21.15.42:0/229254282 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.003+0000 7effb6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb8104bb0 0x7effb819a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38976/0 (socket says 172.21.15.42:38976) 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effb6a00640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effb8102280 0x7effb8199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effb7400640 1 -- 172.21.15.42:0/229254282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb8104bb0 msgr2=0x7effb819a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effb7400640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb8104bb0 0x7effb819a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effb7400640 1 -- 172.21.15.42:0/229254282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effb8102280 msgr2=0x7effb8199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effb7400640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effb8102280 0x7effb8199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effb7400640 1 -- 172.21.15.42:0/229254282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb81a1870 con 0x7effb81074e0 2024-04-25T00:21:28.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effb7400640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effb81074e0 0x7effb81a11d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7effb000e970 tx=0x7effb000ed70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effaf400640 1 -- 172.21.15.42:0/229254282 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb000ccc0 con 0x7effb81074e0 2024-04-25T00:21:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effaf400640 1 -- 172.21.15.42:0/229254282 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb000ce60 con 0x7effb81074e0 2024-04-25T00:21:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effaf400640 1 -- 172.21.15.42:0/229254282 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb0010490 con 0x7effb81074e0 2024-04-25T00:21:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.004+0000 7effbdc00640 1 -- 172.21.15.42:0/229254282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb81a1b60 con 0x7effb81074e0 2024-04-25T00:21:28.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.005+0000 7effbdc00640 1 -- 172.21.15.42:0/229254282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb81a2080 con 0x7effb81074e0 2024-04-25T00:21:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.008+0000 7effaf400640 1 -- 172.21.15.42:0/229254282 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effb0005020 con 0x7effb81074e0 2024-04-25T00:21:28.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.008+0000 7effc0b99740 1 Processor -- start 2024-04-25T00:21:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.008+0000 7effc0b99740 1 -- start start 2024-04-25T00:21:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.008+0000 7effc0b99740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 0x564d250d60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effc0b99740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d6690 0x564d250db190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effc0b99740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250db800 0x564d250ddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effc0b99740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564d25038ab0 con 0x564d250d5ca0 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effc0b99740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564d24fe1640 con 0x564d250d6690 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effc0b99740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564d25034a70 con 0x564d250db800 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d6690 0x564d250db190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d6690 0x564d250db190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44046/0 (socket says 172.21.15.42:44046) 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6a00640 1 -- 172.21.15.42:0/414411714 learned_addr learned my addr 172.21.15.42:0/414411714 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6000640 1 --2- 172.21.15.42:0/414411714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 0x564d250d60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6000640 1 --2- 172.21.15.42:0/414411714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250db800 0x564d250ddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6a00640 1 -- 172.21.15.42:0/414411714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250db800 msgr2=0x564d250ddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6a00640 1 --2- 172.21.15.42:0/414411714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250db800 0x564d250ddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6a00640 1 -- 172.21.15.42:0/414411714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 msgr2=0x564d250d60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6a00640 1 --2- 172.21.15.42:0/414411714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 0x564d250d60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6a00640 1 -- 172.21.15.42:0/414411714 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d24d4d210 con 0x564d250d6690 2024-04-25T00:21:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.009+0000 7effb6000640 1 --2- 172.21.15.42:0/414411714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 0x564d250d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.010+0000 7effb6a00640 1 --2- 172.21.15.42:0/414411714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d6690 0x564d250db190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7effa8000f80 tx=0x7effa80092f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2417c59218f97a86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.010+0000 7effaf400640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff9c0a2cb0 0x7eff9c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.010+0000 7effa7e00640 1 -- 172.21.15.42:0/414411714 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8040030 con 0x564d250d6690 2024-04-25T00:21:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.010+0000 7effa7e00640 1 -- 172.21.15.42:0/414411714 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8030cb0 con 0x564d250d6690 2024-04-25T00:21:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.010+0000 7effa7e00640 1 -- 172.21.15.42:0/414411714 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8036040 con 0x564d250d6690 2024-04-25T00:21:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.010+0000 7effaf400640 1 -- 172.21.15.42:0/229254282 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7effb00cf590 con 0x7effb81074e0 2024-04-25T00:21:28.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.010+0000 7effb6000640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff9c0a2cb0 0x7eff9c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effc0b99740 1 -- 172.21.15.42:0/414411714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d6690 msgr2=0x564d250db190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effc0b99740 1 --2- 172.21.15.42:0/414411714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d6690 0x564d250db190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7effa8000f80 tx=0x7effa80092f0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effb6000640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff9c0a2cb0 0x7eff9c0a5170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x564d250d6ef0 tx=0x7effa0009640 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effc0b99740 1 -- 172.21.15.42:0/414411714 shutdown_connections 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effc0b99740 1 --2- 172.21.15.42:0/414411714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250db800 0x564d250ddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effc0b99740 1 --2- 172.21.15.42:0/414411714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d6690 0x564d250db190 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effc0b99740 1 --2- 172.21.15.42:0/414411714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 0x564d250d60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effc0b99740 1 -- 172.21.15.42:0/414411714 >> 172.21.15.42:0/414411714 conn(0x564d250d1310 msgr2=0x564d250d1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effc0b99740 1 -- 172.21.15.42:0/414411714 shutdown_connections 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effc0b99740 1 -- 172.21.15.42:0/414411714 wait complete. 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.011+0000 7effc0b99740 1 -- start start 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.012+0000 7effc0b99740 1 Processor -- start 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.012+0000 7effc0b99740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 0x564d250dfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.012+0000 7effc0b99740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 0x564d250e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.012+0000 7effc0b99740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 0x564d250e7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.012+0000 7effc0b99740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564d250de750 con 0x564d250d5ca0 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.012+0000 7effc0b99740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564d24ff4990 con 0x564d250db800 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.012+0000 7effc0b99740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564d250de8d0 con 0x564d250d6690 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.012+0000 7effc0b99740 10 client.0 _ll_register_callbacks cb 0x564d24ff6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.012+0000 7effb7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 0x564d250e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.012+0000 7effb6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 0x564d250e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effb7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 0x564d250e7400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44048/0 (socket says 172.21.15.42:44048) 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effb7400640 1 -- 172.21.15.42:0/1885314691 learned_addr learned my addr 172.21.15.42:0/1885314691 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effb6a00640 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 0x564d250dfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effb7400640 1 -- 172.21.15.42:0/1885314691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 msgr2=0x564d250e0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effb7400640 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 0x564d250e0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effb7400640 1 -- 172.21.15.42:0/1885314691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 msgr2=0x564d250dfef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:28.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effb7400640 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 0x564d250dfef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effb7400640 1 -- 172.21.15.42:0/1885314691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d24e19620 con 0x564d250db800 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effb7400640 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 0x564d250e7400 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7effb009bd50 tx=0x7effb009c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6a2c095939af43e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effa4c00640 1 -- 172.21.15.42:0/1885314691 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb00a4a30 con 0x564d250db800 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effa4c00640 1 -- 172.21.15.42:0/1885314691 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7effb00a45a0 con 0x564d250db800 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.013+0000 7effc0b99740 1 -- 172.21.15.42:0/1885314691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa8030cb0 con 0x564d250db800 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effa4c00640 1 -- 172.21.15.42:0/1885314691 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7effb00a47a0 con 0x564d250db800 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effc0b99740 10 client.25397 fetch_fsmap learned FSMap version 25 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effc0b99740 1 -- 172.21.15.42:0/1885314691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564d250ea450 con 0x564d250db800 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effa4c00640 1 -- 172.21.15.42:0/1885314691 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa8030cb0 con 0x564d250db800 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effa4c00640 1 -- 172.21.15.42:0/1885314691 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7effb009ac80 con 0x564d250db800 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effc0b99740 10 client.25397 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effc0b99740 10 client.25397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effc0b99740 20 client.25397 populate_metadata read hostname 'smithi042' 2024-04-25T00:21:28.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effc0b99740 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564d250ebb30 0x564d250edff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effc0b99740 4 client.25397 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.014+0000 7effc0b99740 1 -- 172.21.15.42:0/1885314691 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d25045ff0 con 0x564d250ebb30 2024-04-25T00:21:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.015+0000 7effb6a00640 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564d250ebb30 0x564d250edff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.015+0000 7effa4c00640 10 client.25397 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:28.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.015+0000 7effb6a00640 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564d250ebb30 0x564d250edff0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4325128f6137d5ce server_cookie=593261421056452b in_seq=0 out_seq=0 2024-04-25T00:21:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.016+0000 7effa4c00640 1 -- 172.21.15.42:0/1885314691 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d25045ff0 con 0x564d250ebb30 2024-04-25T00:21:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.016+0000 7effa4c00640 10 client.25397 handle_command_reply: tid=0 2024-04-25T00:21:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.016+0000 7effc0b99740 1 client.25397 shutdown 2024-04-25T00:21:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.016+0000 7effc0b99740 1 -- 172.21.15.42:0/1885314691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 msgr2=0x564d250e7400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.016+0000 7effc0b99740 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 0x564d250e7400 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7effb009bd50 tx=0x7effb009c4c0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.017+0000 7effc0b99740 1 -- 172.21.15.42:0/1885314691 shutdown_connections 2024-04-25T00:21:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.017+0000 7effc0b99740 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564d250ebb30 0x564d250edff0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.017+0000 7effc0b99740 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 0x564d250e7400 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.017+0000 7effc0b99740 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 0x564d250e0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.017+0000 7effc0b99740 1 --2- 172.21.15.42:0/1885314691 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d5ca0 0x564d250dfef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.017+0000 7effa4c00640 0 client.25397 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.017+0000 7effc0b99740 1 -- 172.21.15.42:0/1885314691 >> 172.21.15.42:0/1885314691 conn(0x564d250d1310 msgr2=0x564d250d0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.017+0000 7effc0b99740 1 -- 172.21.15.42:0/1885314691 shutdown_connections 2024-04-25T00:21:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.017+0000 7effc0b99740 1 -- 172.21.15.42:0/1885314691 wait complete. 2024-04-25T00:21:28.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.017+0000 7effc0b99740 20 client.25397 trim_cache size 0 max 16384 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.037+0000 7effc0b99740 1 Processor -- start 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.037+0000 7effc0b99740 1 -- start start 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.037+0000 7effc0b99740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d5ca0 0x564d250e7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.037+0000 7effc0b99740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 0x564d250e73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.037+0000 7effc0b99740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250db800 0x564d250cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.037+0000 7effc0b99740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7effb81ad670 con 0x564d250db800 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.037+0000 7effc0b99740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7effb81aaf40 con 0x564d250d5ca0 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.037+0000 7effc0b99740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7effb81a8d90 con 0x564d250d6690 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.038+0000 7effb6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 0x564d250e73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.038+0000 7effb6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 0x564d250e73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38996/0 (socket says 172.21.15.42:38996) 2024-04-25T00:21:28.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.038+0000 7effb6000640 1 -- 172.21.15.42:0/3718661307 learned_addr learned my addr 172.21.15.42:0/3718661307 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:28.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.038+0000 7effb7400640 1 --2- 172.21.15.42:0/3718661307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250db800 0x564d250cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.038+0000 7effb6000640 1 -- 172.21.15.42:0/3718661307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d5ca0 msgr2=0x564d250e7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.038+0000 7effb6000640 1 --2- 172.21.15.42:0/3718661307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d5ca0 0x564d250e7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.038+0000 7effb6000640 1 -- 172.21.15.42:0/3718661307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250db800 msgr2=0x564d250cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.038+0000 7effb6000640 1 --2- 172.21.15.42:0/3718661307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250db800 0x564d250cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.038+0000 7effb6000640 1 -- 172.21.15.42:0/3718661307 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d250d1140 con 0x564d250d6690 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.038+0000 7effb6000640 1 --2- 172.21.15.42:0/3718661307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 0x564d250e73c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effa00216f0 tx=0x7effa001c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6be5090ce3d7fbad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.039+0000 7effa7400640 1 -- 172.21.15.42:0/3718661307 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0004220 con 0x564d250d6690 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.039+0000 7effa7400640 1 -- 172.21.15.42:0/3718661307 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa00043c0 con 0x564d250d6690 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.039+0000 7effa7400640 1 -- 172.21.15.42:0/3718661307 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa0003c20 con 0x564d250d6690 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.040+0000 7effc0b99740 1 -- 172.21.15.42:0/3718661307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 msgr2=0x564d250e73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.040+0000 7effc0b99740 1 --2- 172.21.15.42:0/3718661307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 0x564d250e73c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7effa00216f0 tx=0x7effa001c000 comp rx=0 tx=0).stop 2024-04-25T00:21:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.040+0000 7effc0b99740 1 -- 172.21.15.42:0/3718661307 shutdown_connections 2024-04-25T00:21:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.040+0000 7effc0b99740 1 --2- 172.21.15.42:0/3718661307 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250db800 0x564d250cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.040+0000 7effc0b99740 1 --2- 172.21.15.42:0/3718661307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d6690 0x564d250e73c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.040+0000 7effc0b99740 1 --2- 172.21.15.42:0/3718661307 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250d5ca0 0x564d250e7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.040+0000 7effc0b99740 1 -- 172.21.15.42:0/3718661307 >> 172.21.15.42:0/3718661307 conn(0x564d250d1310 msgr2=0x564d250daf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.040+0000 7effc0b99740 1 -- 172.21.15.42:0/3718661307 shutdown_connections 2024-04-25T00:21:28.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.040+0000 7effc0b99740 1 -- 172.21.15.42:0/3718661307 wait complete. 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.040+0000 7effc0b99740 1 -- start start 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effc0b99740 1 Processor -- start 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effc0b99740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d5ca0 0x564d250e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effc0b99740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d6690 0x564d250e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effc0b99740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 0x564d250e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effc0b99740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7effb81a9400 con 0x564d250d6690 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effc0b99740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7effb81aa870 con 0x564d250db800 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effc0b99740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7effb81a9ad0 con 0x564d250d5ca0 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effc0b99740 10 client.0 _ll_register_callbacks cb 0x7effb0005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effb6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d5ca0 0x564d250e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effb6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d5ca0 0x564d250e4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39012/0 (socket says 172.21.15.42:39012) 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effb6a00640 1 -- 172.21.15.42:0/3743326862 learned_addr learned my addr 172.21.15.42:0/3743326862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effb6000640 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d6690 0x564d250e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.041+0000 7effb7400640 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 0x564d250e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.042+0000 7effb6a00640 1 -- 172.21.15.42:0/3743326862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 msgr2=0x564d250e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.042+0000 7effb6a00640 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 0x564d250e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.042+0000 7effb6a00640 1 -- 172.21.15.42:0/3743326862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d6690 msgr2=0x564d250e4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.042+0000 7effb6a00640 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d6690 0x564d250e4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.042+0000 7effb6a00640 1 -- 172.21.15.42:0/3743326862 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d250e6520 con 0x564d250d5ca0 2024-04-25T00:21:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.042+0000 7effb6a00640 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d5ca0 0x564d250e4a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7effa802acb0 tx=0x7effa8009500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91175a9ce420e8c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:28.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.042+0000 7effa4c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:21:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.042+0000 7effa4c00640 1 -- 172.21.15.42:0/3743326862 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8053060 con 0x564d250d5ca0 2024-04-25T00:21:28.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.042+0000 7effa4c00640 1 -- 172.21.15.42:0/3743326862 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7effa804ebc0 con 0x564d250d5ca0 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.043+0000 7effc0b99740 1 -- 172.21.15.42:0/3743326862 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa00043c0 con 0x564d250d5ca0 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.043+0000 7effa4c00640 1 -- 172.21.15.42:0/3743326862 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7effa8036040 con 0x564d250d5ca0 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.043+0000 7effc0b99740 10 client.25405 fetch_fsmap learned FSMap version 25 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.043+0000 7effc0b99740 1 -- 172.21.15.42:0/3743326862 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564d25152170 con 0x564d250d5ca0 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.043+0000 7effa4c00640 1 -- 172.21.15.42:0/3743326862 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effa00043c0 con 0x564d250d5ca0 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.043+0000 7effa4c00640 1 -- 172.21.15.42:0/3743326862 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7effa804e3d0 con 0x564d250d5ca0 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.043+0000 7effc0b99740 10 client.25405 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.043+0000 7effc0b99740 10 client.25405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.044+0000 7effc0b99740 20 client.25405 populate_metadata read hostname 'smithi042' 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.044+0000 7effc0b99740 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564d25152680 0x564d25154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.044+0000 7effc0b99740 4 client.25405 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.044+0000 7effc0b99740 1 -- 172.21.15.42:0/3743326862 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d25040340 con 0x564d25152680 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.044+0000 7effb6000640 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564d25152680 0x564d25154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:28.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.044+0000 7effb6000640 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564d25152680 0x564d25154aa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a3283bf4c1b2c2 server_cookie=6ca6695b1005bcff in_seq=0 out_seq=0 2024-04-25T00:21:28.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.044+0000 7effa4c00640 10 client.25405 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:28.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effa4c00640 1 -- 172.21.15.42:0/3743326862 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d25040340 con 0x564d25152680 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effa4c00640 10 client.25405 handle_command_reply: tid=0 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 client.25405 shutdown 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 -- 172.21.15.42:0/3743326862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d5ca0 msgr2=0x564d250e4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d5ca0 0x564d250e4a30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7effa802acb0 tx=0x7effa8009500 comp rx=0 tx=0).stop 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 -- 172.21.15.42:0/3743326862 shutdown_connections 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564d25152680 0x564d25154aa0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564d250db800 0x564d250e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564d250d6690 0x564d250e4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effa4c00640 0 client.25405 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 --2- 172.21.15.42:0/3743326862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564d250d5ca0 0x564d250e4a30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 -- 172.21.15.42:0/3743326862 >> 172.21.15.42:0/3743326862 conn(0x564d250d1310 msgr2=0x564d250dab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 -- 172.21.15.42:0/3743326862 shutdown_connections 2024-04-25T00:21:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 1 -- 172.21.15.42:0/3743326862 wait complete. 2024-04-25T00:21:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.100+0000 7effc0b99740 20 client.25405 trim_cache size 0 max 16384 2024-04-25T00:21:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:21:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:21:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:21:28.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:21:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.101+0000 7effbdc00640 1 -- 172.21.15.42:0/229254282 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff9c0a2cb0 msgr2=0x7eff9c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.101+0000 7effbdc00640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff9c0a2cb0 0x7eff9c0a5170 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x564d250d6ef0 tx=0x7effa0009640 comp rx=0 tx=0).stop 2024-04-25T00:21:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.101+0000 7effbdc00640 1 -- 172.21.15.42:0/229254282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effb81074e0 msgr2=0x7effb81a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.101+0000 7effbdc00640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effb81074e0 0x7effb81a11d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7effb000e970 tx=0x7effb000ed70 comp rx=0 tx=0).stop 2024-04-25T00:21:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.101+0000 7effbdc00640 1 -- 172.21.15.42:0/229254282 shutdown_connections 2024-04-25T00:21:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.101+0000 7effbdc00640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7eff9c0a2cb0 0x7eff9c0a5170 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.101+0000 7effbdc00640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7effb81074e0 0x7effb81a11d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.101+0000 7effbdc00640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effb8104bb0 0x7effb819a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.101+0000 7effbdc00640 1 --2- 172.21.15.42:0/229254282 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7effb8102280 0x7effb8199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.101+0000 7effbdc00640 1 -- 172.21.15.42:0/229254282 >> 172.21.15.42:0/229254282 conn(0x7effb80fbc10 msgr2=0x7effb8101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.102+0000 7effbdc00640 1 -- 172.21.15.42:0/229254282 shutdown_connections 2024-04-25T00:21:28.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:28.102+0000 7effbdc00640 1 -- 172.21.15.42:0/229254282 wait complete. 2024-04-25T00:21:28.209 INFO:tasks.workunit.client.0.smithi042.stdout: 1 102735 7.32 MB/sec warmup 78 sec latency 55.956 ms 2024-04-25T00:21:29.209 INFO:tasks.workunit.client.0.smithi042.stdout: 1 103340 7.29 MB/sec warmup 79 sec latency 35.244 ms 2024-04-25T00:21:29.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:29 smithi042 ceph-mon[28130]: pgmap v469: 97 pgs: 97 active+clean; 43 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-04-25T00:21:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:29 smithi136 ceph-mon[32923]: pgmap v469: 97 pgs: 97 active+clean; 43 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-04-25T00:21:29.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:29 smithi106 ceph-mon[32510]: pgmap v469: 97 pgs: 97 active+clean; 43 MiB data, 636 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-04-25T00:21:30.209 INFO:tasks.workunit.client.0.smithi042.stdout: 1 104313 7.26 MB/sec warmup 80 sec latency 25.524 ms 2024-04-25T00:21:31.209 INFO:tasks.workunit.client.0.smithi042.stdout: 1 105472 7.20 MB/sec warmup 81 sec latency 23.140 ms 2024-04-25T00:21:31.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:31 smithi042 ceph-mon[28130]: pgmap v470: 97 pgs: 97 active+clean; 43 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-04-25T00:21:31.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:31 smithi136 ceph-mon[32923]: pgmap v470: 97 pgs: 97 active+clean; 43 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-04-25T00:21:31.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:31 smithi106 ceph-mon[32510]: pgmap v470: 97 pgs: 97 active+clean; 43 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-04-25T00:21:32.210 INFO:tasks.workunit.client.0.smithi042.stdout: 1 106714 7.25 MB/sec warmup 82 sec latency 19.768 ms 2024-04-25T00:21:33.210 INFO:tasks.workunit.client.0.smithi042.stdout: 1 107240 7.17 MB/sec warmup 83 sec latency 55.545 ms 2024-04-25T00:21:33.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:33 smithi042 ceph-mon[28130]: pgmap v471: 97 pgs: 97 active+clean; 41 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-04-25T00:21:33.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:33 smithi136 ceph-mon[32923]: pgmap v471: 97 pgs: 97 active+clean; 41 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-04-25T00:21:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:33 smithi106 ceph-mon[32510]: pgmap v471: 97 pgs: 97 active+clean; 41 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 237 op/s 2024-04-25T00:21:34.210 INFO:tasks.workunit.client.0.smithi042.stdout: 1 108226 7.14 MB/sec warmup 84 sec latency 78.255 ms 2024-04-25T00:21:35.210 INFO:tasks.workunit.client.0.smithi042.stdout: 1 109829 7.17 MB/sec warmup 85 sec latency 24.144 ms 2024-04-25T00:21:35.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:35 smithi042 ceph-mon[28130]: pgmap v472: 97 pgs: 97 active+clean; 41 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-04-25T00:21:35.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:35 smithi136 ceph-mon[32923]: pgmap v472: 97 pgs: 97 active+clean; 41 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-04-25T00:21:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:35 smithi106 ceph-mon[32510]: pgmap v472: 97 pgs: 97 active+clean; 41 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 169 op/s 2024-04-25T00:21:36.210 INFO:tasks.workunit.client.0.smithi042.stdout: 1 110431 7.14 MB/sec warmup 86 sec latency 19.309 ms 2024-04-25T00:21:37.211 INFO:tasks.workunit.client.0.smithi042.stdout: 1 111259 7.08 MB/sec warmup 87 sec latency 20.205 ms 2024-04-25T00:21:37.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:37 smithi136 ceph-mon[32923]: pgmap v473: 97 pgs: 97 active+clean; 41 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-04-25T00:21:37.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:37 smithi106 ceph-mon[32510]: pgmap v473: 97 pgs: 97 active+clean; 41 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-04-25T00:21:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:37 smithi042 ceph-mon[28130]: pgmap v473: 97 pgs: 97 active+clean; 41 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-04-25T00:21:38.211 INFO:tasks.workunit.client.0.smithi042.stdout: 1 112607 7.07 MB/sec warmup 88 sec latency 33.078 ms 2024-04-25T00:21:39.211 INFO:tasks.workunit.client.0.smithi042.stdout: 1 113951 7.11 MB/sec warmup 89 sec latency 17.059 ms 2024-04-25T00:21:39.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:39 smithi136 ceph-mon[32923]: pgmap v474: 97 pgs: 97 active+clean; 42 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-04-25T00:21:39.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:39 smithi106 ceph-mon[32510]: pgmap v474: 97 pgs: 97 active+clean; 42 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-04-25T00:21:39.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:39 smithi042 ceph-mon[28130]: pgmap v474: 97 pgs: 97 active+clean; 42 MiB data, 654 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-04-25T00:21:40.211 INFO:tasks.workunit.client.0.smithi042.stdout: 1 115049 7.09 MB/sec warmup 90 sec latency 32.897 ms 2024-04-25T00:21:41.211 INFO:tasks.workunit.client.0.smithi042.stdout: 1 116936 7.11 MB/sec warmup 91 sec latency 18.845 ms 2024-04-25T00:21:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.250+0000 7f06eee00640 1 -- 172.21.15.42:0/1662402856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8075700 msgr2=0x7f06e8073b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.250+0000 7f06eee00640 1 --2- 172.21.15.42:0/1662402856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8075700 0x7f06e8073b50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f06d8009900 tx=0x7f06d802a480 comp rx=0 tx=0).stop 2024-04-25T00:21:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.250+0000 7f06eee00640 1 -- 172.21.15.42:0/1662402856 shutdown_connections 2024-04-25T00:21:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.250+0000 7f06eee00640 1 --2- 172.21.15.42:0/1662402856 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06e8109670 0x7f06e810bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.250+0000 7f06eee00640 1 --2- 172.21.15.42:0/1662402856 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06e8074120 0x7f06e8074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.250+0000 7f06eee00640 1 --2- 172.21.15.42:0/1662402856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8075700 0x7f06e8073b50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.250+0000 7f06eee00640 1 -- 172.21.15.42:0/1662402856 >> 172.21.15.42:0/1662402856 conn(0x7f06e80fcd30 msgr2=0x7f06e80ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:41.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.250+0000 7f06eee00640 1 -- 172.21.15.42:0/1662402856 shutdown_connections 2024-04-25T00:21:41.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.250+0000 7f06eee00640 1 -- 172.21.15.42:0/1662402856 wait complete. 2024-04-25T00:21:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.251+0000 7f06eee00640 1 Processor -- start 2024-04-25T00:21:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06eee00640 1 -- start start 2024-04-25T00:21:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06eee00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8074120 0x7f06e8199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06eee00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06e8075700 0x7f06e819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06eee00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06e8109670 0x7f06e81a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06eee00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f06e810f690 con 0x7f06e8109670 2024-04-25T00:21:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06eee00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06e810f510 con 0x7f06e8075700 2024-04-25T00:21:41.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06eee00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06e810f810 con 0x7f06e8074120 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06e7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8074120 0x7f06e8199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06e7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8074120 0x7f06e8199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:42692/0 (socket says 172.21.15.42:42692) 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06e7e00640 1 -- 172.21.15.42:0/2393612732 learned_addr learned my addr 172.21.15.42:0/2393612732 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06e7400640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06e8075700 0x7f06e819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.252+0000 7f06ece00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06e8109670 0x7f06e81a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06e7e00640 1 -- 172.21.15.42:0/2393612732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06e8075700 msgr2=0x7f06e819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06e7e00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06e8075700 0x7f06e819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06e7e00640 1 -- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06e8109670 msgr2=0x7f06e81a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06e7e00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06e8109670 0x7f06e81a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06e7e00640 1 -- 172.21.15.42:0/2393612732 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e81a1850 con 0x7f06e8074120 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06ece00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06e8109670 0x7f06e81a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06e7e00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8074120 0x7f06e8199c40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f06d8031440 tx=0x7f06d8032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06e4c00640 1 -- 172.21.15.42:0/2393612732 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d803f030 con 0x7f06e8074120 2024-04-25T00:21:41.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06e4c00640 1 -- 172.21.15.42:0/2393612732 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06d8003ce0 con 0x7f06e8074120 2024-04-25T00:21:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06eee00640 1 -- 172.21.15.42:0/2393612732 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e81a1ae0 con 0x7f06e8074120 2024-04-25T00:21:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.253+0000 7f06e4c00640 1 -- 172.21.15.42:0/2393612732 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d8035070 con 0x7f06e8074120 2024-04-25T00:21:41.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.254+0000 7f06eee00640 1 -- 172.21.15.42:0/2393612732 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e81a1fd0 con 0x7f06e8074120 2024-04-25T00:21:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.257+0000 7f06e4c00640 1 -- 172.21.15.42:0/2393612732 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f06d8042030 con 0x7f06e8074120 2024-04-25T00:21:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.257+0000 7f06f1bb0740 1 Processor -- start 2024-04-25T00:21:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.257+0000 7f06f1bb0740 1 -- start start 2024-04-25T00:21:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9675ca0 0x5643a96760c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 0x5643a967b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a967b800 0x5643a967dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5643a95d8ab0 con 0x5643a9676690 2024-04-25T00:21:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5643a9581640 con 0x5643a967b800 2024-04-25T00:21:41.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5643a95d4a70 con 0x5643a9675ca0 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06ece00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9675ca0 0x5643a96760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06ece00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9675ca0 0x5643a96760c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:42708/0 (socket says 172.21.15.42:42708) 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06ece00640 1 -- 172.21.15.42:0/1102595053 learned_addr learned my addr 172.21.15.42:0/1102595053 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06e7e00640 1 --2- 172.21.15.42:0/1102595053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 0x5643a967b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06e7400640 1 --2- 172.21.15.42:0/1102595053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a967b800 0x5643a967dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06ece00640 1 -- 172.21.15.42:0/1102595053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a967b800 msgr2=0x5643a967dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06ece00640 1 --2- 172.21.15.42:0/1102595053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a967b800 0x5643a967dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06ece00640 1 -- 172.21.15.42:0/1102595053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 msgr2=0x5643a967b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06ece00640 1 --2- 172.21.15.42:0/1102595053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 0x5643a967b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06ece00640 1 -- 172.21.15.42:0/1102595053 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643a92ed210 con 0x5643a9675ca0 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.258+0000 7f06e7e00640 1 --2- 172.21.15.42:0/1102595053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 0x5643a967b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:41.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.259+0000 7f06ece00640 1 --2- 172.21.15.42:0/1102595053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9675ca0 0x5643a96760c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06e000bcd0 tx=0x7f06e000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dfaf125f3f2842a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.259+0000 7f06e4c00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f06c80a2c30 0x7f06c80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.259+0000 7f06dd600640 1 -- 172.21.15.42:0/1102595053 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0015070 con 0x5643a9675ca0 2024-04-25T00:21:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.259+0000 7f06dd600640 1 -- 172.21.15.42:0/1102595053 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e00035f0 con 0x5643a9675ca0 2024-04-25T00:21:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.259+0000 7f06dd600640 1 -- 172.21.15.42:0/1102595053 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e00116a0 con 0x5643a9675ca0 2024-04-25T00:21:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.259+0000 7f06e4c00640 1 -- 172.21.15.42:0/2393612732 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f06d80ef6e0 con 0x7f06e8074120 2024-04-25T00:21:41.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.259+0000 7f06e7e00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f06c80a2c30 0x7f06c80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.260+0000 7f06f1bb0740 1 -- 172.21.15.42:0/1102595053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9675ca0 msgr2=0x5643a96760c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.260+0000 7f06e7e00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f06c80a2c30 0x7f06c80a50f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f06d80bbc10 tx=0x7f06d802a480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:41.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.260+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/1102595053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9675ca0 0x5643a96760c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f06e000bcd0 tx=0x7f06e000f730 comp rx=0 tx=0).stop 2024-04-25T00:21:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.261+0000 7f06f1bb0740 1 -- 172.21.15.42:0/1102595053 shutdown_connections 2024-04-25T00:21:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.261+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/1102595053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a967b800 0x5643a967dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.261+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/1102595053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 0x5643a967b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.261+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/1102595053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9675ca0 0x5643a96760c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.261+0000 7f06f1bb0740 1 -- 172.21.15.42:0/1102595053 >> 172.21.15.42:0/1102595053 conn(0x5643a9671310 msgr2=0x5643a9671710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.261+0000 7f06f1bb0740 1 -- 172.21.15.42:0/1102595053 shutdown_connections 2024-04-25T00:21:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.261+0000 7f06f1bb0740 1 -- 172.21.15.42:0/1102595053 wait complete. 2024-04-25T00:21:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.261+0000 7f06f1bb0740 1 -- start start 2024-04-25T00:21:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.261+0000 7f06f1bb0740 1 Processor -- start 2024-04-25T00:21:41.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a967ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 0x5643a96804c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a967b800 0x5643a9687500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5643a967e7b0 con 0x5643a9676690 2024-04-25T00:21:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5643a9594990 con 0x5643a9675ca0 2024-04-25T00:21:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5643a967e930 con 0x5643a967b800 2024-04-25T00:21:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06f1bb0740 10 client.0 _ll_register_callbacks cb 0x5643a9596950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:41.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06ece00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a967ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06ece00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a967ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42064/0 (socket says 172.21.15.42:42064) 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06ece00640 1 -- 172.21.15.42:0/3209818311 learned_addr learned my addr 172.21.15.42:0/3209818311 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06ece00640 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a967b800 0x5643a9687500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.262+0000 7f06e7400640 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 0x5643a96804c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.263+0000 7f06ece00640 1 -- 172.21.15.42:0/3209818311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a967b800 msgr2=0x5643a9687500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.263+0000 7f06ece00640 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a967b800 0x5643a9687500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.263+0000 7f06ece00640 1 -- 172.21.15.42:0/3209818311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 msgr2=0x5643a96804c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.263+0000 7f06ece00640 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 0x5643a96804c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.263+0000 7f06ece00640 1 -- 172.21.15.42:0/3209818311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643a93b9620 con 0x5643a9675ca0 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.263+0000 7f06ece00640 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a967ff80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f06e0000c00 tx=0x7f06e000b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb3f552cc9228f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.263+0000 7f06d6000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:21:41.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.263+0000 7f06d6000640 1 -- 172.21.15.42:0/3209818311 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0015070 con 0x5643a9675ca0 2024-04-25T00:21:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.263+0000 7f06d6000640 1 -- 172.21.15.42:0/3209818311 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f06e0005050 con 0x5643a9675ca0 2024-04-25T00:21:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.263+0000 7f06f1bb0740 1 -- 172.21.15.42:0/3209818311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e00035f0 con 0x5643a9675ca0 2024-04-25T00:21:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06d6000640 1 -- 172.21.15.42:0/3209818311 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f06e0022040 con 0x5643a9675ca0 2024-04-25T00:21:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06f1bb0740 10 client.25412 fetch_fsmap learned FSMap version 25 2024-04-25T00:21:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06f1bb0740 1 -- 172.21.15.42:0/3209818311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643a968a4e0 con 0x5643a9675ca0 2024-04-25T00:21:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06d6000640 1 -- 172.21.15.42:0/3209818311 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e00035f0 con 0x5643a9675ca0 2024-04-25T00:21:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06d6000640 1 -- 172.21.15.42:0/3209818311 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f06e0004c50 con 0x5643a9675ca0 2024-04-25T00:21:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06f1bb0740 10 client.25412 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:21:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06f1bb0740 10 client.25412 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:41.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06f1bb0740 20 client.25412 populate_metadata read hostname 'smithi042' 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643a968ba00 0x5643a968de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06f1bb0740 4 client.25412 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.264+0000 7f06f1bb0740 1 -- 172.21.15.42:0/3209818311 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643a95e5ff0 con 0x5643a968ba00 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.265+0000 7f06e7400640 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643a968ba00 0x5643a968de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.265+0000 7f06e7400640 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643a968ba00 0x5643a968de20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92696368c057d914 server_cookie=1c9b736cd1008381 in_seq=0 out_seq=0 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.265+0000 7f06d6000640 10 client.25412 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06d6000640 1 -- 172.21.15.42:0/3209818311 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643a95e5ff0 con 0x5643a968ba00 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06d6000640 10 client.25412 handle_command_reply: tid=0 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06f1bb0740 1 client.25412 shutdown 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06f1bb0740 1 -- 172.21.15.42:0/3209818311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 msgr2=0x5643a967ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a967ff80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f06e0000c00 tx=0x7f06e000b5c0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06f1bb0740 1 -- 172.21.15.42:0/3209818311 shutdown_connections 2024-04-25T00:21:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643a968ba00 0x5643a968de20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a967b800 0x5643a9687500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06d6000640 0 client.25412 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a9676690 0x5643a96804c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/3209818311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a967ff80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.266+0000 7f06f1bb0740 1 -- 172.21.15.42:0/3209818311 >> 172.21.15.42:0/3209818311 conn(0x5643a9671310 msgr2=0x5643a9670480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.267+0000 7f06f1bb0740 1 -- 172.21.15.42:0/3209818311 shutdown_connections 2024-04-25T00:21:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.267+0000 7f06f1bb0740 1 -- 172.21.15.42:0/3209818311 wait complete. 2024-04-25T00:21:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.267+0000 7f06f1bb0740 20 client.25412 trim_cache size 0 max 16384 2024-04-25T00:21:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.285+0000 7f06f1bb0740 1 Processor -- start 2024-04-25T00:21:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.285+0000 7f06f1bb0740 1 -- start start 2024-04-25T00:21:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a9674ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a96802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 0x5643a96926e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81ad610 con 0x5643a967b800 2024-04-25T00:21:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81aaee0 con 0x5643a9675ca0 2024-04-25T00:21:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81a8d30 con 0x5643a9676690 2024-04-25T00:21:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06ece00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a9674ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06ece00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 0x5643a96926e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06e7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a96802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06ece00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a9674ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42072/0 (socket says 172.21.15.42:42072) 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.286+0000 7f06ece00640 1 -- 172.21.15.42:0/1201434908 learned_addr learned my addr 172.21.15.42:0/1201434908 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.287+0000 7f06ece00640 1 -- 172.21.15.42:0/1201434908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 msgr2=0x5643a96802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.287+0000 7f06ece00640 1 --2- 172.21.15.42:0/1201434908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a96802a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.287+0000 7f06ece00640 1 -- 172.21.15.42:0/1201434908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 msgr2=0x5643a96926e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.287+0000 7f06ece00640 1 --2- 172.21.15.42:0/1201434908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 0x5643a96926e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.287+0000 7f06ece00640 1 -- 172.21.15.42:0/1201434908 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643a96774a0 con 0x5643a9675ca0 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.287+0000 7f06e7400640 1 --2- 172.21.15.42:0/1201434908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a96802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.287+0000 7f06ece00640 1 --2- 172.21.15.42:0/1201434908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a9674ab0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f06e000b400 tx=0x7f06e0005970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f9e79e8fc323ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.287+0000 7f06dcc00640 1 -- 172.21.15.42:0/1201434908 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0015040 con 0x5643a9675ca0 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.287+0000 7f06dcc00640 1 -- 172.21.15.42:0/1201434908 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e00115d0 con 0x5643a9675ca0 2024-04-25T00:21:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.287+0000 7f06dcc00640 1 -- 172.21.15.42:0/1201434908 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e0005280 con 0x5643a9675ca0 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.288+0000 7f06f1bb0740 1 -- 172.21.15.42:0/1201434908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 msgr2=0x5643a9674ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.288+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/1201434908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a9674ab0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f06e000b400 tx=0x7f06e0005970 comp rx=0 tx=0).stop 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.288+0000 7f06f1bb0740 1 -- 172.21.15.42:0/1201434908 shutdown_connections 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.288+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/1201434908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 0x5643a96926e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.288+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/1201434908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a96802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.288+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/1201434908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a9674ab0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.288+0000 7f06f1bb0740 1 -- 172.21.15.42:0/1201434908 >> 172.21.15.42:0/1201434908 conn(0x5643a9671310 msgr2=0x5643a967fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.288+0000 7f06f1bb0740 1 -- 172.21.15.42:0/1201434908 shutdown_connections 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.288+0000 7f06f1bb0740 1 -- 172.21.15.42:0/1201434908 wait complete. 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.289+0000 7f06f1bb0740 1 -- start start 2024-04-25T00:21:41.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.289+0000 7f06f1bb0740 1 Processor -- start 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.289+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a96847e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.289+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a9684d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.289+0000 7f06f1bb0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 0x5643a9685d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.289+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81a93a0 con 0x5643a967b800 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.289+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81aa810 con 0x5643a9675ca0 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.289+0000 7f06f1bb0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06e81a9a70 con 0x5643a9676690 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.289+0000 7f06f1bb0740 10 client.0 _ll_register_callbacks cb 0x7f06e000bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06e7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a9684d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06ece00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a96847e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06e7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a9684d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:42736/0 (socket says 172.21.15.42:42736) 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06e7400640 1 -- 172.21.15.42:0/428738427 learned_addr learned my addr 172.21.15.42:0/428738427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06ece00640 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 0x5643a9685d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06e7400640 1 -- 172.21.15.42:0/428738427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 msgr2=0x5643a96847e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06e7400640 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a96847e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06e7400640 1 -- 172.21.15.42:0/428738427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 msgr2=0x5643a9685d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06e7400640 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 0x5643a9685d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06e7400640 1 -- 172.21.15.42:0/428738427 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643a9686320 con 0x5643a9676690 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06ece00640 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 0x5643a9685d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06e7400640 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a9684d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009f30 tx=0x7f06d0013a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4994a521d5793241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06d6000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.290+0000 7f06d6000640 1 -- 172.21.15.42:0/428738427 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0004d20 con 0x5643a9676690 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.291+0000 7f06d6000640 1 -- 172.21.15.42:0/428738427 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7f06d0022440 con 0x5643a9676690 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.291+0000 7f06f1bb0740 1 -- 172.21.15.42:0/428738427 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06e00115d0 con 0x5643a9676690 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.291+0000 7f06d6000640 1 -- 172.21.15.42:0/428738427 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f06d0022640 con 0x5643a9676690 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.291+0000 7f06f1bb0740 10 client.25423 fetch_fsmap learned FSMap version 25 2024-04-25T00:21:41.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.291+0000 7f06f1bb0740 1 -- 172.21.15.42:0/428738427 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5643a96f2730 con 0x5643a9676690 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.291+0000 7f06d6000640 1 -- 172.21.15.42:0/428738427 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06e00115d0 con 0x5643a9676690 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.291+0000 7f06d6000640 1 -- 172.21.15.42:0/428738427 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f06d0005020 con 0x5643a9676690 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.291+0000 7f06f1bb0740 10 client.25423 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.291+0000 7f06f1bb0740 10 client.25423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.291+0000 7f06f1bb0740 20 client.25423 populate_metadata read hostname 'smithi042' 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.292+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643a96f3170 0x5643a96f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.292+0000 7f06f1bb0740 4 client.25423 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.292+0000 7f06f1bb0740 1 -- 172.21.15.42:0/428738427 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643a95e0340 con 0x5643a96f3170 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.292+0000 7f06ece00640 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643a96f3170 0x5643a96f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.292+0000 7f06ece00640 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643a96f3170 0x5643a96f5590 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=551ddd60e2ebfef2 server_cookie=f0ac28c52aa460f3 in_seq=0 out_seq=0 2024-04-25T00:21:41.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.292+0000 7f06d6000640 10 client.25423 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06d6000640 1 -- 172.21.15.42:0/428738427 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643a95e0340 con 0x5643a96f3170 2024-04-25T00:21:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06d6000640 10 client.25423 handle_command_reply: tid=0 2024-04-25T00:21:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 client.25423 shutdown 2024-04-25T00:21:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 -- 172.21.15.42:0/428738427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 msgr2=0x5643a9684d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a9684d20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009f30 tx=0x7f06d0013a70 comp rx=0 tx=0).stop 2024-04-25T00:21:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 -- 172.21.15.42:0/428738427 shutdown_connections 2024-04-25T00:21:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643a96f3170 0x5643a96f5590 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5643a967b800 0x5643a9685d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5643a9676690 0x5643a9684d20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 --2- 172.21.15.42:0/428738427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5643a9675ca0 0x5643a96847e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 -- 172.21.15.42:0/428738427 >> 172.21.15.42:0/428738427 conn(0x5643a9671310 msgr2=0x5643a967fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06d6000640 0 client.25423 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 -- 172.21.15.42:0/428738427 shutdown_connections 2024-04-25T00:21:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.338+0000 7f06f1bb0740 1 -- 172.21.15.42:0/428738427 wait complete. 2024-04-25T00:21:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.339+0000 7f06f1bb0740 20 client.25423 trim_cache size 0 max 16384 2024-04-25T00:21:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:21:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:21:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:21:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:21:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.339+0000 7f06eee00640 1 -- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f06c80a2c30 msgr2=0x7f06c80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.339+0000 7f06eee00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f06c80a2c30 0x7f06c80a50f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f06d80bbc10 tx=0x7f06d802a480 comp rx=0 tx=0).stop 2024-04-25T00:21:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.339+0000 7f06eee00640 1 -- 172.21.15.42:0/2393612732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8074120 msgr2=0x7f06e8199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.339+0000 7f06eee00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8074120 0x7f06e8199c40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f06d8031440 tx=0x7f06d8032ce0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.340+0000 7f06eee00640 1 -- 172.21.15.42:0/2393612732 shutdown_connections 2024-04-25T00:21:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.340+0000 7f06eee00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f06c80a2c30 0x7f06c80a50f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.340+0000 7f06eee00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06e8109670 0x7f06e81a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.340+0000 7f06eee00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06e8075700 0x7f06e819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.340+0000 7f06eee00640 1 --2- 172.21.15.42:0/2393612732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06e8074120 0x7f06e8199c40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.340+0000 7f06eee00640 1 -- 172.21.15.42:0/2393612732 >> 172.21.15.42:0/2393612732 conn(0x7f06e80fcd30 msgr2=0x7f06e8105600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.340+0000 7f06eee00640 1 -- 172.21.15.42:0/2393612732 shutdown_connections 2024-04-25T00:21:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:41.340+0000 7f06eee00640 1 -- 172.21.15.42:0/2393612732 wait complete. 2024-04-25T00:21:41.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:41 smithi136 ceph-mon[32923]: pgmap v475: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-04-25T00:21:41.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:41 smithi106 ceph-mon[32510]: pgmap v475: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-04-25T00:21:41.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:41 smithi042 ceph-mon[28130]: pgmap v475: 97 pgs: 97 active+clean; 42 MiB data, 655 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-04-25T00:21:42.211 INFO:tasks.workunit.client.0.smithi042.stdout: 1 117691 7.09 MB/sec warmup 92 sec latency 39.785 ms 2024-04-25T00:21:43.211 INFO:tasks.workunit.client.0.smithi042.stdout: 1 118866 7.07 MB/sec warmup 93 sec latency 38.625 ms 2024-04-25T00:21:43.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:43 smithi136 ceph-mon[32923]: pgmap v476: 97 pgs: 97 active+clean; 40 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 232 op/s 2024-04-25T00:21:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:43 smithi106 ceph-mon[32510]: pgmap v476: 97 pgs: 97 active+clean; 40 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 232 op/s 2024-04-25T00:21:43.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:43 smithi042 ceph-mon[28130]: pgmap v476: 97 pgs: 97 active+clean; 40 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 232 op/s 2024-04-25T00:21:44.212 INFO:tasks.workunit.client.0.smithi042.stdout: 1 120845 7.14 MB/sec warmup 94 sec latency 14.782 ms 2024-04-25T00:21:45.212 INFO:tasks.workunit.client.0.smithi042.stdout: 1 121501 7.07 MB/sec warmup 95 sec latency 17.324 ms 2024-04-25T00:21:45.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:45 smithi136 ceph-mon[32923]: pgmap v477: 97 pgs: 97 active+clean; 40 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-25T00:21:45.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:45 smithi106 ceph-mon[32510]: pgmap v477: 97 pgs: 97 active+clean; 40 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-25T00:21:45.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:45 smithi042 ceph-mon[28130]: pgmap v477: 97 pgs: 97 active+clean; 40 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-25T00:21:46.212 INFO:tasks.workunit.client.0.smithi042.stdout: 1 123001 7.06 MB/sec warmup 96 sec latency 19.663 ms 2024-04-25T00:21:47.212 INFO:tasks.workunit.client.0.smithi042.stdout: 1 124504 7.11 MB/sec warmup 97 sec latency 15.472 ms 2024-04-25T00:21:47.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:47 smithi136 ceph-mon[32923]: pgmap v478: 97 pgs: 97 active+clean; 42 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-04-25T00:21:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:47 smithi106 ceph-mon[32510]: pgmap v478: 97 pgs: 97 active+clean; 42 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-04-25T00:21:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:47 smithi042 ceph-mon[28130]: pgmap v478: 97 pgs: 97 active+clean; 42 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-04-25T00:21:48.213 INFO:tasks.workunit.client.0.smithi042.stdout: 1 125333 7.06 MB/sec warmup 98 sec latency 22.582 ms 2024-04-25T00:21:49.213 INFO:tasks.workunit.client.0.smithi042.stdout: 1 126794 7.05 MB/sec warmup 99 sec latency 36.442 ms 2024-04-25T00:21:49.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:49 smithi136 ceph-mon[32923]: pgmap v479: 97 pgs: 97 active+clean; 43 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-04-25T00:21:49.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:49 smithi106 ceph-mon[32510]: pgmap v479: 97 pgs: 97 active+clean; 43 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-04-25T00:21:49.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:49 smithi042 ceph-mon[28130]: pgmap v479: 97 pgs: 97 active+clean; 43 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-04-25T00:21:50.213 INFO:tasks.workunit.client.0.smithi042.stdout: 1 128146 7.09 MB/sec warmup 100 sec latency 14.730 ms 2024-04-25T00:21:51.213 INFO:tasks.workunit.client.0.smithi042.stdout: 1 129215 7.07 MB/sec warmup 101 sec latency 28.016 ms 2024-04-25T00:21:51.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:51 smithi136 ceph-mon[32923]: pgmap v480: 97 pgs: 97 active+clean; 43 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 189 op/s 2024-04-25T00:21:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:51 smithi106 ceph-mon[32510]: pgmap v480: 97 pgs: 97 active+clean; 43 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 189 op/s 2024-04-25T00:21:51.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:51 smithi042 ceph-mon[28130]: pgmap v480: 97 pgs: 97 active+clean; 43 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 189 op/s 2024-04-25T00:21:52.213 INFO:tasks.workunit.client.0.smithi042.stdout: 1 131079 7.10 MB/sec warmup 102 sec latency 19.305 ms 2024-04-25T00:21:53.213 INFO:tasks.workunit.client.0.smithi042.stdout: 1 131807 7.07 MB/sec warmup 103 sec latency 42.322 ms 2024-04-25T00:21:53.679 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:53 smithi042 ceph-mon[28130]: pgmap v481: 97 pgs: 97 active+clean; 45 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 254 op/s 2024-04-25T00:21:53.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:53 smithi136 ceph-mon[32923]: pgmap v481: 97 pgs: 97 active+clean; 45 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 254 op/s 2024-04-25T00:21:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:53 smithi106 ceph-mon[32510]: pgmap v481: 97 pgs: 97 active+clean; 45 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 254 op/s 2024-04-25T00:21:54.213 INFO:tasks.workunit.client.0.smithi042.stdout: 1 132925 7.05 MB/sec warmup 104 sec latency 44.195 ms 2024-04-25T00:21:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.502+0000 7ff14cc00640 1 -- 172.21.15.42:0/80866085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff148104bc0 msgr2=0x7ff148105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.502+0000 7ff14cc00640 1 --2- 172.21.15.42:0/80866085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff148104bc0 0x7ff148105040 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff138009be0 tx=0x7ff13802a6c0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.502+0000 7ff14cc00640 1 -- 172.21.15.42:0/80866085 shutdown_connections 2024-04-25T00:21:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.502+0000 7ff14cc00640 1 --2- 172.21.15.42:0/80866085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff148105580 0x7ff14810be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.502+0000 7ff14cc00640 1 --2- 172.21.15.42:0/80866085 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff148104bc0 0x7ff148105040 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.502+0000 7ff14cc00640 1 --2- 172.21.15.42:0/80866085 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1481039d0 0x7ff148103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.502+0000 7ff14cc00640 1 -- 172.21.15.42:0/80866085 >> 172.21.15.42:0/80866085 conn(0x7ff1480fef60 msgr2=0x7ff148101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.503+0000 7ff14cc00640 1 -- 172.21.15.42:0/80866085 shutdown_connections 2024-04-25T00:21:54.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.503+0000 7ff14cc00640 1 -- 172.21.15.42:0/80866085 wait complete. 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.504+0000 7ff14cc00640 1 Processor -- start 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.504+0000 7ff14cc00640 1 -- start start 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.504+0000 7ff14cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1481039d0 0x7ff148199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.504+0000 7ff14cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff148104bc0 0x7ff14819a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.504+0000 7ff14cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff148105580 0x7ff1481a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.504+0000 7ff14cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff14810f650 con 0x7ff1481039d0 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.504+0000 7ff14cc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff14810f4d0 con 0x7ff148104bc0 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.504+0000 7ff14cc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff14810f7d0 con 0x7ff148105580 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff145e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1481039d0 0x7ff148199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff145e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1481039d0 0x7ff148199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43586/0 (socket says 172.21.15.42:43586) 2024-04-25T00:21:54.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff145e00640 1 -- 172.21.15.42:0/833667667 learned_addr learned my addr 172.21.15.42:0/833667667 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff146800640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff148105580 0x7ff1481a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff145400640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff148104bc0 0x7ff14819a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff146800640 1 -- 172.21.15.42:0/833667667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff148104bc0 msgr2=0x7ff14819a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff146800640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff148104bc0 0x7ff14819a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff146800640 1 -- 172.21.15.42:0/833667667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1481039d0 msgr2=0x7ff148199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff146800640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1481039d0 0x7ff148199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff146800640 1 -- 172.21.15.42:0/833667667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1481a17f0 con 0x7ff148105580 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff145e00640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1481039d0 0x7ff148199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.505+0000 7ff146800640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff148105580 0x7ff1481a1180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff14000e970 tx=0x7ff14000ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.506+0000 7ff13ea00640 1 -- 172.21.15.42:0/833667667 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14000ccc0 con 0x7ff148105580 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.506+0000 7ff13ea00640 1 -- 172.21.15.42:0/833667667 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff140005020 con 0x7ff148105580 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.506+0000 7ff14cc00640 1 -- 172.21.15.42:0/833667667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1481a1ae0 con 0x7ff148105580 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.506+0000 7ff13ea00640 1 -- 172.21.15.42:0/833667667 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140004a00 con 0x7ff148105580 2024-04-25T00:21:54.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.506+0000 7ff14cc00640 1 -- 172.21.15.42:0/833667667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1481a2000 con 0x7ff148105580 2024-04-25T00:21:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.509+0000 7ff13ea00640 1 -- 172.21.15.42:0/833667667 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1400051d0 con 0x7ff148105580 2024-04-25T00:21:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.509+0000 7ff14f9c5740 1 Processor -- start 2024-04-25T00:21:54.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.509+0000 7ff14f9c5740 1 -- start start 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdadca0 0x55ab5bdae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdb3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdb3800 0x55ab5bdb5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ab5bd10ab0 con 0x55ab5bdae690 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ab5bcb9640 con 0x55ab5bdadca0 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ab5bd0ca70 con 0x55ab5bdb3800 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff145400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdadca0 0x55ab5bdae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff145400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdadca0 0x55ab5bdae0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38218/0 (socket says 172.21.15.42:38218) 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff145400640 1 -- 172.21.15.42:0/3196049243 learned_addr learned my addr 172.21.15.42:0/3196049243 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff145400640 1 --2- 172.21.15.42:0/3196049243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdb3800 0x55ab5bdb5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.510+0000 7ff145e00640 1 --2- 172.21.15.42:0/3196049243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdb3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff145400640 1 -- 172.21.15.42:0/3196049243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdb3800 msgr2=0x55ab5bdb5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:54.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff145400640 1 --2- 172.21.15.42:0/3196049243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdb3800 0x55ab5bdb5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff145400640 1 -- 172.21.15.42:0/3196049243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 msgr2=0x55ab5bdb3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff145400640 1 --2- 172.21.15.42:0/3196049243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdb3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff145400640 1 -- 172.21.15.42:0/3196049243 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab5ba25210 con 0x55ab5bdadca0 2024-04-25T00:21:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff145400640 1 --2- 172.21.15.42:0/3196049243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdadca0 0x55ab5bdae0c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff138009be0 tx=0x7ff138000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd7d8d9b05c515f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff13ea00640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff12c0a2c30 0x7ff12c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff137000640 1 -- 172.21.15.42:0/3196049243 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138036040 con 0x55ab5bdadca0 2024-04-25T00:21:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff137000640 1 -- 172.21.15.42:0/3196049243 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff138003e50 con 0x55ab5bdadca0 2024-04-25T00:21:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff137000640 1 -- 172.21.15.42:0/3196049243 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138004830 con 0x55ab5bdadca0 2024-04-25T00:21:54.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.511+0000 7ff13ea00640 1 -- 172.21.15.42:0/833667667 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff1400cefa0 con 0x7ff148105580 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.512+0000 7ff145e00640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff12c0a2c30 0x7ff12c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.512+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3196049243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdadca0 msgr2=0x55ab5bdae0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.512+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3196049243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdadca0 0x55ab5bdae0c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff138009be0 tx=0x7ff138000f40 comp rx=0 tx=0).stop 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.512+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3196049243 shutdown_connections 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.512+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3196049243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdb3800 0x55ab5bdb5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.512+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3196049243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdb3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.512+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3196049243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdadca0 0x55ab5bdae0c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.512+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3196049243 >> 172.21.15.42:0/3196049243 conn(0x55ab5bda9310 msgr2=0x55ab5bda9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.512+0000 7ff145e00640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff12c0a2c30 0x7ff12c0a50f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55ab5bdaea90 tx=0x7ff130009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.513+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3196049243 shutdown_connections 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.513+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3196049243 wait complete. 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.513+0000 7ff14f9c5740 1 -- start start 2024-04-25T00:21:54.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.513+0000 7ff14f9c5740 1 Processor -- start 2024-04-25T00:21:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdb7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdae690 0x55ab5bdb8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdb3800 0x55ab5bdbf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ab5bdb6750 con 0x55ab5bdb3800 2024-04-25T00:21:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ab5bccc990 con 0x55ab5bdae690 2024-04-25T00:21:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ab5bdb68d0 con 0x55ab5bdadca0 2024-04-25T00:21:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff14f9c5740 10 client.0 _ll_register_callbacks cb 0x55ab5bcce950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff146800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdb3800 0x55ab5bdbf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff146800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdb3800 0x55ab5bdbf470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43606/0 (socket says 172.21.15.42:43606) 2024-04-25T00:21:54.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff146800640 1 -- 172.21.15.42:0/3218417305 learned_addr learned my addr 172.21.15.42:0/3218417305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff145400640 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdae690 0x55ab5bdb8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.514+0000 7ff145e00640 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdb7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.515+0000 7ff145400640 1 -- 172.21.15.42:0/3218417305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 msgr2=0x55ab5bdb7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.515+0000 7ff145400640 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdb7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.515+0000 7ff145400640 1 -- 172.21.15.42:0/3218417305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdb3800 msgr2=0x55ab5bdbf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.515+0000 7ff145400640 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdb3800 0x55ab5bdbf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.515+0000 7ff145400640 1 -- 172.21.15.42:0/3218417305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab5baf1620 con 0x55ab5bdae690 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.515+0000 7ff145400640 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdae690 0x55ab5bdb8430 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff138030960 tx=0x7ff138009b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74359b4e05f98dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.515+0000 7ff12b400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.515+0000 7ff12b400640 1 -- 172.21.15.42:0/3218417305 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff138036070 con 0x55ab5bdae690 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.515+0000 7ff12b400640 1 -- 172.21.15.42:0/3218417305 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff138030480 con 0x55ab5bdae690 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.515+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3218417305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff138003e50 con 0x55ab5bdae690 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.516+0000 7ff12b400640 1 -- 172.21.15.42:0/3218417305 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff138030680 con 0x55ab5bdae690 2024-04-25T00:21:54.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.516+0000 7ff14f9c5740 10 client.25430 fetch_fsmap learned FSMap version 25 2024-04-25T00:21:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.516+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3218417305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab5bdc2450 con 0x55ab5bdae690 2024-04-25T00:21:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.516+0000 7ff12b400640 1 -- 172.21.15.42:0/3218417305 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff138003e50 con 0x55ab5bdae690 2024-04-25T00:21:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.516+0000 7ff12b400640 1 -- 172.21.15.42:0/3218417305 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff13803c040 con 0x55ab5bdae690 2024-04-25T00:21:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.516+0000 7ff14f9c5740 10 client.25430 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:21:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.516+0000 7ff14f9c5740 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.516+0000 7ff14f9c5740 20 client.25430 populate_metadata read hostname 'smithi042' 2024-04-25T00:21:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.517+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ab5bdc37a0 0x55ab5bdc5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.517+0000 7ff14f9c5740 4 client.25430 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:21:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.517+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3218417305 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab5bd1dff0 con 0x55ab5bdc37a0 2024-04-25T00:21:54.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.517+0000 7ff145e00640 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ab5bdc37a0 0x55ab5bdc5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.517+0000 7ff145e00640 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ab5bdc37a0 0x55ab5bdc5b90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bb4c3140213fdda server_cookie=e629c30d66e5c6fb in_seq=0 out_seq=0 2024-04-25T00:21:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.517+0000 7ff12b400640 10 client.25430 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.518+0000 7ff12b400640 1 -- 172.21.15.42:0/3218417305 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab5bd1dff0 con 0x55ab5bdc37a0 2024-04-25T00:21:54.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.518+0000 7ff12b400640 10 client.25430 handle_command_reply: tid=0 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.518+0000 7ff14f9c5740 1 client.25430 shutdown 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.518+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3218417305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdae690 msgr2=0x55ab5bdb8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.518+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdae690 0x55ab5bdb8430 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff138030960 tx=0x7ff138009b30 comp rx=0 tx=0).stop 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.518+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3218417305 shutdown_connections 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.518+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ab5bdc37a0 0x55ab5bdc5b90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.519+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdb3800 0x55ab5bdbf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.519+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdae690 0x55ab5bdb8430 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.519+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3218417305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdb7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.519+0000 7ff12b400640 0 client.25430 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.519+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3218417305 >> 172.21.15.42:0/3218417305 conn(0x55ab5bda9310 msgr2=0x55ab5bda8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.519+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3218417305 shutdown_connections 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.519+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3218417305 wait complete. 2024-04-25T00:21:54.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.519+0000 7ff14f9c5740 20 client.25430 trim_cache size 0 max 16384 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.538+0000 7ff14f9c5740 1 Processor -- start 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff14f9c5740 1 -- start start 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdac220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdbf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdac760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481ad520 con 0x55ab5bdae690 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481aadf0 con 0x55ab5bdb3800 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481a8c40 con 0x55ab5bdadca0 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff146800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdac760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff145400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdbf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff146800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdac760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38232/0 (socket says 172.21.15.42:38232) 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.539+0000 7ff146800640 1 -- 172.21.15.42:0/3559997354 learned_addr learned my addr 172.21.15.42:0/3559997354 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:54.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.540+0000 7ff145e00640 1 --2- 172.21.15.42:0/3559997354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.540+0000 7ff146800640 1 -- 172.21.15.42:0/3559997354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 msgr2=0x55ab5bdac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.540+0000 7ff146800640 1 --2- 172.21.15.42:0/3559997354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.540+0000 7ff146800640 1 -- 172.21.15.42:0/3559997354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 msgr2=0x55ab5bdbf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.540+0000 7ff146800640 1 --2- 172.21.15.42:0/3559997354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdbf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.540+0000 7ff146800640 1 -- 172.21.15.42:0/3559997354 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab5bdad8e0 con 0x55ab5bdb3800 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.540+0000 7ff146800640 1 --2- 172.21.15.42:0/3559997354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdac760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff14000e940 tx=0x7ff140005560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2ab7aa8c581286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.540+0000 7ff136600640 1 -- 172.21.15.42:0/3559997354 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14001c040 con 0x55ab5bdb3800 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.541+0000 7ff136600640 1 -- 172.21.15.42:0/3559997354 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1400b4070 con 0x55ab5bdb3800 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.541+0000 7ff136600640 1 -- 172.21.15.42:0/3559997354 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140097e40 con 0x55ab5bdb3800 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.541+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3559997354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 msgr2=0x55ab5bdac760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.541+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3559997354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdac760 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff14000e940 tx=0x7ff140005560 comp rx=0 tx=0).stop 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.541+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3559997354 shutdown_connections 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.541+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3559997354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdac760 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.541+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3559997354 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdbf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.541+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/3559997354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.541+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3559997354 >> 172.21.15.42:0/3559997354 conn(0x55ab5bda9310 msgr2=0x55ab5bdaf950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.542+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3559997354 shutdown_connections 2024-04-25T00:21:54.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.542+0000 7ff14f9c5740 1 -- 172.21.15.42:0/3559997354 wait complete. 2024-04-25T00:21:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.542+0000 7ff14f9c5740 1 -- start start 2024-04-25T00:21:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.542+0000 7ff14f9c5740 1 Processor -- start 2024-04-25T00:21:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdbc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdbce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff14f9c5740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdbde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481a92b0 con 0x55ab5bdae690 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481aa720 con 0x55ab5bdb3800 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff14f9c5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1481a9980 con 0x55ab5bdadca0 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff14f9c5740 10 client.0 _ll_register_callbacks cb 0x7ff130009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff145e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdbc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff146800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdbde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff145e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdbc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60374/0 (socket says 172.21.15.42:60374) 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff145e00640 1 -- 172.21.15.42:0/1321501082 learned_addr learned my addr 172.21.15.42:0/1321501082 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff145400640 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdbce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff145e00640 1 -- 172.21.15.42:0/1321501082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 msgr2=0x55ab5bdbde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff145e00640 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdbde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff145e00640 1 -- 172.21.15.42:0/1321501082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 msgr2=0x55ab5bdbce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff145e00640 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdbce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff145e00640 1 -- 172.21.15.42:0/1321501082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab5bda9140 con 0x55ab5bdadca0 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.543+0000 7ff146800640 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdbde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.544+0000 7ff145e00640 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdbc900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff130005ab0 tx=0x7ff130004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52676851bc4778d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.544+0000 7ff12b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.544+0000 7ff12b400640 1 -- 172.21.15.42:0/1321501082 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff13002c580 con 0x55ab5bdadca0 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.544+0000 7ff12b400640 1 -- 172.21.15.42:0/1321501082 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8182+0+0 (secure 0 0 0) 0x7ff13002c720 con 0x55ab5bdadca0 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.544+0000 7ff14f9c5740 1 -- 172.21.15.42:0/1321501082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1400b4070 con 0x55ab5bdadca0 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.544+0000 7ff12b400640 1 -- 172.21.15.42:0/1321501082 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff13002c920 con 0x55ab5bdadca0 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.544+0000 7ff14f9c5740 10 client.25441 fetch_fsmap learned FSMap version 25 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.544+0000 7ff14f9c5740 1 -- 172.21.15.42:0/1321501082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ab5bdbf650 con 0x55ab5bdadca0 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.545+0000 7ff12b400640 1 -- 172.21.15.42:0/1321501082 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1400b4070 con 0x55ab5bdadca0 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.545+0000 7ff12b400640 1 -- 172.21.15.42:0/1321501082 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7ff130033020 con 0x55ab5bdadca0 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.545+0000 7ff14f9c5740 10 client.25441 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.545+0000 7ff14f9c5740 10 client.25441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:21:54.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.545+0000 7ff14f9c5740 20 client.25441 populate_metadata read hostname 'smithi042' 2024-04-25T00:21:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.545+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ab5be2a280 0x55ab5be2c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:21:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.545+0000 7ff14f9c5740 4 client.25441 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:21:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.545+0000 7ff14f9c5740 1 -- 172.21.15.42:0/1321501082 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab5bd18340 con 0x55ab5be2a280 2024-04-25T00:21:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.545+0000 7ff145400640 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ab5be2a280 0x55ab5be2c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:21:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.546+0000 7ff145400640 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ab5be2a280 0x55ab5be2c6a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40bfd69ae2702de server_cookie=e6a14e3635a9fc50 in_seq=0 out_seq=0 2024-04-25T00:21:54.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.546+0000 7ff12b400640 10 client.25441 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.593+0000 7ff12b400640 1 -- 172.21.15.42:0/1321501082 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ab5bd18340 con 0x55ab5be2a280 2024-04-25T00:21:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff12b400640 10 client.25441 handle_command_reply: tid=0 2024-04-25T00:21:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 client.25441 shutdown 2024-04-25T00:21:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 -- 172.21.15.42:0/1321501082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 msgr2=0x55ab5bdbc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdbc900 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff130005ab0 tx=0x7ff130004730 comp rx=0 tx=0).stop 2024-04-25T00:21:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 -- 172.21.15.42:0/1321501082 shutdown_connections 2024-04-25T00:21:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ab5be2a280 0x55ab5be2c6a0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ab5bdb3800 0x55ab5bdbde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff12b400640 0 client.25441 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:21:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ab5bdae690 0x55ab5bdbce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 --2- 172.21.15.42:0/1321501082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab5bdadca0 0x55ab5bdbc900 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 -- 172.21.15.42:0/1321501082 >> 172.21.15.42:0/1321501082 conn(0x55ab5bda9310 msgr2=0x55ab5bdb7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 -- 172.21.15.42:0/1321501082 shutdown_connections 2024-04-25T00:21:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 1 -- 172.21.15.42:0/1321501082 wait complete. 2024-04-25T00:21:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.594+0000 7ff14f9c5740 20 client.25441 trim_cache size 0 max 16384 2024-04-25T00:21:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:21:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:21:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:21:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:21:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.595+0000 7ff14cc00640 1 -- 172.21.15.42:0/833667667 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff12c0a2c30 msgr2=0x7ff12c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.595+0000 7ff14cc00640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff12c0a2c30 0x7ff12c0a50f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55ab5bdaea90 tx=0x7ff130009260 comp rx=0 tx=0).stop 2024-04-25T00:21:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.595+0000 7ff14cc00640 1 -- 172.21.15.42:0/833667667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff148105580 msgr2=0x7ff1481a1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:21:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.595+0000 7ff14cc00640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff148105580 0x7ff1481a1180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff14000e970 tx=0x7ff14000ee30 comp rx=0 tx=0).stop 2024-04-25T00:21:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.596+0000 7ff14cc00640 1 -- 172.21.15.42:0/833667667 shutdown_connections 2024-04-25T00:21:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.596+0000 7ff14cc00640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff12c0a2c30 0x7ff12c0a50f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.596+0000 7ff14cc00640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff148105580 0x7ff1481a1180 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.596+0000 7ff14cc00640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff148104bc0 0x7ff14819a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.596+0000 7ff14cc00640 1 --2- 172.21.15.42:0/833667667 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff1481039d0 0x7ff148199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:21:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.596+0000 7ff14cc00640 1 -- 172.21.15.42:0/833667667 >> 172.21.15.42:0/833667667 conn(0x7ff1480fef60 msgr2=0x7ff148107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:21:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.596+0000 7ff14cc00640 1 -- 172.21.15.42:0/833667667 shutdown_connections 2024-04-25T00:21:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:21:54.596+0000 7ff14cc00640 1 -- 172.21.15.42:0/833667667 wait complete. 2024-04-25T00:21:55.213 INFO:tasks.workunit.client.0.smithi042.stdout: 1 134952 7.11 MB/sec warmup 105 sec latency 21.601 ms 2024-04-25T00:21:55.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:55 smithi136 ceph-mon[32923]: pgmap v482: 97 pgs: 97 active+clean; 45 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-04-25T00:21:55.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:55 smithi136 ceph-mon[32923]: daemon mds.i restarted 2024-04-25T00:21:55.763 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:21:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: ignoring --setuser ceph since I am not root 2024-04-25T00:21:55.764 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:21:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: ignoring --setgroup ceph since I am not root 2024-04-25T00:21:55.764 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:21:55 smithi136 ceph-mds[63607]: -- 172.21.15.136:0/3686449747 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557137f13a40 con 0x557137ee6c00 2024-04-25T00:21:55.764 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:21:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: starting mds.i at 2024-04-25T00:21:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:55 smithi106 ceph-mon[32510]: pgmap v482: 97 pgs: 97 active+clean; 45 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-04-25T00:21:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:55 smithi106 ceph-mon[32510]: daemon mds.i restarted 2024-04-25T00:21:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:55 smithi042 ceph-mon[28130]: pgmap v482: 97 pgs: 97 active+clean; 45 MiB data, 739 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-04-25T00:21:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:55 smithi042 ceph-mon[28130]: daemon mds.i restarted 2024-04-25T00:21:56.214 INFO:tasks.workunit.client.0.smithi042.stdout: 1 135555 7.06 MB/sec warmup 106 sec latency 23.326 ms 2024-04-25T00:21:56.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:56 smithi136 ceph-mon[32923]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:21:56.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:56 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6838/2800537284,v1:172.21.15.136:6839/2800537284] up:boot 2024-04-25T00:21:56.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:56 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:21:56.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:21:56.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:56 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:21:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:56 smithi106 ceph-mon[32510]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:21:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:56 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6838/2800537284,v1:172.21.15.136:6839/2800537284] up:boot 2024-04-25T00:21:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:56 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:21:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:21:56.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:56 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:21:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:56 smithi042 ceph-mon[28130]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T00:21:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:56 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6838/2800537284,v1:172.21.15.136:6839/2800537284] up:boot 2024-04-25T00:21:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:56 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:21:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:21:56.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:56 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:21:57.214 INFO:tasks.workunit.client.0.smithi042.stdout: 1 137005 7.04 MB/sec warmup 107 sec latency 20.954 ms 2024-04-25T00:21:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:57 smithi136 ceph-mon[32923]: pgmap v484: 97 pgs: 97 active+clean; 43 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-04-25T00:21:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:57 smithi106 ceph-mon[32510]: pgmap v484: 97 pgs: 97 active+clean; 43 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-04-25T00:21:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:57 smithi042 ceph-mon[28130]: pgmap v484: 97 pgs: 97 active+clean; 43 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-04-25T00:21:58.214 INFO:tasks.workunit.client.0.smithi042.stdout: 1 138665 7.10 MB/sec warmup 108 sec latency 13.963 ms 2024-04-25T00:21:59.214 INFO:tasks.workunit.client.0.smithi042.stdout: 1 139408 7.05 MB/sec warmup 109 sec latency 56.896 ms 2024-04-25T00:21:59.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:21:59 smithi136 ceph-mon[32923]: pgmap v485: 97 pgs: 97 active+clean; 43 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-04-25T00:21:59.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:21:59 smithi106 ceph-mon[32510]: pgmap v485: 97 pgs: 97 active+clean; 43 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-04-25T00:21:59.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:21:59 smithi042 ceph-mon[28130]: pgmap v485: 97 pgs: 97 active+clean; 43 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 213 op/s 2024-04-25T00:22:00.214 INFO:tasks.workunit.client.0.smithi042.stdout: 1 141119 7.07 MB/sec warmup 110 sec latency 28.354 ms 2024-04-25T00:22:00.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:00 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:00.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:00 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:00.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:00 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:22:01.214 INFO:tasks.workunit.client.0.smithi042.stdout: 1 142355 7.08 MB/sec warmup 111 sec latency 13.227 ms 2024-04-25T00:22:01.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:01 smithi136 ceph-mon[32923]: pgmap v486: 97 pgs: 97 active+clean; 43 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2024-04-25T00:22:01.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:01.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:01.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:22:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:01 smithi106 ceph-mon[32510]: pgmap v486: 97 pgs: 97 active+clean; 43 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2024-04-25T00:22:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:22:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:01 smithi042 ceph-mon[28130]: pgmap v486: 97 pgs: 97 active+clean; 43 MiB data, 702 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.5 MiB/s wr, 216 op/s 2024-04-25T00:22:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:22:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:22:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:22:02.214 INFO:tasks.workunit.client.0.smithi042.stdout: 1 143441 7.06 MB/sec warmup 112 sec latency 26.652 ms 2024-04-25T00:22:03.214 INFO:tasks.workunit.client.0.smithi042.stdout: 1 145286 7.08 MB/sec warmup 113 sec latency 18.939 ms 2024-04-25T00:22:03.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:03 smithi136 ceph-mon[32923]: pgmap v487: 97 pgs: 97 active+clean; 46 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-04-25T00:22:03.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:03 smithi106 ceph-mon[32510]: pgmap v487: 97 pgs: 97 active+clean; 46 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-04-25T00:22:03.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:03 smithi042 ceph-mon[28130]: pgmap v487: 97 pgs: 97 active+clean; 46 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 220 op/s 2024-04-25T00:22:04.214 INFO:tasks.workunit.client.0.smithi042.stdout: 1 146009 7.06 MB/sec warmup 114 sec latency 50.457 ms 2024-04-25T00:22:05.215 INFO:tasks.workunit.client.0.smithi042.stdout: 1 147118 7.05 MB/sec warmup 115 sec latency 26.236 ms 2024-04-25T00:22:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:05 smithi136 ceph-mon[32923]: pgmap v488: 97 pgs: 97 active+clean; 46 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-04-25T00:22:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:05 smithi106 ceph-mon[32510]: pgmap v488: 97 pgs: 97 active+clean; 46 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-04-25T00:22:05.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:05 smithi042 ceph-mon[28130]: pgmap v488: 97 pgs: 97 active+clean; 46 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 221 op/s 2024-04-25T00:22:06.215 INFO:tasks.workunit.client.0.smithi042.stdout: 1 149000 7.09 MB/sec warmup 116 sec latency 68.711 ms 2024-04-25T00:22:07.215 INFO:tasks.workunit.client.0.smithi042.stdout: 1 149594 7.04 MB/sec warmup 117 sec latency 18.714 ms 2024-04-25T00:22:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.743+0000 7f63b5800640 1 -- 172.21.15.42:0/564180112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b0103800 msgr2=0x7f63b0103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.743+0000 7f63b5800640 1 --2- 172.21.15.42:0/564180112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b0103800 0x7f63b0103c20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f63a8009be0 tx=0x7f63a802a6d0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.743+0000 7f63b5800640 1 -- 172.21.15.42:0/564180112 shutdown_connections 2024-04-25T00:22:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.743+0000 7f63b5800640 1 --2- 172.21.15.42:0/564180112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b01053b0 0x7f63b010bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.743+0000 7f63b5800640 1 --2- 172.21.15.42:0/564180112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63b01049f0 0x7f63b0104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.743+0000 7f63b5800640 1 --2- 172.21.15.42:0/564180112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b0103800 0x7f63b0103c20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.743+0000 7f63b5800640 1 -- 172.21.15.42:0/564180112 >> 172.21.15.42:0/564180112 conn(0x7f63b00fedb0 msgr2=0x7f63b01011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.743+0000 7f63b5800640 1 -- 172.21.15.42:0/564180112 shutdown_connections 2024-04-25T00:22:07.744 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.743+0000 7f63b5800640 1 -- 172.21.15.42:0/564180112 wait complete. 2024-04-25T00:22:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.744+0000 7f63b5800640 1 Processor -- start 2024-04-25T00:22:07.745 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.744+0000 7f63b5800640 1 -- start start 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.744+0000 7f63b5800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63b0103800 0x7f63b0199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.744+0000 7f63b5800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b01049f0 0x7f63b019a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.744+0000 7f63b5800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b01053b0 0x7f63b01a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.744+0000 7f63b5800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010f450 con 0x7f63b01049f0 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.744+0000 7f63b5800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010f2d0 con 0x7f63b01053b0 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.744+0000 7f63b5800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63b010f5d0 con 0x7f63b0103800 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63afe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63b0103800 0x7f63b0199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63afe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63b0103800 0x7f63b0199b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41576/0 (socket says 172.21.15.42:41576) 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63afe00640 1 -- 172.21.15.42:0/1177217496 learned_addr learned my addr 172.21.15.42:0/1177217496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63b4e00640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b01053b0 0x7f63b01a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63af400640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b01049f0 0x7f63b019a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63afe00640 1 -- 172.21.15.42:0/1177217496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b01053b0 msgr2=0x7f63b01a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63afe00640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b01053b0 0x7f63b01a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63afe00640 1 -- 172.21.15.42:0/1177217496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b01049f0 msgr2=0x7f63b019a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63afe00640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b01049f0 0x7f63b019a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63afe00640 1 -- 172.21.15.42:0/1177217496 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b01a1750 con 0x7f63b0103800 2024-04-25T00:22:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.745+0000 7f63afe00640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63b0103800 0x7f63b0199b30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f63a802acc0 tx=0x7f63a80305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.746+0000 7f63acc00640 1 -- 172.21.15.42:0/1177217496 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8041020 con 0x7f63b0103800 2024-04-25T00:22:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.746+0000 7f63acc00640 1 -- 172.21.15.42:0/1177217496 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a8031920 con 0x7f63b0103800 2024-04-25T00:22:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.746+0000 7f63acc00640 1 -- 172.21.15.42:0/1177217496 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a8036040 con 0x7f63b0103800 2024-04-25T00:22:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.746+0000 7f63b5800640 1 -- 172.21.15.42:0/1177217496 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b01a19e0 con 0x7f63b0103800 2024-04-25T00:22:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.746+0000 7f63b5800640 1 -- 172.21.15.42:0/1177217496 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b01a1e80 con 0x7f63b0103800 2024-04-25T00:22:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.749+0000 7f63acc00640 1 -- 172.21.15.42:0/1177217496 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f63a8044020 con 0x7f63b0103800 2024-04-25T00:22:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.749+0000 7f63b9c43740 1 Processor -- start 2024-04-25T00:22:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.749+0000 7f63b9c43740 1 -- start start 2024-04-25T00:22:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d1549e50 0x55a0d154a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154a840 0x55a0d154f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 0x55a0d1551e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63b9c43740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a0d14acab0 con 0x55a0d154a840 2024-04-25T00:22:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63b9c43740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a0d1455640 con 0x55a0d154f9b0 2024-04-25T00:22:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63b9c43740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0d14a8a70 con 0x55a0d1549e50 2024-04-25T00:22:07.751 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63b4e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d1549e50 0x55a0d154a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154a840 0x55a0d154f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63af400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 0x55a0d1551e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63b4e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d1549e50 0x55a0d154a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41584/0 (socket says 172.21.15.42:41584) 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.750+0000 7f63b4e00640 1 -- 172.21.15.42:0/2285523182 learned_addr learned my addr 172.21.15.42:0/2285523182 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63b4e00640 1 -- 172.21.15.42:0/2285523182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 msgr2=0x55a0d1551e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63b4e00640 1 --2- 172.21.15.42:0/2285523182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 0x55a0d1551e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63b4e00640 1 -- 172.21.15.42:0/2285523182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154a840 msgr2=0x55a0d154f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63b4e00640 1 --2- 172.21.15.42:0/2285523182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154a840 0x55a0d154f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63b4e00640 1 -- 172.21.15.42:0/2285523182 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d11c1210 con 0x55a0d1549e50 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63af400640 1 --2- 172.21.15.42:0/2285523182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 0x55a0d1551e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:07.752 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63b4e00640 1 --2- 172.21.15.42:0/2285523182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d1549e50 0x55a0d154a270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009c90 tx=0x7f63a0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=edd94b62f266f82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63acc00640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f63940a2c00 0x7f63940a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63a5600640 1 -- 172.21.15.42:0/2285523182 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a0004230 con 0x55a0d1549e50 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63a5600640 1 -- 172.21.15.42:0/2285523182 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63a0009e40 con 0x55a0d1549e50 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63a5600640 1 -- 172.21.15.42:0/2285523182 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63a00197e0 con 0x55a0d1549e50 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.751+0000 7f63acc00640 1 -- 172.21.15.42:0/1177217496 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f63a80efda0 con 0x7f63b0103800 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63afe00640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f63940a2c00 0x7f63940a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63b9c43740 1 -- 172.21.15.42:0/2285523182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d1549e50 msgr2=0x55a0d154a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63b9c43740 1 --2- 172.21.15.42:0/2285523182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d1549e50 0x55a0d154a270 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009c90 tx=0x7f63a0011810 comp rx=0 tx=0).stop 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63afe00640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f63940a2c00 0x7f63940a50c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63a8006010 tx=0x7f63a80f37e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63b9c43740 1 -- 172.21.15.42:0/2285523182 shutdown_connections 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63b9c43740 1 --2- 172.21.15.42:0/2285523182 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 0x55a0d1551e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63b9c43740 1 --2- 172.21.15.42:0/2285523182 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154a840 0x55a0d154f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63b9c43740 1 --2- 172.21.15.42:0/2285523182 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d1549e50 0x55a0d154a270 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63b9c43740 1 -- 172.21.15.42:0/2285523182 >> 172.21.15.42:0/2285523182 conn(0x55a0d15454c0 msgr2=0x55a0d15458c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63b9c43740 1 -- 172.21.15.42:0/2285523182 shutdown_connections 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.752+0000 7f63b9c43740 1 -- 172.21.15.42:0/2285523182 wait complete. 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.753+0000 7f63b9c43740 1 -- start start 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.753+0000 7f63b9c43740 1 Processor -- start 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 0x55a0d15540f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d1554630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 0x55a0d155b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b9c43740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a0d1552960 con 0x55a0d154f9b0 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b9c43740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a0d1468990 con 0x55a0d1549e50 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b9c43740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0d1552ae0 con 0x55a0d154a840 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b9c43740 10 client.0 _ll_register_callbacks cb 0x55a0d146a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b4e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 0x55a0d15540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63af400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d1554630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b4e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 0x55a0d15540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:47430/0 (socket says 172.21.15.42:47430) 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b4e00640 1 -- 172.21.15.42:0/1735417876 learned_addr learned my addr 172.21.15.42:0/1735417876 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63b4e00640 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 0x55a0d155b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63af400640 1 -- 172.21.15.42:0/1735417876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 msgr2=0x55a0d15540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63af400640 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 0x55a0d15540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63af400640 1 -- 172.21.15.42:0/1735417876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 msgr2=0x55a0d155b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63af400640 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 0x55a0d155b670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.754+0000 7f63af400640 1 -- 172.21.15.42:0/1735417876 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d128d620 con 0x55a0d154a840 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.755+0000 7f63af400640 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d1554630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63b019b040 tx=0x7f6398011b30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c218737a9e2caa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.755+0000 7f639e000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.755+0000 7f639e000640 1 -- 172.21.15.42:0/1735417876 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639801e060 con 0x55a0d154a840 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.755+0000 7f639e000640 1 -- 172.21.15.42:0/1735417876 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f639800fb60 con 0x55a0d154a840 2024-04-25T00:22:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.755+0000 7f63b9c43740 1 -- 172.21.15.42:0/1735417876 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a0d14b9ff0 con 0x55a0d154a840 2024-04-25T00:22:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.755+0000 7f639e000640 1 -- 172.21.15.42:0/1735417876 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f639800fd60 con 0x55a0d154a840 2024-04-25T00:22:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.755+0000 7f63b9c43740 10 client.25459 fetch_fsmap learned FSMap version 27 2024-04-25T00:22:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.755+0000 7f63b9c43740 1 -- 172.21.15.42:0/1735417876 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0d155e720 con 0x55a0d154a840 2024-04-25T00:22:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.756+0000 7f639e000640 1 -- 172.21.15.42:0/1735417876 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a0d14b9ff0 con 0x55a0d154a840 2024-04-25T00:22:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.756+0000 7f639e000640 1 -- 172.21.15.42:0/1735417876 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6398019070 con 0x55a0d154a840 2024-04-25T00:22:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.756+0000 7f63b9c43740 10 client.25459 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:22:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.756+0000 7f63b9c43740 10 client.25459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.756+0000 7f63b9c43740 20 client.25459 populate_metadata read hostname 'smithi042' 2024-04-25T00:22:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.756+0000 7f63b9c43740 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0d155fed0 0x55a0d15622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.756+0000 7f63b9c43740 4 client.25459 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.756+0000 7f63b9c43740 1 -- 172.21.15.42:0/1735417876 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0d14b4340 con 0x55a0d155fed0 2024-04-25T00:22:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.756+0000 7f63b4e00640 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0d155fed0 0x55a0d15622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.757+0000 7f63b4e00640 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0d155fed0 0x55a0d15622c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4a00b4cdc0f15d9 server_cookie=8a17b7268d95fdd2 in_seq=0 out_seq=0 2024-04-25T00:22:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.757+0000 7f639e000640 10 client.25459 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.758+0000 7f639e000640 1 -- 172.21.15.42:0/1735417876 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0d14b4340 con 0x55a0d155fed0 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.758+0000 7f639e000640 10 client.25459 handle_command_reply: tid=0 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.758+0000 7f63b9c43740 1 client.25459 shutdown 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 1 -- 172.21.15.42:0/1735417876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 msgr2=0x55a0d1554630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d1554630 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f63b019b040 tx=0x7f6398011b30 comp rx=0 tx=0).stop 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 1 -- 172.21.15.42:0/1735417876 shutdown_connections 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0d155fed0 0x55a0d15622c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 0x55a0d155b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d1554630 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 1 --2- 172.21.15.42:0/1735417876 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 0x55a0d15540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f639e000640 0 client.25459 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 1 -- 172.21.15.42:0/1735417876 >> 172.21.15.42:0/1735417876 conn(0x55a0d15454c0 msgr2=0x55a0d1544620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 1 -- 172.21.15.42:0/1735417876 shutdown_connections 2024-04-25T00:22:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 1 -- 172.21.15.42:0/1735417876 wait complete. 2024-04-25T00:22:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.759+0000 7f63b9c43740 20 client.25459 trim_cache size 0 max 16384 2024-04-25T00:22:07.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:07 smithi136 ceph-mon[32923]: pgmap v489: 97 pgs: 97 active+clean; 48 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2024-04-25T00:22:07.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.776+0000 7f63b9c43740 1 Processor -- start 2024-04-25T00:22:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63b9c43740 1 -- start start 2024-04-25T00:22:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d1549e50 0x55a0d1552eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d155b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 0x55a0d15484b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63b9c43740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01ad550 con 0x55a0d1549e50 2024-04-25T00:22:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63b9c43740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aadc0 con 0x55a0d154f9b0 2024-04-25T00:22:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63b9c43740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01a8c10 con 0x55a0d154a840 2024-04-25T00:22:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63b4e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d1549e50 0x55a0d1552eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63af400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d155b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63af400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d155b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41594/0 (socket says 172.21.15.42:41594) 2024-04-25T00:22:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63af400640 1 -- 172.21.15.42:0/3923000753 learned_addr learned my addr 172.21.15.42:0/3923000753 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:07.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.777+0000 7f63b4e00640 1 --2- 172.21.15.42:0/3923000753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 0x55a0d15484b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.778+0000 7f63af400640 1 -- 172.21.15.42:0/3923000753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 msgr2=0x55a0d15484b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.778+0000 7f63af400640 1 --2- 172.21.15.42:0/3923000753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 0x55a0d15484b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.778+0000 7f63af400640 1 -- 172.21.15.42:0/3923000753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d1549e50 msgr2=0x55a0d1552eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.778+0000 7f63af400640 1 --2- 172.21.15.42:0/3923000753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d1549e50 0x55a0d1552eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.778+0000 7f63af400640 1 -- 172.21.15.42:0/3923000753 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d1562940 con 0x55a0d154a840 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.778+0000 7f63b4e00640 1 --2- 172.21.15.42:0/3923000753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d1549e50 0x55a0d1552eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.778+0000 7f63af400640 1 --2- 172.21.15.42:0/3923000753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d155b630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6398009f20 tx=0x7f639800fcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d54f4f9bbead4263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.778+0000 7f63a4c00640 1 -- 172.21.15.42:0/3923000753 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398003fa0 con 0x55a0d154a840 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.778+0000 7f63a4c00640 1 -- 172.21.15.42:0/3923000753 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6398005510 con 0x55a0d154a840 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.778+0000 7f63a4c00640 1 -- 172.21.15.42:0/3923000753 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398003a20 con 0x55a0d154a840 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.779+0000 7f63b9c43740 1 -- 172.21.15.42:0/3923000753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 msgr2=0x55a0d155b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.779+0000 7f63b9c43740 1 --2- 172.21.15.42:0/3923000753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d155b630 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6398009f20 tx=0x7f639800fcf0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.779+0000 7f63b9c43740 1 -- 172.21.15.42:0/3923000753 shutdown_connections 2024-04-25T00:22:07.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.779+0000 7f63b9c43740 1 --2- 172.21.15.42:0/3923000753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d154f9b0 0x55a0d15484b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.779+0000 7f63b9c43740 1 --2- 172.21.15.42:0/3923000753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d155b630 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.779+0000 7f63b9c43740 1 --2- 172.21.15.42:0/3923000753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d1549e50 0x55a0d1552eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.779+0000 7f63b9c43740 1 -- 172.21.15.42:0/3923000753 >> 172.21.15.42:0/3923000753 conn(0x55a0d15454c0 msgr2=0x55a0d154bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.779+0000 7f63b9c43740 1 -- 172.21.15.42:0/3923000753 shutdown_connections 2024-04-25T00:22:07.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.779+0000 7f63b9c43740 1 -- 172.21.15.42:0/3923000753 wait complete. 2024-04-25T00:22:07.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.780+0000 7f63b9c43740 1 -- start start 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.780+0000 7f63b9c43740 1 Processor -- start 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.780+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 0x55a0d1558d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d15592d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63b9c43740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 0x55a0d155a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63b9c43740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01a9280 con 0x55a0d154f9b0 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63b9c43740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01aa6f0 con 0x55a0d1549e50 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63b9c43740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01a9950 con 0x55a0d154a840 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63b9c43740 10 client.0 _ll_register_callbacks cb 0x7f6398009890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63b4e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 0x55a0d1558d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63af400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d15592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63af400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d15592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41608/0 (socket says 172.21.15.42:41608) 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63af400640 1 -- 172.21.15.42:0/3500966194 learned_addr learned my addr 172.21.15.42:0/3500966194 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:07.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63b4e00640 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 0x55a0d155a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63af400640 1 -- 172.21.15.42:0/3500966194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 msgr2=0x55a0d1558d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63af400640 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 0x55a0d1558d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63af400640 1 -- 172.21.15.42:0/3500966194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 msgr2=0x55a0d155a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63af400640 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 0x55a0d155a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63af400640 1 -- 172.21.15.42:0/3500966194 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d155a8d0 con 0x55a0d154a840 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.781+0000 7f63b4e00640 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 0x55a0d155a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.782+0000 7f63af400640 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d15592d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6398009280 tx=0x7f6398001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5051a4c7feed52cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.782+0000 7f639e000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.782+0000 7f639e000640 1 -- 172.21.15.42:0/3500966194 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639803a020 con 0x55a0d154a840 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.782+0000 7f639e000640 1 -- 172.21.15.42:0/3500966194 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6398033070 con 0x55a0d154a840 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.782+0000 7f63b9c43740 1 -- 172.21.15.42:0/3500966194 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6398005510 con 0x55a0d154a840 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.782+0000 7f639e000640 1 -- 172.21.15.42:0/3500966194 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f63980042f0 con 0x55a0d154a840 2024-04-25T00:22:07.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.782+0000 7f63b9c43740 10 client.25465 fetch_fsmap learned FSMap version 27 2024-04-25T00:22:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.782+0000 7f63b9c43740 1 -- 172.21.15.42:0/3500966194 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a0d15c6500 con 0x55a0d154a840 2024-04-25T00:22:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.783+0000 7f639e000640 1 -- 172.21.15.42:0/3500966194 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6398005510 con 0x55a0d154a840 2024-04-25T00:22:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.783+0000 7f639e000640 1 -- 172.21.15.42:0/3500966194 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6398019070 con 0x55a0d154a840 2024-04-25T00:22:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.783+0000 7f63b9c43740 10 client.25465 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:22:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.783+0000 7f63b9c43740 10 client.25465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.783+0000 7f63b9c43740 20 client.25465 populate_metadata read hostname 'smithi042' 2024-04-25T00:22:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.783+0000 7f63b9c43740 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0d15c6e60 0x55a0d15c9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.783+0000 7f63b9c43740 4 client.25465 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:22:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.783+0000 7f63b9c43740 1 -- 172.21.15.42:0/3500966194 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0d14327a0 con 0x55a0d15c6e60 2024-04-25T00:22:07.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.784+0000 7f63b4e00640 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0d15c6e60 0x55a0d15c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.784+0000 7f63b4e00640 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0d15c6e60 0x55a0d15c9280 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=633c8266a639d32a server_cookie=d4886c553ee27bb6 in_seq=0 out_seq=0 2024-04-25T00:22:07.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.784+0000 7f639e000640 10 client.25465 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.828+0000 7f639e000640 1 -- 172.21.15.42:0/3500966194 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0d14327a0 con 0x55a0d15c6e60 2024-04-25T00:22:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.828+0000 7f639e000640 10 client.25465 handle_command_reply: tid=0 2024-04-25T00:22:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.828+0000 7f63b9c43740 1 client.25465 shutdown 2024-04-25T00:22:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 1 -- 172.21.15.42:0/3500966194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 msgr2=0x55a0d15592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d15592d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6398009280 tx=0x7f6398001e50 comp rx=0 tx=0).stop 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 1 -- 172.21.15.42:0/3500966194 shutdown_connections 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0d15c6e60 0x55a0d15c9280 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0d154f9b0 0x55a0d155a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d154a840 0x55a0d15592d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 1 --2- 172.21.15.42:0/3500966194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0d1549e50 0x55a0d1558d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f639e000640 0 client.25465 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 1 -- 172.21.15.42:0/3500966194 >> 172.21.15.42:0/3500966194 conn(0x55a0d15454c0 msgr2=0x55a0d15484b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 1 -- 172.21.15.42:0/3500966194 shutdown_connections 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 1 -- 172.21.15.42:0/3500966194 wait complete. 2024-04-25T00:22:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.829+0000 7f63b9c43740 20 client.25465 trim_cache size 0 max 16384 2024-04-25T00:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 -- 172.21.15.42:0/1177217496 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f63940a2c00 msgr2=0x7f63940a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f63940a2c00 0x7f63940a50c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63a8006010 tx=0x7f63a80f37e0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 -- 172.21.15.42:0/1177217496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63b0103800 msgr2=0x7f63b0199b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63b0103800 0x7f63b0199b30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f63a802acc0 tx=0x7f63a80305d0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 -- 172.21.15.42:0/1177217496 shutdown_connections 2024-04-25T00:22:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f63940a2c00 0x7f63940a50c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b01053b0 0x7f63b01a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f63b01049f0 0x7f63b019a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 --2- 172.21.15.42:0/1177217496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f63b0103800 0x7f63b0199b30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 -- 172.21.15.42:0/1177217496 >> 172.21.15.42:0/1177217496 conn(0x7f63b00fedb0 msgr2=0x7f63b0107d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 -- 172.21.15.42:0/1177217496 shutdown_connections 2024-04-25T00:22:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:07.830+0000 7f63b5800640 1 -- 172.21.15.42:0/1177217496 wait complete. 2024-04-25T00:22:07.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:07 smithi106 ceph-mon[32510]: pgmap v489: 97 pgs: 97 active+clean; 48 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2024-04-25T00:22:07.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:07 smithi042 ceph-mon[28130]: pgmap v489: 97 pgs: 97 active+clean; 48 MiB data, 730 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.8 MiB/s wr, 251 op/s 2024-04-25T00:22:08.215 INFO:tasks.workunit.client.0.smithi042.stdout: 1 150967 7.03 MB/sec warmup 118 sec latency 37.975 ms 2024-04-25T00:22:09.215 INFO:tasks.workunit.client.0.smithi042.stdout: 1 152758 7.08 MB/sec warmup 119 sec latency 102.455 ms 2024-04-25T00:22:09.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:09 smithi136 ceph-mon[32923]: pgmap v490: 97 pgs: 97 active+clean; 48 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:22:09.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:09 smithi106 ceph-mon[32510]: pgmap v490: 97 pgs: 97 active+clean; 48 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:22:09.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:09 smithi042 ceph-mon[28130]: pgmap v490: 97 pgs: 97 active+clean; 48 MiB data, 719 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:22:11.215 INFO:tasks.workunit.client.0.smithi042.stdout: 1 154699 5.32 MB/sec execute 1 sec latency 27.773 ms 2024-04-25T00:22:11.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:11 smithi136 ceph-mon[32923]: pgmap v491: 97 pgs: 97 active+clean; 48 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-04-25T00:22:11.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:11 smithi106 ceph-mon[32510]: pgmap v491: 97 pgs: 97 active+clean; 48 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-04-25T00:22:11.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:11 smithi042 ceph-mon[28130]: pgmap v491: 97 pgs: 97 active+clean; 48 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-04-25T00:22:12.215 INFO:tasks.workunit.client.0.smithi042.stdout: 1 156362 9.20 MB/sec execute 2 sec latency 14.251 ms 2024-04-25T00:22:13.216 INFO:tasks.workunit.client.0.smithi042.stdout: 1 157152 6.88 MB/sec execute 3 sec latency 20.423 ms 2024-04-25T00:22:13.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:13 smithi136 ceph-mon[32923]: pgmap v492: 97 pgs: 97 active+clean; 51 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 256 op/s 2024-04-25T00:22:13.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:13 smithi106 ceph-mon[32510]: pgmap v492: 97 pgs: 97 active+clean; 51 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 256 op/s 2024-04-25T00:22:13.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:13 smithi042 ceph-mon[28130]: pgmap v492: 97 pgs: 97 active+clean; 51 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 256 op/s 2024-04-25T00:22:14.216 INFO:tasks.workunit.client.0.smithi042.stdout: 1 158702 6.69 MB/sec execute 4 sec latency 39.191 ms 2024-04-25T00:22:15.216 INFO:tasks.workunit.client.0.smithi042.stdout: 1 160056 7.56 MB/sec execute 5 sec latency 17.105 ms 2024-04-25T00:22:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:15 smithi136 ceph-mon[32923]: pgmap v493: 97 pgs: 97 active+clean; 51 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-04-25T00:22:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:15 smithi106 ceph-mon[32510]: pgmap v493: 97 pgs: 97 active+clean; 51 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-04-25T00:22:15.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:15 smithi042 ceph-mon[28130]: pgmap v493: 97 pgs: 97 active+clean; 51 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-04-25T00:22:16.216 INFO:tasks.workunit.client.0.smithi042.stdout: 1 161111 7.15 MB/sec execute 6 sec latency 30.077 ms 2024-04-25T00:22:17.216 INFO:tasks.workunit.client.0.smithi042.stdout: 1 162909 7.46 MB/sec execute 7 sec latency 21.245 ms 2024-04-25T00:22:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:17 smithi136 ceph-mon[32923]: pgmap v494: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-04-25T00:22:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:17 smithi106 ceph-mon[32510]: pgmap v494: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-04-25T00:22:17.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:17 smithi042 ceph-mon[28130]: pgmap v494: 97 pgs: 97 active+clean; 53 MiB data, 755 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.0 MiB/s wr, 248 op/s 2024-04-25T00:22:18.216 INFO:tasks.workunit.client.0.smithi042.stdout: 1 163776 7.12 MB/sec execute 8 sec latency 17.809 ms 2024-04-25T00:22:19.216 INFO:tasks.workunit.client.0.smithi042.stdout: 1 164875 6.91 MB/sec execute 9 sec latency 49.191 ms 2024-04-25T00:22:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:19 smithi136 ceph-mon[32923]: pgmap v495: 97 pgs: 97 active+clean; 53 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-04-25T00:22:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:19 smithi106 ceph-mon[32510]: pgmap v495: 97 pgs: 97 active+clean; 53 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-04-25T00:22:19.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:19 smithi042 ceph-mon[28130]: pgmap v495: 97 pgs: 97 active+clean; 53 MiB data, 760 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-04-25T00:22:20.216 INFO:tasks.workunit.client.0.smithi042.stdout: 1 166818 7.46 MB/sec execute 10 sec latency 18.797 ms 2024-04-25T00:22:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.981+0000 7fc9f5600640 1 -- 172.21.15.42:0/2015329298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9f01039c0 msgr2=0x7fc9f0103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.981+0000 7fc9f5600640 1 --2- 172.21.15.42:0/2015329298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9f01039c0 0x7fc9f0103de0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009ca0 tx=0x7fc9e002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.981+0000 7fc9f5600640 1 -- 172.21.15.42:0/2015329298 shutdown_connections 2024-04-25T00:22:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.981+0000 7fc9f5600640 1 --2- 172.21.15.42:0/2015329298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9f0105570 0x7fc9f010be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.981+0000 7fc9f5600640 1 --2- 172.21.15.42:0/2015329298 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9f0104bb0 0x7fc9f0105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.981+0000 7fc9f5600640 1 --2- 172.21.15.42:0/2015329298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9f01039c0 0x7fc9f0103de0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.981+0000 7fc9f5600640 1 -- 172.21.15.42:0/2015329298 >> 172.21.15.42:0/2015329298 conn(0x7fc9f00fef30 msgr2=0x7fc9f0101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.981+0000 7fc9f5600640 1 -- 172.21.15.42:0/2015329298 shutdown_connections 2024-04-25T00:22:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.981+0000 7fc9f5600640 1 -- 172.21.15.42:0/2015329298 wait complete. 2024-04-25T00:22:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.982+0000 7fc9f5600640 1 Processor -- start 2024-04-25T00:22:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.982+0000 7fc9f5600640 1 -- start start 2024-04-25T00:22:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9f5600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9f01039c0 0x7fc9f0199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9f5600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9f0104bb0 0x7fc9f019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9f5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9f0105570 0x7fc9f01a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9f5600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f010f660 con 0x7fc9f0105570 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9f5600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f010f4e0 con 0x7fc9f01039c0 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9f5600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f010f7e0 con 0x7fc9f0104bb0 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9f0104bb0 0x7fc9f019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9f0104bb0 0x7fc9f019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33716/0 (socket says 172.21.15.42:33716) 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9f01039c0 0x7fc9f0199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9ee000640 1 -- 172.21.15.42:0/241645621 learned_addr learned my addr 172.21.15.42:0/241645621 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9ef400640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9f0105570 0x7fc9f01a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9ee000640 1 -- 172.21.15.42:0/241645621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9f01039c0 msgr2=0x7fc9f0199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9ee000640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9f01039c0 0x7fc9f0199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9ee000640 1 -- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9f0105570 msgr2=0x7fc9f01a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9ee000640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9f0105570 0x7fc9f01a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.983+0000 7fc9ee000640 1 -- 172.21.15.42:0/241645621 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f01a1850 con 0x7fc9f0104bb0 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.984+0000 7fc9ef400640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9f0105570 0x7fc9f01a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:20.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.984+0000 7fc9ee000640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9f0104bb0 0x7fc9f019a180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00d950 tx=0x7fc9dc00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.984+0000 7fc9e7400640 1 -- 172.21.15.42:0/241645621 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc017030 con 0x7fc9f0104bb0 2024-04-25T00:22:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.984+0000 7fc9e7400640 1 -- 172.21.15.42:0/241645621 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9dc004020 con 0x7fc9f0104bb0 2024-04-25T00:22:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.984+0000 7fc9e7400640 1 -- 172.21.15.42:0/241645621 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc0047a0 con 0x7fc9f0104bb0 2024-04-25T00:22:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.984+0000 7fc9f5600640 1 -- 172.21.15.42:0/241645621 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f01a1b40 con 0x7fc9f0104bb0 2024-04-25T00:22:20.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.984+0000 7fc9f5600640 1 -- 172.21.15.42:0/241645621 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f01a2010 con 0x7fc9f0104bb0 2024-04-25T00:22:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.989+0000 7fc9f8406740 1 Processor -- start 2024-04-25T00:22:20.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.989+0000 7fc9f8406740 1 -- start start 2024-04-25T00:22:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f110c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f16190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f18c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9f8406740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8e73ab0 con 0x55c5a8f10ca0 2024-04-25T00:22:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9f8406740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8e1c640 con 0x55c5a8f16800 2024-04-25T00:22:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9f8406740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8e6fa70 con 0x55c5a8f11690 2024-04-25T00:22:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f16190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f16190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33728/0 (socket says 172.21.15.42:33728) 2024-04-25T00:22:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9eea00640 1 -- 172.21.15.42:0/3959067534 learned_addr learned my addr 172.21.15.42:0/3959067534 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9ef400640 1 --2- 172.21.15.42:0/3959067534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.990+0000 7fc9e7400640 1 -- 172.21.15.42:0/241645621 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc9dc0041d0 con 0x7fc9f0104bb0 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9eea00640 1 -- 172.21.15.42:0/3959067534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 msgr2=0x55c5a8f18c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9ee000640 1 --2- 172.21.15.42:0/3959067534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f18c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9eea00640 1 --2- 172.21.15.42:0/3959067534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f18c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9eea00640 1 -- 172.21.15.42:0/3959067534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 msgr2=0x55c5a8f110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9eea00640 1 --2- 172.21.15.42:0/3959067534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9eea00640 1 -- 172.21.15.42:0/3959067534 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a8b88210 con 0x55c5a8f11690 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9ee000640 1 --2- 172.21.15.42:0/3959067534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f18c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9eea00640 1 --2- 172.21.15.42:0/3959067534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f16190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0031c10 tx=0x7fc9e0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b8f38dca17f92bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9db400640 1 -- 172.21.15.42:0/3959067534 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0031650 con 0x55c5a8f11690 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9db400640 1 -- 172.21.15.42:0/3959067534 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e0036030 con 0x55c5a8f11690 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.991+0000 7fc9db400640 1 -- 172.21.15.42:0/3959067534 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0004830 con 0x55c5a8f11690 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9e7400640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9d40a2cb0 0x7fc9d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9f8406740 1 -- 172.21.15.42:0/3959067534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 msgr2=0x55c5a8f16190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:20.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9f8406740 1 --2- 172.21.15.42:0/3959067534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f16190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0031c10 tx=0x7fc9e0009920 comp rx=0 tx=0).stop 2024-04-25T00:22:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9e7400640 1 -- 172.21.15.42:0/241645621 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fc9dc0ceaa0 con 0x7fc9f0104bb0 2024-04-25T00:22:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9f8406740 1 -- 172.21.15.42:0/3959067534 shutdown_connections 2024-04-25T00:22:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9f8406740 1 --2- 172.21.15.42:0/3959067534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f18c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9f8406740 1 --2- 172.21.15.42:0/3959067534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f16190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9f8406740 1 --2- 172.21.15.42:0/3959067534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9f8406740 1 -- 172.21.15.42:0/3959067534 >> 172.21.15.42:0/3959067534 conn(0x55c5a8f0c310 msgr2=0x55c5a8f0c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9ef400640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9d40a2cb0 0x7fc9d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9f8406740 1 -- 172.21.15.42:0/3959067534 shutdown_connections 2024-04-25T00:22:20.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.992+0000 7fc9f8406740 1 -- 172.21.15.42:0/3959067534 wait complete. 2024-04-25T00:22:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.993+0000 7fc9f8406740 1 -- start start 2024-04-25T00:22:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.993+0000 7fc9ef400640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9d40a2cb0 0x7fc9d40a5170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55c5a8f11ef0 tx=0x7fc9e800d4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.993+0000 7fc9f8406740 1 Processor -- start 2024-04-25T00:22:20.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f1ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f11690 0x55c5a8f1b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f222a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9f8406740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8f195d0 con 0x55c5a8f11690 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9f8406740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8e2f990 con 0x55c5a8f16800 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9f8406740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c5a8f19750 con 0x55c5a8f10ca0 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9f8406740 10 client.0 _ll_register_callbacks cb 0x55c5a8e31950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f1ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f1ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33744/0 (socket says 172.21.15.42:33744) 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9eea00640 1 -- 172.21.15.42:0/3334401737 learned_addr learned my addr 172.21.15.42:0/3334401737 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9ef400640 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9ee000640 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f11690 0x55c5a8f1b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:20.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9eea00640 1 -- 172.21.15.42:0/3334401737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 msgr2=0x55c5a8f222a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.994+0000 7fc9eea00640 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f222a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.995+0000 7fc9eea00640 1 -- 172.21.15.42:0/3334401737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f11690 msgr2=0x55c5a8f1b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.995+0000 7fc9eea00640 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f11690 0x55c5a8f1b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.995+0000 7fc9eea00640 1 -- 172.21.15.42:0/3334401737 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a8c54620 con 0x55c5a8f10ca0 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.995+0000 7fc9ef400640 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.995+0000 7fc9eea00640 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f1ad90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009ca0 tx=0x7fc9e0031680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8206918cf2634907 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.995+0000 7fc9cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.995+0000 7fc9cfe00640 1 -- 172.21.15.42:0/3334401737 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e004a040 con 0x55c5a8f10ca0 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.995+0000 7fc9cfe00640 1 -- 172.21.15.42:0/3334401737 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fc9e0003970 con 0x55c5a8f10ca0 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.995+0000 7fc9f8406740 1 -- 172.21.15.42:0/3334401737 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e0036030 con 0x55c5a8f10ca0 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9cfe00640 1 -- 172.21.15.42:0/3334401737 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc9e0031870 con 0x55c5a8f10ca0 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9f8406740 10 client.25477 fetch_fsmap learned FSMap version 27 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9f8406740 1 -- 172.21.15.42:0/3334401737 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5a8f195d0 con 0x55c5a8f10ca0 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9cfe00640 1 -- 172.21.15.42:0/3334401737 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e0036030 con 0x55c5a8f10ca0 2024-04-25T00:22:20.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9cfe00640 1 -- 172.21.15.42:0/3334401737 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc9e004d030 con 0x55c5a8f10ca0 2024-04-25T00:22:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9f8406740 10 client.25477 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:22:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9f8406740 10 client.25477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9f8406740 20 client.25477 populate_metadata read hostname 'smithi042' 2024-04-25T00:22:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9f8406740 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5a8f265f0 0x55c5a8f28a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9f8406740 4 client.25477 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.996+0000 7fc9f8406740 1 -- 172.21.15.42:0/3334401737 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5a8e80ff0 con 0x55c5a8f265f0 2024-04-25T00:22:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.997+0000 7fc9ee000640 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5a8f265f0 0x55c5a8f28a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.997+0000 7fc9ee000640 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5a8f265f0 0x55c5a8f28a00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d707dcd0ae483479 server_cookie=55b972032cf41e0b in_seq=0 out_seq=0 2024-04-25T00:22:21.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.997+0000 7fc9cfe00640 10 client.25477 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9cfe00640 1 -- 172.21.15.42:0/3334401737 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5a8e80ff0 con 0x55c5a8f265f0 2024-04-25T00:22:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9cfe00640 10 client.25477 handle_command_reply: tid=0 2024-04-25T00:22:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9f8406740 1 client.25477 shutdown 2024-04-25T00:22:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9f8406740 1 -- 172.21.15.42:0/3334401737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f10ca0 msgr2=0x55c5a8f1ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:21.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9f8406740 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f1ad90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009ca0 tx=0x7fc9e0031680 comp rx=0 tx=0).stop 2024-04-25T00:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9f8406740 1 -- 172.21.15.42:0/3334401737 shutdown_connections 2024-04-25T00:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9f8406740 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5a8f265f0 0x55c5a8f28a00 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9f8406740 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9f8406740 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f11690 0x55c5a8f1b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9cfe00640 0 client.25477 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9f8406740 1 --2- 172.21.15.42:0/3334401737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f1ad90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:20.999+0000 7fc9f8406740 1 -- 172.21.15.42:0/3334401737 >> 172.21.15.42:0/3334401737 conn(0x55c5a8f0c310 msgr2=0x55c5a8f0b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.000+0000 7fc9f8406740 1 -- 172.21.15.42:0/3334401737 shutdown_connections 2024-04-25T00:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.000+0000 7fc9f8406740 1 -- 172.21.15.42:0/3334401737 wait complete. 2024-04-25T00:22:21.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.000+0000 7fc9f8406740 20 client.25477 trim_cache size 0 max 16384 2024-04-25T00:22:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.017+0000 7fc9f8406740 1 Processor -- start 2024-04-25T00:22:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.017+0000 7fc9f8406740 1 -- start start 2024-04-25T00:22:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.017+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f0f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.017+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f22260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:21.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.017+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f1a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.017+0000 7fc9f8406740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01ad5e0 con 0x55c5a8f10ca0 2024-04-25T00:22:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.017+0000 7fc9f8406740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01aae50 con 0x55c5a8f16800 2024-04-25T00:22:21.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.017+0000 7fc9f8406740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01a8ca0 con 0x55c5a8f11690 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ef400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f1a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f22260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f0f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ef400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f1a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46464/0 (socket says 172.21.15.42:46464) 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ef400640 1 -- 172.21.15.42:0/504849645 learned_addr learned my addr 172.21.15.42:0/504849645 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f22260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33746/0 (socket says 172.21.15.42:33746) 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ef400640 1 -- 172.21.15.42:0/504849645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 msgr2=0x55c5a8f22260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ef400640 1 --2- 172.21.15.42:0/504849645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f22260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ef400640 1 -- 172.21.15.42:0/504849645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 msgr2=0x55c5a8f0f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ef400640 1 --2- 172.21.15.42:0/504849645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f0f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ef400640 1 -- 172.21.15.42:0/504849645 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5a8f07620 con 0x55c5a8f16800 2024-04-25T00:22:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9eea00640 1 --2- 172.21.15.42:0/504849645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f0f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.018+0000 7fc9ef400640 1 --2- 172.21.15.42:0/504849645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f1a3b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8020530 tx=0x7fc9e8015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=215e6d5cf743e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.019+0000 7fc9daa00640 1 -- 172.21.15.42:0/504849645 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8010690 con 0x55c5a8f16800 2024-04-25T00:22:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.019+0000 7fc9daa00640 1 -- 172.21.15.42:0/504849645 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e8010d10 con 0x55c5a8f16800 2024-04-25T00:22:21.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.019+0000 7fc9daa00640 1 -- 172.21.15.42:0/504849645 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8028e10 con 0x55c5a8f16800 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.019+0000 7fc9f8406740 1 -- 172.21.15.42:0/504849645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 msgr2=0x55c5a8f1a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.019+0000 7fc9f8406740 1 --2- 172.21.15.42:0/504849645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f1a3b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8020530 tx=0x7fc9e8015000 comp rx=0 tx=0).stop 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.019+0000 7fc9f8406740 1 -- 172.21.15.42:0/504849645 shutdown_connections 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.019+0000 7fc9f8406740 1 --2- 172.21.15.42:0/504849645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f1a3b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.019+0000 7fc9f8406740 1 --2- 172.21.15.42:0/504849645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f22260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.019+0000 7fc9f8406740 1 --2- 172.21.15.42:0/504849645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f0f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.019+0000 7fc9f8406740 1 -- 172.21.15.42:0/504849645 >> 172.21.15.42:0/504849645 conn(0x55c5a8f0c310 msgr2=0x55c5a8f0f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.020+0000 7fc9f8406740 1 -- 172.21.15.42:0/504849645 shutdown_connections 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.020+0000 7fc9f8406740 1 -- 172.21.15.42:0/504849645 wait complete. 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.020+0000 7fc9f8406740 1 -- start start 2024-04-25T00:22:21.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.020+0000 7fc9f8406740 1 Processor -- start 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f1f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f1fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9f8406740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f20d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9f8406740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01a9310 con 0x55c5a8f10ca0 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9f8406740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01aa780 con 0x55c5a8f16800 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9f8406740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01a99e0 con 0x55c5a8f11690 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9f8406740 10 client.0 _ll_register_callbacks cb 0x7fc9e800d360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9eea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f1f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9ef400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f20d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9ef400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f20d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46466/0 (socket says 172.21.15.42:46466) 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9ee000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f1fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9ef400640 1 -- 172.21.15.42:0/2489067782 learned_addr learned my addr 172.21.15.42:0/2489067782 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9ef400640 1 -- 172.21.15.42:0/2489067782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 msgr2=0x55c5a8f1fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9ef400640 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f1fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9ef400640 1 -- 172.21.15.42:0/2489067782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 msgr2=0x55c5a8f1f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:21.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9ef400640 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f1f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.021+0000 7fc9ef400640 1 -- 172.21.15.42:0/2489067782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5a8f21310 con 0x55c5a8f16800 2024-04-25T00:22:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.022+0000 7fc9ef400640 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f20d20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8020dc0 tx=0x7fc9e800de50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b100bcc08ebfe4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:21.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.022+0000 7fc9cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:22:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.022+0000 7fc9cfe00640 1 -- 172.21.15.42:0/2489067782 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8036070 con 0x55c5a8f16800 2024-04-25T00:22:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.022+0000 7fc9cfe00640 1 -- 172.21.15.42:0/2489067782 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fc9e80106a0 con 0x55c5a8f16800 2024-04-25T00:22:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.022+0000 7fc9f8406740 1 -- 172.21.15.42:0/2489067782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e8010d10 con 0x55c5a8f16800 2024-04-25T00:22:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.022+0000 7fc9cfe00640 1 -- 172.21.15.42:0/2489067782 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc9e8042050 con 0x55c5a8f16800 2024-04-25T00:22:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.022+0000 7fc9f8406740 10 client.25475 fetch_fsmap learned FSMap version 27 2024-04-25T00:22:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.022+0000 7fc9f8406740 1 -- 172.21.15.42:0/2489067782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c5a8f8cb80 con 0x55c5a8f16800 2024-04-25T00:22:21.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.023+0000 7fc9cfe00640 1 -- 172.21.15.42:0/2489067782 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9e8010d10 con 0x55c5a8f16800 2024-04-25T00:22:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.023+0000 7fc9cfe00640 1 -- 172.21.15.42:0/2489067782 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc9e8027070 con 0x55c5a8f16800 2024-04-25T00:22:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.023+0000 7fc9f8406740 10 client.25475 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:22:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.023+0000 7fc9f8406740 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.023+0000 7fc9f8406740 20 client.25475 populate_metadata read hostname 'smithi042' 2024-04-25T00:22:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.023+0000 7fc9f8406740 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5a8f8d3a0 0x55c5a8f8f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.023+0000 7fc9f8406740 4 client.25475 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:22:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.023+0000 7fc9f8406740 1 -- 172.21.15.42:0/2489067782 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5a8f16530 con 0x55c5a8f8d3a0 2024-04-25T00:22:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.023+0000 7fc9eea00640 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5a8f8d3a0 0x55c5a8f8f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.024+0000 7fc9eea00640 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5a8f8d3a0 0x55c5a8f8f7c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e074faf13e3d9509 server_cookie=d581f10ab5ebb9ad in_seq=0 out_seq=0 2024-04-25T00:22:21.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.024+0000 7fc9cfe00640 10 client.25475 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.070+0000 7fc9cfe00640 1 -- 172.21.15.42:0/2489067782 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c5a8f16530 con 0x55c5a8f8d3a0 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.070+0000 7fc9cfe00640 10 client.25475 handle_command_reply: tid=0 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.070+0000 7fc9f8406740 1 client.25475 shutdown 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 1 -- 172.21.15.42:0/2489067782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 msgr2=0x55c5a8f20d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f20d20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8020dc0 tx=0x7fc9e800de50 comp rx=0 tx=0).stop 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 1 -- 172.21.15.42:0/2489067782 shutdown_connections 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5a8f8d3a0 0x55c5a8f8f7c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5a8f16800 0x55c5a8f20d20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5a8f11690 0x55c5a8f1fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 1 --2- 172.21.15.42:0/2489067782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5a8f10ca0 0x55c5a8f1f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9cfe00640 0 client.25475 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 1 -- 172.21.15.42:0/2489067782 >> 172.21.15.42:0/2489067782 conn(0x55c5a8f0c310 msgr2=0x55c5a8f0f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 1 -- 172.21.15.42:0/2489067782 shutdown_connections 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 1 -- 172.21.15.42:0/2489067782 wait complete. 2024-04-25T00:22:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.071+0000 7fc9f8406740 20 client.25475 trim_cache size 0 max 16384 2024-04-25T00:22:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:22:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:22:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:22:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:22:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 -- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9d40a2cb0 msgr2=0x7fc9d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9d40a2cb0 0x7fc9d40a5170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x55c5a8f11ef0 tx=0x7fc9e800d4c0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 -- 172.21.15.42:0/241645621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9f0104bb0 msgr2=0x7fc9f019a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9f0104bb0 0x7fc9f019a180 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc00d950 tx=0x7fc9dc00de10 comp rx=0 tx=0).stop 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 -- 172.21.15.42:0/241645621 shutdown_connections 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc9d40a2cb0 0x7fc9d40a5170 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc9f0105570 0x7fc9f01a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc9f0104bb0 0x7fc9f019a180 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 --2- 172.21.15.42:0/241645621 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9f01039c0 0x7fc9f0199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 -- 172.21.15.42:0/241645621 >> 172.21.15.42:0/241645621 conn(0x7fc9f00fef30 msgr2=0x7fc9f0107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 -- 172.21.15.42:0/241645621 shutdown_connections 2024-04-25T00:22:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:21.072+0000 7fc9f5600640 1 -- 172.21.15.42:0/241645621 wait complete. 2024-04-25T00:22:21.216 INFO:tasks.workunit.client.0.smithi042.stdout: 1 167454 6.94 MB/sec execute 11 sec latency 18.312 ms 2024-04-25T00:22:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:21 smithi106 ceph-mon[32510]: pgmap v496: 97 pgs: 97 active+clean; 53 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-04-25T00:22:21.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:21 smithi042 ceph-mon[28130]: pgmap v496: 97 pgs: 97 active+clean; 53 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-04-25T00:22:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:21 smithi136 ceph-mon[32923]: pgmap v496: 97 pgs: 97 active+clean; 53 MiB data, 784 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 197 op/s 2024-04-25T00:22:22.217 INFO:tasks.workunit.client.0.smithi042.stdout: 1 168594 6.79 MB/sec execute 12 sec latency 23.935 ms 2024-04-25T00:22:23.217 INFO:tasks.workunit.client.0.smithi042.stdout: 1 170485 7.30 MB/sec execute 13 sec latency 26.915 ms 2024-04-25T00:22:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:23 smithi106 ceph-mon[32510]: pgmap v497: 97 pgs: 97 active+clean; 55 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-04-25T00:22:23.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:23 smithi042 ceph-mon[28130]: pgmap v497: 97 pgs: 97 active+clean; 55 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-04-25T00:22:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:23 smithi136 ceph-mon[32923]: pgmap v497: 97 pgs: 97 active+clean; 55 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-04-25T00:22:24.217 INFO:tasks.workunit.client.0.smithi042.stdout: 1 171014 6.82 MB/sec execute 14 sec latency 79.259 ms 2024-04-25T00:22:25.217 INFO:tasks.workunit.client.0.smithi042.stdout: 1 172457 6.73 MB/sec execute 15 sec latency 25.228 ms 2024-04-25T00:22:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:25 smithi106 ceph-mon[32510]: pgmap v498: 97 pgs: 97 active+clean; 55 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-04-25T00:22:25.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:25 smithi042 ceph-mon[28130]: pgmap v498: 97 pgs: 97 active+clean; 55 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-04-25T00:22:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:25 smithi136 ceph-mon[32923]: pgmap v498: 97 pgs: 97 active+clean; 55 MiB data, 790 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.4 MiB/s wr, 169 op/s 2024-04-25T00:22:26.217 INFO:tasks.workunit.client.0.smithi042.stdout: 1 174099 7.12 MB/sec execute 16 sec latency 15.297 ms 2024-04-25T00:22:27.217 INFO:tasks.workunit.client.0.smithi042.stdout: 1 174802 6.77 MB/sec execute 17 sec latency 16.735 ms 2024-04-25T00:22:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:27 smithi106 ceph-mon[32510]: pgmap v499: 97 pgs: 97 active+clean; 53 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-25T00:22:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:27 smithi042 ceph-mon[28130]: pgmap v499: 97 pgs: 97 active+clean; 53 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-25T00:22:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:27 smithi136 ceph-mon[32923]: pgmap v499: 97 pgs: 97 active+clean; 53 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.0 MiB/s wr, 250 op/s 2024-04-25T00:22:28.217 INFO:tasks.workunit.client.0.smithi042.stdout: 1 176240 6.75 MB/sec execute 18 sec latency 22.986 ms 2024-04-25T00:22:29.217 INFO:tasks.workunit.client.0.smithi042.stdout: 1 177667 7.01 MB/sec execute 19 sec latency 37.607 ms 2024-04-25T00:22:29.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:29 smithi106 ceph-mon[32510]: pgmap v500: 97 pgs: 97 active+clean; 53 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-04-25T00:22:29.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:29 smithi042 ceph-mon[28130]: pgmap v500: 97 pgs: 97 active+clean; 53 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-04-25T00:22:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:29 smithi136 ceph-mon[32923]: pgmap v500: 97 pgs: 97 active+clean; 53 MiB data, 802 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-04-25T00:22:30.217 INFO:tasks.workunit.client.0.smithi042.stdout: 1 178344 6.71 MB/sec execute 20 sec latency 26.422 ms 2024-04-25T00:22:31.218 INFO:tasks.workunit.client.0.smithi042.stdout: 1 179767 6.69 MB/sec execute 21 sec latency 23.477 ms 2024-04-25T00:22:31.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:31 smithi106 ceph-mon[32510]: pgmap v501: 97 pgs: 97 active+clean; 53 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-25T00:22:31.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:31 smithi042 ceph-mon[28130]: pgmap v501: 97 pgs: 97 active+clean; 53 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-25T00:22:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:31 smithi136 ceph-mon[32923]: pgmap v501: 97 pgs: 97 active+clean; 53 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-25T00:22:32.218 INFO:tasks.workunit.client.0.smithi042.stdout: 1 181182 6.92 MB/sec execute 22 sec latency 15.150 ms 2024-04-25T00:22:33.218 INFO:tasks.workunit.client.0.smithi042.stdout: 1 181799 6.66 MB/sec execute 23 sec latency 22.631 ms 2024-04-25T00:22:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:33 smithi106 ceph-mon[32510]: pgmap v502: 97 pgs: 97 active+clean; 54 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 250 op/s 2024-04-25T00:22:33.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:33 smithi042 ceph-mon[28130]: pgmap v502: 97 pgs: 97 active+clean; 54 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 250 op/s 2024-04-25T00:22:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:33 smithi136 ceph-mon[32923]: pgmap v502: 97 pgs: 97 active+clean; 54 MiB data, 819 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 250 op/s 2024-04-25T00:22:34.218 INFO:tasks.workunit.client.0.smithi042.stdout: 1 183003 6.59 MB/sec execute 24 sec latency 30.687 ms 2024-04-25T00:22:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.227+0000 7f725fe00640 1 -- 172.21.15.42:0/4166170882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72601031b0 msgr2=0x7f7260109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.227+0000 7f725fe00640 1 --2- 172.21.15.42:0/4166170882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72601031b0 0x7f7260109a40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7258009be0 tx=0x7f725802a6c0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.227+0000 7f725fe00640 1 -- 172.21.15.42:0/4166170882 shutdown_connections 2024-04-25T00:22:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.227+0000 7f725fe00640 1 --2- 172.21.15.42:0/4166170882 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72601031b0 0x7f7260109a40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.227+0000 7f725fe00640 1 --2- 172.21.15.42:0/4166170882 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72601027f0 0x7f7260102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.227+0000 7f725fe00640 1 --2- 172.21.15.42:0/4166170882 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7260101600 0x7f7260101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.227+0000 7f725fe00640 1 -- 172.21.15.42:0/4166170882 >> 172.21.15.42:0/4166170882 conn(0x7f72600fcb70 msgr2=0x7f72600fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.228+0000 7f725fe00640 1 -- 172.21.15.42:0/4166170882 shutdown_connections 2024-04-25T00:22:34.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.228+0000 7f725fe00640 1 -- 172.21.15.42:0/4166170882 wait complete. 2024-04-25T00:22:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.229+0000 7f725fe00640 1 Processor -- start 2024-04-25T00:22:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.229+0000 7f725fe00640 1 -- start start 2024-04-25T00:22:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.229+0000 7f725fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7260101600 0x7f7260199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.229+0000 7f725fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72601027f0 0x7f726019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.229+0000 7f725fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72601031b0 0x7f72601a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.229+0000 7f725fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f726010d260 con 0x7f7260101600 2024-04-25T00:22:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.229+0000 7f725fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f726010d0e0 con 0x7f72601027f0 2024-04-25T00:22:34.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.229+0000 7f725fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f726010d3e0 con 0x7f72601031b0 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.229+0000 7f725e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72601027f0 0x7f726019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72601027f0 0x7f726019a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43368/0 (socket says 172.21.15.42:43368) 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725e000640 1 -- 172.21.15.42:0/3672554360 learned_addr learned my addr 172.21.15.42:0/3672554360 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725f400640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72601031b0 0x7f72601a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725ea00640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7260101600 0x7f7260199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725e000640 1 -- 172.21.15.42:0/3672554360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72601031b0 msgr2=0x7f72601a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725e000640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72601031b0 0x7f72601a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725e000640 1 -- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7260101600 msgr2=0x7f7260199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725e000640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7260101600 0x7f7260199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725e000640 1 -- 172.21.15.42:0/3672554360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72601a1730 con 0x7f72601027f0 2024-04-25T00:22:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725ea00640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7260101600 0x7f7260199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.230+0000 7f725e000640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72601027f0 0x7f726019a050 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f724c00e9c0 tx=0x7f724c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.231+0000 7f7264a00640 1 -- 172.21.15.42:0/3672554360 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c00ccc0 con 0x7f72601027f0 2024-04-25T00:22:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.231+0000 7f7264a00640 1 -- 172.21.15.42:0/3672554360 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f724c005500 con 0x7f72601027f0 2024-04-25T00:22:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.231+0000 7f725fe00640 1 -- 172.21.15.42:0/3672554360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72601a1a20 con 0x7f72601027f0 2024-04-25T00:22:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.231+0000 7f7264a00640 1 -- 172.21.15.42:0/3672554360 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c010430 con 0x7f72601027f0 2024-04-25T00:22:34.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.231+0000 7f725fe00640 1 -- 172.21.15.42:0/3672554360 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72601a1f70 con 0x7f72601027f0 2024-04-25T00:22:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.234+0000 7f7264a00640 1 -- 172.21.15.42:0/3672554360 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f724c0046c0 con 0x7f72601027f0 2024-04-25T00:22:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.235+0000 7f7268bea740 1 Processor -- start 2024-04-25T00:22:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f7268bea740 1 -- start start 2024-04-25T00:22:34.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f7268bea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbc9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f7268bea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbce340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f7268bea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbce9b0 0x5601fcbd0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f7268bea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5601fcb2bab0 con 0x5601fcbc8e50 2024-04-25T00:22:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f7268bea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5601fcad4640 con 0x5601fcbc9840 2024-04-25T00:22:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f7268bea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601fcb27a70 con 0x5601fcbce9b0 2024-04-25T00:22:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f7264a00640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72440a2b80 0x7f72440a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f725ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbce340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f7264a00640 1 -- 172.21.15.42:0/3672554360 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f724c0cf420 con 0x7f72601027f0 2024-04-25T00:22:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f725ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbce340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43382/0 (socket says 172.21.15.42:43382) 2024-04-25T00:22:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f725ea00640 1 -- 172.21.15.42:0/4192741395 learned_addr learned my addr 172.21.15.42:0/4192741395 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.236+0000 7f725f400640 1 --2- 172.21.15.42:0/4192741395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbc9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.238 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.237+0000 7f725e000640 1 --2- 172.21.15.42:0/4192741395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbce9b0 0x5601fcbd0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.237+0000 7f725f400640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72440a2b80 0x7f72440a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.237+0000 7f725ea00640 1 -- 172.21.15.42:0/4192741395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbce9b0 msgr2=0x5601fcbd0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.237+0000 7f725ea00640 1 --2- 172.21.15.42:0/4192741395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbce9b0 0x5601fcbd0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.237+0000 7f725ea00640 1 -- 172.21.15.42:0/4192741395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 msgr2=0x5601fcbc9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.237+0000 7f725ea00640 1 --2- 172.21.15.42:0/4192741395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbc9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.237+0000 7f725ea00640 1 -- 172.21.15.42:0/4192741395 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601fc840210 con 0x5601fcbc9840 2024-04-25T00:22:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.237+0000 7f725ea00640 1 --2- 172.21.15.42:0/4192741395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbce340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7250009c30 tx=0x7f725000e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=850e8558e471291b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.238+0000 7f724be00640 1 -- 172.21.15.42:0/4192741395 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250016070 con 0x5601fcbc9840 2024-04-25T00:22:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.238+0000 7f724be00640 1 -- 172.21.15.42:0/4192741395 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7250005500 con 0x5601fcbc9840 2024-04-25T00:22:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.238+0000 7f724be00640 1 -- 172.21.15.42:0/4192741395 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7250011470 con 0x5601fcbc9840 2024-04-25T00:22:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f7268bea740 1 -- 172.21.15.42:0/4192741395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 msgr2=0x5601fcbce340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f725f400640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72440a2b80 0x7f72440a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f725802abe0 tx=0x7f7258005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f7268bea740 1 --2- 172.21.15.42:0/4192741395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbce340 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7250009c30 tx=0x7f725000e6c0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f7268bea740 1 -- 172.21.15.42:0/4192741395 shutdown_connections 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f7268bea740 1 --2- 172.21.15.42:0/4192741395 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbce9b0 0x5601fcbd0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f7268bea740 1 --2- 172.21.15.42:0/4192741395 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbce340 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f7268bea740 1 --2- 172.21.15.42:0/4192741395 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbc9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f7268bea740 1 -- 172.21.15.42:0/4192741395 >> 172.21.15.42:0/4192741395 conn(0x5601fcbc44c0 msgr2=0x5601fcbc48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f7268bea740 1 -- 172.21.15.42:0/4192741395 shutdown_connections 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f7268bea740 1 -- 172.21.15.42:0/4192741395 wait complete. 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.239+0000 7f7268bea740 1 -- start start 2024-04-25T00:22:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.240+0000 7f7268bea740 1 Processor -- start 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.240+0000 7f7268bea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc8e50 0x5601fcbd3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.240+0000 7f7268bea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc9840 0x5601fcbd35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.240+0000 7f7268bea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbda5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.240+0000 7f7268bea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5601fcbd1960 con 0x5601fcbc9840 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.240+0000 7f7268bea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5601fcae7990 con 0x5601fcbce9b0 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.240+0000 7f7268bea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601fcbd1ae0 con 0x5601fcbc8e50 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.240+0000 7f7268bea740 10 client.0 _ll_register_callbacks cb 0x5601fcae9950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.241+0000 7f725f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbda5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.241+0000 7f725e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc9840 0x5601fcbd35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.241+0000 7f725f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbda5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43394/0 (socket says 172.21.15.42:43394) 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.241+0000 7f725f400640 1 -- 172.21.15.42:0/3493970479 learned_addr learned my addr 172.21.15.42:0/3493970479 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.241+0000 7f725ea00640 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc8e50 0x5601fcbd3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.241+0000 7f725f400640 1 -- 172.21.15.42:0/3493970479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc8e50 msgr2=0x5601fcbd3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.241+0000 7f725f400640 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc8e50 0x5601fcbd3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.241+0000 7f725f400640 1 -- 172.21.15.42:0/3493970479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc9840 msgr2=0x5601fcbd35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.241+0000 7f725f400640 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc9840 0x5601fcbd35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.241+0000 7f725f400640 1 -- 172.21.15.42:0/3493970479 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601fc90c620 con 0x5601fcbce9b0 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.242+0000 7f725f400640 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbda5f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7258009be0 tx=0x7f7258009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab78f7aa77dd4cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.242+0000 7f7248c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.242+0000 7f7248c00640 1 -- 172.21.15.42:0/3493970479 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f725804e020 con 0x5601fcbce9b0 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.242+0000 7f7248c00640 1 -- 172.21.15.42:0/3493970479 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f7258003c30 con 0x5601fcbce9b0 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.242+0000 7f7268bea740 1 -- 172.21.15.42:0/3493970479 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7250005500 con 0x5601fcbce9b0 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.242+0000 7f7248c00640 1 -- 172.21.15.42:0/3493970479 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7258036040 con 0x5601fcbce9b0 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.242+0000 7f7268bea740 10 client.25487 fetch_fsmap learned FSMap version 27 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.242+0000 7f7268bea740 1 -- 172.21.15.42:0/3493970479 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601fcbdd640 con 0x5601fcbce9b0 2024-04-25T00:22:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.243+0000 7f7248c00640 1 -- 172.21.15.42:0/3493970479 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7250005500 con 0x5601fcbce9b0 2024-04-25T00:22:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.243+0000 7f7248c00640 1 -- 172.21.15.42:0/3493970479 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f7258039030 con 0x5601fcbce9b0 2024-04-25T00:22:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.243+0000 7f7268bea740 10 client.25487 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:22:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.243+0000 7f7268bea740 10 client.25487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:34.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.243+0000 7f7268bea740 20 client.25487 populate_metadata read hostname 'smithi042' 2024-04-25T00:22:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.243+0000 7f7268bea740 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5601fcbdede0 0x5601fcbe11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.243+0000 7f7268bea740 4 client.25487 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.243+0000 7f7268bea740 1 -- 172.21.15.42:0/3493970479 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601fcb38ff0 con 0x5601fcbdede0 2024-04-25T00:22:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.243+0000 7f725ea00640 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5601fcbdede0 0x5601fcbe11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.244+0000 7f725ea00640 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5601fcbdede0 0x5601fcbe11f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de32d5203dc4b14e server_cookie=7cdfeb70ae36aa00 in_seq=0 out_seq=0 2024-04-25T00:22:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.244+0000 7f7248c00640 10 client.25487 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.244+0000 7f7248c00640 1 -- 172.21.15.42:0/3493970479 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601fcb38ff0 con 0x5601fcbdede0 2024-04-25T00:22:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.244+0000 7f7248c00640 10 client.25487 handle_command_reply: tid=0 2024-04-25T00:22:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 client.25487 shutdown 2024-04-25T00:22:34.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 -- 172.21.15.42:0/3493970479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 msgr2=0x5601fcbda5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbda5f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7258009be0 tx=0x7f7258009500 comp rx=0 tx=0).stop 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 -- 172.21.15.42:0/3493970479 shutdown_connections 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5601fcbdede0 0x5601fcbe11f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbda5f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc9840 0x5601fcbd35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 --2- 172.21.15.42:0/3493970479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc8e50 0x5601fcbd3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 -- 172.21.15.42:0/3493970479 >> 172.21.15.42:0/3493970479 conn(0x5601fcbc44c0 msgr2=0x5601fcbc3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7248c00640 0 client.25487 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 -- 172.21.15.42:0/3493970479 shutdown_connections 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 1 -- 172.21.15.42:0/3493970479 wait complete. 2024-04-25T00:22:34.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.245+0000 7f7268bea740 20 client.25487 trim_cache size 0 max 16384 2024-04-25T00:22:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.298+0000 7f7268bea740 1 Processor -- start 2024-04-25T00:22:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.298+0000 7f7268bea740 1 -- start start 2024-04-25T00:22:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f7268bea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f7268bea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc9840 0x5601fcbda5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f7268bea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbcaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f7268bea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72601a9200 con 0x5601fcbc8e50 2024-04-25T00:22:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f7268bea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f72601a6a70 con 0x5601fcbce9b0 2024-04-25T00:22:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f7268bea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f72601a4900 con 0x5601fcbc9840 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f725ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f725ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59834/0 (socket says 172.21.15.42:59834) 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f725ea00640 1 -- 172.21.15.42:0/2798808492 learned_addr learned my addr 172.21.15.42:0/2798808492 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f725e000640 1 --2- 172.21.15.42:0/2798808492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc9840 0x5601fcbda5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.299+0000 7f725f400640 1 --2- 172.21.15.42:0/2798808492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbcaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.300+0000 7f725e000640 1 -- 172.21.15.42:0/2798808492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 msgr2=0x5601fcbcaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.300+0000 7f725e000640 1 --2- 172.21.15.42:0/2798808492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbcaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.300+0000 7f725e000640 1 -- 172.21.15.42:0/2798808492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 msgr2=0x5601fcbce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.300+0000 7f725e000640 1 --2- 172.21.15.42:0/2798808492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.300+0000 7f725e000640 1 -- 172.21.15.42:0/2798808492 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601fcbca510 con 0x5601fcbc9840 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.300+0000 7f725f400640 1 --2- 172.21.15.42:0/2798808492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbcaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.300+0000 7f725e000640 1 --2- 172.21.15.42:0/2798808492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc9840 0x5601fcbda5b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c6a0 tx=0x7f724c004b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4522e1aa559086a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.300+0000 7f724b400640 1 -- 172.21.15.42:0/2798808492 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c0b1070 con 0x5601fcbc9840 2024-04-25T00:22:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.300+0000 7f724b400640 1 -- 172.21.15.42:0/2798808492 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f724c0a48c0 con 0x5601fcbc9840 2024-04-25T00:22:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.300+0000 7f724b400640 1 -- 172.21.15.42:0/2798808492 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c0a4430 con 0x5601fcbc9840 2024-04-25T00:22:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.301+0000 7f7268bea740 1 -- 172.21.15.42:0/2798808492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc9840 msgr2=0x5601fcbda5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.301+0000 7f7268bea740 1 --2- 172.21.15.42:0/2798808492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc9840 0x5601fcbda5b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f724c00c6a0 tx=0x7f724c004b00 comp rx=0 tx=0).stop 2024-04-25T00:22:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.301+0000 7f7268bea740 1 -- 172.21.15.42:0/2798808492 shutdown_connections 2024-04-25T00:22:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.301+0000 7f7268bea740 1 --2- 172.21.15.42:0/2798808492 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbce9b0 0x5601fcbcaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.301+0000 7f7268bea740 1 --2- 172.21.15.42:0/2798808492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbc9840 0x5601fcbda5b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.301+0000 7f7268bea740 1 --2- 172.21.15.42:0/2798808492 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.301+0000 7f7268bea740 1 -- 172.21.15.42:0/2798808492 >> 172.21.15.42:0/2798808492 conn(0x5601fcbc44c0 msgr2=0x5601fcbc48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.302+0000 7f7268bea740 1 -- 172.21.15.42:0/2798808492 shutdown_connections 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.302+0000 7f7268bea740 1 -- 172.21.15.42:0/2798808492 wait complete. 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.302+0000 7f7268bea740 1 -- start start 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.302+0000 7f7268bea740 1 Processor -- start 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f7268bea740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbd76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f7268bea740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbd7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f7268bea740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbce9b0 0x5601fcbd8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f7268bea740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f72601a4f50 con 0x5601fcbc8e50 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f7268bea740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f72601a63a0 con 0x5601fcbc9840 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f7268bea740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f72601a5600 con 0x5601fcbce9b0 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f7268bea740 10 client.0 _ll_register_callbacks cb 0x7f724c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f725e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbd7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f725ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbd76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f725e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbd7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43414/0 (socket says 172.21.15.42:43414) 2024-04-25T00:22:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f725e000640 1 -- 172.21.15.42:0/283373126 learned_addr learned my addr 172.21.15.42:0/283373126 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.303+0000 7f725f400640 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbce9b0 0x5601fcbd8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f725e000640 1 -- 172.21.15.42:0/283373126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbce9b0 msgr2=0x5601fcbd8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f725e000640 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbce9b0 0x5601fcbd8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f725e000640 1 -- 172.21.15.42:0/283373126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 msgr2=0x5601fcbd76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f725e000640 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbd76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f725e000640 1 -- 172.21.15.42:0/283373126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601fcbd91e0 con 0x5601fcbc9840 2024-04-25T00:22:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f725ea00640 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbd76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:22:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f725e000640 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbd7be0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f724c0bf010 tx=0x7f724c0cd000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60981b88c2107a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f7248c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:22:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f7248c00640 1 -- 172.21.15.42:0/283373126 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f724c0b1030 con 0x5601fcbc9840 2024-04-25T00:22:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f7248c00640 1 -- 172.21.15.42:0/283373126 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f724c0a4630 con 0x5601fcbc9840 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.304+0000 7f7268bea740 1 -- 172.21.15.42:0/283373126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f724c0a48c0 con 0x5601fcbc9840 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7248c00640 1 -- 172.21.15.42:0/283373126 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f724c0bf0b0 con 0x5601fcbc9840 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7268bea740 10 client.25493 fetch_fsmap learned FSMap version 27 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7268bea740 1 -- 172.21.15.42:0/283373126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5601fcbda4f0 con 0x5601fcbc9840 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7248c00640 1 -- 172.21.15.42:0/283373126 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f724c0a48c0 con 0x5601fcbc9840 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7248c00640 1 -- 172.21.15.42:0/283373126 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f724c005000 con 0x5601fcbc9840 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7268bea740 10 client.25493 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7268bea740 10 client.25493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7268bea740 20 client.25493 populate_metadata read hostname 'smithi042' 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7268bea740 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5601fcc56740 0x5601fcc58b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7268bea740 4 client.25493 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.305+0000 7f7268bea740 1 -- 172.21.15.42:0/283373126 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601fcb33340 con 0x5601fcc56740 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.306+0000 7f725ea00640 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5601fcc56740 0x5601fcc58b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.306+0000 7f725ea00640 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5601fcc56740 0x5601fcc58b60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac07912d99a74d5 server_cookie=89ebd03d89bc034b in_seq=0 out_seq=0 2024-04-25T00:22:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.306+0000 7f7248c00640 10 client.25493 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7248c00640 1 -- 172.21.15.42:0/283373126 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601fcb33340 con 0x5601fcc56740 2024-04-25T00:22:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7248c00640 10 client.25493 handle_command_reply: tid=0 2024-04-25T00:22:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 client.25493 shutdown 2024-04-25T00:22:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 -- 172.21.15.42:0/283373126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 msgr2=0x5601fcbd7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbd7be0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f724c0bf010 tx=0x7f724c0cd000 comp rx=0 tx=0).stop 2024-04-25T00:22:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 -- 172.21.15.42:0/283373126 shutdown_connections 2024-04-25T00:22:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5601fcc56740 0x5601fcc58b60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601fcbce9b0 0x5601fcbd8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.370 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5601fcbc9840 0x5601fcbd7be0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 --2- 172.21.15.42:0/283373126 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5601fcbc8e50 0x5601fcbd76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7248c00640 0 client.25493 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 -- 172.21.15.42:0/283373126 >> 172.21.15.42:0/283373126 conn(0x5601fcbc44c0 msgr2=0x5601fcbd3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 -- 172.21.15.42:0/283373126 shutdown_connections 2024-04-25T00:22:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.369+0000 7f7268bea740 1 -- 172.21.15.42:0/283373126 wait complete. 2024-04-25T00:22:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.370+0000 7f7268bea740 20 client.25493 trim_cache size 0 max 16384 2024-04-25T00:22:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:22:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:22:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:22:34.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:22:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.370+0000 7f725fe00640 1 -- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72440a2b80 msgr2=0x7f72440a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.370+0000 7f725fe00640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72440a2b80 0x7f72440a5040 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f725802abe0 tx=0x7f7258005fd0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.370+0000 7f725fe00640 1 -- 172.21.15.42:0/3672554360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72601027f0 msgr2=0x7f726019a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.370+0000 7f725fe00640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72601027f0 0x7f726019a050 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f724c00e9c0 tx=0x7f724c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:22:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.371+0000 7f725fe00640 1 -- 172.21.15.42:0/3672554360 shutdown_connections 2024-04-25T00:22:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.371+0000 7f725fe00640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72440a2b80 0x7f72440a5040 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.371+0000 7f725fe00640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72601031b0 0x7f72601a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.371+0000 7f725fe00640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72601027f0 0x7f726019a050 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.371+0000 7f725fe00640 1 --2- 172.21.15.42:0/3672554360 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7260101600 0x7f7260199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.371+0000 7f725fe00640 1 -- 172.21.15.42:0/3672554360 >> 172.21.15.42:0/3672554360 conn(0x7f72600fcb70 msgr2=0x7f7260073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.371+0000 7f725fe00640 1 -- 172.21.15.42:0/3672554360 shutdown_connections 2024-04-25T00:22:34.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:34.371+0000 7f725fe00640 1 -- 172.21.15.42:0/3672554360 wait complete. 2024-04-25T00:22:35.218 INFO:tasks.workunit.client.0.smithi042.stdout: 1 184684 6.85 MB/sec execute 25 sec latency 22.118 ms 2024-04-25T00:22:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:35 smithi106 ceph-mon[32510]: pgmap v503: 97 pgs: 97 active+clean; 54 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-25T00:22:35.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:35 smithi042 ceph-mon[28130]: pgmap v503: 97 pgs: 97 active+clean; 54 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-25T00:22:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:35 smithi136 ceph-mon[32923]: pgmap v503: 97 pgs: 97 active+clean; 54 MiB data, 803 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-25T00:22:36.218 INFO:tasks.workunit.client.0.smithi042.stdout: 1 185246 6.62 MB/sec execute 26 sec latency 30.324 ms 2024-04-25T00:22:37.218 INFO:tasks.workunit.client.0.smithi042.stdout: 1 186435 6.56 MB/sec execute 27 sec latency 31.809 ms 2024-04-25T00:22:37.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:37 smithi106 ceph-mon[32510]: pgmap v504: 97 pgs: 97 active+clean; 53 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 247 op/s 2024-04-25T00:22:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:37 smithi042 ceph-mon[28130]: pgmap v504: 97 pgs: 97 active+clean; 53 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 247 op/s 2024-04-25T00:22:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:37 smithi136 ceph-mon[32923]: pgmap v504: 97 pgs: 97 active+clean; 53 MiB data, 816 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 247 op/s 2024-04-25T00:22:38.218 INFO:tasks.workunit.client.0.smithi042.stdout: 1 188142 6.77 MB/sec execute 28 sec latency 22.484 ms 2024-04-25T00:22:39.219 INFO:tasks.workunit.client.0.smithi042.stdout: 1 188673 6.58 MB/sec execute 29 sec latency 25.814 ms 2024-04-25T00:22:39.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:39 smithi106 ceph-mon[32510]: pgmap v505: 97 pgs: 97 active+clean; 53 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-04-25T00:22:39.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:39 smithi042 ceph-mon[28130]: pgmap v505: 97 pgs: 97 active+clean; 53 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-04-25T00:22:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:39 smithi136 ceph-mon[32923]: pgmap v505: 97 pgs: 97 active+clean; 53 MiB data, 831 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-04-25T00:22:40.219 INFO:tasks.workunit.client.0.smithi042.stdout: 1 189852 6.54 MB/sec execute 30 sec latency 55.071 ms 2024-04-25T00:22:41.219 INFO:tasks.workunit.client.0.smithi042.stdout: 1 191656 6.73 MB/sec execute 31 sec latency 17.636 ms 2024-04-25T00:22:41.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:41 smithi106 ceph-mon[32510]: pgmap v506: 97 pgs: 97 active+clean; 54 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-04-25T00:22:41.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:41 smithi042 ceph-mon[28130]: pgmap v506: 97 pgs: 97 active+clean; 54 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-04-25T00:22:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:41 smithi136 ceph-mon[32923]: pgmap v506: 97 pgs: 97 active+clean; 54 MiB data, 837 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 204 op/s 2024-04-25T00:22:42.219 INFO:tasks.workunit.client.0.smithi042.stdout: 1 192149 6.56 MB/sec execute 32 sec latency 27.610 ms 2024-04-25T00:22:43.219 INFO:tasks.workunit.client.0.smithi042.stdout: 1 193169 6.52 MB/sec execute 33 sec latency 26.361 ms 2024-04-25T00:22:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:43 smithi106 ceph-mon[32510]: pgmap v507: 97 pgs: 97 active+clean; 54 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-04-25T00:22:43.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:43 smithi042 ceph-mon[28130]: pgmap v507: 97 pgs: 97 active+clean; 54 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-04-25T00:22:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:43 smithi136 ceph-mon[32923]: pgmap v507: 97 pgs: 97 active+clean; 54 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 239 op/s 2024-04-25T00:22:44.219 INFO:tasks.workunit.client.0.smithi042.stdout: 1 194905 6.60 MB/sec execute 34 sec latency 27.098 ms 2024-04-25T00:22:45.219 INFO:tasks.workunit.client.0.smithi042.stdout: 1 195571 6.54 MB/sec execute 35 sec latency 53.458 ms 2024-04-25T00:22:45.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:45 smithi106 ceph-mon[32510]: pgmap v508: 97 pgs: 97 active+clean; 54 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-04-25T00:22:45.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:45 smithi042 ceph-mon[28130]: pgmap v508: 97 pgs: 97 active+clean; 54 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-04-25T00:22:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:45 smithi136 ceph-mon[32923]: pgmap v508: 97 pgs: 97 active+clean; 54 MiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-04-25T00:22:46.219 INFO:tasks.workunit.client.0.smithi042.stdout: 1 196619 6.50 MB/sec execute 36 sec latency 26.192 ms 2024-04-25T00:22:46.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:46 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:47.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:46 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:47.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:46 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:47.219 INFO:tasks.workunit.client.0.smithi042.stdout: 1 198104 6.53 MB/sec execute 37 sec latency 25.242 ms 2024-04-25T00:22:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.529+0000 7ff547e00640 1 -- 172.21.15.42:0/2522990867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff548103190 msgr2=0x7ff548109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.529+0000 7ff547e00640 1 --2- 172.21.15.42:0/2522990867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff548103190 0x7ff548109a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff540009ca0 tx=0x7ff54002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.529+0000 7ff547e00640 1 -- 172.21.15.42:0/2522990867 shutdown_connections 2024-04-25T00:22:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.529+0000 7ff547e00640 1 --2- 172.21.15.42:0/2522990867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff548103190 0x7ff548109a20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.529+0000 7ff547e00640 1 --2- 172.21.15.42:0/2522990867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5481027d0 0x7ff548102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.529+0000 7ff547e00640 1 --2- 172.21.15.42:0/2522990867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5481015e0 0x7ff548101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.529+0000 7ff547e00640 1 -- 172.21.15.42:0/2522990867 >> 172.21.15.42:0/2522990867 conn(0x7ff5480fcb50 msgr2=0x7ff5480fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.529+0000 7ff547e00640 1 -- 172.21.15.42:0/2522990867 shutdown_connections 2024-04-25T00:22:47.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.529+0000 7ff547e00640 1 -- 172.21.15.42:0/2522990867 wait complete. 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.530+0000 7ff547e00640 1 Processor -- start 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff547e00640 1 -- start start 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff547e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5481015e0 0x7ff5480754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff547e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5481027d0 0x7ff548073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff547e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff548103190 0x7ff548074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff547e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff54810d2b0 con 0x7ff5481027d0 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff547e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff54810d130 con 0x7ff548103190 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff547e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff54810d430 con 0x7ff5481015e0 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff546000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5481027d0 0x7ff548073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff546a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5481015e0 0x7ff5480754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff546000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5481027d0 0x7ff548073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35756/0 (socket says 172.21.15.42:35756) 2024-04-25T00:22:47.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.531+0000 7ff546000640 1 -- 172.21.15.42:0/3051637177 learned_addr learned my addr 172.21.15.42:0/3051637177 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:47.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.532+0000 7ff547400640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff548103190 0x7ff548074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.532+0000 7ff546a00640 1 -- 172.21.15.42:0/3051637177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff548103190 msgr2=0x7ff548074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.532+0000 7ff546a00640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff548103190 0x7ff548074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.532+0000 7ff546a00640 1 -- 172.21.15.42:0/3051637177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5481027d0 msgr2=0x7ff548073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.532+0000 7ff546a00640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5481027d0 0x7ff548073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.532+0000 7ff546a00640 1 -- 172.21.15.42:0/3051637177 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff548074900 con 0x7ff5481015e0 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.532+0000 7ff546a00640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5481015e0 0x7ff5480754f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff53800e9c0 tx=0x7ff53800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.533+0000 7ff53f400640 1 -- 172.21.15.42:0/3051637177 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53800ccc0 con 0x7ff5481015e0 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.533+0000 7ff53f400640 1 -- 172.21.15.42:0/3051637177 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5380035f0 con 0x7ff5481015e0 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.533+0000 7ff53f400640 1 -- 172.21.15.42:0/3051637177 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53800ce60 con 0x7ff5481015e0 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.533+0000 7ff547e00640 1 -- 172.21.15.42:0/3051637177 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5481a1a10 con 0x7ff5481015e0 2024-04-25T00:22:47.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.533+0000 7ff547e00640 1 -- 172.21.15.42:0/3051637177 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5481a1f10 con 0x7ff5481015e0 2024-04-25T00:22:47.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.536+0000 7ff53f400640 1 -- 172.21.15.42:0/3051637177 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff5380037a0 con 0x7ff5481015e0 2024-04-25T00:22:47.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.536+0000 7ff5503f0740 1 Processor -- start 2024-04-25T00:22:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.537+0000 7ff5503f0740 1 -- start start 2024-04-25T00:22:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.537+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 0x55e94dc60270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.537+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc65340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.537+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc67e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.537+0000 7ff5503f0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e94dbc2ab0 con 0x55e94dc659b0 2024-04-25T00:22:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.537+0000 7ff5503f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e94db6b640 con 0x55e94dc60840 2024-04-25T00:22:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.537+0000 7ff5503f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e94dbbea70 con 0x55e94dc5fe50 2024-04-25T00:22:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.538+0000 7ff546a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc65340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.538+0000 7ff547400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 0x55e94dc60270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.538+0000 7ff546a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc65340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53842/0 (socket says 172.21.15.42:53842) 2024-04-25T00:22:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.538+0000 7ff546a00640 1 -- 172.21.15.42:0/3320232780 learned_addr learned my addr 172.21.15.42:0/3320232780 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:47.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.538+0000 7ff546000640 1 --2- 172.21.15.42:0/3320232780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc67e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.538+0000 7ff546a00640 1 -- 172.21.15.42:0/3320232780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 msgr2=0x55e94dc60270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.538+0000 7ff546a00640 1 --2- 172.21.15.42:0/3320232780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 0x55e94dc60270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.538+0000 7ff546a00640 1 -- 172.21.15.42:0/3320232780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 msgr2=0x55e94dc67e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:47.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.538+0000 7ff546a00640 1 --2- 172.21.15.42:0/3320232780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc67e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.538+0000 7ff546a00640 1 -- 172.21.15.42:0/3320232780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e94d8d7210 con 0x55e94dc60840 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.539+0000 7ff546a00640 1 --2- 172.21.15.42:0/3320232780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc65340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff5380d54c0 tx=0x7ff5380d5980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=677893611c46ef6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.539+0000 7ff53f400640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff52c0a2c00 0x7ff52c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.539+0000 7ff533400640 1 -- 172.21.15.42:0/3320232780 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5380e2070 con 0x55e94dc60840 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.539+0000 7ff547400640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff52c0a2c00 0x7ff52c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.539+0000 7ff533400640 1 -- 172.21.15.42:0/3320232780 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff538014c30 con 0x55e94dc60840 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.539+0000 7ff533400640 1 -- 172.21.15.42:0/3320232780 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff538004000 con 0x55e94dc60840 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.539+0000 7ff53f400640 1 -- 172.21.15.42:0/3051637177 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff5380cf570 con 0x7ff5481015e0 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.540+0000 7ff5503f0740 1 -- 172.21.15.42:0/3320232780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 msgr2=0x55e94dc65340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.540+0000 7ff547400640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff52c0a2c00 0x7ff52c0a50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff540005e30 tx=0x7ff540031bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.540+0000 7ff5503f0740 1 --2- 172.21.15.42:0/3320232780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc65340 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff5380d54c0 tx=0x7ff5380d5980 comp rx=0 tx=0).stop 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.540+0000 7ff5503f0740 1 -- 172.21.15.42:0/3320232780 shutdown_connections 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.540+0000 7ff5503f0740 1 --2- 172.21.15.42:0/3320232780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc67e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.540+0000 7ff5503f0740 1 --2- 172.21.15.42:0/3320232780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc65340 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.540+0000 7ff5503f0740 1 --2- 172.21.15.42:0/3320232780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 0x55e94dc60270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.540+0000 7ff5503f0740 1 -- 172.21.15.42:0/3320232780 >> 172.21.15.42:0/3320232780 conn(0x55e94dc5b4c0 msgr2=0x55e94dc5b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.540+0000 7ff5503f0740 1 -- 172.21.15.42:0/3320232780 shutdown_connections 2024-04-25T00:22:47.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.540+0000 7ff5503f0740 1 -- 172.21.15.42:0/3320232780 wait complete. 2024-04-25T00:22:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.541+0000 7ff5503f0740 1 -- start start 2024-04-25T00:22:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.541+0000 7ff5503f0740 1 Processor -- start 2024-04-25T00:22:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.541+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc6a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.541+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 0x55e94dc6a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc71610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff5503f0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e94dc68960 con 0x55e94dc659b0 2024-04-25T00:22:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff5503f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e94db7e990 con 0x55e94dc5fe50 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff5503f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e94dc68ae0 con 0x55e94dc60840 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff5503f0740 10 client.0 _ll_register_callbacks cb 0x55e94db80950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff546a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc6a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff546000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 0x55e94dc6a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff546a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc6a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53848/0 (socket says 172.21.15.42:53848) 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff546a00640 1 -- 172.21.15.42:0/1504080678 learned_addr learned my addr 172.21.15.42:0/1504080678 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff547400640 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc71610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff546a00640 1 -- 172.21.15.42:0/1504080678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 msgr2=0x55e94dc6a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff546a00640 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 0x55e94dc6a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff546a00640 1 -- 172.21.15.42:0/1504080678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 msgr2=0x55e94dc71610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff546a00640 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc71610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff546a00640 1 -- 172.21.15.42:0/1504080678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e94d9a3620 con 0x55e94dc5fe50 2024-04-25T00:22:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff547400640 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc71610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff546a00640 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc6a090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff538098200 tx=0x7ff538007f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f21065045c724b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.542+0000 7ff527e00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff527e00640 1 -- 172.21.15.42:0/1504080678 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5380191f0 con 0x55e94dc5fe50 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff527e00640 1 -- 172.21.15.42:0/1504080678 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff53809a0a0 con 0x55e94dc5fe50 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff5503f0740 1 -- 172.21.15.42:0/1504080678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff538014c30 con 0x55e94dc5fe50 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff527e00640 1 -- 172.21.15.42:0/1504080678 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff5380193d0 con 0x55e94dc5fe50 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff5503f0740 10 client.25505 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff5503f0740 1 -- 172.21.15.42:0/1504080678 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e94dc68960 con 0x55e94dc5fe50 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff527e00640 1 -- 172.21.15.42:0/1504080678 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff538014c30 con 0x55e94dc5fe50 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff527e00640 1 -- 172.21.15.42:0/1504080678 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7ff538098410 con 0x55e94dc5fe50 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff5503f0740 10 client.25505 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff5503f0740 10 client.25505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff5503f0740 20 client.25505 populate_metadata read hostname 'smithi042' 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff5503f0740 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e94dc75810 0x55e94dc77c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff5503f0740 4 client.25505 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.543+0000 7ff5503f0740 1 -- 172.21.15.42:0/1504080678 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e94dbcfff0 con 0x55e94dc75810 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.544+0000 7ff546000640 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e94dc75810 0x55e94dc77c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.544+0000 7ff546000640 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e94dc75810 0x55e94dc77c00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75701ccf51284330 server_cookie=858a10a1c06dc7b4 in_seq=0 out_seq=0 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.544+0000 7ff527e00640 10 client.25505 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff527e00640 1 -- 172.21.15.42:0/1504080678 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e94dbcfff0 con 0x55e94dc75810 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff527e00640 10 client.25505 handle_command_reply: tid=0 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff5503f0740 1 client.25505 shutdown 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff5503f0740 1 -- 172.21.15.42:0/1504080678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 msgr2=0x55e94dc6a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff5503f0740 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc6a090 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff538098200 tx=0x7ff538007f20 comp rx=0 tx=0).stop 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff5503f0740 1 -- 172.21.15.42:0/1504080678 shutdown_connections 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff5503f0740 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e94dc75810 0x55e94dc77c00 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff5503f0740 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc71610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff5503f0740 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 0x55e94dc6a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff5503f0740 1 --2- 172.21.15.42:0/1504080678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc6a090 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff5503f0740 1 -- 172.21.15.42:0/1504080678 >> 172.21.15.42:0/1504080678 conn(0x55e94dc5b4c0 msgr2=0x55e94dc5a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.545+0000 7ff527e00640 0 client.25505 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.546+0000 7ff5503f0740 1 -- 172.21.15.42:0/1504080678 shutdown_connections 2024-04-25T00:22:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.546+0000 7ff5503f0740 1 -- 172.21.15.42:0/1504080678 wait complete. 2024-04-25T00:22:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.546+0000 7ff5503f0740 20 client.25505 trim_cache size 0 max 16384 2024-04-25T00:22:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.565+0000 7ff5503f0740 1 Processor -- start 2024-04-25T00:22:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.565+0000 7ff5503f0740 1 -- start start 2024-04-25T00:22:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.565+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc5e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.565+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 0x55e94dc5e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.565+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc696b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.565+0000 7ff5503f0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481ad550 con 0x55e94dc659b0 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.565+0000 7ff5503f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481aadf0 con 0x55e94dc5fe50 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.565+0000 7ff5503f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481a8c40 con 0x55e94dc60840 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc5e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 0x55e94dc5e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc5e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53860/0 (socket says 172.21.15.42:53860) 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546a00640 1 -- 172.21.15.42:0/3809382787 learned_addr learned my addr 172.21.15.42:0/3809382787 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff547400640 1 --2- 172.21.15.42:0/3809382787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546a00640 1 -- 172.21.15.42:0/3809382787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 msgr2=0x55e94dc5e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546a00640 1 --2- 172.21.15.42:0/3809382787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 0x55e94dc5e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546a00640 1 -- 172.21.15.42:0/3809382787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 msgr2=0x55e94dc696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546a00640 1 --2- 172.21.15.42:0/3809382787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546a00640 1 -- 172.21.15.42:0/3809382787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e94dc78280 con 0x55e94dc5fe50 2024-04-25T00:22:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546000640 1 --2- 172.21.15.42:0/3809382787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 0x55e94dc5e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.566+0000 7ff546a00640 1 --2- 172.21.15.42:0/3809382787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc5e3d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff538009fd0 tx=0x7ff538003d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be160067b494263c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.567+0000 7ff532a00640 1 -- 172.21.15.42:0/3809382787 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5380195b0 con 0x55e94dc5fe50 2024-04-25T00:22:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.567+0000 7ff532a00640 1 -- 172.21.15.42:0/3809382787 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff53809c4a0 con 0x55e94dc5fe50 2024-04-25T00:22:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.567+0000 7ff532a00640 1 -- 172.21.15.42:0/3809382787 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5380234f0 con 0x55e94dc5fe50 2024-04-25T00:22:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.567+0000 7ff5503f0740 1 -- 172.21.15.42:0/3809382787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 msgr2=0x55e94dc5e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.567+0000 7ff5503f0740 1 --2- 172.21.15.42:0/3809382787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc5e3d0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff538009fd0 tx=0x7ff538003d70 comp rx=0 tx=0).stop 2024-04-25T00:22:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.568+0000 7ff5503f0740 1 -- 172.21.15.42:0/3809382787 shutdown_connections 2024-04-25T00:22:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.568+0000 7ff5503f0740 1 --2- 172.21.15.42:0/3809382787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.568+0000 7ff5503f0740 1 --2- 172.21.15.42:0/3809382787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc60840 0x55e94dc5e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.568+0000 7ff5503f0740 1 --2- 172.21.15.42:0/3809382787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc5fe50 0x55e94dc5e3d0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.568+0000 7ff5503f0740 1 -- 172.21.15.42:0/3809382787 >> 172.21.15.42:0/3809382787 conn(0x55e94dc5b4c0 msgr2=0x55e94dc56110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.568+0000 7ff5503f0740 1 -- 172.21.15.42:0/3809382787 shutdown_connections 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.568+0000 7ff5503f0740 1 -- 172.21.15.42:0/3809382787 wait complete. 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.568+0000 7ff5503f0740 1 -- start start 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.568+0000 7ff5503f0740 1 Processor -- start 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 0x55e94dc6e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc6e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff5503f0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc6f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff5503f0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481a92b0 con 0x55e94dc659b0 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff5503f0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481aa720 con 0x55e94dc60840 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff5503f0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5481a9980 con 0x55e94dc5fe50 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff5503f0740 10 client.0 _ll_register_callbacks cb 0x7ff5380e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:22:47.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff546000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc6e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff546a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 0x55e94dc6e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff547400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc6f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff546000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc6e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53876/0 (socket says 172.21.15.42:53876) 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff546a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 0x55e94dc6e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44958/0 (socket says 172.21.15.42:44958) 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff546000640 1 -- 172.21.15.42:0/86556660 learned_addr learned my addr 172.21.15.42:0/86556660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff546000640 1 -- 172.21.15.42:0/86556660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 msgr2=0x55e94dc6e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff546000640 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 0x55e94dc6e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff546000640 1 -- 172.21.15.42:0/86556660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 msgr2=0x55e94dc6f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff546000640 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc6f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.569+0000 7ff546000640 1 -- 172.21.15.42:0/86556660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e94dc5b2f0 con 0x55e94dc60840 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.570+0000 7ff546a00640 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 0x55e94dc6e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:22:47.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.570+0000 7ff546000640 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc6e9e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55e94dc5c390 tx=0x7ff534013910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64c372fb24575d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.570+0000 7ff527e00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.570+0000 7ff527e00640 1 -- 172.21.15.42:0/86556660 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff534026070 con 0x55e94dc60840 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.570+0000 7ff527e00640 1 -- 172.21.15.42:0/86556660 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff534004250 con 0x55e94dc60840 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.570+0000 7ff5503f0740 1 -- 172.21.15.42:0/86556660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff53809c4a0 con 0x55e94dc60840 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff527e00640 1 -- 172.21.15.42:0/86556660 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ff534004450 con 0x55e94dc60840 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff5503f0740 10 client.25511 fetch_fsmap learned FSMap version 28 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff5503f0740 1 -- 172.21.15.42:0/86556660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e94dcdbe20 con 0x55e94dc60840 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff527e00640 1 -- 172.21.15.42:0/86556660 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff53809c4a0 con 0x55e94dc60840 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff527e00640 1 -- 172.21.15.42:0/86556660 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 28) ==== 8342+0+0 (secure 0 0 0) 0x7ff534029020 con 0x55e94dc60840 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff5503f0740 10 client.25511 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff5503f0740 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:22:47.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff5503f0740 20 client.25511 populate_metadata read hostname 'smithi042' 2024-04-25T00:22:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff5503f0740 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e94dcdc880 0x55e94dcdeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:22:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff5503f0740 4 client.25511 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:22:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff5503f0740 1 -- 172.21.15.42:0/86556660 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e94dbca340 con 0x55e94dcdc880 2024-04-25T00:22:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.571+0000 7ff546a00640 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e94dcdc880 0x55e94dcdeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:22:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.572+0000 7ff546a00640 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e94dcdc880 0x55e94dcdeca0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b34a5eece5eeba server_cookie=201124100a6940d1 in_seq=0 out_seq=0 2024-04-25T00:22:47.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.572+0000 7ff527e00640 10 client.25511 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff527e00640 1 -- 172.21.15.42:0/86556660 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e94dbca340 con 0x55e94dcdc880 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff527e00640 10 client.25511 handle_command_reply: tid=0 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 client.25511 shutdown 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 -- 172.21.15.42:0/86556660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 msgr2=0x55e94dc6e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc6e9e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55e94dc5c390 tx=0x7ff534013910 comp rx=0 tx=0).stop 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 -- 172.21.15.42:0/86556660 shutdown_connections 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e94dcdc880 0x55e94dcdeca0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e94dc659b0 0x55e94dc6f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e94dc60840 0x55e94dc6e9e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff527e00640 0 client.25511 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:22:47.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 --2- 172.21.15.42:0/86556660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e94dc5fe50 0x55e94dc6e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 -- 172.21.15.42:0/86556660 >> 172.21.15.42:0/86556660 conn(0x55e94dc5b4c0 msgr2=0x55e94dc5e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 -- 172.21.15.42:0/86556660 shutdown_connections 2024-04-25T00:22:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.623+0000 7ff5503f0740 1 -- 172.21.15.42:0/86556660 wait complete. 2024-04-25T00:22:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.624+0000 7ff5503f0740 20 client.25511 trim_cache size 0 max 16384 2024-04-25T00:22:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:22:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:22:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:22:47.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:22:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.624+0000 7ff547e00640 1 -- 172.21.15.42:0/3051637177 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff52c0a2c00 msgr2=0x7ff52c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.624+0000 7ff547e00640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff52c0a2c00 0x7ff52c0a50c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff540005e30 tx=0x7ff540031bb0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.624+0000 7ff547e00640 1 -- 172.21.15.42:0/3051637177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5481015e0 msgr2=0x7ff5480754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:22:47.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.624+0000 7ff547e00640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5481015e0 0x7ff5480754f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff53800e9c0 tx=0x7ff53800ee80 comp rx=0 tx=0).stop 2024-04-25T00:22:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.625+0000 7ff547e00640 1 -- 172.21.15.42:0/3051637177 shutdown_connections 2024-04-25T00:22:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.625+0000 7ff547e00640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff52c0a2c00 0x7ff52c0a50c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.625+0000 7ff547e00640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff548103190 0x7ff548074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.625+0000 7ff547e00640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff5481027d0 0x7ff548073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.625+0000 7ff547e00640 1 --2- 172.21.15.42:0/3051637177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5481015e0 0x7ff5480754f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:22:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.625+0000 7ff547e00640 1 -- 172.21.15.42:0/3051637177 >> 172.21.15.42:0/3051637177 conn(0x7ff5480fcb50 msgr2=0x7ff548105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:22:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.625+0000 7ff547e00640 1 -- 172.21.15.42:0/3051637177 shutdown_connections 2024-04-25T00:22:47.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:22:47.625+0000 7ff547e00640 1 -- 172.21.15.42:0/3051637177 wait complete. 2024-04-25T00:22:47.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:47 smithi042 ceph-mon[28130]: pgmap v509: 97 pgs: 97 active+clean; 56 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 234 op/s 2024-04-25T00:22:47.989 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:47 smithi106 ceph-mon[32510]: pgmap v509: 97 pgs: 97 active+clean; 56 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 234 op/s 2024-04-25T00:22:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:47 smithi136 ceph-mon[32923]: pgmap v509: 97 pgs: 97 active+clean; 56 MiB data, 869 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 234 op/s 2024-04-25T00:22:48.219 INFO:tasks.workunit.client.0.smithi042.stdout: 1 199081 6.53 MB/sec execute 38 sec latency 35.770 ms 2024-04-25T00:22:49.220 INFO:tasks.workunit.client.0.smithi042.stdout: 1 200064 6.49 MB/sec execute 39 sec latency 24.706 ms 2024-04-25T00:22:49.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:49 smithi042 ceph-mon[28130]: pgmap v510: 97 pgs: 97 active+clean; 57 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-04-25T00:22:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:49 smithi136 ceph-mon[32923]: pgmap v510: 97 pgs: 97 active+clean; 57 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-04-25T00:22:50.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:49 smithi106 ceph-mon[32510]: pgmap v510: 97 pgs: 97 active+clean; 57 MiB data, 875 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-04-25T00:22:50.220 INFO:tasks.workunit.client.0.smithi042.stdout: 1 201398 6.48 MB/sec execute 40 sec latency 40.206 ms 2024-04-25T00:22:51.220 INFO:tasks.workunit.client.0.smithi042.stdout: 1 202553 6.52 MB/sec execute 41 sec latency 29.254 ms 2024-04-25T00:22:51.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:51 smithi042 ceph-mon[28130]: pgmap v511: 97 pgs: 97 active+clean; 54 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-04-25T00:22:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:51 smithi136 ceph-mon[32923]: pgmap v511: 97 pgs: 97 active+clean; 54 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-04-25T00:22:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:51 smithi106 ceph-mon[32510]: pgmap v511: 97 pgs: 97 active+clean; 54 MiB data, 876 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-04-25T00:22:52.220 INFO:tasks.workunit.client.0.smithi042.stdout: 1 203258 6.41 MB/sec execute 42 sec latency 29.995 ms 2024-04-25T00:22:53.223 INFO:tasks.workunit.client.0.smithi042.stdout: 1 204564 6.38 MB/sec execute 43 sec latency 27.936 ms 2024-04-25T00:22:53.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:53 smithi042 ceph-mon[28130]: pgmap v512: 97 pgs: 97 active+clean; 56 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-04-25T00:22:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:53 smithi136 ceph-mon[32923]: pgmap v512: 97 pgs: 97 active+clean; 56 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-04-25T00:22:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:53 smithi106 ceph-mon[32510]: pgmap v512: 97 pgs: 97 active+clean; 56 MiB data, 901 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-04-25T00:22:54.220 INFO:tasks.workunit.client.0.smithi042.stdout: 1 205970 6.50 MB/sec execute 44 sec latency 16.378 ms 2024-04-25T00:22:55.220 INFO:tasks.workunit.client.0.smithi042.stdout: 1 206451 6.37 MB/sec execute 45 sec latency 74.804 ms 2024-04-25T00:22:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:55 smithi136 ceph-mon[32923]: pgmap v513: 97 pgs: 97 active+clean; 56 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-04-25T00:22:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:55 smithi106 ceph-mon[32510]: pgmap v513: 97 pgs: 97 active+clean; 56 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-04-25T00:22:56.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:55 smithi042 ceph-mon[28130]: pgmap v513: 97 pgs: 97 active+clean; 56 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-04-25T00:22:56.220 INFO:tasks.workunit.client.0.smithi042.stdout: 1 207463 6.34 MB/sec execute 46 sec latency 33.436 ms 2024-04-25T00:22:57.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:56 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:57.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:56 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:57.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:56 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:22:57.220 INFO:tasks.workunit.client.0.smithi042.stdout: 1 209248 6.46 MB/sec execute 47 sec latency 19.554 ms 2024-04-25T00:22:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:57 smithi136 ceph-mon[32923]: pgmap v514: 97 pgs: 97 active+clean; 58 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 235 op/s 2024-04-25T00:22:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:57 smithi106 ceph-mon[32510]: pgmap v514: 97 pgs: 97 active+clean; 58 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 235 op/s 2024-04-25T00:22:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:57 smithi042 ceph-mon[28130]: pgmap v514: 97 pgs: 97 active+clean; 58 MiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 235 op/s 2024-04-25T00:22:58.221 INFO:tasks.workunit.client.0.smithi042.stdout: 1 209830 6.36 MB/sec execute 48 sec latency 15.800 ms 2024-04-25T00:22:59.221 INFO:tasks.workunit.client.0.smithi042.stdout: 1 210663 6.34 MB/sec execute 49 sec latency 229.285 ms 2024-04-25T00:23:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:22:59 smithi136 ceph-mon[32923]: pgmap v515: 97 pgs: 97 active+clean; 58 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:23:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:22:59 smithi106 ceph-mon[32510]: pgmap v515: 97 pgs: 97 active+clean; 58 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:23:00.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:22:59 smithi042 ceph-mon[28130]: pgmap v515: 97 pgs: 97 active+clean; 58 MiB data, 963 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:23:00.221 INFO:tasks.workunit.client.0.smithi042.stdout: 1 211854 6.27 MB/sec execute 50 sec latency 29.993 ms 2024-04-25T00:23:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.776+0000 7f6193200640 1 -- 172.21.15.42:0/1172081501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61940ffe40 msgr2=0x7f6194100260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.776+0000 7f6193200640 1 --2- 172.21.15.42:0/1172081501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61940ffe40 0x7f6194100260 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6184009ca0 tx=0x7f618402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.776+0000 7f6193200640 1 -- 172.21.15.42:0/1172081501 shutdown_connections 2024-04-25T00:23:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.776+0000 7f6193200640 1 --2- 172.21.15.42:0/1172081501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6194101140 0x7f61941015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.776+0000 7f6193200640 1 --2- 172.21.15.42:0/1172081501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f61941007a0 0x7f6194100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.776+0000 7f6193200640 1 --2- 172.21.15.42:0/1172081501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61940ffe40 0x7f6194100260 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.776+0000 7f6193200640 1 -- 172.21.15.42:0/1172081501 >> 172.21.15.42:0/1172081501 conn(0x7f61940fba40 msgr2=0x7f61940fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.777+0000 7f6193200640 1 -- 172.21.15.42:0/1172081501 shutdown_connections 2024-04-25T00:23:00.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.777+0000 7f6193200640 1 -- 172.21.15.42:0/1172081501 wait complete. 2024-04-25T00:23:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.778+0000 7f6193200640 1 Processor -- start 2024-04-25T00:23:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.778+0000 7f6193200640 1 -- start start 2024-04-25T00:23:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.778+0000 7f6193200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61940ffe40 0x7f6194199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.778+0000 7f6193200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61941007a0 0x7f619419a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.778+0000 7f6193200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6194101140 0x7f61941a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.778+0000 7f6193200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f619410f480 con 0x7f6194101140 2024-04-25T00:23:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.778+0000 7f6193200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f619410f300 con 0x7f61940ffe40 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.778+0000 7f6193200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f619410f600 con 0x7f61941007a0 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.778+0000 7f6189400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61941007a0 0x7f619419a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.778+0000 7f6189400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61941007a0 0x7f619419a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41384/0 (socket says 172.21.15.42:41384) 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f6189400640 1 -- 172.21.15.42:0/1483336795 learned_addr learned my addr 172.21.15.42:0/1483336795 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f6191e00640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61940ffe40 0x7f6194199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f6192800640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6194101140 0x7f61941a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f6189400640 1 -- 172.21.15.42:0/1483336795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61940ffe40 msgr2=0x7f6194199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f6189400640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61940ffe40 0x7f6194199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f6189400640 1 -- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6194101140 msgr2=0x7f61941a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f6189400640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6194101140 0x7f61941a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f6189400640 1 -- 172.21.15.42:0/1483336795 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61941a1710 con 0x7f61941007a0 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f6192800640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6194101140 0x7f61941a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:00.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f6189400640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61941007a0 0x7f619419a030 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f617c00ea60 tx=0x7f617c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f618b400640 1 -- 172.21.15.42:0/1483336795 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c00cde0 con 0x7f61941007a0 2024-04-25T00:23:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.779+0000 7f618b400640 1 -- 172.21.15.42:0/1483336795 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f617c0035f0 con 0x7f61941007a0 2024-04-25T00:23:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.780+0000 7f6193200640 1 -- 172.21.15.42:0/1483336795 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61941a1a00 con 0x7f61941007a0 2024-04-25T00:23:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.780+0000 7f618b400640 1 -- 172.21.15.42:0/1483336795 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c010670 con 0x7f61941007a0 2024-04-25T00:23:00.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.780+0000 7f6193200640 1 -- 172.21.15.42:0/1483336795 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61941a1fa0 con 0x7f61941007a0 2024-04-25T00:23:00.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.783+0000 7f618b400640 1 -- 172.21.15.42:0/1483336795 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f617c0037a0 con 0x7f61941007a0 2024-04-25T00:23:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.783+0000 7f619b831740 1 Processor -- start 2024-04-25T00:23:00.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f619b831740 1 -- start start 2024-04-25T00:23:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f619b831740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f619b831740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 0x55b8acbb8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f619b831740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 0x55b8acbbae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f619b831740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b8acb15ab0 con 0x55b8acbb3840 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f619b831740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8acabe640 con 0x55b8acbb2e50 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f619b831740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8acb11a70 con 0x55b8acbb89b0 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f6192800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f6192800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:36778/0 (socket says 172.21.15.42:36778) 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f6192800640 1 -- 172.21.15.42:0/3040409815 learned_addr learned my addr 172.21.15.42:0/3040409815 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f6189400640 1 --2- 172.21.15.42:0/3040409815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 0x55b8acbbae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.784+0000 7f6191e00640 1 --2- 172.21.15.42:0/3040409815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 0x55b8acbb8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f6192800640 1 -- 172.21.15.42:0/3040409815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 msgr2=0x55b8acbbae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f6192800640 1 --2- 172.21.15.42:0/3040409815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 0x55b8acbbae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f6192800640 1 -- 172.21.15.42:0/3040409815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 msgr2=0x55b8acbb8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f6192800640 1 --2- 172.21.15.42:0/3040409815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 0x55b8acbb8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f6192800640 1 -- 172.21.15.42:0/3040409815 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8ac82a210 con 0x55b8acbb2e50 2024-04-25T00:23:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f6192800640 1 --2- 172.21.15.42:0/3040409815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb3270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f618c009bb0 tx=0x7f618c010740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbacd412307d928c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f618b400640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f61780a2c00 0x7f61780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f6183000640 1 -- 172.21.15.42:0/3040409815 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c004020 con 0x55b8acbb2e50 2024-04-25T00:23:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f6183000640 1 -- 172.21.15.42:0/3040409815 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f618c0041c0 con 0x55b8acbb2e50 2024-04-25T00:23:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f618b400640 1 -- 172.21.15.42:0/1483336795 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f617c0cf3b0 con 0x7f61941007a0 2024-04-25T00:23:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.785+0000 7f6183000640 1 -- 172.21.15.42:0/3040409815 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c012650 con 0x55b8acbb2e50 2024-04-25T00:23:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.786+0000 7f6191e00640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f61780a2c00 0x7f61780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.786+0000 7f619b831740 1 -- 172.21.15.42:0/3040409815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 msgr2=0x55b8acbb3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.786+0000 7f619b831740 1 --2- 172.21.15.42:0/3040409815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb3270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f618c009bb0 tx=0x7f618c010740 comp rx=0 tx=0).stop 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.786+0000 7f619b831740 1 -- 172.21.15.42:0/3040409815 shutdown_connections 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.786+0000 7f619b831740 1 --2- 172.21.15.42:0/3040409815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 0x55b8acbbae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.786+0000 7f619b831740 1 --2- 172.21.15.42:0/3040409815 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 0x55b8acbb8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.786+0000 7f619b831740 1 --2- 172.21.15.42:0/3040409815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb3270 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.787+0000 7f619b831740 1 -- 172.21.15.42:0/3040409815 >> 172.21.15.42:0/3040409815 conn(0x55b8acbae4c0 msgr2=0x55b8acbae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.787+0000 7f619b831740 1 -- 172.21.15.42:0/3040409815 shutdown_connections 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.787+0000 7f619b831740 1 -- 172.21.15.42:0/3040409815 wait complete. 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.787+0000 7f6191e00640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f61780a2c00 0x7f61780a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6184005e30 tx=0x7f6184005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.787+0000 7f619b831740 1 -- start start 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.787+0000 7f619b831740 1 Processor -- start 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f619b831740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb2e50 0x55b8acbbd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f619b831740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb3840 0x55b8acbbd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f619b831740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb89b0 0x55b8acbc4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f619b831740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b8acbbb960 con 0x55b8acbb2e50 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f619b831740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b8acad1990 con 0x55b8acbb89b0 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f619b831740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b8acbbbae0 con 0x55b8acbb3840 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f619b831740 10 client.0 _ll_register_callbacks cb 0x55b8acad3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f6191e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb2e50 0x55b8acbbd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f6189400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb3840 0x55b8acbbd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f6189400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb3840 0x55b8acbbd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41404/0 (socket says 172.21.15.42:41404) 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f6189400640 1 -- 172.21.15.42:0/1671453566 learned_addr learned my addr 172.21.15.42:0/1671453566 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f6191e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb2e50 0x55b8acbbd0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42866/0 (socket says 172.21.15.42:42866) 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.788+0000 7f6192800640 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb89b0 0x55b8acbc4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6189400640 1 -- 172.21.15.42:0/1671453566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb89b0 msgr2=0x55b8acbc4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6189400640 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb89b0 0x55b8acbc4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6189400640 1 -- 172.21.15.42:0/1671453566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb2e50 msgr2=0x55b8acbbd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6189400640 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb2e50 0x55b8acbbd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6189400640 1 -- 172.21.15.42:0/1671453566 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8ac8f6620 con 0x55b8acbb3840 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6192800640 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb89b0 0x55b8acbc4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6189400640 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb3840 0x55b8acbbd5e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f617c0985b0 tx=0x7f617c0d29e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54fdffad8b430e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6177400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6177400640 1 -- 172.21.15.42:0/1671453566 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c086050 con 0x55b8acbb3840 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6177400640 1 -- 172.21.15.42:0/1671453566 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f617c098250 con 0x55b8acbb3840 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f619b831740 1 -- 172.21.15.42:0/1671453566 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618c0041c0 con 0x55b8acbb3840 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f6177400640 1 -- 172.21.15.42:0/1671453566 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f617c09b540 con 0x55b8acbb3840 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f619b831740 10 client.25531 fetch_fsmap learned FSMap version 29 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.789+0000 7f619b831740 1 -- 172.21.15.42:0/1671453566 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b8acbc76c0 con 0x55b8acbb3840 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.790+0000 7f6177400640 1 -- 172.21.15.42:0/1671453566 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f618c0041c0 con 0x55b8acbb3840 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.790+0000 7f6177400640 1 -- 172.21.15.42:0/1671453566 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f617c09b030 con 0x55b8acbb3840 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.790+0000 7f619b831740 10 client.25531 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.790+0000 7f619b831740 10 client.25531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:00.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.790+0000 7f619b831740 20 client.25531 populate_metadata read hostname 'smithi042' 2024-04-25T00:23:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.790+0000 7f619b831740 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8acbc8d90 0x55b8acbcb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.790+0000 7f619b831740 4 client.25531 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.790+0000 7f619b831740 1 -- 172.21.15.42:0/1671453566 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8acb22ff0 con 0x55b8acbc8d90 2024-04-25T00:23:00.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.790+0000 7f6191e00640 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8acbc8d90 0x55b8acbcb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.791+0000 7f6191e00640 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8acbc8d90 0x55b8acbcb1c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e4ade57916bae2a server_cookie=3324dd43a5e237c5 in_seq=0 out_seq=0 2024-04-25T00:23:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.791+0000 7f6177400640 10 client.25531 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:00.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f6177400640 1 -- 172.21.15.42:0/1671453566 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b8acb22ff0 con 0x55b8acbc8d90 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f6177400640 10 client.25531 handle_command_reply: tid=0 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 client.25531 shutdown 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 -- 172.21.15.42:0/1671453566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb3840 msgr2=0x55b8acbbd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb3840 0x55b8acbbd5e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f617c0985b0 tx=0x7f617c0d29e0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 -- 172.21.15.42:0/1671453566 shutdown_connections 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8acbc8d90 0x55b8acbcb1c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb89b0 0x55b8acbc4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb3840 0x55b8acbbd5e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 --2- 172.21.15.42:0/1671453566 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb2e50 0x55b8acbbd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 -- 172.21.15.42:0/1671453566 >> 172.21.15.42:0/1671453566 conn(0x55b8acbae4c0 msgr2=0x55b8acbad620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f6177400640 0 client.25531 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 -- 172.21.15.42:0/1671453566 shutdown_connections 2024-04-25T00:23:00.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.792+0000 7f619b831740 1 -- 172.21.15.42:0/1671453566 wait complete. 2024-04-25T00:23:00.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.793+0000 7f619b831740 20 client.25531 trim_cache size 0 max 16384 2024-04-25T00:23:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f619b831740 1 Processor -- start 2024-04-25T00:23:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f619b831740 1 -- start start 2024-04-25T00:23:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f619b831740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f619b831740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 0x55b8acbb4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f619b831740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 0x55b8acbb4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f619b831740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61941ad450 con 0x55b8acbb3840 2024-04-25T00:23:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f619b831740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f61941aad20 con 0x55b8acbb2e50 2024-04-25T00:23:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f619b831740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f61941a8b70 con 0x55b8acbb89b0 2024-04-25T00:23:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f6189400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 0x55b8acbb4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f6192800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 0x55b8acbb4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f6189400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 0x55b8acbb4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42882/0 (socket says 172.21.15.42:42882) 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f6192800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 0x55b8acbb4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41416/0 (socket says 172.21.15.42:41416) 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.811+0000 7f6189400640 1 -- 172.21.15.42:0/1488190550 learned_addr learned my addr 172.21.15.42:0/1488190550 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.812+0000 7f6191e00640 1 --2- 172.21.15.42:0/1488190550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.812+0000 7f6192800640 1 -- 172.21.15.42:0/1488190550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 msgr2=0x55b8acbb80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.812+0000 7f6192800640 1 --2- 172.21.15.42:0/1488190550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.812+0000 7f6192800640 1 -- 172.21.15.42:0/1488190550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 msgr2=0x55b8acbb4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.812+0000 7f6192800640 1 --2- 172.21.15.42:0/1488190550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 0x55b8acbb4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.812+0000 7f6192800640 1 -- 172.21.15.42:0/1488190550 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8acbaca10 con 0x55b8acbb89b0 2024-04-25T00:23:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.812+0000 7f6191e00640 1 --2- 172.21.15.42:0/1488190550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.812+0000 7f6192800640 1 --2- 172.21.15.42:0/1488190550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 0x55b8acbb4fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f618c0093d0 tx=0x7f618c009980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=633fb5d7ca7984d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.813+0000 7f6182600640 1 -- 172.21.15.42:0/1488190550 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c02b070 con 0x55b8acbb89b0 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.813+0000 7f6182600640 1 -- 172.21.15.42:0/1488190550 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f618c012930 con 0x55b8acbb89b0 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.813+0000 7f6182600640 1 -- 172.21.15.42:0/1488190550 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c026900 con 0x55b8acbb89b0 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.813+0000 7f619b831740 1 -- 172.21.15.42:0/1488190550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 msgr2=0x55b8acbb4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.813+0000 7f619b831740 1 --2- 172.21.15.42:0/1488190550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 0x55b8acbb4fe0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f618c0093d0 tx=0x7f618c009980 comp rx=0 tx=0).stop 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.813+0000 7f619b831740 1 -- 172.21.15.42:0/1488190550 shutdown_connections 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.813+0000 7f619b831740 1 --2- 172.21.15.42:0/1488190550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb89b0 0x55b8acbb4fe0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.813+0000 7f619b831740 1 --2- 172.21.15.42:0/1488190550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb3840 0x55b8acbb4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.813+0000 7f619b831740 1 --2- 172.21.15.42:0/1488190550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb2e50 0x55b8acbb80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.813+0000 7f619b831740 1 -- 172.21.15.42:0/1488190550 >> 172.21.15.42:0/1488190550 conn(0x55b8acbae4c0 msgr2=0x55b8acbae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.814+0000 7f619b831740 1 -- 172.21.15.42:0/1488190550 shutdown_connections 2024-04-25T00:23:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.814+0000 7f619b831740 1 -- 172.21.15.42:0/1488190550 wait complete. 2024-04-25T00:23:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.814+0000 7f619b831740 1 -- start start 2024-04-25T00:23:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.814+0000 7f619b831740 1 Processor -- start 2024-04-25T00:23:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f619b831740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb2e50 0x55b8acbc1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f619b831740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb3840 0x55b8acbc1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f619b831740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb89b0 0x55b8acbc2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f619b831740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f61941a91e0 con 0x55b8acbb89b0 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f619b831740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f61941aa650 con 0x55b8acbb3840 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f619b831740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f61941a98b0 con 0x55b8acbb2e50 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f619b831740 10 client.0 _ll_register_callbacks cb 0x7f617c00e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f6191e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb2e50 0x55b8acbc1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f6191e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb2e50 0x55b8acbc1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41420/0 (socket says 172.21.15.42:41420) 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f6191e00640 1 -- 172.21.15.42:0/1570060631 learned_addr learned my addr 172.21.15.42:0/1570060631 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f6189400640 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb3840 0x55b8acbc1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.815+0000 7f6192800640 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb89b0 0x55b8acbc2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6191e00640 1 -- 172.21.15.42:0/1570060631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb3840 msgr2=0x55b8acbc1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6191e00640 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb3840 0x55b8acbc1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6191e00640 1 -- 172.21.15.42:0/1570060631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb89b0 msgr2=0x55b8acbc2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6191e00640 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb89b0 0x55b8acbc2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6191e00640 1 -- 172.21.15.42:0/1570060631 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8acbc2b60 con 0x55b8acbb2e50 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6189400640 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb3840 0x55b8acbc1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6192800640 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb89b0 0x55b8acbc2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6191e00640 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb2e50 0x55b8acbc1020 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6184009ca0 tx=0x7f618402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89ffcb5a49022f1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6177400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6177400640 1 -- 172.21.15.42:0/1570060631 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6184003af0 con 0x55b8acbb2e50 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f619b831740 1 -- 172.21.15.42:0/1570060631 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618c012930 con 0x55b8acbb2e50 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.816+0000 7f6177400640 1 -- 172.21.15.42:0/1570060631 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6184003c90 con 0x55b8acbb2e50 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f6177400640 1 -- 172.21.15.42:0/1570060631 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6184063050 con 0x55b8acbb2e50 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f619b831740 10 client.25537 fetch_fsmap learned FSMap version 29 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f619b831740 1 -- 172.21.15.42:0/1570060631 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b8acbc3ee0 con 0x55b8acbb2e50 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f6177400640 1 -- 172.21.15.42:0/1570060631 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f618c012930 con 0x55b8acbb2e50 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f6177400640 1 -- 172.21.15.42:0/1570060631 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f61840045d0 con 0x55b8acbb2e50 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f619b831740 10 client.25537 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:23:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f619b831740 10 client.25537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f619b831740 20 client.25537 populate_metadata read hostname 'smithi042' 2024-04-25T00:23:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f619b831740 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8acc2fba0 0x55b8acc31fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f619b831740 4 client.25537 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:23:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f619b831740 1 -- 172.21.15.42:0/1570060631 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b8acb1d340 con 0x55b8acc2fba0 2024-04-25T00:23:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.817+0000 7f6189400640 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8acc2fba0 0x55b8acc31fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.818+0000 7f6189400640 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8acc2fba0 0x55b8acc31fc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38c3b1c88c8bbafb server_cookie=21861ec7d285a1da in_seq=0 out_seq=0 2024-04-25T00:23:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.818+0000 7f6177400640 10 client.25537 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f6177400640 1 -- 172.21.15.42:0/1570060631 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b8acb1d340 con 0x55b8acc2fba0 2024-04-25T00:23:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f6177400640 10 client.25537 handle_command_reply: tid=0 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 client.25537 shutdown 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 -- 172.21.15.42:0/1570060631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb2e50 msgr2=0x55b8acbc1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb2e50 0x55b8acbc1020 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6184009ca0 tx=0x7f618402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 -- 172.21.15.42:0/1570060631 shutdown_connections 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b8acc2fba0 0x55b8acc31fc0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b8acbb89b0 0x55b8acbc2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b8acbb3840 0x55b8acbc1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f6177400640 0 client.25537 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 --2- 172.21.15.42:0/1570060631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b8acbb2e50 0x55b8acbc1020 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 -- 172.21.15.42:0/1570060631 >> 172.21.15.42:0/1570060631 conn(0x55b8acbae4c0 msgr2=0x55b8acbb4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 -- 172.21.15.42:0/1570060631 shutdown_connections 2024-04-25T00:23:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.866+0000 7f619b831740 1 -- 172.21.15.42:0/1570060631 wait complete. 2024-04-25T00:23:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.867+0000 7f619b831740 20 client.25537 trim_cache size 0 max 16384 2024-04-25T00:23:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:23:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:23:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:23:00.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:23:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.867+0000 7f6193200640 1 -- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f61780a2c00 msgr2=0x7f61780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.867+0000 7f6193200640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f61780a2c00 0x7f61780a50c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6184005e30 tx=0x7f6184005f90 comp rx=0 tx=0).stop 2024-04-25T00:23:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.867+0000 7f6193200640 1 -- 172.21.15.42:0/1483336795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61941007a0 msgr2=0x7f619419a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:00.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.867+0000 7f6193200640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61941007a0 0x7f619419a030 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f617c00ea60 tx=0x7f617c00ef20 comp rx=0 tx=0).stop 2024-04-25T00:23:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.868+0000 7f6193200640 1 -- 172.21.15.42:0/1483336795 shutdown_connections 2024-04-25T00:23:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.868+0000 7f6193200640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f61780a2c00 0x7f61780a50c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.868+0000 7f6193200640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6194101140 0x7f61941a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.868+0000 7f6193200640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f61941007a0 0x7f619419a030 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.868+0000 7f6193200640 1 --2- 172.21.15.42:0/1483336795 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f61940ffe40 0x7f6194199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.868+0000 7f6193200640 1 -- 172.21.15.42:0/1483336795 >> 172.21.15.42:0/1483336795 conn(0x7f61940fba40 msgr2=0x7f61940fd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.868+0000 7f6193200640 1 -- 172.21.15.42:0/1483336795 shutdown_connections 2024-04-25T00:23:00.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:00.868+0000 7f6193200640 1 -- 172.21.15.42:0/1483336795 wait complete. 2024-04-25T00:23:01.221 INFO:tasks.workunit.client.0.smithi042.stdout: 1 213183 6.36 MB/sec execute 51 sec latency 25.916 ms 2024-04-25T00:23:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:01 smithi136 ceph-mon[32923]: pgmap v516: 97 pgs: 97 active+clean; 59 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 196 op/s 2024-04-25T00:23:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:01 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:01 smithi106 ceph-mon[32510]: pgmap v516: 97 pgs: 97 active+clean; 59 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 196 op/s 2024-04-25T00:23:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:01 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:01 smithi042 ceph-mon[28130]: pgmap v516: 97 pgs: 97 active+clean; 59 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 196 op/s 2024-04-25T00:23:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:01 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:23:02.221 INFO:tasks.workunit.client.0.smithi042.stdout: 1 214051 6.28 MB/sec execute 52 sec latency 30.549 ms 2024-04-25T00:23:03.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:02 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:03.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:02 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:03.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:02 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:23:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:02 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:02 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:02 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:23:03.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:02 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:23:03.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:02 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:23:03.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:02 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:23:03.222 INFO:tasks.workunit.client.0.smithi042.stdout: 1 215803 6.36 MB/sec execute 53 sec latency 26.927 ms 2024-04-25T00:23:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:03 smithi136 ceph-mon[32923]: pgmap v517: 97 pgs: 97 active+clean; 60 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-04-25T00:23:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:03 smithi106 ceph-mon[32510]: pgmap v517: 97 pgs: 97 active+clean; 60 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-04-25T00:23:04.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:03 smithi042 ceph-mon[28130]: pgmap v517: 97 pgs: 97 active+clean; 60 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-04-25T00:23:04.222 INFO:tasks.workunit.client.0.smithi042.stdout: 1 216807 6.37 MB/sec execute 54 sec latency 17.534 ms 2024-04-25T00:23:05.222 INFO:tasks.workunit.client.0.smithi042.stdout: 1 217530 6.29 MB/sec execute 55 sec latency 51.591 ms 2024-04-25T00:23:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:05 smithi136 ceph-mon[32923]: pgmap v518: 97 pgs: 97 active+clean; 60 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-04-25T00:23:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:05 smithi106 ceph-mon[32510]: pgmap v518: 97 pgs: 97 active+clean; 60 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-04-25T00:23:06.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:05 smithi042 ceph-mon[28130]: pgmap v518: 97 pgs: 97 active+clean; 60 MiB data, 977 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-04-25T00:23:06.222 INFO:tasks.workunit.client.0.smithi042.stdout: 1 219004 6.28 MB/sec execute 56 sec latency 27.829 ms 2024-04-25T00:23:07.225 INFO:tasks.workunit.client.0.smithi042.stdout: 1 220356 6.37 MB/sec execute 57 sec latency 15.508 ms 2024-04-25T00:23:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:07 smithi136 ceph-mon[32923]: pgmap v519: 97 pgs: 97 active+clean; 60 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-04-25T00:23:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:07 smithi106 ceph-mon[32510]: pgmap v519: 97 pgs: 97 active+clean; 60 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-04-25T00:23:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:07 smithi042 ceph-mon[28130]: pgmap v519: 97 pgs: 97 active+clean; 60 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-04-25T00:23:08.222 INFO:tasks.workunit.client.0.smithi042.stdout: 1 221197 6.29 MB/sec execute 58 sec latency 61.679 ms 2024-04-25T00:23:09.222 INFO:tasks.workunit.client.0.smithi042.stdout: 1 222863 6.35 MB/sec execute 59 sec latency 24.542 ms 2024-04-25T00:23:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:09 smithi136 ceph-mon[32923]: pgmap v520: 97 pgs: 97 active+clean; 60 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-04-25T00:23:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:09 smithi106 ceph-mon[32510]: pgmap v520: 97 pgs: 97 active+clean; 60 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-04-25T00:23:10.130 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:09 smithi042 ceph-mon[28130]: pgmap v520: 97 pgs: 97 active+clean; 60 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.5 MiB/s wr, 186 op/s 2024-04-25T00:23:10.222 INFO:tasks.workunit.client.0.smithi042.stdout: 1 223913 6.37 MB/sec execute 60 sec latency 46.391 ms 2024-04-25T00:23:11.223 INFO:tasks.workunit.client.0.smithi042.stdout: 1 225009 6.35 MB/sec execute 61 sec latency 31.227 ms 2024-04-25T00:23:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:11 smithi136 ceph-mon[32923]: pgmap v521: 97 pgs: 97 active+clean; 61 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-04-25T00:23:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:11 smithi106 ceph-mon[32510]: pgmap v521: 97 pgs: 97 active+clean; 61 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-04-25T00:23:12.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:11 smithi042 ceph-mon[28130]: pgmap v521: 97 pgs: 97 active+clean; 61 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-04-25T00:23:12.223 INFO:tasks.workunit.client.0.smithi042.stdout: 1 226723 6.38 MB/sec execute 62 sec latency 24.189 ms 2024-04-25T00:23:13.223 INFO:tasks.workunit.client.0.smithi042.stdout: 1 227625 6.37 MB/sec execute 63 sec latency 14.514 ms 2024-04-25T00:23:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:13 smithi136 ceph-mon[32923]: pgmap v522: 97 pgs: 97 active+clean; 63 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 226 op/s 2024-04-25T00:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.026+0000 7f2c83200640 1 -- 172.21.15.42:0/4121639513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c841037e0 msgr2=0x7f2c84103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.026+0000 7f2c83200640 1 --2- 172.21.15.42:0/4121639513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c841037e0 0x7f2c84103c00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2c74009900 tx=0x7f2c7402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.026+0000 7f2c83200640 1 -- 172.21.15.42:0/4121639513 shutdown_connections 2024-04-25T00:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.026+0000 7f2c83200640 1 --2- 172.21.15.42:0/4121639513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c84105390 0x7f2c8410bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.026+0000 7f2c83200640 1 --2- 172.21.15.42:0/4121639513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c841049d0 0x7f2c84104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.026+0000 7f2c83200640 1 --2- 172.21.15.42:0/4121639513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c841037e0 0x7f2c84103c00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.026+0000 7f2c83200640 1 -- 172.21.15.42:0/4121639513 >> 172.21.15.42:0/4121639513 conn(0x7f2c840fed50 msgr2=0x7f2c84101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.026+0000 7f2c83200640 1 -- 172.21.15.42:0/4121639513 shutdown_connections 2024-04-25T00:23:14.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.026+0000 7f2c83200640 1 -- 172.21.15.42:0/4121639513 wait complete. 2024-04-25T00:23:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.028+0000 7f2c83200640 1 Processor -- start 2024-04-25T00:23:14.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.028+0000 7f2c83200640 1 -- start start 2024-04-25T00:23:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.028+0000 7f2c83200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c841037e0 0x7f2c84199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.028+0000 7f2c83200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c841049d0 0x7f2c8419a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.028+0000 7f2c83200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c84105390 0x7f2c841a10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.028+0000 7f2c83200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410f480 con 0x7f2c84105390 2024-04-25T00:23:14.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.028+0000 7f2c83200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410f300 con 0x7f2c841049d0 2024-04-25T00:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.028+0000 7f2c83200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8410f600 con 0x7f2c841037e0 2024-04-25T00:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c79400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c841049d0 0x7f2c8419a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c81e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c841037e0 0x7f2c84199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c79400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c841049d0 0x7f2c8419a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46638/0 (socket says 172.21.15.42:46638) 2024-04-25T00:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c79400640 1 -- 172.21.15.42:0/364478567 learned_addr learned my addr 172.21.15.42:0/364478567 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c81e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c841037e0 0x7f2c84199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:58310/0 (socket says 172.21.15.42:58310) 2024-04-25T00:23:14.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c82800640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c84105390 0x7f2c841a10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c79400640 1 -- 172.21.15.42:0/364478567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c841037e0 msgr2=0x7f2c84199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c79400640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c841037e0 0x7f2c84199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c79400640 1 -- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c84105390 msgr2=0x7f2c841a10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c79400640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c84105390 0x7f2c841a10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c79400640 1 -- 172.21.15.42:0/364478567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c841a1740 con 0x7f2c841049d0 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c81e00640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c841037e0 0x7f2c84199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c82800640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c84105390 0x7f2c841a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.029+0000 7f2c79400640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c841049d0 0x7f2c8419a060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00e9f0 tx=0x7f2c6c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.030+0000 7f2c7b400640 1 -- 172.21.15.42:0/364478567 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c00cd80 con 0x7f2c841049d0 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.030+0000 7f2c83200640 1 -- 172.21.15.42:0/364478567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c841a1a30 con 0x7f2c841049d0 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.030+0000 7f2c7b400640 1 -- 172.21.15.42:0/364478567 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c6c010910 con 0x7f2c841049d0 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.030+0000 7f2c7b400640 1 -- 172.21.15.42:0/364478567 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0188f0 con 0x7f2c841049d0 2024-04-25T00:23:14.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.030+0000 7f2c83200640 1 -- 172.21.15.42:0/364478567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c841a1f80 con 0x7f2c841049d0 2024-04-25T00:23:14.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.033+0000 7f2c7b400640 1 -- 172.21.15.42:0/364478567 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2c6c010ac0 con 0x7f2c841049d0 2024-04-25T00:23:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c8bb64740 1 Processor -- start 2024-04-25T00:23:14.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c8bb64740 1 -- start start 2024-04-25T00:23:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb0a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 0x56343eb0f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb11e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56343ea6cab0 con 0x56343eb0a840 2024-04-25T00:23:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56343ea15640 con 0x56343eb0f9b0 2024-04-25T00:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56343ea68a70 con 0x56343eb09e50 2024-04-25T00:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c82800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb0a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c82800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb0a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:58312/0 (socket says 172.21.15.42:58312) 2024-04-25T00:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c82800640 1 -- 172.21.15.42:0/4594174 learned_addr learned my addr 172.21.15.42:0/4594174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.034+0000 7f2c81e00640 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 0x56343eb0f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c82800640 1 -- 172.21.15.42:0/4594174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 msgr2=0x56343eb11e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c79400640 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb11e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c82800640 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb11e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c82800640 1 -- 172.21.15.42:0/4594174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 msgr2=0x56343eb0f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c82800640 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 0x56343eb0f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c82800640 1 -- 172.21.15.42:0/4594174 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343e781210 con 0x56343eb09e50 2024-04-25T00:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c81e00640 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 0x56343eb0f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c79400640 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb11e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c7b400640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2c680a2b80 0x7f2c680a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c82800640 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb0a270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00f410 tx=0x7f2c7c00f8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=870772265792c708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c7b400640 1 -- 172.21.15.42:0/364478567 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f2c6c0cf4d0 con 0x7f2c841049d0 2024-04-25T00:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.035+0000 7f2c81e00640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2c680a2b80 0x7f2c680a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.036+0000 7f2c73400640 1 -- 172.21.15.42:0/4594174 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c015070 con 0x56343eb09e50 2024-04-25T00:23:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.036+0000 7f2c73400640 1 -- 172.21.15.42:0/4594174 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c011be0 con 0x56343eb09e50 2024-04-25T00:23:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.036+0000 7f2c73400640 1 -- 172.21.15.42:0/4594174 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c010d10 con 0x56343eb09e50 2024-04-25T00:23:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c8bb64740 1 -- 172.21.15.42:0/4594174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 msgr2=0x56343eb0a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:14.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb0a270 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c00f410 tx=0x7f2c7c00f8d0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c81e00640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2c680a2b80 0x7f2c680a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c7402a9e0 tx=0x7f2c740315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c8bb64740 1 -- 172.21.15.42:0/4594174 shutdown_connections 2024-04-25T00:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb11e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 0x56343eb0f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/4594174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb0a270 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c8bb64740 1 -- 172.21.15.42:0/4594174 >> 172.21.15.42:0/4594174 conn(0x56343eb054c0 msgr2=0x56343eb058c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c8bb64740 1 -- 172.21.15.42:0/4594174 shutdown_connections 2024-04-25T00:23:14.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c8bb64740 1 -- 172.21.15.42:0/4594174 wait complete. 2024-04-25T00:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.037+0000 7f2c8bb64740 1 -- start start 2024-04-25T00:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c8bb64740 1 Processor -- start 2024-04-25T00:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb140d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 0x56343eb14610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb1b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56343eb12960 con 0x56343eb0a840 2024-04-25T00:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56343ea28990 con 0x56343eb0f9b0 2024-04-25T00:23:14.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56343eb12ae0 con 0x56343eb09e50 2024-04-25T00:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c8bb64740 10 client.0 _ll_register_callbacks cb 0x56343ea2a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c81e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c82800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb1b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c79400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 0x56343eb14610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c82800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb1b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46656/0 (socket says 172.21.15.42:46656) 2024-04-25T00:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c81e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:58328/0 (socket says 172.21.15.42:58328) 2024-04-25T00:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.038+0000 7f2c82800640 1 -- 172.21.15.42:0/3847985999 learned_addr learned my addr 172.21.15.42:0/3847985999 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:14.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.039+0000 7f2c82800640 1 -- 172.21.15.42:0/3847985999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 msgr2=0x56343eb140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.039+0000 7f2c82800640 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.039+0000 7f2c82800640 1 -- 172.21.15.42:0/3847985999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 msgr2=0x56343eb14610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.039+0000 7f2c82800640 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 0x56343eb14610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.039+0000 7f2c82800640 1 -- 172.21.15.42:0/3847985999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56343e84d620 con 0x56343eb0f9b0 2024-04-25T00:23:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.039+0000 7f2c81e00640 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.039+0000 7f2c67e00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:23:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.039+0000 7f2c82800640 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb1b650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009d90 tx=0x7f2c7c012fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29926388ed408e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:14.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.040+0000 7f2c67e00640 1 -- 172.21.15.42:0/3847985999 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c027020 con 0x56343eb0f9b0 2024-04-25T00:23:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.040+0000 7f2c67e00640 1 -- 172.21.15.42:0/3847985999 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f2c7c004a40 con 0x56343eb0f9b0 2024-04-25T00:23:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.040+0000 7f2c8bb64740 1 -- 172.21.15.42:0/3847985999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7c011be0 con 0x56343eb0f9b0 2024-04-25T00:23:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.040+0000 7f2c67e00640 1 -- 172.21.15.42:0/3847985999 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c7c015040 con 0x56343eb0f9b0 2024-04-25T00:23:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.040+0000 7f2c8bb64740 10 client.25538 fetch_fsmap learned FSMap version 29 2024-04-25T00:23:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.040+0000 7f2c8bb64740 1 -- 172.21.15.42:0/3847985999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56343eb1e5a0 con 0x56343eb0f9b0 2024-04-25T00:23:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.040+0000 7f2c67e00640 1 -- 172.21.15.42:0/3847985999 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7c011be0 con 0x56343eb0f9b0 2024-04-25T00:23:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.040+0000 7f2c67e00640 1 -- 172.21.15.42:0/3847985999 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2c7c022070 con 0x56343eb0f9b0 2024-04-25T00:23:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.041+0000 7f2c8bb64740 10 client.25538 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:23:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.041+0000 7f2c8bb64740 10 client.25538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.041+0000 7f2c8bb64740 20 client.25538 populate_metadata read hostname 'smithi042' 2024-04-25T00:23:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.041+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56343eb1f9a0 0x56343eb21d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.041+0000 7f2c8bb64740 4 client.25538 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.041+0000 7f2c8bb64740 1 -- 172.21.15.42:0/3847985999 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56343ea79ff0 con 0x56343eb1f9a0 2024-04-25T00:23:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.041+0000 7f2c81e00640 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56343eb1f9a0 0x56343eb21d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.042+0000 7f2c67e00640 10 client.25538 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.042+0000 7f2c81e00640 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56343eb1f9a0 0x56343eb21d90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15cb4aa053f40be7 server_cookie=1e59bb1230aaeba2 in_seq=0 out_seq=0 2024-04-25T00:23:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.042+0000 7f2c67e00640 1 -- 172.21.15.42:0/3847985999 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56343ea79ff0 con 0x56343eb1f9a0 2024-04-25T00:23:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.042+0000 7f2c67e00640 10 client.25538 handle_command_reply: tid=0 2024-04-25T00:23:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.042+0000 7f2c8bb64740 1 client.25538 shutdown 2024-04-25T00:23:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 1 -- 172.21.15.42:0/3847985999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 msgr2=0x56343eb1b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb1b650 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009d90 tx=0x7f2c7c012fb0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 1 -- 172.21.15.42:0/3847985999 shutdown_connections 2024-04-25T00:23:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56343eb1f9a0 0x56343eb21d90 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0f9b0 0x56343eb1b650 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0a840 0x56343eb14610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c67e00640 0 client.25538 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/3847985999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 1 -- 172.21.15.42:0/3847985999 >> 172.21.15.42:0/3847985999 conn(0x56343eb054c0 msgr2=0x56343eb04620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 1 -- 172.21.15.42:0/3847985999 shutdown_connections 2024-04-25T00:23:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 1 -- 172.21.15.42:0/3847985999 wait complete. 2024-04-25T00:23:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.043+0000 7f2c8bb64740 20 client.25538 trim_cache size 0 max 16384 2024-04-25T00:23:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.061+0000 7f2c8bb64740 1 Processor -- start 2024-04-25T00:23:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.061+0000 7f2c8bb64740 1 -- start start 2024-04-25T00:23:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.061+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb09e50 0x56343eb03d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.061+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb0a840 0x56343eb1b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 0x56343eb042d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841ad580 con 0x56343eb0f9b0 2024-04-25T00:23:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841aae50 con 0x56343eb09e50 2024-04-25T00:23:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841a8ca0 con 0x56343eb0a840 2024-04-25T00:23:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c81e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb09e50 0x56343eb03d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c81e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb09e50 0x56343eb03d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46660/0 (socket says 172.21.15.42:46660) 2024-04-25T00:23:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c81e00640 1 -- 172.21.15.42:0/616616753 learned_addr learned my addr 172.21.15.42:0/616616753 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c82800640 1 --2- 172.21.15.42:0/616616753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 0x56343eb042d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c79400640 1 --2- 172.21.15.42:0/616616753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb0a840 0x56343eb1b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c81e00640 1 -- 172.21.15.42:0/616616753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb0a840 msgr2=0x56343eb1b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c81e00640 1 --2- 172.21.15.42:0/616616753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb0a840 0x56343eb1b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c81e00640 1 -- 172.21.15.42:0/616616753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 msgr2=0x56343eb042d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c81e00640 1 --2- 172.21.15.42:0/616616753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 0x56343eb042d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.062+0000 7f2c81e00640 1 -- 172.21.15.42:0/616616753 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56343eb223c0 con 0x56343eb09e50 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.063+0000 7f2c81e00640 1 --2- 172.21.15.42:0/616616753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb09e50 0x56343eb03d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2c74000c00 tx=0x7f2c7403f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac244a43d481016c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.063+0000 7f2c72a00640 1 -- 172.21.15.42:0/616616753 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74035040 con 0x56343eb09e50 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.063+0000 7f2c72a00640 1 -- 172.21.15.42:0/616616753 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7405a070 con 0x56343eb09e50 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.063+0000 7f2c72a00640 1 -- 172.21.15.42:0/616616753 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c74031b80 con 0x56343eb09e50 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.064+0000 7f2c8bb64740 1 -- 172.21.15.42:0/616616753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb09e50 msgr2=0x56343eb03d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.064+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/616616753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb09e50 0x56343eb03d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2c74000c00 tx=0x7f2c7403f000 comp rx=0 tx=0).stop 2024-04-25T00:23:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.064+0000 7f2c8bb64740 1 -- 172.21.15.42:0/616616753 shutdown_connections 2024-04-25T00:23:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.064+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/616616753 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 0x56343eb042d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.064+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/616616753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb0a840 0x56343eb1b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.064+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/616616753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb09e50 0x56343eb03d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.064+0000 7f2c8bb64740 1 -- 172.21.15.42:0/616616753 >> 172.21.15.42:0/616616753 conn(0x56343eb054c0 msgr2=0x56343eb140d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.064+0000 7f2c8bb64740 1 -- 172.21.15.42:0/616616753 shutdown_connections 2024-04-25T00:23:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.064+0000 7f2c8bb64740 1 -- 172.21.15.42:0/616616753 wait complete. 2024-04-25T00:23:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.064+0000 7f2c8bb64740 1 -- start start 2024-04-25T00:23:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.065+0000 7f2c8bb64740 1 Processor -- start 2024-04-25T00:23:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.065+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb19070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.065+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0a840 0x56343eb195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.065+0000 7f2c8bb64740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 0x56343eb1a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.065+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841a9310 con 0x56343eb0f9b0 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.065+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841aa780 con 0x56343eb0a840 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.065+0000 7f2c8bb64740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2c841a99e0 con 0x56343eb09e50 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c8bb64740 10 client.0 _ll_register_callbacks cb 0x7f2c7c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c79400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0a840 0x56343eb195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c81e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb19070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c79400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0a840 0x56343eb195b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46674/0 (socket says 172.21.15.42:46674) 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c79400640 1 -- 172.21.15.42:0/2088078751 learned_addr learned my addr 172.21.15.42:0/2088078751 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c82800640 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 0x56343eb1a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c79400640 1 -- 172.21.15.42:0/2088078751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 msgr2=0x56343eb19070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c79400640 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb19070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c79400640 1 -- 172.21.15.42:0/2088078751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 msgr2=0x56343eb1a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c79400640 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 0x56343eb1a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c79400640 1 -- 172.21.15.42:0/2088078751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56343eb0f770 con 0x56343eb0a840 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c82800640 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 0x56343eb1a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c81e00640 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb19070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c79400640 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0a840 0x56343eb195b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c098110 tx=0x7f2c6c09c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76c7f6eda92aece2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.066+0000 7f2c67e00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c67e00640 1 -- 172.21.15.42:0/2088078751 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c0cf2b0 con 0x56343eb0a840 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c67e00640 1 -- 172.21.15.42:0/2088078751 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f2c6c09b150 con 0x56343eb0a840 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c8bb64740 1 -- 172.21.15.42:0/2088078751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7405a070 con 0x56343eb0a840 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c67e00640 1 -- 172.21.15.42:0/2088078751 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c6c09b350 con 0x56343eb0a840 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c8bb64740 10 client.25544 fetch_fsmap learned FSMap version 29 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c8bb64740 1 -- 172.21.15.42:0/2088078751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56343eb09a90 con 0x56343eb0a840 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c67e00640 1 -- 172.21.15.42:0/2088078751 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2c7405a070 con 0x56343eb0a840 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c67e00640 1 -- 172.21.15.42:0/2088078751 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f2c6c097c30 con 0x56343eb0a840 2024-04-25T00:23:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c8bb64740 10 client.25544 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:23:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c8bb64740 10 client.25544 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c8bb64740 20 client.25544 populate_metadata read hostname 'smithi042' 2024-04-25T00:23:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56343eb86760 0x56343eb88b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c8bb64740 4 client.25544 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:23:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.067+0000 7f2c8bb64740 1 -- 172.21.15.42:0/2088078751 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56343ea74340 con 0x56343eb86760 2024-04-25T00:23:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.068+0000 7f2c81e00640 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56343eb86760 0x56343eb88b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.068+0000 7f2c81e00640 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56343eb86760 0x56343eb88b80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29e2a3b515499f95 server_cookie=791a09e154970426 in_seq=0 out_seq=0 2024-04-25T00:23:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.068+0000 7f2c67e00640 10 client.25544 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:13 smithi106 ceph-mon[32510]: pgmap v522: 97 pgs: 97 active+clean; 63 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 226 op/s 2024-04-25T00:23:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c67e00640 1 -- 172.21.15.42:0/2088078751 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56343ea74340 con 0x56343eb86760 2024-04-25T00:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c67e00640 10 client.25544 handle_command_reply: tid=0 2024-04-25T00:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 client.25544 shutdown 2024-04-25T00:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 -- 172.21.15.42:0/2088078751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0a840 msgr2=0x56343eb195b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0a840 0x56343eb195b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c098110 tx=0x7f2c6c09c540 comp rx=0 tx=0).stop 2024-04-25T00:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 -- 172.21.15.42:0/2088078751 shutdown_connections 2024-04-25T00:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56343eb86760 0x56343eb88b80 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56343eb0f9b0 0x56343eb1a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56343eb0a840 0x56343eb195b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 --2- 172.21.15.42:0/2088078751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56343eb09e50 0x56343eb19070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c67e00640 0 client.25544 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 -- 172.21.15.42:0/2088078751 >> 172.21.15.42:0/2088078751 conn(0x56343eb054c0 msgr2=0x56343eb081a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 -- 172.21.15.42:0/2088078751 shutdown_connections 2024-04-25T00:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 1 -- 172.21.15.42:0/2088078751 wait complete. 2024-04-25T00:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.116+0000 7f2c8bb64740 20 client.25544 trim_cache size 0 max 16384 2024-04-25T00:23:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:23:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:23:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:23:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:23:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.117+0000 7f2c83200640 1 -- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2c680a2b80 msgr2=0x7f2c680a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.117+0000 7f2c83200640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2c680a2b80 0x7f2c680a5040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2c7402a9e0 tx=0x7f2c740315b0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.117+0000 7f2c83200640 1 -- 172.21.15.42:0/364478567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c841049d0 msgr2=0x7f2c8419a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.117+0000 7f2c83200640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c841049d0 0x7f2c8419a060 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c00e9f0 tx=0x7f2c6c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.117+0000 7f2c83200640 1 -- 172.21.15.42:0/364478567 shutdown_connections 2024-04-25T00:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.117+0000 7f2c83200640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f2c680a2b80 0x7f2c680a5040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.117+0000 7f2c83200640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2c84105390 0x7f2c841a10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.117+0000 7f2c83200640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c841049d0 0x7f2c8419a060 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.117+0000 7f2c83200640 1 --2- 172.21.15.42:0/364478567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2c841037e0 0x7f2c84199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.118+0000 7f2c83200640 1 -- 172.21.15.42:0/364478567 >> 172.21.15.42:0/364478567 conn(0x7f2c840fed50 msgr2=0x7f2c84107d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.118+0000 7f2c83200640 1 -- 172.21.15.42:0/364478567 shutdown_connections 2024-04-25T00:23:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:14.118+0000 7f2c83200640 1 -- 172.21.15.42:0/364478567 wait complete. 2024-04-25T00:23:14.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:13 smithi042 ceph-mon[28130]: pgmap v522: 97 pgs: 97 active+clean; 63 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 226 op/s 2024-04-25T00:23:14.223 INFO:tasks.workunit.client.0.smithi042.stdout: 1 228882 6.35 MB/sec execute 64 sec latency 42.404 ms 2024-04-25T00:23:15.223 INFO:tasks.workunit.client.0.smithi042.stdout: 1 230817 6.46 MB/sec execute 65 sec latency 17.565 ms 2024-04-25T00:23:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:15 smithi136 ceph-mon[32923]: pgmap v523: 97 pgs: 97 active+clean; 63 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-25T00:23:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:15 smithi106 ceph-mon[32510]: pgmap v523: 97 pgs: 97 active+clean; 63 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-25T00:23:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:15 smithi042 ceph-mon[28130]: pgmap v523: 97 pgs: 97 active+clean; 63 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-25T00:23:16.223 INFO:tasks.workunit.client.0.smithi042.stdout: 1 231407 6.38 MB/sec execute 66 sec latency 30.445 ms 2024-04-25T00:23:17.223 INFO:tasks.workunit.client.0.smithi042.stdout: 1 232897 6.37 MB/sec execute 67 sec latency 23.052 ms 2024-04-25T00:23:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:17 smithi136 ceph-mon[32923]: pgmap v524: 97 pgs: 97 active+clean; 60 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 248 op/s 2024-04-25T00:23:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:17 smithi106 ceph-mon[32510]: pgmap v524: 97 pgs: 97 active+clean; 60 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 248 op/s 2024-04-25T00:23:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:17 smithi042 ceph-mon[28130]: pgmap v524: 97 pgs: 97 active+clean; 60 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 248 op/s 2024-04-25T00:23:18.223 INFO:tasks.workunit.client.0.smithi042.stdout: 1 234447 6.46 MB/sec execute 68 sec latency 13.181 ms 2024-04-25T00:23:19.223 INFO:tasks.workunit.client.0.smithi042.stdout: 1 235240 6.40 MB/sec execute 69 sec latency 23.781 ms 2024-04-25T00:23:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:19 smithi136 ceph-mon[32923]: pgmap v525: 97 pgs: 97 active+clean; 60 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-04-25T00:23:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:19 smithi106 ceph-mon[32510]: pgmap v525: 97 pgs: 97 active+clean; 60 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-04-25T00:23:20.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:19 smithi042 ceph-mon[28130]: pgmap v525: 97 pgs: 97 active+clean; 60 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-04-25T00:23:20.224 INFO:tasks.workunit.client.0.smithi042.stdout: 1 236396 6.36 MB/sec execute 70 sec latency 24.269 ms 2024-04-25T00:23:21.224 INFO:tasks.workunit.client.0.smithi042.stdout: 1 237957 6.45 MB/sec execute 71 sec latency 62.215 ms 2024-04-25T00:23:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:21 smithi136 ceph-mon[32923]: pgmap v526: 97 pgs: 97 active+clean; 60 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-04-25T00:23:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:21 smithi106 ceph-mon[32510]: pgmap v526: 97 pgs: 97 active+clean; 60 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-04-25T00:23:22.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:21 smithi042 ceph-mon[28130]: pgmap v526: 97 pgs: 97 active+clean; 60 MiB data, 1006 MiB used, 1.0 TiB / 1.0 TiB avail; 736 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-04-25T00:23:22.224 INFO:tasks.workunit.client.0.smithi042.stdout: 1 238675 6.38 MB/sec execute 72 sec latency 19.778 ms 2024-04-25T00:23:23.224 INFO:tasks.workunit.client.0.smithi042.stdout: 1 240193 6.39 MB/sec execute 73 sec latency 22.810 ms 2024-04-25T00:23:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:23 smithi136 ceph-mon[32923]: pgmap v527: 97 pgs: 97 active+clean; 62 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-04-25T00:23:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:23 smithi106 ceph-mon[32510]: pgmap v527: 97 pgs: 97 active+clean; 62 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-04-25T00:23:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:23 smithi042 ceph-mon[28130]: pgmap v527: 97 pgs: 97 active+clean; 62 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 1.9 MiB/s wr, 226 op/s 2024-04-25T00:23:24.224 INFO:tasks.workunit.client.0.smithi042.stdout: 1 241552 6.45 MB/sec execute 74 sec latency 14.412 ms 2024-04-25T00:23:25.224 INFO:tasks.workunit.client.0.smithi042.stdout: 1 242238 6.38 MB/sec execute 75 sec latency 27.119 ms 2024-04-25T00:23:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:25 smithi136 ceph-mon[32923]: pgmap v528: 97 pgs: 97 active+clean; 63 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-04-25T00:23:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:25 smithi106 ceph-mon[32510]: pgmap v528: 97 pgs: 97 active+clean; 63 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-04-25T00:23:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:25 smithi042 ceph-mon[28130]: pgmap v528: 97 pgs: 97 active+clean; 63 MiB data, 1017 MiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-04-25T00:23:26.224 INFO:tasks.workunit.client.0.smithi042.stdout: 1 243679 6.38 MB/sec execute 76 sec latency 76.736 ms 2024-04-25T00:23:27.224 INFO:tasks.workunit.client.0.smithi042.stdout: 1 245083 6.45 MB/sec execute 77 sec latency 24.588 ms 2024-04-25T00:23:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.273+0000 7f19fde00640 1 -- 172.21.15.42:0/1255196874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8104b50 msgr2=0x7f19f8106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.273+0000 7f19fde00640 1 --2- 172.21.15.42:0/1255196874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8104b50 0x7f19f8106f40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009900 tx=0x7f19e002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.273+0000 7f19fde00640 1 -- 172.21.15.42:0/1255196874 shutdown_connections 2024-04-25T00:23:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.273+0000 7f19fde00640 1 --2- 172.21.15.42:0/1255196874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f8107480 0x7f19f81098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.273+0000 7f19fde00640 1 --2- 172.21.15.42:0/1255196874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8104b50 0x7f19f8106f40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.273+0000 7f19fde00640 1 --2- 172.21.15.42:0/1255196874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f8102220 0x7f19f8104610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.273+0000 7f19fde00640 1 -- 172.21.15.42:0/1255196874 >> 172.21.15.42:0/1255196874 conn(0x7f19f80fbc10 msgr2=0x7f19f80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.273+0000 7f19fde00640 1 -- 172.21.15.42:0/1255196874 shutdown_connections 2024-04-25T00:23:27.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.273+0000 7f19fde00640 1 -- 172.21.15.42:0/1255196874 wait complete. 2024-04-25T00:23:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.274+0000 7f19fde00640 1 Processor -- start 2024-04-25T00:23:27.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.274+0000 7f19fde00640 1 -- start start 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19fde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8102220 0x7f19f8101b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19fde00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f8104b50 0x7f19f81001d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19fde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f8107480 0x7f19f8100710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19fde00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19f810d430 con 0x7f19f8102220 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19fde00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19f810d2b0 con 0x7f19f8104b50 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19fde00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19f810d5b0 con 0x7f19f8107480 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19ee000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f8104b50 0x7f19f81001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19ee000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f8104b50 0x7f19f81001d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34370/0 (socket says 172.21.15.42:34370) 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19ee000640 1 -- 172.21.15.42:0/4049634219 learned_addr learned my addr 172.21.15.42:0/4049634219 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19f7400640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f8107480 0x7f19f8100710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19f6a00640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8102220 0x7f19f8101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19ee000640 1 -- 172.21.15.42:0/4049634219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f8107480 msgr2=0x7f19f8100710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:27.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19ee000640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f8107480 0x7f19f8100710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19ee000640 1 -- 172.21.15.42:0/4049634219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8102220 msgr2=0x7f19f8101b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.275+0000 7f19ee000640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8102220 0x7f19f8101b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.276+0000 7f19ee000640 1 -- 172.21.15.42:0/4049634219 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f8100fa0 con 0x7f19f8104b50 2024-04-25T00:23:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.276+0000 7f19f7400640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f8107480 0x7f19f8100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.276+0000 7f19ee000640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f8104b50 0x7f19f81001d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19e0031610 tx=0x7f19e0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.276+0000 7f19fca00640 1 -- 172.21.15.42:0/4049634219 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e003f030 con 0x7f19f8104b50 2024-04-25T00:23:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.276+0000 7f19fde00640 1 -- 172.21.15.42:0/4049634219 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f81a1b10 con 0x7f19f8104b50 2024-04-25T00:23:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.276+0000 7f19fca00640 1 -- 172.21.15.42:0/4049634219 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e0003ad0 con 0x7f19f8104b50 2024-04-25T00:23:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.276+0000 7f19fca00640 1 -- 172.21.15.42:0/4049634219 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0035070 con 0x7f19f8104b50 2024-04-25T00:23:27.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.276+0000 7f19fde00640 1 -- 172.21.15.42:0/4049634219 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f81a2060 con 0x7f19f8104b50 2024-04-25T00:23:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f1a00baf740 1 Processor -- start 2024-04-25T00:23:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f19fca00640 1 -- 172.21.15.42:0/4049634219 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19e0042030 con 0x7f19f8104b50 2024-04-25T00:23:27.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f1a00baf740 1 -- start start 2024-04-25T00:23:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ad6ca0 0x55a119ad70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad7690 0x55a119adc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 0x55a119adec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f1a00baf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a119a39ab0 con 0x55a119adc800 2024-04-25T00:23:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f1a00baf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a1199e2640 con 0x55a119ad6ca0 2024-04-25T00:23:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f1a00baf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a119a35a70 con 0x55a119ad7690 2024-04-25T00:23:27.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f19f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad7690 0x55a119adc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f19f7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ad6ca0 0x55a119ad70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f19f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad7690 0x55a119adc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41476/0 (socket says 172.21.15.42:41476) 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.280+0000 7f19f6a00640 1 -- 172.21.15.42:0/1132106266 learned_addr learned my addr 172.21.15.42:0/1132106266 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.281+0000 7f19ee000640 1 --2- 172.21.15.42:0/1132106266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 0x55a119adec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.281+0000 7f19f6a00640 1 -- 172.21.15.42:0/1132106266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ad6ca0 msgr2=0x55a119ad70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.281+0000 7f19f6a00640 1 --2- 172.21.15.42:0/1132106266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ad6ca0 0x55a119ad70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.281+0000 7f19f6a00640 1 -- 172.21.15.42:0/1132106266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 msgr2=0x55a119adec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.281+0000 7f19f6a00640 1 --2- 172.21.15.42:0/1132106266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 0x55a119adec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.281+0000 7f19f6a00640 1 -- 172.21.15.42:0/1132106266 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a11974e210 con 0x55a119ad7690 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.281+0000 7f19f6a00640 1 --2- 172.21.15.42:0/1132106266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad7690 0x55a119adc190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009d20 tx=0x7f19e8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d48f5fc8cf78f5a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.281+0000 7f19e7e00640 1 -- 172.21.15.42:0/1132106266 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e801e070 con 0x55a119ad7690 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.282+0000 7f19e7e00640 1 -- 172.21.15.42:0/1132106266 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8019b10 con 0x55a119ad7690 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.282+0000 7f19fca00640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f19dc0a2cb0 0x7f19dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.282+0000 7f19e7e00640 1 -- 172.21.15.42:0/1132106266 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800ebb0 con 0x55a119ad7690 2024-04-25T00:23:27.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.282+0000 7f19fca00640 1 -- 172.21.15.42:0/4049634219 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f19e00ef740 con 0x7f19f8104b50 2024-04-25T00:23:27.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.282+0000 7f19f7400640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f19dc0a2cb0 0x7f19dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.282+0000 7f1a00baf740 1 -- 172.21.15.42:0/1132106266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad7690 msgr2=0x55a119adc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.282+0000 7f1a00baf740 1 --2- 172.21.15.42:0/1132106266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad7690 0x55a119adc190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009d20 tx=0x7f19e8011810 comp rx=0 tx=0).stop 2024-04-25T00:23:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.283+0000 7f1a00baf740 1 -- 172.21.15.42:0/1132106266 shutdown_connections 2024-04-25T00:23:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.283+0000 7f1a00baf740 1 --2- 172.21.15.42:0/1132106266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 0x55a119adec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.283+0000 7f1a00baf740 1 --2- 172.21.15.42:0/1132106266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad7690 0x55a119adc190 secure :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009d20 tx=0x7f19e8011810 comp rx=0 tx=0).stop 2024-04-25T00:23:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.283+0000 7f1a00baf740 1 --2- 172.21.15.42:0/1132106266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ad6ca0 0x55a119ad70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.283+0000 7f1a00baf740 1 -- 172.21.15.42:0/1132106266 >> 172.21.15.42:0/1132106266 conn(0x55a119ad2310 msgr2=0x55a119ad2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.283+0000 7f19f7400640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f19dc0a2cb0 0x7f19dc0a5170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55a119ad7ef0 tx=0x7f19f00096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.283+0000 7f1a00baf740 1 -- 172.21.15.42:0/1132106266 shutdown_connections 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.283+0000 7f1a00baf740 1 -- 172.21.15.42:0/1132106266 wait complete. 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.283+0000 7f1a00baf740 1 -- start start 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.283+0000 7f1a00baf740 1 Processor -- start 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119ae1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 0x55a119ae1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ae8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f1a00baf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a119adf7b0 con 0x55a119adc800 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f1a00baf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a1199f5990 con 0x55a119ae6690 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f1a00baf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a119adf930 con 0x55a119ad6ca0 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f19f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119ae1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f19f7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ae8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f1a00baf740 10 client.0 _ll_register_callbacks cb 0x55a1199f7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f19f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119ae1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41478/0 (socket says 172.21.15.42:41478) 2024-04-25T00:23:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f19f6a00640 1 -- 172.21.15.42:0/2891187425 learned_addr learned my addr 172.21.15.42:0/2891187425 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.284+0000 7f19ee000640 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 0x55a119ae1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f19f7400640 1 -- 172.21.15.42:0/2891187425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 msgr2=0x55a119ae1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f19f7400640 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119ae1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f19f7400640 1 -- 172.21.15.42:0/2891187425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 msgr2=0x55a119ae1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f19f7400640 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 0x55a119ae1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f19f7400640 1 -- 172.21.15.42:0/2891187425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a11981a620 con 0x55a119ae6690 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f19ee000640 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 0x55a119ae1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f19f7400640 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ae8a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19f0010b40 tx=0x7f19f0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e83428d1cffac160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f19e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f19e4c00640 1 -- 172.21.15.42:0/2891187425 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f001fc80 con 0x55a119ae6690 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f19e4c00640 1 -- 172.21.15.42:0/2891187425 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f19f0017030 con 0x55a119ae6690 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.285+0000 7f1a00baf740 1 -- 172.21.15.42:0/2891187425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19e8019b10 con 0x55a119ae6690 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f19e4c00640 1 -- 172.21.15.42:0/2891187425 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f19f002d020 con 0x55a119ae6690 2024-04-25T00:23:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f1a00baf740 10 client.25556 fetch_fsmap learned FSMap version 29 2024-04-25T00:23:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f1a00baf740 1 -- 172.21.15.42:0/2891187425 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a119aebad0 con 0x55a119ae6690 2024-04-25T00:23:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f19e4c00640 1 -- 172.21.15.42:0/2891187425 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19e8019b10 con 0x55a119ae6690 2024-04-25T00:23:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f19e4c00640 1 -- 172.21.15.42:0/2891187425 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f19f0004670 con 0x55a119ae6690 2024-04-25T00:23:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f1a00baf740 10 client.25556 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:23:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f1a00baf740 10 client.25556 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f1a00baf740 20 client.25556 populate_metadata read hostname 'smithi042' 2024-04-25T00:23:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f1a00baf740 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a119aed1d0 0x55a119aef630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f1a00baf740 4 client.25556 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.286+0000 7f1a00baf740 1 -- 172.21.15.42:0/2891187425 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a119a46ff0 con 0x55a119aed1d0 2024-04-25T00:23:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.287+0000 7f19f6a00640 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a119aed1d0 0x55a119aef630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.287+0000 7f19f6a00640 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a119aed1d0 0x55a119aef630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcf703e7466a0a90 server_cookie=6e9b4afcf044ddd5 in_seq=0 out_seq=0 2024-04-25T00:23:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.287+0000 7f19e4c00640 10 client.25556 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f19e4c00640 1 -- 172.21.15.42:0/2891187425 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a119a46ff0 con 0x55a119aed1d0 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f19e4c00640 10 client.25556 handle_command_reply: tid=0 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 client.25556 shutdown 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 -- 172.21.15.42:0/2891187425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 msgr2=0x55a119ae8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ae8a80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f19f0010b40 tx=0x7f19f0004ab0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 -- 172.21.15.42:0/2891187425 shutdown_connections 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a119aed1d0 0x55a119aef630 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ae8a80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119adc800 0x55a119ae1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 --2- 172.21.15.42:0/2891187425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119ae1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 -- 172.21.15.42:0/2891187425 >> 172.21.15.42:0/2891187425 conn(0x55a119ad2310 msgr2=0x55a119ad1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f19e4c00640 0 client.25556 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 -- 172.21.15.42:0/2891187425 shutdown_connections 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.288+0000 7f1a00baf740 1 -- 172.21.15.42:0/2891187425 wait complete. 2024-04-25T00:23:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.289+0000 7f1a00baf740 20 client.25556 trim_cache size 0 max 16384 2024-04-25T00:23:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.307+0000 7f1a00baf740 1 Processor -- start 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.307+0000 7f1a00baf740 1 -- start start 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.307+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119ad6ca0 0x55a119accfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119adc800 0x55a119ad8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ad9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f1a00baf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81ad5d0 con 0x55a119ad6ca0 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f1a00baf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81aaea0 con 0x55a119ae6690 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f1a00baf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81a8cf0 con 0x55a119adc800 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f19f7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ad9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f19ee000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119adc800 0x55a119ad8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f19f7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ad9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34406/0 (socket says 172.21.15.42:34406) 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f19f7400640 1 -- 172.21.15.42:0/3278214323 learned_addr learned my addr 172.21.15.42:0/3278214323 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f19f6a00640 1 --2- 172.21.15.42:0/3278214323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119ad6ca0 0x55a119accfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f19f7400640 1 -- 172.21.15.42:0/3278214323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119adc800 msgr2=0x55a119ad8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f19f7400640 1 --2- 172.21.15.42:0/3278214323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119adc800 0x55a119ad8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f19f7400640 1 -- 172.21.15.42:0/3278214323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119ad6ca0 msgr2=0x55a119accfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f19f7400640 1 --2- 172.21.15.42:0/3278214323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119ad6ca0 0x55a119accfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.308+0000 7f19f7400640 1 -- 172.21.15.42:0/3278214323 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a119ad7a70 con 0x55a119ae6690 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.309+0000 7f19f7400640 1 --2- 172.21.15.42:0/3278214323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ad9070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f19f0000f80 tx=0x7f19f001f930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf68d63779ef2412 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.309+0000 7f19e7400640 1 -- 172.21.15.42:0/3278214323 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0011610 con 0x55a119ae6690 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.309+0000 7f19e7400640 1 -- 172.21.15.42:0/3278214323 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f0003970 con 0x55a119ae6690 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.309+0000 7f19e7400640 1 -- 172.21.15.42:0/3278214323 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f0004040 con 0x55a119ae6690 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.310+0000 7f1a00baf740 1 -- 172.21.15.42:0/3278214323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 msgr2=0x55a119ad9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.310+0000 7f1a00baf740 1 --2- 172.21.15.42:0/3278214323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ad9070 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f19f0000f80 tx=0x7f19f001f930 comp rx=0 tx=0).stop 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.310+0000 7f1a00baf740 1 -- 172.21.15.42:0/3278214323 shutdown_connections 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.310+0000 7f1a00baf740 1 --2- 172.21.15.42:0/3278214323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119ae6690 0x55a119ad9070 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.310+0000 7f1a00baf740 1 --2- 172.21.15.42:0/3278214323 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119adc800 0x55a119ad8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.310+0000 7f1a00baf740 1 --2- 172.21.15.42:0/3278214323 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119ad6ca0 0x55a119accfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.310+0000 7f1a00baf740 1 -- 172.21.15.42:0/3278214323 >> 172.21.15.42:0/3278214323 conn(0x55a119ad2310 msgr2=0x55a119ad26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.310+0000 7f1a00baf740 1 -- 172.21.15.42:0/3278214323 shutdown_connections 2024-04-25T00:23:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.310+0000 7f1a00baf740 1 -- 172.21.15.42:0/3278214323 wait complete. 2024-04-25T00:23:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.310+0000 7f1a00baf740 1 -- start start 2024-04-25T00:23:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f1a00baf740 1 Processor -- start 2024-04-25T00:23:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119afe6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119adc800 0x55a119afec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f1a00baf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119ae6690 0x55a119affc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f1a00baf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81a9360 con 0x55a119ae6690 2024-04-25T00:23:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f1a00baf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81aa7d0 con 0x55a119adc800 2024-04-25T00:23:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f1a00baf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19f81a9a30 con 0x55a119ad6ca0 2024-04-25T00:23:27.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f1a00baf740 10 client.0 _ll_register_callbacks cb 0x7f19f0010a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f19f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119afe6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f19f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119afe6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41492/0 (socket says 172.21.15.42:41492) 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.311+0000 7f19f6a00640 1 -- 172.21.15.42:0/3099096322 learned_addr learned my addr 172.21.15.42:0/3099096322 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.312+0000 7f19f7400640 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119ae6690 0x55a119affc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.312+0000 7f19ee000640 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119adc800 0x55a119afec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.312+0000 7f19f6a00640 1 -- 172.21.15.42:0/3099096322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119adc800 msgr2=0x55a119afec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.312+0000 7f19f6a00640 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119adc800 0x55a119afec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.312+0000 7f19f6a00640 1 -- 172.21.15.42:0/3099096322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119ae6690 msgr2=0x55a119affc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.312+0000 7f19f6a00640 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119ae6690 0x55a119affc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.312+0000 7f19f6a00640 1 -- 172.21.15.42:0/3099096322 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a119b00220 con 0x55a119ad6ca0 2024-04-25T00:23:27.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.312+0000 7f19f6a00640 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119afe6e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009c40 tx=0x7f19e8005fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c8f51caf849dacf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.312+0000 7f19e4c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:23:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.312+0000 7f19e4c00640 1 -- 172.21.15.42:0/3099096322 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e801e040 con 0x55a119ad6ca0 2024-04-25T00:23:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f19e4c00640 1 -- 172.21.15.42:0/3099096322 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f19e80052d0 con 0x55a119ad6ca0 2024-04-25T00:23:27.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f1a00baf740 1 -- 172.21.15.42:0/3099096322 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f0003970 con 0x55a119ad6ca0 2024-04-25T00:23:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f19e4c00640 1 -- 172.21.15.42:0/3099096322 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f19e801ae50 con 0x55a119ad6ca0 2024-04-25T00:23:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f1a00baf740 10 client.25573 fetch_fsmap learned FSMap version 29 2024-04-25T00:23:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f1a00baf740 1 -- 172.21.15.42:0/3099096322 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a119b537d0 con 0x55a119ad6ca0 2024-04-25T00:23:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f19e4c00640 1 -- 172.21.15.42:0/3099096322 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f0003970 con 0x55a119ad6ca0 2024-04-25T00:23:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f19e4c00640 1 -- 172.21.15.42:0/3099096322 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f19e80194c0 con 0x55a119ad6ca0 2024-04-25T00:23:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f1a00baf740 10 client.25573 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:23:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f1a00baf740 10 client.25573 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:27.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f1a00baf740 20 client.25573 populate_metadata read hostname 'smithi042' 2024-04-25T00:23:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f1a00baf740 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a119b53f40 0x55a119b56360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f1a00baf740 4 client.25573 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:23:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.313+0000 7f1a00baf740 1 -- 172.21.15.42:0/3099096322 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a119a41340 con 0x55a119b53f40 2024-04-25T00:23:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.314+0000 7f19ee000640 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a119b53f40 0x55a119b56360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.314+0000 7f19ee000640 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a119b53f40 0x55a119b56360 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=344e63ba3bb1fc5d server_cookie=183dec3431c57f32 in_seq=0 out_seq=0 2024-04-25T00:23:27.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.314+0000 7f19e4c00640 10 client.25573 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f19e4c00640 1 -- 172.21.15.42:0/3099096322 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a119a41340 con 0x55a119b53f40 2024-04-25T00:23:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f19e4c00640 10 client.25573 handle_command_reply: tid=0 2024-04-25T00:23:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f1a00baf740 1 client.25573 shutdown 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f1a00baf740 1 -- 172.21.15.42:0/3099096322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 msgr2=0x55a119afe6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f1a00baf740 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119afe6e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f19e8009c40 tx=0x7f19e8005fa0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f1a00baf740 1 -- 172.21.15.42:0/3099096322 shutdown_connections 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f1a00baf740 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a119b53f40 0x55a119b56360 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f1a00baf740 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a119ae6690 0x55a119affc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f1a00baf740 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a119adc800 0x55a119afec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f1a00baf740 1 --2- 172.21.15.42:0/3099096322 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a119ad6ca0 0x55a119afe6e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f19e4c00640 0 client.25573 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.362+0000 7f1a00baf740 1 -- 172.21.15.42:0/3099096322 >> 172.21.15.42:0/3099096322 conn(0x55a119ad2310 msgr2=0x55a119accfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.363+0000 7f1a00baf740 1 -- 172.21.15.42:0/3099096322 shutdown_connections 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.363+0000 7f1a00baf740 1 -- 172.21.15.42:0/3099096322 wait complete. 2024-04-25T00:23:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.363+0000 7f1a00baf740 20 client.25573 trim_cache size 0 max 16384 2024-04-25T00:23:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:23:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:23:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:23:27.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:23:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 -- 172.21.15.42:0/4049634219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f19dc0a2cb0 msgr2=0x7f19dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f19dc0a2cb0 0x7f19dc0a5170 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x55a119ad7ef0 tx=0x7f19f00096a0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 -- 172.21.15.42:0/4049634219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f8104b50 msgr2=0x7f19f81001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f8104b50 0x7f19f81001d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f19e0031610 tx=0x7f19e0004270 comp rx=0 tx=0).stop 2024-04-25T00:23:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 -- 172.21.15.42:0/4049634219 shutdown_connections 2024-04-25T00:23:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f19dc0a2cb0 0x7f19dc0a5170 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f19f8107480 0x7f19f8100710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f8104b50 0x7f19f81001d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 --2- 172.21.15.42:0/4049634219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f19f8102220 0x7f19f8101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 -- 172.21.15.42:0/4049634219 >> 172.21.15.42:0/4049634219 conn(0x7f19f80fbc10 msgr2=0x7f19f8102b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 -- 172.21.15.42:0/4049634219 shutdown_connections 2024-04-25T00:23:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:27.364+0000 7f19fde00640 1 -- 172.21.15.42:0/4049634219 wait complete. 2024-04-25T00:23:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:27 smithi136 ceph-mon[32923]: pgmap v529: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-04-25T00:23:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:27 smithi106 ceph-mon[32510]: pgmap v529: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-04-25T00:23:28.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:27 smithi042 ceph-mon[28130]: pgmap v529: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 2.0 MiB/s wr, 232 op/s 2024-04-25T00:23:28.225 INFO:tasks.workunit.client.0.smithi042.stdout: 1 245699 6.38 MB/sec execute 78 sec latency 33.993 ms 2024-04-25T00:23:29.225 INFO:tasks.workunit.client.0.smithi042.stdout: 1 246753 6.36 MB/sec execute 79 sec latency 25.418 ms 2024-04-25T00:23:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:29 smithi136 ceph-mon[32923]: pgmap v530: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-04-25T00:23:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:29 smithi106 ceph-mon[32510]: pgmap v530: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-04-25T00:23:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:29 smithi042 ceph-mon[28130]: pgmap v530: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-04-25T00:23:30.225 INFO:tasks.workunit.client.0.smithi042.stdout: 1 248463 6.43 MB/sec execute 80 sec latency 20.467 ms 2024-04-25T00:23:31.225 INFO:tasks.workunit.client.0.smithi042.stdout: 1 249046 6.37 MB/sec execute 81 sec latency 55.522 ms 2024-04-25T00:23:32.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:31 smithi136 ceph-mon[32923]: pgmap v531: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-04-25T00:23:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:31 smithi106 ceph-mon[32510]: pgmap v531: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-04-25T00:23:32.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:31 smithi042 ceph-mon[28130]: pgmap v531: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 783 KiB/s rd, 1.6 MiB/s wr, 175 op/s 2024-04-25T00:23:32.225 INFO:tasks.workunit.client.0.smithi042.stdout: 1 250172 6.36 MB/sec execute 82 sec latency 26.802 ms 2024-04-25T00:23:33.225 INFO:tasks.workunit.client.0.smithi042.stdout: 1 251937 6.43 MB/sec execute 83 sec latency 19.851 ms 2024-04-25T00:23:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:33 smithi136 ceph-mon[32923]: pgmap v532: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-04-25T00:23:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:33 smithi106 ceph-mon[32510]: pgmap v532: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-04-25T00:23:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:33 smithi042 ceph-mon[28130]: pgmap v532: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 942 KiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-04-25T00:23:34.226 INFO:tasks.workunit.client.0.smithi042.stdout: 1 252472 6.37 MB/sec execute 84 sec latency 19.119 ms 2024-04-25T00:23:35.225 INFO:tasks.workunit.client.0.smithi042.stdout: 1 253489 6.35 MB/sec execute 85 sec latency 29.940 ms 2024-04-25T00:23:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:35 smithi106 ceph-mon[32510]: pgmap v533: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-25T00:23:36.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:35 smithi042 ceph-mon[28130]: pgmap v533: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-25T00:23:36.226 INFO:tasks.workunit.client.0.smithi042.stdout: 1 255157 6.39 MB/sec execute 86 sec latency 24.390 ms 2024-04-25T00:23:36.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:35 smithi136 ceph-mon[32923]: pgmap v533: 97 pgs: 97 active+clean; 64 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-04-25T00:23:37.226 INFO:tasks.workunit.client.0.smithi042.stdout: 1 255897 6.37 MB/sec execute 87 sec latency 21.424 ms 2024-04-25T00:23:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:37 smithi106 ceph-mon[32510]: pgmap v534: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.7 MiB/s wr, 218 op/s 2024-04-25T00:23:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:37 smithi042 ceph-mon[28130]: pgmap v534: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.7 MiB/s wr, 218 op/s 2024-04-25T00:23:38.226 INFO:tasks.workunit.client.0.smithi042.stdout: 1 256930 6.35 MB/sec execute 88 sec latency 28.127 ms 2024-04-25T00:23:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:37 smithi136 ceph-mon[32923]: pgmap v534: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 899 KiB/s rd, 1.7 MiB/s wr, 218 op/s 2024-04-25T00:23:39.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:38 smithi106 ceph-mon[32510]: pgmap v535: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-04-25T00:23:39.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:38 smithi042 ceph-mon[28130]: pgmap v535: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-04-25T00:23:39.226 INFO:tasks.workunit.client.0.smithi042.stdout: 1 258426 6.37 MB/sec execute 89 sec latency 22.569 ms 2024-04-25T00:23:39.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:38 smithi136 ceph-mon[32923]: pgmap v535: 97 pgs: 97 active+clean; 65 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-04-25T00:23:40.226 INFO:tasks.workunit.client.0.smithi042.stdout: 1 259394 6.37 MB/sec execute 90 sec latency 14.534 ms 2024-04-25T00:23:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.522+0000 7fa90f600640 1 -- 172.21.15.42:0/3366861071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa908100070 msgr2=0x7fa908100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.522+0000 7fa90f600640 1 --2- 172.21.15.42:0/3366861071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa908100070 0x7fa908100490 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8009860 tx=0x7fa8f802a340 comp rx=0 tx=0).stop 2024-04-25T00:23:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.522+0000 7fa90f600640 1 -- 172.21.15.42:0/3366861071 shutdown_connections 2024-04-25T00:23:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.522+0000 7fa90f600640 1 --2- 172.21.15.42:0/3366861071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa908101370 0x7fa9081017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.522+0000 7fa90f600640 1 --2- 172.21.15.42:0/3366861071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa9081009d0 0x7fa908100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.522+0000 7fa90f600640 1 --2- 172.21.15.42:0/3366861071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa908100070 0x7fa908100490 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.522+0000 7fa90f600640 1 -- 172.21.15.42:0/3366861071 >> 172.21.15.42:0/3366861071 conn(0x7fa9080fbc30 msgr2=0x7fa9080fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.523+0000 7fa90f600640 1 -- 172.21.15.42:0/3366861071 shutdown_connections 2024-04-25T00:23:40.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.523+0000 7fa90f600640 1 -- 172.21.15.42:0/3366861071 wait complete. 2024-04-25T00:23:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.524+0000 7fa90f600640 1 Processor -- start 2024-04-25T00:23:40.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.524+0000 7fa90f600640 1 -- start start 2024-04-25T00:23:40.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90f600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa908100070 0x7fa908199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90f600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9081009d0 0x7fa90819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90f600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa908101370 0x7fa9081a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90f600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa90810f5e0 con 0x7fa908101370 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90f600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa90810f460 con 0x7fa908100070 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90f600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa90810f760 con 0x7fa9081009d0 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa908100070 0x7fa908199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa908100070 0x7fa908199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51732/0 (socket says 172.21.15.42:51732) 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90cc00640 1 -- 172.21.15.42:0/1681085467 learned_addr learned my addr 172.21.15.42:0/1681085467 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa907e00640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9081009d0 0x7fa90819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90d600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa908101370 0x7fa9081a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90cc00640 1 -- 172.21.15.42:0/1681085467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9081009d0 msgr2=0x7fa90819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90cc00640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9081009d0 0x7fa90819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90cc00640 1 -- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa908101370 msgr2=0x7fa9081a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90cc00640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa908101370 0x7fa9081a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90cc00640 1 -- 172.21.15.42:0/1681085467 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9081a17e0 con 0x7fa908100070 2024-04-25T00:23:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.525+0000 7fa90d600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa908101370 0x7fa9081a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:23:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.526+0000 7fa90cc00640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa908100070 0x7fa908199bc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8035040 tx=0x7fa8f8033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.526+0000 7fa905600640 1 -- 172.21.15.42:0/1681085467 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8037040 con 0x7fa908100070 2024-04-25T00:23:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.526+0000 7fa905600640 1 -- 172.21.15.42:0/1681085467 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f8030850 con 0x7fa908100070 2024-04-25T00:23:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.526+0000 7fa905600640 1 -- 172.21.15.42:0/1681085467 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8004920 con 0x7fa908100070 2024-04-25T00:23:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.526+0000 7fa90f600640 1 -- 172.21.15.42:0/1681085467 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9081a1a70 con 0x7fa908100070 2024-04-25T00:23:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.526+0000 7fa90f600640 1 -- 172.21.15.42:0/1681085467 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9081a1f10 con 0x7fa908100070 2024-04-25T00:23:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.530+0000 7fa905600640 1 -- 172.21.15.42:0/1681085467 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa8f8004050 con 0x7fa908100070 2024-04-25T00:23:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.530+0000 7fa9125b9740 1 Processor -- start 2024-04-25T00:23:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.530+0000 7fa9125b9740 1 -- start start 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x5581341a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a2690 0x5581341a7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a7800 0x5581341a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa9125b9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x558134104ab0 con 0x5581341a7800 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa9125b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5581340ad640 con 0x5581341a2690 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa9125b9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558134100a70 con 0x5581341a1ca0 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a2690 0x5581341a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a2690 0x5581341a7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51736/0 (socket says 172.21.15.42:51736) 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa90cc00640 1 -- 172.21.15.42:0/3415586201 learned_addr learned my addr 172.21.15.42:0/3415586201 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa90d600640 1 --2- 172.21.15.42:0/3415586201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x5581341a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa907e00640 1 --2- 172.21.15.42:0/3415586201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a7800 0x5581341a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa90cc00640 1 -- 172.21.15.42:0/3415586201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 msgr2=0x5581341a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa90cc00640 1 --2- 172.21.15.42:0/3415586201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x5581341a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa90cc00640 1 -- 172.21.15.42:0/3415586201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a7800 msgr2=0x5581341a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa90cc00640 1 --2- 172.21.15.42:0/3415586201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a7800 0x5581341a9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.531+0000 7fa90cc00640 1 -- 172.21.15.42:0/3415586201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558133e19210 con 0x5581341a2690 2024-04-25T00:23:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.532+0000 7fa905600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8ec0a2cb0 0x7fa8ec0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.532+0000 7fa90cc00640 1 --2- 172.21.15.42:0/3415586201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a2690 0x5581341a7190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80f2eb0 tx=0x7fa8f80f70e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6b99dda6d720eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.532+0000 7fa90d600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8ec0a2cb0 0x7fa8ec0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.532+0000 7fa8fdc00640 1 -- 172.21.15.42:0/3415586201 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80b7a70 con 0x5581341a2690 2024-04-25T00:23:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.532+0000 7fa8fdc00640 1 -- 172.21.15.42:0/3415586201 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f80fc070 con 0x5581341a2690 2024-04-25T00:23:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.532+0000 7fa905600640 1 -- 172.21.15.42:0/1681085467 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa8f80eefa0 con 0x7fa908100070 2024-04-25T00:23:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.532+0000 7fa8fdc00640 1 -- 172.21.15.42:0/3415586201 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80cb810 con 0x5581341a2690 2024-04-25T00:23:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.533+0000 7fa9125b9740 1 -- 172.21.15.42:0/3415586201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a2690 msgr2=0x5581341a7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.533+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3415586201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a2690 0x5581341a7190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80f2eb0 tx=0x7fa8f80f70e0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.533+0000 7fa90d600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8ec0a2cb0 0x7fa8ec0a5170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5581341a2ef0 tx=0x7fa9000095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.533+0000 7fa9125b9740 1 -- 172.21.15.42:0/3415586201 shutdown_connections 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.533+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3415586201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a7800 0x5581341a9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.533+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3415586201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a2690 0x5581341a7190 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.533+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3415586201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x5581341a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.533+0000 7fa9125b9740 1 -- 172.21.15.42:0/3415586201 >> 172.21.15.42:0/3415586201 conn(0x55813419d310 msgr2=0x55813419d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.533+0000 7fa9125b9740 1 -- 172.21.15.42:0/3415586201 shutdown_connections 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.533+0000 7fa9125b9740 1 -- 172.21.15.42:0/3415586201 wait complete. 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.534+0000 7fa9125b9740 1 -- start start 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.534+0000 7fa9125b9740 1 Processor -- start 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.534+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x5581341abc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.534+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.534+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a7800 0x5581341b31a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.534+0000 7fa9125b9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5581341aa570 con 0x5581341a2690 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.534+0000 7fa9125b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5581340c0990 con 0x5581341a7800 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.534+0000 7fa9125b9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5581341aa6f0 con 0x5581341a1ca0 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.534+0000 7fa9125b9740 10 client.0 _ll_register_callbacks cb 0x5581340c2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x5581341abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa90d600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a7800 0x5581341b31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x5581341abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57526/0 (socket says 172.21.15.42:57526) 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa90cc00640 1 -- 172.21.15.42:0/3440864237 learned_addr learned my addr 172.21.15.42:0/3440864237 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa907e00640 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa90cc00640 1 -- 172.21.15.42:0/3440864237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a7800 msgr2=0x5581341b31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa90cc00640 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a7800 0x5581341b31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa90cc00640 1 -- 172.21.15.42:0/3440864237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 msgr2=0x5581341ac1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa90cc00640 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341ac1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa90cc00640 1 -- 172.21.15.42:0/3440864237 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558133ee5620 con 0x5581341a1ca0 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa907e00640 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa90cc00640 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x5581341abc90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80f75e0 tx=0x7fa8f80f27b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb037556d40f43c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa8f6000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa8f6000640 1 -- 172.21.15.42:0/3440864237 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80cbf50 con 0x5581341a1ca0 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.535+0000 7fa8f6000640 1 -- 172.21.15.42:0/3440864237 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa8f80a60e0 con 0x5581341a1ca0 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa9125b9740 1 -- 172.21.15.42:0/3440864237 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f80fc070 con 0x5581341a1ca0 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa8f6000640 1 -- 172.21.15.42:0/3440864237 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa8f80cca60 con 0x5581341a1ca0 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa9125b9740 10 client.25585 fetch_fsmap learned FSMap version 29 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa9125b9740 1 -- 172.21.15.42:0/3440864237 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581341aa570 con 0x5581341a1ca0 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa8f6000640 1 -- 172.21.15.42:0/3440864237 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f80fc070 con 0x5581341a1ca0 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa8f6000640 1 -- 172.21.15.42:0/3440864237 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa8f803a070 con 0x5581341a1ca0 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa9125b9740 10 client.25585 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:23:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa9125b9740 10 client.25585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa9125b9740 20 client.25585 populate_metadata read hostname 'smithi042' 2024-04-25T00:23:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5581341b73c0 0x5581341b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa9125b9740 4 client.25585 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.536+0000 7fa9125b9740 1 -- 172.21.15.42:0/3440864237 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558134111ff0 con 0x5581341b73c0 2024-04-25T00:23:40.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.537+0000 7fa907e00640 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5581341b73c0 0x5581341b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.537+0000 7fa907e00640 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5581341b73c0 0x5581341b97f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd1c879d43a79b33 server_cookie=b3650f539210edd7 in_seq=0 out_seq=0 2024-04-25T00:23:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.537+0000 7fa8f6000640 10 client.25585 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa8f6000640 1 -- 172.21.15.42:0/3440864237 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558134111ff0 con 0x5581341b73c0 2024-04-25T00:23:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa8f6000640 10 client.25585 handle_command_reply: tid=0 2024-04-25T00:23:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa9125b9740 1 client.25585 shutdown 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa9125b9740 1 -- 172.21.15.42:0/3440864237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 msgr2=0x5581341abc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x5581341abc90 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80f75e0 tx=0x7fa8f80f27b0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa9125b9740 1 -- 172.21.15.42:0/3440864237 shutdown_connections 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5581341b73c0 0x5581341b97f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a7800 0x5581341b31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3440864237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x5581341abc90 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa8f6000640 0 client.25585 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa9125b9740 1 -- 172.21.15.42:0/3440864237 >> 172.21.15.42:0/3440864237 conn(0x55813419d310 msgr2=0x55813419c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.538+0000 7fa9125b9740 1 -- 172.21.15.42:0/3440864237 shutdown_connections 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.539+0000 7fa9125b9740 1 -- 172.21.15.42:0/3440864237 wait complete. 2024-04-25T00:23:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.539+0000 7fa9125b9740 20 client.25585 trim_cache size 0 max 16384 2024-04-25T00:23:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa9125b9740 1 Processor -- start 2024-04-25T00:23:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa9125b9740 1 -- start start 2024-04-25T00:23:40.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x55813419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341a0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a7800 0x5581341a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa9125b9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa9081ad590 con 0x5581341a2690 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa9125b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9081aae60 con 0x5581341a7800 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa9125b9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa9081a8cb0 con 0x5581341a1ca0 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x55813419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x55813419d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57542/0 (socket says 172.21.15.42:57542) 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.556+0000 7fa90cc00640 1 -- 172.21.15.42:0/4019442891 learned_addr learned my addr 172.21.15.42:0/4019442891 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa907e00640 1 --2- 172.21.15.42:0/4019442891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa90d600640 1 --2- 172.21.15.42:0/4019442891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a7800 0x5581341a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa90cc00640 1 -- 172.21.15.42:0/4019442891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a7800 msgr2=0x5581341a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa90cc00640 1 --2- 172.21.15.42:0/4019442891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a7800 0x5581341a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa90cc00640 1 -- 172.21.15.42:0/4019442891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 msgr2=0x5581341a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:40.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa90cc00640 1 --2- 172.21.15.42:0/4019442891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341a0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa90cc00640 1 -- 172.21.15.42:0/4019442891 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581341b9e70 con 0x5581341a1ca0 2024-04-25T00:23:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa907e00640 1 --2- 172.21.15.42:0/4019442891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa90cc00640 1 --2- 172.21.15.42:0/4019442891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x55813419d6f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8030410 tx=0x7fa8f80b9ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bd70fe4194ce3c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa8fd200640 1 -- 172.21.15.42:0/4019442891 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80c4690 con 0x5581341a1ca0 2024-04-25T00:23:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa8fd200640 1 -- 172.21.15.42:0/4019442891 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f80fc070 con 0x5581341a1ca0 2024-04-25T00:23:40.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.557+0000 7fa8fd200640 1 -- 172.21.15.42:0/4019442891 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80bb130 con 0x5581341a1ca0 2024-04-25T00:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.558+0000 7fa9125b9740 1 -- 172.21.15.42:0/4019442891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 msgr2=0x55813419d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.558+0000 7fa9125b9740 1 --2- 172.21.15.42:0/4019442891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x55813419d6f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8030410 tx=0x7fa8f80b9ce0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.558+0000 7fa9125b9740 1 -- 172.21.15.42:0/4019442891 shutdown_connections 2024-04-25T00:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.558+0000 7fa9125b9740 1 --2- 172.21.15.42:0/4019442891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a7800 0x5581341a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.558+0000 7fa9125b9740 1 --2- 172.21.15.42:0/4019442891 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.558+0000 7fa9125b9740 1 --2- 172.21.15.42:0/4019442891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a1ca0 0x55813419d6f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.558+0000 7fa9125b9740 1 -- 172.21.15.42:0/4019442891 >> 172.21.15.42:0/4019442891 conn(0x55813419d310 msgr2=0x5581341a6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.558+0000 7fa9125b9740 1 -- 172.21.15.42:0/4019442891 shutdown_connections 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.558+0000 7fa9125b9740 1 -- 172.21.15.42:0/4019442891 wait complete. 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.559+0000 7fa9125b9740 1 -- start start 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.559+0000 7fa9125b9740 1 Processor -- start 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.559+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a1ca0 0x5581341afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.559+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.559+0000 7fa9125b9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a7800 0x5581341b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.559+0000 7fa9125b9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa9081a9320 con 0x5581341a2690 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.559+0000 7fa9125b9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa9081aa790 con 0x5581341a1ca0 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.559+0000 7fa9125b9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa9081a99f0 con 0x5581341a7800 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.559+0000 7fa9125b9740 10 client.0 _ll_register_callbacks cb 0x7fa8f80bf070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a1ca0 0x5581341afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa90d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a7800 0x5581341b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa90cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a1ca0 0x5581341afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51778/0 (socket says 172.21.15.42:51778) 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa90cc00640 1 -- 172.21.15.42:0/3026563738 learned_addr learned my addr 172.21.15.42:0/3026563738 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa907e00640 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa90cc00640 1 -- 172.21.15.42:0/3026563738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a7800 msgr2=0x5581341b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa90cc00640 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a7800 0x5581341b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa90cc00640 1 -- 172.21.15.42:0/3026563738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 msgr2=0x5581341b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa90cc00640 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa90cc00640 1 -- 172.21.15.42:0/3026563738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55813419d140 con 0x5581341a1ca0 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa90cc00640 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a1ca0 0x5581341afeb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80b7c30 tx=0x7fa8f80b7470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5503f60a1a0d9ab1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa8f6000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.560+0000 7fa8f6000640 1 -- 172.21.15.42:0/3026563738 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80d4060 con 0x5581341a1ca0 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa8f6000640 1 -- 172.21.15.42:0/3026563738 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa8f80f20a0 con 0x5581341a1ca0 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa9125b9740 1 -- 172.21.15.42:0/3026563738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f80fc070 con 0x5581341a1ca0 2024-04-25T00:23:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa8f6000640 1 -- 172.21.15.42:0/3026563738 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa8f80bb310 con 0x5581341a1ca0 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa9125b9740 10 client.25580 fetch_fsmap learned FSMap version 29 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa9125b9740 1 -- 172.21.15.42:0/3026563738 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55813419d140 con 0x5581341a1ca0 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa8f6000640 1 -- 172.21.15.42:0/3026563738 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8f80fc070 con 0x5581341a1ca0 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa8f6000640 1 -- 172.21.15.42:0/3026563738 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa8f803a070 con 0x5581341a1ca0 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa9125b9740 10 client.25580 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa9125b9740 10 client.25580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa9125b9740 20 client.25580 populate_metadata read hostname 'smithi042' 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55813421dd80 0x5581342201a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa9125b9740 4 client.25580 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.561+0000 7fa9125b9740 1 -- 172.21.15.42:0/3026563738 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55813410c340 con 0x55813421dd80 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.562+0000 7fa907e00640 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55813421dd80 0x5581342201a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.562+0000 7fa907e00640 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55813421dd80 0x5581342201a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ba6bdc63150816e server_cookie=c6128363ba9940ec in_seq=0 out_seq=0 2024-04-25T00:23:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.562+0000 7fa8f6000640 10 client.25580 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa8f6000640 1 -- 172.21.15.42:0/3026563738 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55813410c340 con 0x55813421dd80 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa8f6000640 10 client.25580 handle_command_reply: tid=0 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 client.25580 shutdown 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 -- 172.21.15.42:0/3026563738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a1ca0 msgr2=0x5581341afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a1ca0 0x5581341afeb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80b7c30 tx=0x7fa8f80b7470 comp rx=0 tx=0).stop 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 -- 172.21.15.42:0/3026563738 shutdown_connections 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55813421dd80 0x5581342201a0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5581341a7800 0x5581341b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5581341a2690 0x5581341b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa8f6000640 0 client.25580 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 --2- 172.21.15.42:0/3026563738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5581341a1ca0 0x5581341afeb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 -- 172.21.15.42:0/3026563738 >> 172.21.15.42:0/3026563738 conn(0x55813419d310 msgr2=0x5581341a3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 -- 172.21.15.42:0/3026563738 shutdown_connections 2024-04-25T00:23:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 1 -- 172.21.15.42:0/3026563738 wait complete. 2024-04-25T00:23:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.614+0000 7fa9125b9740 20 client.25580 trim_cache size 0 max 16384 2024-04-25T00:23:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:23:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:23:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:23:40.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:23:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 -- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8ec0a2cb0 msgr2=0x7fa8ec0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8ec0a2cb0 0x7fa8ec0a5170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x5581341a2ef0 tx=0x7fa9000095c0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 -- 172.21.15.42:0/1681085467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa908100070 msgr2=0x7fa908199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa908100070 0x7fa908199bc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8035040 tx=0x7fa8f8033d90 comp rx=0 tx=0).stop 2024-04-25T00:23:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 -- 172.21.15.42:0/1681085467 shutdown_connections 2024-04-25T00:23:40.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa8ec0a2cb0 0x7fa8ec0a5170 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa908101370 0x7fa9081a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa9081009d0 0x7fa90819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 --2- 172.21.15.42:0/1681085467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa908100070 0x7fa908199bc0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 -- 172.21.15.42:0/1681085467 >> 172.21.15.42:0/1681085467 conn(0x7fa9080fbc30 msgr2=0x7fa9080fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 -- 172.21.15.42:0/1681085467 shutdown_connections 2024-04-25T00:23:40.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:40.615+0000 7fa90f600640 1 -- 172.21.15.42:0/1681085467 wait complete. 2024-04-25T00:23:41.226 INFO:tasks.workunit.client.0.smithi042.stdout: 1 260394 6.35 MB/sec execute 91 sec latency 41.801 ms 2024-04-25T00:23:41.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:41 smithi136 ceph-mon[32923]: pgmap v536: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-25T00:23:41.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:41 smithi106 ceph-mon[32510]: pgmap v536: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-25T00:23:41.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:41 smithi042 ceph-mon[28130]: pgmap v536: 97 pgs: 97 active+clean; 66 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 746 KiB/s rd, 1.4 MiB/s wr, 191 op/s 2024-04-25T00:23:42.226 INFO:tasks.workunit.client.0.smithi042.stdout: 1 261991 6.37 MB/sec execute 92 sec latency 27.403 ms 2024-04-25T00:23:43.227 INFO:tasks.workunit.client.0.smithi042.stdout: 1 263069 6.37 MB/sec execute 93 sec latency 12.558 ms 2024-04-25T00:23:43.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:43 smithi136 ceph-mon[32923]: pgmap v537: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-04-25T00:23:43.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:43 smithi106 ceph-mon[32510]: pgmap v537: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-04-25T00:23:43.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:43 smithi042 ceph-mon[28130]: pgmap v537: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-04-25T00:23:44.227 INFO:tasks.workunit.client.0.smithi042.stdout: 1 264180 6.36 MB/sec execute 94 sec latency 21.460 ms 2024-04-25T00:23:45.227 INFO:tasks.workunit.client.0.smithi042.stdout: 1 265861 6.39 MB/sec execute 95 sec latency 20.656 ms 2024-04-25T00:23:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:45 smithi136 ceph-mon[32923]: pgmap v538: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-04-25T00:23:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:45 smithi106 ceph-mon[32510]: pgmap v538: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-04-25T00:23:45.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:45 smithi042 ceph-mon[28130]: pgmap v538: 97 pgs: 97 active+clean; 70 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-04-25T00:23:46.227 INFO:tasks.workunit.client.0.smithi042.stdout: 1 266652 6.37 MB/sec execute 96 sec latency 35.364 ms 2024-04-25T00:23:47.227 INFO:tasks.workunit.client.0.smithi042.stdout: 1 267836 6.36 MB/sec execute 97 sec latency 28.795 ms 2024-04-25T00:23:47.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:47 smithi136 ceph-mon[32923]: pgmap v539: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-25T00:23:47.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:47 smithi106 ceph-mon[32510]: pgmap v539: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-25T00:23:47.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:47 smithi042 ceph-mon[28130]: pgmap v539: 97 pgs: 97 active+clean; 70 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-25T00:23:48.227 INFO:tasks.workunit.client.0.smithi042.stdout: 1 269749 6.42 MB/sec execute 98 sec latency 22.199 ms 2024-04-25T00:23:49.227 INFO:tasks.workunit.client.0.smithi042.stdout: 1 270486 6.38 MB/sec execute 99 sec latency 17.690 ms 2024-04-25T00:23:49.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:49 smithi136 ceph-mon[32923]: pgmap v540: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.5 MiB/s wr, 199 op/s 2024-04-25T00:23:49.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:49 smithi106 ceph-mon[32510]: pgmap v540: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.5 MiB/s wr, 199 op/s 2024-04-25T00:23:49.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:49 smithi042 ceph-mon[28130]: pgmap v540: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.5 MiB/s wr, 199 op/s 2024-04-25T00:23:50.227 INFO:tasks.workunit.client.0.smithi042.stdout: 1 271582 6.36 MB/sec execute 100 sec latency 24.889 ms 2024-04-25T00:23:51.227 INFO:tasks.workunit.client.0.smithi042.stdout: 1 273364 6.43 MB/sec execute 101 sec latency 55.155 ms 2024-04-25T00:23:51.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:51 smithi136 ceph-mon[32923]: pgmap v541: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-04-25T00:23:51.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:51 smithi106 ceph-mon[32510]: pgmap v541: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-04-25T00:23:51.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:51 smithi042 ceph-mon[28130]: pgmap v541: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-04-25T00:23:52.227 INFO:tasks.workunit.client.0.smithi042.stdout: 1 274054 6.38 MB/sec execute 102 sec latency 22.964 ms 2024-04-25T00:23:53.228 INFO:tasks.workunit.client.0.smithi042.stdout: 1 275569 6.38 MB/sec execute 103 sec latency 28.500 ms 2024-04-25T00:23:53.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:53 smithi136 ceph-mon[32923]: pgmap v542: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-04-25T00:23:53.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:53 smithi106 ceph-mon[32510]: pgmap v542: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-04-25T00:23:53.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:53 smithi042 ceph-mon[28130]: pgmap v542: 97 pgs: 97 active+clean; 72 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 824 KiB/s rd, 2.1 MiB/s wr, 223 op/s 2024-04-25T00:23:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.766+0000 7f329ae00640 1 -- 172.21.15.42:0/2576788733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32941097a0 msgr2=0x7f3294110030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.766+0000 7f329ae00640 1 --2- 172.21.15.42:0/2576788733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32941097a0 0x7f3294110030 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f328c009ca0 tx=0x7f328c02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.766+0000 7f329ae00640 1 -- 172.21.15.42:0/2576788733 shutdown_connections 2024-04-25T00:23:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.766+0000 7f329ae00640 1 --2- 172.21.15.42:0/2576788733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32941097a0 0x7f3294110030 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.766+0000 7f329ae00640 1 --2- 172.21.15.42:0/2576788733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3294108de0 0x7f3294109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.766+0000 7f329ae00640 1 --2- 172.21.15.42:0/2576788733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3294107bf0 0x7f3294108010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.766+0000 7f329ae00640 1 -- 172.21.15.42:0/2576788733 >> 172.21.15.42:0/2576788733 conn(0x7f3294076bd0 msgr2=0x7f3294078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.766+0000 7f329ae00640 1 -- 172.21.15.42:0/2576788733 shutdown_connections 2024-04-25T00:23:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.766+0000 7f329ae00640 1 -- 172.21.15.42:0/2576788733 wait complete. 2024-04-25T00:23:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.767+0000 7f329ae00640 1 Processor -- start 2024-04-25T00:23:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.767+0000 7f329ae00640 1 -- start start 2024-04-25T00:23:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f329ae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3294107bf0 0x7f329419de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f329ae00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3294108de0 0x7f329419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f329ae00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32941097a0 0x7f32941a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f329ae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32941138c0 con 0x7f3294107bf0 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f329ae00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3294113740 con 0x7f32941097a0 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f329ae00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3294113a40 con 0x7f3294108de0 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f3299000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3294108de0 0x7f329419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f329a400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32941097a0 0x7f32941a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f3299a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3294107bf0 0x7f329419de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f3299000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3294108de0 0x7f329419e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39696/0 (socket says 172.21.15.42:39696) 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f3299000640 1 -- 172.21.15.42:0/3302312736 learned_addr learned my addr 172.21.15.42:0/3302312736 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f3299000640 1 -- 172.21.15.42:0/3302312736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32941097a0 msgr2=0x7f32941a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f3299000640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32941097a0 0x7f32941a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f3299000640 1 -- 172.21.15.42:0/3302312736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3294107bf0 msgr2=0x7f329419de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f3299000640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3294107bf0 0x7f329419de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f3299000640 1 -- 172.21.15.42:0/3302312736 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32941a5a90 con 0x7f3294108de0 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.768+0000 7f329a400640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32941097a0 0x7f32941a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.769+0000 7f3299000640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3294108de0 0x7f329419e3b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f328000e9f0 tx=0x7f328000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.769+0000 7f3292000640 1 -- 172.21.15.42:0/3302312736 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328000cd80 con 0x7f3294108de0 2024-04-25T00:23:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.769+0000 7f3292000640 1 -- 172.21.15.42:0/3302312736 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3280005500 con 0x7f3294108de0 2024-04-25T00:23:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.769+0000 7f329ae00640 1 -- 172.21.15.42:0/3302312736 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32941a5d80 con 0x7f3294108de0 2024-04-25T00:23:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.769+0000 7f3292000640 1 -- 172.21.15.42:0/3302312736 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3280010690 con 0x7f3294108de0 2024-04-25T00:23:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.769+0000 7f329ae00640 1 -- 172.21.15.42:0/3302312736 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32941a6220 con 0x7f3294108de0 2024-04-25T00:23:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.772+0000 7f329f206740 1 Processor -- start 2024-04-25T00:23:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f329f206740 1 -- start start 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f329f206740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bade50 0x563ee7bae270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f329f206740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bae840 0x563ee7bb3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f3292000640 1 -- 172.21.15.42:0/3302312736 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32800046c0 con 0x7f3294108de0 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f329f206740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 0x563ee7bb5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f329f206740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563ee7b10ab0 con 0x563ee7bade50 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f329f206740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563ee7ab9640 con 0x563ee7bae840 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f329f206740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563ee7b0ca70 con 0x563ee7bb39b0 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f3299000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 0x563ee7bb5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f3299a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bae840 0x563ee7bb3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f329a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bade50 0x563ee7bae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f3299a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bae840 0x563ee7bb3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49974/0 (socket says 172.21.15.42:49974) 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.773+0000 7f3299a00640 1 -- 172.21.15.42:0/2976040918 learned_addr learned my addr 172.21.15.42:0/2976040918 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.774+0000 7f3299a00640 1 -- 172.21.15.42:0/2976040918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 msgr2=0x563ee7bb5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.774+0000 7f3299a00640 1 --2- 172.21.15.42:0/2976040918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 0x563ee7bb5e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.774+0000 7f3299a00640 1 -- 172.21.15.42:0/2976040918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bade50 msgr2=0x563ee7bae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.774+0000 7f3299a00640 1 --2- 172.21.15.42:0/2976040918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bade50 0x563ee7bae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.774+0000 7f3299a00640 1 -- 172.21.15.42:0/2976040918 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee7825210 con 0x563ee7bae840 2024-04-25T00:23:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.774+0000 7f3299a00640 1 --2- 172.21.15.42:0/2976040918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bae840 0x563ee7bb3340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3284009ca0 tx=0x7f3284011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=846c5013344b31de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.774+0000 7f328aa00640 1 -- 172.21.15.42:0/2976040918 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284004220 con 0x563ee7bae840 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.774+0000 7f328aa00640 1 -- 172.21.15.42:0/2976040918 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32840043c0 con 0x563ee7bae840 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.774+0000 7f328aa00640 1 -- 172.21.15.42:0/2976040918 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284019930 con 0x563ee7bae840 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.775+0000 7f3292000640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f32780a2b80 0x7f32780a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.775+0000 7f3292000640 1 -- 172.21.15.42:0/3302312736 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f32800cf430 con 0x7f3294108de0 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.775+0000 7f329a400640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f32780a2b80 0x7f32780a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.775+0000 7f329f206740 1 -- 172.21.15.42:0/2976040918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bae840 msgr2=0x563ee7bb3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.775+0000 7f329f206740 1 --2- 172.21.15.42:0/2976040918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bae840 0x563ee7bb3340 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3284009ca0 tx=0x7f3284011810 comp rx=0 tx=0).stop 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.775+0000 7f329f206740 1 -- 172.21.15.42:0/2976040918 shutdown_connections 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.775+0000 7f329f206740 1 --2- 172.21.15.42:0/2976040918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 0x563ee7bb5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.775+0000 7f329f206740 1 --2- 172.21.15.42:0/2976040918 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bae840 0x563ee7bb3340 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.775+0000 7f329f206740 1 --2- 172.21.15.42:0/2976040918 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bade50 0x563ee7bae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.775+0000 7f329f206740 1 -- 172.21.15.42:0/2976040918 >> 172.21.15.42:0/2976040918 conn(0x563ee7ba94c0 msgr2=0x563ee7ba98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.776+0000 7f329f206740 1 -- 172.21.15.42:0/2976040918 shutdown_connections 2024-04-25T00:23:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.776+0000 7f329f206740 1 -- 172.21.15.42:0/2976040918 wait complete. 2024-04-25T00:23:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.776+0000 7f329a400640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f32780a2b80 0x7f32780a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f328c005e30 tx=0x7f328c031bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.776+0000 7f329f206740 1 -- start start 2024-04-25T00:23:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f329f206740 1 Processor -- start 2024-04-25T00:23:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f329f206740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bb8060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f329f206740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bae840 0x563ee7bb85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f329f206740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7bbf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f329f206740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563ee7bb6960 con 0x563ee7bb39b0 2024-04-25T00:23:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f329f206740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563ee7acc990 con 0x563ee7bade50 2024-04-25T00:23:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f329f206740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563ee7bb6ae0 con 0x563ee7bae840 2024-04-25T00:23:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f329f206740 10 client.0 _ll_register_callbacks cb 0x563ee7ace950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f3299a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bb8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f3299000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bae840 0x563ee7bb85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f3299a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bb8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49978/0 (socket says 172.21.15.42:49978) 2024-04-25T00:23:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.777+0000 7f3299a00640 1 -- 172.21.15.42:0/2956897365 learned_addr learned my addr 172.21.15.42:0/2956897365 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f329a400640 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7bbf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f3299a00640 1 -- 172.21.15.42:0/2956897365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bae840 msgr2=0x563ee7bb85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f3299a00640 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bae840 0x563ee7bb85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f3299a00640 1 -- 172.21.15.42:0/2956897365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 msgr2=0x563ee7bbf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f3299a00640 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7bbf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f3299a00640 1 -- 172.21.15.42:0/2956897365 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee78f1620 con 0x563ee7bade50 2024-04-25T00:23:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f329a400640 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7bbf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f3299000640 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bae840 0x563ee7bb85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f3299a00640 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bb8060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3284011d10 tx=0x7f3284003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245783834998b571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f327f400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:23:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f327f400640 1 -- 172.21.15.42:0/2956897365 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3284028070 con 0x563ee7bade50 2024-04-25T00:23:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f327f400640 1 -- 172.21.15.42:0/2956897365 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f3284005040 con 0x563ee7bade50 2024-04-25T00:23:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.778+0000 7f329f206740 1 -- 172.21.15.42:0/2956897365 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32840043c0 con 0x563ee7bade50 2024-04-25T00:23:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f327f400640 1 -- 172.21.15.42:0/2956897365 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3284034050 con 0x563ee7bade50 2024-04-25T00:23:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f329f206740 10 client.25592 fetch_fsmap learned FSMap version 29 2024-04-25T00:23:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f329f206740 1 -- 172.21.15.42:0/2956897365 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ee7bc25c0 con 0x563ee7bade50 2024-04-25T00:23:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f327f400640 1 -- 172.21.15.42:0/2956897365 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32840043c0 con 0x563ee7bade50 2024-04-25T00:23:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f327f400640 1 -- 172.21.15.42:0/2956897365 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f3284004900 con 0x563ee7bade50 2024-04-25T00:23:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f329f206740 10 client.25592 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:23:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f329f206740 10 client.25592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f329f206740 20 client.25592 populate_metadata read hostname 'smithi042' 2024-04-25T00:23:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f329f206740 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563ee7bc3a90 0x563ee7bc5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f329f206740 4 client.25592 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:23:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.779+0000 7f329f206740 1 -- 172.21.15.42:0/2956897365 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ee7b1dff0 con 0x563ee7bc3a90 2024-04-25T00:23:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.780+0000 7f3299000640 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563ee7bc3a90 0x563ee7bc5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.780+0000 7f3299000640 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563ee7bc3a90 0x563ee7bc5e80 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5088534d0a22f7 server_cookie=8b59360fee762b1b in_seq=0 out_seq=0 2024-04-25T00:23:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.780+0000 7f327f400640 10 client.25592 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f327f400640 1 -- 172.21.15.42:0/2956897365 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563ee7b1dff0 con 0x563ee7bc3a90 2024-04-25T00:23:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f327f400640 10 client.25592 handle_command_reply: tid=0 2024-04-25T00:23:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 client.25592 shutdown 2024-04-25T00:23:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 -- 172.21.15.42:0/2956897365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 msgr2=0x563ee7bb8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bb8060 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3284011d10 tx=0x7f3284003820 comp rx=0 tx=0).stop 2024-04-25T00:23:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 -- 172.21.15.42:0/2956897365 shutdown_connections 2024-04-25T00:23:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563ee7bc3a90 0x563ee7bc5e80 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7bbf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bae840 0x563ee7bb85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 --2- 172.21.15.42:0/2956897365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bb8060 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 -- 172.21.15.42:0/2956897365 >> 172.21.15.42:0/2956897365 conn(0x563ee7ba94c0 msgr2=0x563ee7ba8620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f327f400640 0 client.25592 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 -- 172.21.15.42:0/2956897365 shutdown_connections 2024-04-25T00:23:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.781+0000 7f329f206740 1 -- 172.21.15.42:0/2956897365 wait complete. 2024-04-25T00:23:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.782+0000 7f329f206740 20 client.25592 trim_cache size 0 max 16384 2024-04-25T00:23:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.799+0000 7f329f206740 1 Processor -- start 2024-04-25T00:23:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.799+0000 7f329f206740 1 -- start start 2024-04-25T00:23:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.799+0000 7f329f206740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bb3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.799+0000 7f329f206740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bae840 0x563ee7bbf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.799+0000 7f329f206740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7baca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.799+0000 7f329f206740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32941b1870 con 0x563ee7bb39b0 2024-04-25T00:23:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.799+0000 7f329f206740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f32941af140 con 0x563ee7bade50 2024-04-25T00:23:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.799+0000 7f329f206740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32941acf90 con 0x563ee7bae840 2024-04-25T00:23:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.800+0000 7f329a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7baca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.800+0000 7f329a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7baca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58780/0 (socket says 172.21.15.42:58780) 2024-04-25T00:23:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.800+0000 7f329a400640 1 -- 172.21.15.42:0/2155281741 learned_addr learned my addr 172.21.15.42:0/2155281741 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.800+0000 7f3299a00640 1 --2- 172.21.15.42:0/2155281741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bb3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.800+0000 7f329a400640 1 -- 172.21.15.42:0/2155281741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bae840 msgr2=0x563ee7bbf5a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:23:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.800+0000 7f329a400640 1 --2- 172.21.15.42:0/2155281741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bae840 0x563ee7bbf5a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.800+0000 7f329a400640 1 -- 172.21.15.42:0/2155281741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 msgr2=0x563ee7bb3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.800+0000 7f329a400640 1 --2- 172.21.15.42:0/2155281741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bb3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.800+0000 7f329a400640 1 -- 172.21.15.42:0/2155281741 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ee7bc61b0 con 0x563ee7bb39b0 2024-04-25T00:23:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.801+0000 7f329a400640 1 --2- 172.21.15.42:0/2155281741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7baca30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f328c035000 tx=0x7f328c042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d413c4e96b89df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.801+0000 7f328a000640 1 -- 172.21.15.42:0/2155281741 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c031830 con 0x563ee7bb39b0 2024-04-25T00:23:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.801+0000 7f328a000640 1 -- 172.21.15.42:0/2155281741 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c036040 con 0x563ee7bb39b0 2024-04-25T00:23:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.801+0000 7f328a000640 1 -- 172.21.15.42:0/2155281741 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328c02f960 con 0x563ee7bb39b0 2024-04-25T00:23:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.802+0000 7f329f206740 1 -- 172.21.15.42:0/2155281741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 msgr2=0x563ee7baca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.802+0000 7f329f206740 1 --2- 172.21.15.42:0/2155281741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7baca30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f328c035000 tx=0x7f328c042000 comp rx=0 tx=0).stop 2024-04-25T00:23:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.802+0000 7f329f206740 1 -- 172.21.15.42:0/2155281741 shutdown_connections 2024-04-25T00:23:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.802+0000 7f329f206740 1 --2- 172.21.15.42:0/2155281741 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bb39b0 0x563ee7baca30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.802+0000 7f329f206740 1 --2- 172.21.15.42:0/2155281741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bae840 0x563ee7bbf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.802+0000 7f329f206740 1 --2- 172.21.15.42:0/2155281741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bb3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.802+0000 7f329f206740 1 -- 172.21.15.42:0/2155281741 >> 172.21.15.42:0/2155281741 conn(0x563ee7ba94c0 msgr2=0x563ee7ba98a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.802+0000 7f329f206740 1 -- 172.21.15.42:0/2155281741 shutdown_connections 2024-04-25T00:23:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.802+0000 7f329f206740 1 -- 172.21.15.42:0/2155281741 wait complete. 2024-04-25T00:23:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.802+0000 7f329f206740 1 -- start start 2024-04-25T00:23:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f329f206740 1 Processor -- start 2024-04-25T00:23:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f329f206740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bbc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f329f206740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bae840 0x563ee7bbcd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f329f206740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 0x563ee7bbdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f329f206740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32941ad600 con 0x563ee7bae840 2024-04-25T00:23:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f329f206740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f32941aea70 con 0x563ee7bade50 2024-04-25T00:23:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f329f206740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32941adcd0 con 0x563ee7bb39b0 2024-04-25T00:23:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f329f206740 10 client.0 _ll_register_callbacks cb 0x7f3280005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:23:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f3299a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bbc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f329a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 0x563ee7bbdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f3299a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bbc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49988/0 (socket says 172.21.15.42:49988) 2024-04-25T00:23:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f3299a00640 1 -- 172.21.15.42:0/2137384731 learned_addr learned my addr 172.21.15.42:0/2137384731 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:23:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.803+0000 7f3299000640 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bae840 0x563ee7bbcd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f3299a00640 1 -- 172.21.15.42:0/2137384731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 msgr2=0x563ee7bbdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f3299a00640 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 0x563ee7bbdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f3299a00640 1 -- 172.21.15.42:0/2137384731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bae840 msgr2=0x563ee7bbcd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:23:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f3299a00640 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bae840 0x563ee7bbcd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f3299a00640 1 -- 172.21.15.42:0/2137384731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ee7bbe2b0 con 0x563ee7bade50 2024-04-25T00:23:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f329a400640 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 0x563ee7bbdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:23:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f3299000640 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bae840 0x563ee7bbcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:23:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f3299a00640 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bbc7e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3284004ec0 tx=0x7f32840099d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2954611520127cc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:23:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f327f400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:23:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f327f400640 1 -- 172.21.15.42:0/2137384731 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f328403d060 con 0x563ee7bade50 2024-04-25T00:23:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f327f400640 1 -- 172.21.15.42:0/2137384731 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f3284019630 con 0x563ee7bade50 2024-04-25T00:23:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.804+0000 7f329f206740 1 -- 172.21.15.42:0/2137384731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f328c036040 con 0x563ee7bade50 2024-04-25T00:23:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f327f400640 1 -- 172.21.15.42:0/2137384731 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3284028040 con 0x563ee7bade50 2024-04-25T00:23:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f329f206740 10 client.25598 fetch_fsmap learned FSMap version 29 2024-04-25T00:23:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f329f206740 1 -- 172.21.15.42:0/2137384731 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563ee7bbf5c0 con 0x563ee7bade50 2024-04-25T00:23:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f327f400640 1 -- 172.21.15.42:0/2137384731 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f328c036040 con 0x563ee7bade50 2024-04-25T00:23:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f327f400640 1 -- 172.21.15.42:0/2137384731 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f3284005350 con 0x563ee7bade50 2024-04-25T00:23:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f329f206740 10 client.25598 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:23:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f329f206740 10 client.25598 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:23:53.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f329f206740 20 client.25598 populate_metadata read hostname 'smithi042' 2024-04-25T00:23:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f329f206740 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563ee7c2a410 0x563ee7c2c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:23:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f329f206740 4 client.25598 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:23:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f329f206740 1 -- 172.21.15.42:0/2137384731 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ee7b18340 con 0x563ee7c2a410 2024-04-25T00:23:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.805+0000 7f3299000640 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563ee7c2a410 0x563ee7c2c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:23:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.806+0000 7f3299000640 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563ee7c2a410 0x563ee7c2c830 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=807f5296cc13325a server_cookie=b02f4ec923006303 in_seq=0 out_seq=0 2024-04-25T00:23:53.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.806+0000 7f327f400640 10 client.25598 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f327f400640 1 -- 172.21.15.42:0/2137384731 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563ee7b18340 con 0x563ee7c2a410 2024-04-25T00:23:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f327f400640 10 client.25598 handle_command_reply: tid=0 2024-04-25T00:23:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 client.25598 shutdown 2024-04-25T00:23:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 -- 172.21.15.42:0/2137384731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 msgr2=0x563ee7bbc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:53.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bbc7e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3284004ec0 tx=0x7f32840099d0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 -- 172.21.15.42:0/2137384731 shutdown_connections 2024-04-25T00:23:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563ee7c2a410 0x563ee7c2c830 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ee7bb39b0 0x563ee7bbdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563ee7bae840 0x563ee7bbcd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f327f400640 0 client.25598 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:23:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 --2- 172.21.15.42:0/2137384731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563ee7bade50 0x563ee7bbc7e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 -- 172.21.15.42:0/2137384731 >> 172.21.15.42:0/2137384731 conn(0x563ee7ba94c0 msgr2=0x563ee7ba7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 -- 172.21.15.42:0/2137384731 shutdown_connections 2024-04-25T00:23:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 1 -- 172.21.15.42:0/2137384731 wait complete. 2024-04-25T00:23:53.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.852+0000 7f329f206740 20 client.25598 trim_cache size 0 max 16384 2024-04-25T00:23:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:23:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:23:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:23:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:23:53.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.853+0000 7f329ae00640 1 -- 172.21.15.42:0/3302312736 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f32780a2b80 msgr2=0x7f32780a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.853+0000 7f329ae00640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f32780a2b80 0x7f32780a5040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f328c005e30 tx=0x7f328c031bb0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.853+0000 7f329ae00640 1 -- 172.21.15.42:0/3302312736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3294108de0 msgr2=0x7f329419e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.853+0000 7f329ae00640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3294108de0 0x7f329419e3b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f328000e9f0 tx=0x7f328000edf0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.853+0000 7f329ae00640 1 -- 172.21.15.42:0/3302312736 shutdown_connections 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.853+0000 7f329ae00640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f32780a2b80 0x7f32780a5040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.853+0000 7f329ae00640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32941097a0 0x7f32941a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.853+0000 7f329ae00640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3294108de0 0x7f329419e3b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.853+0000 7f329ae00640 1 --2- 172.21.15.42:0/3302312736 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3294107bf0 0x7f329419de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.853+0000 7f329ae00640 1 -- 172.21.15.42:0/3302312736 >> 172.21.15.42:0/3302312736 conn(0x7f3294076bd0 msgr2=0x7f329410c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.854+0000 7f329ae00640 1 -- 172.21.15.42:0/3302312736 shutdown_connections 2024-04-25T00:23:53.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:23:53.854+0000 7f329ae00640 1 -- 172.21.15.42:0/3302312736 wait complete. 2024-04-25T00:23:54.228 INFO:tasks.workunit.client.0.smithi042.stdout: 1 277069 6.43 MB/sec execute 104 sec latency 43.796 ms 2024-04-25T00:23:55.228 INFO:tasks.workunit.client.0.smithi042.stdout: 1 278103 6.42 MB/sec execute 105 sec latency 35.100 ms 2024-04-25T00:23:55.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:55 smithi136 ceph-mon[32923]: pgmap v543: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-25T00:23:55.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:55 smithi106 ceph-mon[32510]: pgmap v543: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-25T00:23:55.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:55 smithi042 ceph-mon[28130]: pgmap v543: 97 pgs: 97 active+clean; 73 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-25T00:23:56.228 INFO:tasks.workunit.client.0.smithi042.stdout: 1 279172 6.38 MB/sec execute 106 sec latency 55.209 ms 2024-04-25T00:23:57.228 INFO:tasks.workunit.client.0.smithi042.stdout: 1 280657 6.43 MB/sec execute 107 sec latency 55.778 ms 2024-04-25T00:23:57.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:57 smithi136 ceph-mon[32923]: pgmap v544: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-04-25T00:23:57.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:57 smithi106 ceph-mon[32510]: pgmap v544: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-04-25T00:23:57.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:57 smithi042 ceph-mon[28130]: pgmap v544: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-04-25T00:23:58.228 INFO:tasks.workunit.client.0.smithi042.stdout: 1 281761 6.42 MB/sec execute 108 sec latency 21.865 ms 2024-04-25T00:23:59.228 INFO:tasks.workunit.client.0.smithi042.stdout: 1 283463 6.43 MB/sec execute 109 sec latency 20.770 ms 2024-04-25T00:23:59.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:23:59 smithi136 ceph-mon[32923]: pgmap v545: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-04-25T00:23:59.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:23:59 smithi106 ceph-mon[32510]: pgmap v545: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-04-25T00:23:59.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:23:59 smithi042 ceph-mon[28130]: pgmap v545: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 1.8 MiB/s wr, 228 op/s 2024-04-25T00:24:00.228 INFO:tasks.workunit.client.0.smithi042.stdout: 1 284328 6.43 MB/sec execute 110 sec latency 13.880 ms 2024-04-25T00:24:01.229 INFO:tasks.workunit.client.0.smithi042.stdout: 1 285291 6.42 MB/sec execute 111 sec latency 32.132 ms 2024-04-25T00:24:01.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:01 smithi136 ceph-mon[32923]: pgmap v546: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-04-25T00:24:01.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:01 smithi106 ceph-mon[32510]: pgmap v546: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-04-25T00:24:01.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:01 smithi042 ceph-mon[28130]: pgmap v546: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-04-25T00:24:02.229 INFO:tasks.workunit.client.0.smithi042.stdout: 1 287133 6.44 MB/sec execute 112 sec latency 41.200 ms 2024-04-25T00:24:03.229 INFO:tasks.workunit.client.0.smithi042.stdout: 1 287887 6.43 MB/sec execute 113 sec latency 19.387 ms 2024-04-25T00:24:03.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:03 smithi136 ceph-mon[32923]: pgmap v547: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-04-25T00:24:03.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:03 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:03 smithi106 ceph-mon[32510]: pgmap v547: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-04-25T00:24:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:03 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:03.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:03 smithi042 ceph-mon[28130]: pgmap v547: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 889 KiB/s rd, 1.9 MiB/s wr, 239 op/s 2024-04-25T00:24:03.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:03 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:24:04.229 INFO:tasks.workunit.client.0.smithi042.stdout: 1 289310 6.42 MB/sec execute 114 sec latency 24.934 ms 2024-04-25T00:24:05.229 INFO:tasks.workunit.client.0.smithi042.stdout: 1 291099 6.48 MB/sec execute 115 sec latency 26.003 ms 2024-04-25T00:24:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:05 smithi136 ceph-mon[32923]: pgmap v548: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-04-25T00:24:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:05.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:24:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:05 smithi106 ceph-mon[32510]: pgmap v548: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-04-25T00:24:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:05.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:24:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:05 smithi042 ceph-mon[28130]: pgmap v548: 97 pgs: 97 active+clean; 71 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1.3 MiB/s wr, 190 op/s 2024-04-25T00:24:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:24:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:24:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:24:06.229 INFO:tasks.workunit.client.0.smithi042.stdout: 1 291656 6.43 MB/sec execute 116 sec latency 23.905 ms 2024-04-25T00:24:07.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.016+0000 7ffa2cc00640 1 -- 172.21.15.42:0/1792738312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa281053b0 msgr2=0x7ffa2810bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.016+0000 7ffa2cc00640 1 --2- 172.21.15.42:0/1792738312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa281053b0 0x7ffa2810bc40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffa20009ca0 tx=0x7ffa2002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.016+0000 7ffa2cc00640 1 -- 172.21.15.42:0/1792738312 shutdown_connections 2024-04-25T00:24:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.016+0000 7ffa2cc00640 1 --2- 172.21.15.42:0/1792738312 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa281053b0 0x7ffa2810bc40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.016+0000 7ffa2cc00640 1 --2- 172.21.15.42:0/1792738312 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa281049f0 0x7ffa28104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.016+0000 7ffa2cc00640 1 --2- 172.21.15.42:0/1792738312 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa28103800 0x7ffa28103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.016+0000 7ffa2cc00640 1 -- 172.21.15.42:0/1792738312 >> 172.21.15.42:0/1792738312 conn(0x7ffa280fed70 msgr2=0x7ffa281011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.016+0000 7ffa2cc00640 1 -- 172.21.15.42:0/1792738312 shutdown_connections 2024-04-25T00:24:07.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.016+0000 7ffa2cc00640 1 -- 172.21.15.42:0/1792738312 wait complete. 2024-04-25T00:24:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.018+0000 7ffa2cc00640 1 Processor -- start 2024-04-25T00:24:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.018+0000 7ffa2cc00640 1 -- start start 2024-04-25T00:24:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.018+0000 7ffa2cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa28103800 0x7ffa28199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.018+0000 7ffa2cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa281049f0 0x7ffa28199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.018+0000 7ffa2cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa281053b0 0x7ffa281a1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.018+0000 7ffa2cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2810f4d0 con 0x7ffa281053b0 2024-04-25T00:24:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.018+0000 7ffa2cc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2810f350 con 0x7ffa281049f0 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.018+0000 7ffa2cc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa2810f650 con 0x7ffa28103800 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa27400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa28103800 0x7ffa28199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa27400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa28103800 0x7ffa28199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48720/0 (socket says 172.21.15.42:48720) 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa27400640 1 -- 172.21.15.42:0/583397203 learned_addr learned my addr 172.21.15.42:0/583397203 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa26a00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa281049f0 0x7ffa28199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa27e00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa281053b0 0x7ffa281a1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa26a00640 1 -- 172.21.15.42:0/583397203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa28103800 msgr2=0x7ffa28199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa26a00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa28103800 0x7ffa28199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa26a00640 1 -- 172.21.15.42:0/583397203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa281053b0 msgr2=0x7ffa281a1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa26a00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa281053b0 0x7ffa281a1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa26a00640 1 -- 172.21.15.42:0/583397203 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa281a16d0 con 0x7ffa281049f0 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa27400640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa28103800 0x7ffa28199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.019+0000 7ffa26a00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa281049f0 0x7ffa28199ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400e9f0 tx=0x7ffa1400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.020+0000 7ffa1fe00640 1 -- 172.21.15.42:0/583397203 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1400ccc0 con 0x7ffa281049f0 2024-04-25T00:24:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.020+0000 7ffa1fe00640 1 -- 172.21.15.42:0/583397203 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa14005020 con 0x7ffa281049f0 2024-04-25T00:24:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.020+0000 7ffa2cc00640 1 -- 172.21.15.42:0/583397203 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa281a19c0 con 0x7ffa281049f0 2024-04-25T00:24:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.020+0000 7ffa1fe00640 1 -- 172.21.15.42:0/583397203 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa140049e0 con 0x7ffa281049f0 2024-04-25T00:24:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.020+0000 7ffa2cc00640 1 -- 172.21.15.42:0/583397203 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa281a1ec0 con 0x7ffa281049f0 2024-04-25T00:24:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa311d7740 1 Processor -- start 2024-04-25T00:24:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa311d7740 1 -- start start 2024-04-25T00:24:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f86e50 0x55dda0f87270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f87840 0x55dda0f8c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 0x55dda0f8ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa311d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dda0ee9ab0 con 0x55dda0f87840 2024-04-25T00:24:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa311d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dda0e92640 con 0x55dda0f8c9b0 2024-04-25T00:24:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa311d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dda0ee5a70 con 0x55dda0f86e50 2024-04-25T00:24:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f86e50 0x55dda0f87270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa1fe00640 1 -- 172.21.15.42:0/583397203 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa14005560 con 0x7ffa281049f0 2024-04-25T00:24:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f86e50 0x55dda0f87270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48728/0 (socket says 172.21.15.42:48728) 2024-04-25T00:24:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.024+0000 7ffa27e00640 1 -- 172.21.15.42:0/1319875617 learned_addr learned my addr 172.21.15.42:0/1319875617 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.025+0000 7ffa27400640 1 --2- 172.21.15.42:0/1319875617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f87840 0x55dda0f8c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.025+0000 7ffa27e00640 1 -- 172.21.15.42:0/1319875617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 msgr2=0x55dda0f8ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.025+0000 7ffa27e00640 1 --2- 172.21.15.42:0/1319875617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 0x55dda0f8ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.025+0000 7ffa27e00640 1 -- 172.21.15.42:0/1319875617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f87840 msgr2=0x55dda0f8c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.025+0000 7ffa27e00640 1 --2- 172.21.15.42:0/1319875617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f87840 0x55dda0f8c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.025+0000 7ffa27e00640 1 -- 172.21.15.42:0/1319875617 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dda0bfe210 con 0x55dda0f86e50 2024-04-25T00:24:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.025+0000 7ffa27400640 1 --2- 172.21.15.42:0/1319875617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f87840 0x55dda0f8c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.025+0000 7ffa27e00640 1 --2- 172.21.15.42:0/1319875617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f86e50 0x55dda0f87270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa20031c10 tx=0x7ffa20005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3778489795eb3e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.026+0000 7ffa13e00640 1 -- 172.21.15.42:0/1319875617 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20031650 con 0x55dda0f86e50 2024-04-25T00:24:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.026+0000 7ffa13e00640 1 -- 172.21.15.42:0/1319875617 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa20036040 con 0x55dda0f86e50 2024-04-25T00:24:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.026+0000 7ffa13e00640 1 -- 172.21.15.42:0/1319875617 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa20004710 con 0x55dda0f86e50 2024-04-25T00:24:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.026+0000 7ffa1fe00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa0c0a2b80 0x7ffa0c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.026+0000 7ffa1fe00640 1 -- 172.21.15.42:0/583397203 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ffa140cf030 con 0x7ffa281049f0 2024-04-25T00:24:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.027+0000 7ffa311d7740 1 -- 172.21.15.42:0/1319875617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f86e50 msgr2=0x55dda0f87270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.027+0000 7ffa311d7740 1 --2- 172.21.15.42:0/1319875617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f86e50 0x55dda0f87270 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa20031c10 tx=0x7ffa20005ef0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.027+0000 7ffa27400640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa0c0a2b80 0x7ffa0c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.027+0000 7ffa311d7740 1 -- 172.21.15.42:0/1319875617 shutdown_connections 2024-04-25T00:24:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.027+0000 7ffa311d7740 1 --2- 172.21.15.42:0/1319875617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 0x55dda0f8ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.027+0000 7ffa311d7740 1 --2- 172.21.15.42:0/1319875617 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f87840 0x55dda0f8c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.027+0000 7ffa311d7740 1 --2- 172.21.15.42:0/1319875617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f86e50 0x55dda0f87270 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.027+0000 7ffa311d7740 1 -- 172.21.15.42:0/1319875617 >> 172.21.15.42:0/1319875617 conn(0x55dda0f824c0 msgr2=0x55dda0f828c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.027+0000 7ffa311d7740 1 -- 172.21.15.42:0/1319875617 shutdown_connections 2024-04-25T00:24:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.027+0000 7ffa311d7740 1 -- 172.21.15.42:0/1319875617 wait complete. 2024-04-25T00:24:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.028+0000 7ffa27400640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa0c0a2b80 0x7ffa0c0a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55dda0f83390 tx=0x7ffa18005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.028+0000 7ffa311d7740 1 -- start start 2024-04-25T00:24:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.028+0000 7ffa311d7740 1 Processor -- start 2024-04-25T00:24:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 0x55dda0f91100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f87840 0x55dda0f91640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa311d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dda0f8f900 con 0x55dda0f86e50 2024-04-25T00:24:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa311d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dda0ea5990 con 0x55dda0f8c9b0 2024-04-25T00:24:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa311d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dda0f8fa80 con 0x55dda0f87840 2024-04-25T00:24:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa311d7740 10 client.0 _ll_register_callbacks cb 0x55dda0ea7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:48628/0 (socket says 172.21.15.42:48628) 2024-04-25T00:24:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa27e00640 1 -- 172.21.15.42:0/3283491193 learned_addr learned my addr 172.21.15.42:0/3283491193 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa26a00640 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f87840 0x55dda0f91640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa27400640 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 0x55dda0f91100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa26a00640 1 -- 172.21.15.42:0/3283491193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 msgr2=0x55dda0f98680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa26a00640 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa26a00640 1 -- 172.21.15.42:0/3283491193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 msgr2=0x55dda0f91100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.029+0000 7ffa26a00640 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 0x55dda0f91100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.030+0000 7ffa26a00640 1 -- 172.21.15.42:0/3283491193 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dda0cca620 con 0x55dda0f87840 2024-04-25T00:24:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.030+0000 7ffa27e00640 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.030+0000 7ffa26a00640 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f87840 0x55dda0f91640 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400c6a0 tx=0x7ffa14005290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe289ae83f05978c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.030+0000 7ffa10c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:24:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.030+0000 7ffa10c00640 1 -- 172.21.15.42:0/3283491193 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1401c040 con 0x55dda0f87840 2024-04-25T00:24:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.030+0000 7ffa10c00640 1 -- 172.21.15.42:0/3283491193 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ffa140d10f0 con 0x55dda0f87840 2024-04-25T00:24:07.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.030+0000 7ffa311d7740 1 -- 172.21.15.42:0/3283491193 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa20036040 con 0x55dda0f87840 2024-04-25T00:24:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.030+0000 7ffa10c00640 1 -- 172.21.15.42:0/3283491193 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa140862d0 con 0x55dda0f87840 2024-04-25T00:24:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.030+0000 7ffa311d7740 10 client.25618 fetch_fsmap learned FSMap version 29 2024-04-25T00:24:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.030+0000 7ffa311d7740 1 -- 172.21.15.42:0/3283491193 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dda0f9b720 con 0x55dda0f87840 2024-04-25T00:24:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.031+0000 7ffa10c00640 1 -- 172.21.15.42:0/3283491193 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa20036040 con 0x55dda0f87840 2024-04-25T00:24:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.031+0000 7ffa10c00640 1 -- 172.21.15.42:0/3283491193 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ffa140a40a0 con 0x55dda0f87840 2024-04-25T00:24:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.031+0000 7ffa311d7740 10 client.25618 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:24:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.031+0000 7ffa311d7740 10 client.25618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.031+0000 7ffa311d7740 20 client.25618 populate_metadata read hostname 'smithi042' 2024-04-25T00:24:07.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.031+0000 7ffa311d7740 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dda0f9ce40 0x55dda0f9f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.031+0000 7ffa311d7740 4 client.25618 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.031+0000 7ffa311d7740 1 -- 172.21.15.42:0/3283491193 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dda0ef6ff0 con 0x55dda0f9ce40 2024-04-25T00:24:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.031+0000 7ffa27400640 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dda0f9ce40 0x55dda0f9f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.032+0000 7ffa27400640 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dda0f9ce40 0x55dda0f9f230 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4207597ed9ae81 server_cookie=9ab6d467c78d084e in_seq=0 out_seq=0 2024-04-25T00:24:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.032+0000 7ffa10c00640 10 client.25618 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa10c00640 1 -- 172.21.15.42:0/3283491193 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dda0ef6ff0 con 0x55dda0f9ce40 2024-04-25T00:24:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa10c00640 10 client.25618 handle_command_reply: tid=0 2024-04-25T00:24:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa311d7740 1 client.25618 shutdown 2024-04-25T00:24:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa311d7740 1 -- 172.21.15.42:0/3283491193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f87840 msgr2=0x55dda0f91640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa311d7740 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f87840 0x55dda0f91640 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400c6a0 tx=0x7ffa14005290 comp rx=0 tx=0).stop 2024-04-25T00:24:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa311d7740 1 -- 172.21.15.42:0/3283491193 shutdown_connections 2024-04-25T00:24:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa311d7740 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dda0f9ce40 0x55dda0f9f230 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa311d7740 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa10c00640 0 client.25618 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa311d7740 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f87840 0x55dda0f91640 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa311d7740 1 --2- 172.21.15.42:0/3283491193 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 0x55dda0f91100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.033+0000 7ffa311d7740 1 -- 172.21.15.42:0/3283491193 >> 172.21.15.42:0/3283491193 conn(0x55dda0f824c0 msgr2=0x55dda0f81620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.034+0000 7ffa311d7740 1 -- 172.21.15.42:0/3283491193 shutdown_connections 2024-04-25T00:24:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.034+0000 7ffa311d7740 1 -- 172.21.15.42:0/3283491193 wait complete. 2024-04-25T00:24:07.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.034+0000 7ffa311d7740 20 client.25618 trim_cache size 0 max 16384 2024-04-25T00:24:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.053+0000 7ffa311d7740 1 Processor -- start 2024-04-25T00:24:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.053+0000 7ffa311d7740 1 -- start start 2024-04-25T00:24:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.053+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f86e50 0x55dda0f85ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.053+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.053+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0fa3af0 0x55dda0fa3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.053+0000 7ffa311d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281ad4d0 con 0x55dda0fa3af0 2024-04-25T00:24:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa311d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281aada0 con 0x55dda0f86e50 2024-04-25T00:24:07.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa311d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281a8bf0 con 0x55dda0f8c9b0 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa27400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f86e50 0x55dda0f85ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa27e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0fa3af0 0x55dda0fa3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa27400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f86e50 0x55dda0f85ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:48630/0 (socket says 172.21.15.42:48630) 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa27400640 1 -- 172.21.15.42:0/1597846311 learned_addr learned my addr 172.21.15.42:0/1597846311 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa26a00640 1 --2- 172.21.15.42:0/1597846311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa27400640 1 -- 172.21.15.42:0/1597846311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 msgr2=0x55dda0f98640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa27400640 1 --2- 172.21.15.42:0/1597846311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa27400640 1 -- 172.21.15.42:0/1597846311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0fa3af0 msgr2=0x55dda0fa3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa27400640 1 --2- 172.21.15.42:0/1597846311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0fa3af0 0x55dda0fa3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa27400640 1 -- 172.21.15.42:0/1597846311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dda0f86a90 con 0x55dda0f86e50 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.054+0000 7ffa26a00640 1 --2- 172.21.15.42:0/1597846311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:24:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.055+0000 7ffa27400640 1 --2- 172.21.15.42:0/1597846311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f86e50 0x55dda0f85ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55dda0f87c40 tx=0x7ffa1801b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=604fc1a427269071 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.055+0000 7ffa13400640 1 -- 172.21.15.42:0/1597846311 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1802b530 con 0x55dda0f86e50 2024-04-25T00:24:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.055+0000 7ffa13400640 1 -- 172.21.15.42:0/1597846311 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa18005500 con 0x55dda0f86e50 2024-04-25T00:24:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.055+0000 7ffa13400640 1 -- 172.21.15.42:0/1597846311 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18004b00 con 0x55dda0f86e50 2024-04-25T00:24:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.056+0000 7ffa311d7740 1 -- 172.21.15.42:0/1597846311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f86e50 msgr2=0x55dda0f85ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.056+0000 7ffa311d7740 1 --2- 172.21.15.42:0/1597846311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f86e50 0x55dda0f85ae0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55dda0f87c40 tx=0x7ffa1801b000 comp rx=0 tx=0).stop 2024-04-25T00:24:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.056+0000 7ffa311d7740 1 -- 172.21.15.42:0/1597846311 shutdown_connections 2024-04-25T00:24:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.056+0000 7ffa311d7740 1 --2- 172.21.15.42:0/1597846311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0fa3af0 0x55dda0fa3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.056+0000 7ffa311d7740 1 --2- 172.21.15.42:0/1597846311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f98640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.056+0000 7ffa311d7740 1 --2- 172.21.15.42:0/1597846311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0f86e50 0x55dda0f85ae0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.056+0000 7ffa311d7740 1 -- 172.21.15.42:0/1597846311 >> 172.21.15.42:0/1597846311 conn(0x55dda0f824c0 msgr2=0x55dda0f853d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.056+0000 7ffa311d7740 1 -- 172.21.15.42:0/1597846311 shutdown_connections 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.056+0000 7ffa311d7740 1 -- 172.21.15.42:0/1597846311 wait complete. 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.056+0000 7ffa311d7740 1 -- start start 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.057+0000 7ffa311d7740 1 Processor -- start 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.057+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 0x55dda0f98600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.057+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f96c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.057+0000 7ffa311d7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0fa3af0 0x55dda0f971b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.057+0000 7ffa311d7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281a9260 con 0x55dda0f86e50 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.057+0000 7ffa311d7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281aa6d0 con 0x55dda0fa3af0 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.057+0000 7ffa311d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa281a9930 con 0x55dda0f8c9b0 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa311d7740 10 client.0 _ll_register_callbacks cb 0x7ffa140053d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa26a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f96c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa26a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f96c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48764/0 (socket says 172.21.15.42:48764) 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa26a00640 1 -- 172.21.15.42:0/659750863 learned_addr learned my addr 172.21.15.42:0/659750863 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa27e00640 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0fa3af0 0x55dda0f971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa27400640 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 0x55dda0f98600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa26a00640 1 -- 172.21.15.42:0/659750863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0fa3af0 msgr2=0x55dda0f971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa26a00640 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0fa3af0 0x55dda0f971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa26a00640 1 -- 172.21.15.42:0/659750863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 msgr2=0x55dda0f98600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa26a00640 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 0x55dda0f98600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa26a00640 1 -- 172.21.15.42:0/659750863 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dda0f979e0 con 0x55dda0f8c9b0 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.058+0000 7ffa27400640 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 0x55dda0f98600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.059+0000 7ffa26a00640 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f96c10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffa140cced0 tx=0x7ffa140096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912c26b50e942fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.059+0000 7ffa10c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.059+0000 7ffa10c00640 1 -- 172.21.15.42:0/659750863 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1401c070 con 0x55dda0f8c9b0 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.059+0000 7ffa10c00640 1 -- 172.21.15.42:0/659750863 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ffa1409ade0 con 0x55dda0f8c9b0 2024-04-25T00:24:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.059+0000 7ffa311d7740 1 -- 172.21.15.42:0/659750863 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa18005500 con 0x55dda0f8c9b0 2024-04-25T00:24:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.059+0000 7ffa10c00640 1 -- 172.21.15.42:0/659750863 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa14097b60 con 0x55dda0f8c9b0 2024-04-25T00:24:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.059+0000 7ffa311d7740 10 client.25621 fetch_fsmap learned FSMap version 29 2024-04-25T00:24:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.059+0000 7ffa311d7740 1 -- 172.21.15.42:0/659750863 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dda1004430 con 0x55dda0f8c9b0 2024-04-25T00:24:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.059+0000 7ffa10c00640 1 -- 172.21.15.42:0/659750863 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa18005500 con 0x55dda0f8c9b0 2024-04-25T00:24:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.060+0000 7ffa10c00640 1 -- 172.21.15.42:0/659750863 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ffa1409a780 con 0x55dda0f8c9b0 2024-04-25T00:24:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.060+0000 7ffa311d7740 10 client.25621 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:24:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.060+0000 7ffa311d7740 10 client.25621 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.060+0000 7ffa311d7740 20 client.25621 populate_metadata read hostname 'smithi042' 2024-04-25T00:24:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.060+0000 7ffa311d7740 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dda1004db0 0x55dda10071d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.060+0000 7ffa311d7740 4 client.25621 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:24:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.060+0000 7ffa311d7740 1 -- 172.21.15.42:0/659750863 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dda0ef1340 con 0x55dda1004db0 2024-04-25T00:24:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.061+0000 7ffa27400640 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dda1004db0 0x55dda10071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.061+0000 7ffa27400640 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dda1004db0 0x55dda10071d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=608289723212bc31 server_cookie=d971eab9a60e774d in_seq=0 out_seq=0 2024-04-25T00:24:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.061+0000 7ffa10c00640 10 client.25621 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa10c00640 1 -- 172.21.15.42:0/659750863 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dda0ef1340 con 0x55dda1004db0 2024-04-25T00:24:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa10c00640 10 client.25621 handle_command_reply: tid=0 2024-04-25T00:24:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa311d7740 1 client.25621 shutdown 2024-04-25T00:24:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa311d7740 1 -- 172.21.15.42:0/659750863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 msgr2=0x55dda0f96c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa311d7740 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f96c10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffa140cced0 tx=0x7ffa140096e0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa311d7740 1 -- 172.21.15.42:0/659750863 shutdown_connections 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa311d7740 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dda1004db0 0x55dda10071d0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa311d7740 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dda0fa3af0 0x55dda0f971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa311d7740 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dda0f8c9b0 0x55dda0f96c10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa311d7740 1 --2- 172.21.15.42:0/659750863 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dda0f86e50 0x55dda0f98600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa10c00640 0 client.25621 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.106+0000 7ffa311d7740 1 -- 172.21.15.42:0/659750863 >> 172.21.15.42:0/659750863 conn(0x55dda0f824c0 msgr2=0x55dda0f80d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.107+0000 7ffa311d7740 1 -- 172.21.15.42:0/659750863 shutdown_connections 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.107+0000 7ffa311d7740 1 -- 172.21.15.42:0/659750863 wait complete. 2024-04-25T00:24:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.107+0000 7ffa311d7740 20 client.25621 trim_cache size 0 max 16384 2024-04-25T00:24:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:24:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:24:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:24:07.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.107+0000 7ffa2cc00640 1 -- 172.21.15.42:0/583397203 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa0c0a2b80 msgr2=0x7ffa0c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.107+0000 7ffa2cc00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa0c0a2b80 0x7ffa0c0a5040 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x55dda0f83390 tx=0x7ffa18005f90 comp rx=0 tx=0).stop 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.107+0000 7ffa2cc00640 1 -- 172.21.15.42:0/583397203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa281049f0 msgr2=0x7ffa28199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.107+0000 7ffa2cc00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa281049f0 0x7ffa28199ff0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400e9f0 tx=0x7ffa1400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.108+0000 7ffa2cc00640 1 -- 172.21.15.42:0/583397203 shutdown_connections 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.108+0000 7ffa2cc00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa0c0a2b80 0x7ffa0c0a5040 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.108+0000 7ffa2cc00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffa281053b0 0x7ffa281a1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.108+0000 7ffa2cc00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa281049f0 0x7ffa28199ff0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.108+0000 7ffa2cc00640 1 --2- 172.21.15.42:0/583397203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa28103800 0x7ffa28199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.108+0000 7ffa2cc00640 1 -- 172.21.15.42:0/583397203 >> 172.21.15.42:0/583397203 conn(0x7ffa280fed70 msgr2=0x7ffa28107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:07.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.108+0000 7ffa2cc00640 1 -- 172.21.15.42:0/583397203 shutdown_connections 2024-04-25T00:24:07.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:07.108+0000 7ffa2cc00640 1 -- 172.21.15.42:0/583397203 wait complete. 2024-04-25T00:24:07.230 INFO:tasks.workunit.client.0.smithi042.stdout: 1 293252 6.43 MB/sec execute 117 sec latency 41.044 ms 2024-04-25T00:24:07.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:07 smithi136 ceph-mon[32923]: pgmap v549: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-04-25T00:24:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:07 smithi106 ceph-mon[32510]: pgmap v549: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-04-25T00:24:07.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:07 smithi042 ceph-mon[28130]: pgmap v549: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.8 MiB/s wr, 220 op/s 2024-04-25T00:24:08.229 INFO:tasks.workunit.client.0.smithi042.stdout: 1 294715 6.48 MB/sec execute 118 sec latency 13.902 ms 2024-04-25T00:24:09.229 INFO:tasks.workunit.client.0.smithi042.stdout: 1 295405 6.43 MB/sec execute 119 sec latency 26.672 ms 2024-04-25T00:24:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:09 smithi136 ceph-mon[32923]: pgmap v550: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-04-25T00:24:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:09 smithi106 ceph-mon[32510]: pgmap v550: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-04-25T00:24:09.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:09 smithi042 ceph-mon[28130]: pgmap v550: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 797 KiB/s rd, 1.6 MiB/s wr, 202 op/s 2024-04-25T00:24:10.230 INFO:tasks.workunit.client.0.smithi042.stdout: 1 296830 6.43 MB/sec execute 120 sec latency 27.580 ms 2024-04-25T00:24:11.230 INFO:tasks.workunit.client.0.smithi042.stdout: 1 298253 6.47 MB/sec execute 121 sec latency 16.358 ms 2024-04-25T00:24:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:11 smithi136 ceph-mon[32923]: pgmap v551: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-04-25T00:24:11.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:11 smithi106 ceph-mon[32510]: pgmap v551: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-04-25T00:24:11.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:11 smithi042 ceph-mon[28130]: pgmap v551: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.5 MiB/s wr, 156 op/s 2024-04-25T00:24:12.230 INFO:tasks.workunit.client.0.smithi042.stdout: 1 298954 6.43 MB/sec execute 122 sec latency 29.110 ms 2024-04-25T00:24:13.230 INFO:tasks.workunit.client.0.smithi042.stdout: 1 300344 6.43 MB/sec execute 123 sec latency 38.103 ms 2024-04-25T00:24:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:13 smithi136 ceph-mon[32923]: pgmap v552: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-04-25T00:24:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:13 smithi106 ceph-mon[32510]: pgmap v552: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-04-25T00:24:13.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:13 smithi042 ceph-mon[28130]: pgmap v552: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-04-25T00:24:14.230 INFO:tasks.workunit.client.0.smithi042.stdout: 1 301768 6.47 MB/sec execute 124 sec latency 15.868 ms 2024-04-25T00:24:15.230 INFO:tasks.workunit.client.0.smithi042.stdout: 1 302425 6.43 MB/sec execute 125 sec latency 22.952 ms 2024-04-25T00:24:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:15 smithi136 ceph-mon[32923]: pgmap v553: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-04-25T00:24:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:15 smithi106 ceph-mon[32510]: pgmap v553: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-04-25T00:24:15.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:15 smithi042 ceph-mon[28130]: pgmap v553: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-04-25T00:24:16.231 INFO:tasks.workunit.client.0.smithi042.stdout: 1 303393 6.41 MB/sec execute 126 sec latency 30.242 ms 2024-04-25T00:24:17.231 INFO:tasks.workunit.client.0.smithi042.stdout: 1 305063 6.46 MB/sec execute 127 sec latency 66.319 ms 2024-04-25T00:24:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:17 smithi106 ceph-mon[32510]: pgmap v554: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-04-25T00:24:17.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:17 smithi042 ceph-mon[28130]: pgmap v554: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-04-25T00:24:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:17 smithi136 ceph-mon[32923]: pgmap v554: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 817 KiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-04-25T00:24:18.231 INFO:tasks.workunit.client.0.smithi042.stdout: 1 305685 6.42 MB/sec execute 128 sec latency 20.509 ms 2024-04-25T00:24:19.231 INFO:tasks.workunit.client.0.smithi042.stdout: 1 306829 6.41 MB/sec execute 129 sec latency 30.277 ms 2024-04-25T00:24:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:19 smithi106 ceph-mon[32510]: pgmap v555: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-04-25T00:24:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:19 smithi042 ceph-mon[28130]: pgmap v555: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-04-25T00:24:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:19 smithi136 ceph-mon[32923]: pgmap v555: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.5 MiB/s wr, 195 op/s 2024-04-25T00:24:20.231 INFO:tasks.workunit.client.0.smithi042.stdout: 1 308638 6.46 MB/sec execute 130 sec latency 20.534 ms 2024-04-25T00:24:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.263+0000 7efd9aa00640 1 -- 172.21.15.42:0/3848387052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd941006d0 msgr2=0x7efd94100af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.263+0000 7efd9aa00640 1 --2- 172.21.15.42:0/3848387052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd941006d0 0x7efd94100af0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efd84009ca0 tx=0x7efd8402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.263+0000 7efd9aa00640 1 -- 172.21.15.42:0/3848387052 shutdown_connections 2024-04-25T00:24:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.263+0000 7efd9aa00640 1 --2- 172.21.15.42:0/3848387052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd94102280 0x7efd94108b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.263+0000 7efd9aa00640 1 --2- 172.21.15.42:0/3848387052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd941018c0 0x7efd94101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.263+0000 7efd9aa00640 1 --2- 172.21.15.42:0/3848387052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd941006d0 0x7efd94100af0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.263+0000 7efd9aa00640 1 -- 172.21.15.42:0/3848387052 >> 172.21.15.42:0/3848387052 conn(0x7efd940fbc40 msgr2=0x7efd940fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.263+0000 7efd9aa00640 1 -- 172.21.15.42:0/3848387052 shutdown_connections 2024-04-25T00:24:20.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.263+0000 7efd9aa00640 1 -- 172.21.15.42:0/3848387052 wait complete. 2024-04-25T00:24:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.264+0000 7efd9aa00640 1 Processor -- start 2024-04-25T00:24:20.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd9aa00640 1 -- start start 2024-04-25T00:24:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd9aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd941006d0 0x7efd9410b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd9aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd941018c0 0x7efd94109690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd9aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd94102280 0x7efd94109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd9aa00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd9410d4d0 con 0x7efd941018c0 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd9aa00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd9410d350 con 0x7efd941006d0 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd9aa00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd9410d650 con 0x7efd94102280 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd93e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd941006d0 0x7efd9410b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd98a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd94102280 0x7efd94109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd93e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd941006d0 0x7efd9410b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40034/0 (socket says 172.21.15.42:40034) 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd93e00640 1 -- 172.21.15.42:0/50625342 learned_addr learned my addr 172.21.15.42:0/50625342 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.265+0000 7efd8b400640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd941018c0 0x7efd94109690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.266+0000 7efd93e00640 1 -- 172.21.15.42:0/50625342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd94102280 msgr2=0x7efd94109bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.266+0000 7efd93e00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd94102280 0x7efd94109bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.266+0000 7efd93e00640 1 -- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd941018c0 msgr2=0x7efd94109690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.266+0000 7efd93e00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd941018c0 0x7efd94109690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.266+0000 7efd93e00640 1 -- 172.21.15.42:0/50625342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9410a460 con 0x7efd941006d0 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.266+0000 7efd8b400640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd941018c0 0x7efd94109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:20.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.266+0000 7efd93e00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd941006d0 0x7efd9410b050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd84000f80 tx=0x7efd840319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.267+0000 7efd91600640 1 -- 172.21.15.42:0/50625342 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84003b70 con 0x7efd941006d0 2024-04-25T00:24:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.267+0000 7efd91600640 1 -- 172.21.15.42:0/50625342 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd84003d10 con 0x7efd941006d0 2024-04-25T00:24:20.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.267+0000 7efd91600640 1 -- 172.21.15.42:0/50625342 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8402fc50 con 0x7efd941006d0 2024-04-25T00:24:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.267+0000 7efd9aa00640 1 -- 172.21.15.42:0/50625342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd941a1b30 con 0x7efd941006d0 2024-04-25T00:24:20.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.267+0000 7efd9aa00640 1 -- 172.21.15.42:0/50625342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd941a2030 con 0x7efd941006d0 2024-04-25T00:24:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.271+0000 7efd91600640 1 -- 172.21.15.42:0/50625342 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efd84004040 con 0x7efd941006d0 2024-04-25T00:24:20.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.271+0000 7efd9d829740 1 Processor -- start 2024-04-25T00:24:20.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd9d829740 1 -- start start 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd9d829740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f1ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd9d829740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f1f3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd9d829740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f1f5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd9d829740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bb7f150ab0 con 0x55bb7f1edca0 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd9d829740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bb7f0f9640 con 0x55bb7f1f3800 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd9d829740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bb7f14ca70 con 0x55bb7f1ee690 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd93e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f1f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd98a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f1ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd8b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f1f5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd93e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f1f3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:36290/0 (socket says 172.21.15.42:36290) 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd93e00640 1 -- 172.21.15.42:0/3464514909 learned_addr learned my addr 172.21.15.42:0/3464514909 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:20.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.272+0000 7efd8b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f1f5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40050/0 (socket says 172.21.15.42:40050) 2024-04-25T00:24:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.273+0000 7efd93e00640 1 -- 172.21.15.42:0/3464514909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 msgr2=0x55bb7f1f5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.273+0000 7efd93e00640 1 --2- 172.21.15.42:0/3464514909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f1f5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.273+0000 7efd93e00640 1 -- 172.21.15.42:0/3464514909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 msgr2=0x55bb7f1ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:20.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.273+0000 7efd93e00640 1 --2- 172.21.15.42:0/3464514909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f1ee0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.273+0000 7efd93e00640 1 -- 172.21.15.42:0/3464514909 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7ee65210 con 0x55bb7f1ee690 2024-04-25T00:24:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.274+0000 7efd93e00640 1 --2- 172.21.15.42:0/3464514909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f1f3190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efd840f65a0 tx=0x7efd840f6690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d94631d5b9ce5938 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.274+0000 7efd91600640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efd780a2c30 0x7efd780a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.274+0000 7efd91600640 1 -- 172.21.15.42:0/50625342 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7efd840f05c0 con 0x7efd941006d0 2024-04-25T00:24:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.274+0000 7efd98a00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efd780a2c30 0x7efd780a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.274+0000 7efd89200640 1 -- 172.21.15.42:0/3464514909 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd840b90f0 con 0x55bb7f1ee690 2024-04-25T00:24:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.274+0000 7efd89200640 1 -- 172.21.15.42:0/3464514909 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd840fd030 con 0x55bb7f1ee690 2024-04-25T00:24:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.274+0000 7efd89200640 1 -- 172.21.15.42:0/3464514909 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd840bc7b0 con 0x55bb7f1ee690 2024-04-25T00:24:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd9d829740 1 -- 172.21.15.42:0/3464514909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 msgr2=0x55bb7f1f3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd9d829740 1 --2- 172.21.15.42:0/3464514909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f1f3190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efd840f65a0 tx=0x7efd840f6690 comp rx=0 tx=0).stop 2024-04-25T00:24:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd98a00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efd780a2c30 0x7efd780a50f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55bb7f1eeef0 tx=0x7efd8c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd9d829740 1 -- 172.21.15.42:0/3464514909 shutdown_connections 2024-04-25T00:24:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd9d829740 1 --2- 172.21.15.42:0/3464514909 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f1f5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd9d829740 1 --2- 172.21.15.42:0/3464514909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f1f3190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd9d829740 1 --2- 172.21.15.42:0/3464514909 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f1ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd9d829740 1 -- 172.21.15.42:0/3464514909 >> 172.21.15.42:0/3464514909 conn(0x55bb7f1e9310 msgr2=0x55bb7f1e9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd9d829740 1 -- 172.21.15.42:0/3464514909 shutdown_connections 2024-04-25T00:24:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd9d829740 1 -- 172.21.15.42:0/3464514909 wait complete. 2024-04-25T00:24:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.276+0000 7efd9d829740 1 -- start start 2024-04-25T00:24:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd9d829740 1 Processor -- start 2024-04-25T00:24:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd9d829740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f2002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd9d829740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f200830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd9d829740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f207870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd9d829740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bb7f14ca70 con 0x55bb7f1edca0 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd9d829740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bb7f10c990 con 0x55bb7f1f3800 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd9d829740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bb7f1fed80 con 0x55bb7f1ee690 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd9d829740 10 client.0 _ll_register_callbacks cb 0x55bb7f10e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd93e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f2002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd98a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f207870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd93e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f2002f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:51096/0 (socket says 172.21.15.42:51096) 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd93e00640 1 -- 172.21.15.42:0/2189571416 learned_addr learned my addr 172.21.15.42:0/2189571416 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.277+0000 7efd8b400640 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f200830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.278+0000 7efd8b400640 1 -- 172.21.15.42:0/2189571416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 msgr2=0x55bb7f207870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.278+0000 7efd8b400640 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f207870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.278+0000 7efd8b400640 1 -- 172.21.15.42:0/2189571416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 msgr2=0x55bb7f2002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.278+0000 7efd8b400640 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f2002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.278+0000 7efd8b400640 1 -- 172.21.15.42:0/2189571416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7ef31620 con 0x55bb7f1ee690 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.278+0000 7efd93e00640 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f2002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.278+0000 7efd98a00640 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f207870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.278+0000 7efd8b400640 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f200830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efd9410adc0 tx=0x7efd7c0039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6432c50a7183038f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.278+0000 7efd81600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.278+0000 7efd81600640 1 -- 172.21.15.42:0/2189571416 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c012860 con 0x55bb7f1ee690 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.279+0000 7efd9d829740 1 -- 172.21.15.42:0/2189571416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd840fd030 con 0x55bb7f1ee690 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.279+0000 7efd81600640 1 -- 172.21.15.42:0/2189571416 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7efd7c012a00 con 0x55bb7f1ee690 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.279+0000 7efd81600640 1 -- 172.21.15.42:0/2189571416 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efd7c022070 con 0x55bb7f1ee690 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.279+0000 7efd9d829740 10 client.25633 fetch_fsmap learned FSMap version 29 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.279+0000 7efd9d829740 1 -- 172.21.15.42:0/2189571416 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb7f20a970 con 0x55bb7f1ee690 2024-04-25T00:24:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.279+0000 7efd81600640 1 -- 172.21.15.42:0/2189571416 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd840fd030 con 0x55bb7f1ee690 2024-04-25T00:24:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.280+0000 7efd81600640 1 -- 172.21.15.42:0/2189571416 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7efd7c01a030 con 0x55bb7f1ee690 2024-04-25T00:24:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.280+0000 7efd9d829740 10 client.25633 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:24:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.280+0000 7efd9d829740 10 client.25633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.280+0000 7efd9d829740 20 client.25633 populate_metadata read hostname 'smithi042' 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.280+0000 7efd9d829740 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7f20c070 0x55bb7f20e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.280+0000 7efd9d829740 4 client.25633 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.280+0000 7efd9d829740 1 -- 172.21.15.42:0/2189571416 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb7f15dff0 con 0x55bb7f20c070 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.280+0000 7efd93e00640 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7f20c070 0x55bb7f20e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.281+0000 7efd93e00640 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7f20c070 0x55bb7f20e4f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4006ea300c0ce11 server_cookie=2e1ec67594743112 in_seq=0 out_seq=0 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.281+0000 7efd81600640 10 client.25633 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.281+0000 7efd81600640 1 -- 172.21.15.42:0/2189571416 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb7f15dff0 con 0x55bb7f20c070 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.281+0000 7efd81600640 10 client.25633 handle_command_reply: tid=0 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.281+0000 7efd9d829740 1 client.25633 shutdown 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 1 -- 172.21.15.42:0/2189571416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 msgr2=0x55bb7f200830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f200830 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efd9410adc0 tx=0x7efd7c0039c0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 1 -- 172.21.15.42:0/2189571416 shutdown_connections 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7f20c070 0x55bb7f20e4f0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f207870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1ee690 0x55bb7f200830 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd81600640 0 client.25633 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 1 --2- 172.21.15.42:0/2189571416 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f2002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 1 -- 172.21.15.42:0/2189571416 >> 172.21.15.42:0/2189571416 conn(0x55bb7f1e9310 msgr2=0x55bb7f1e8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 1 -- 172.21.15.42:0/2189571416 shutdown_connections 2024-04-25T00:24:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 1 -- 172.21.15.42:0/2189571416 wait complete. 2024-04-25T00:24:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.282+0000 7efd9d829740 20 client.25633 trim_cache size 0 max 16384 2024-04-25T00:24:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.299+0000 7efd9d829740 1 Processor -- start 2024-04-25T00:24:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.299+0000 7efd9d829740 1 -- start start 2024-04-25T00:24:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.299+0000 7efd9d829740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f1ebff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.299+0000 7efd9d829740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1ee690 0x55bb7f207830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.299+0000 7efd9d829740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1f3800 0x55bb7f1ec530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.299+0000 7efd9d829740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd941a91a0 con 0x55bb7f1edca0 2024-04-25T00:24:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.299+0000 7efd9d829740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd941a6a70 con 0x55bb7f1ee690 2024-04-25T00:24:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.299+0000 7efd9d829740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd941a48c0 con 0x55bb7f1f3800 2024-04-25T00:24:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd98a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1f3800 0x55bb7f1ec530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd93e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f1ebff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd98a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1f3800 0x55bb7f1ec530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:36310/0 (socket says 172.21.15.42:36310) 2024-04-25T00:24:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd98a00640 1 -- 172.21.15.42:0/1598811225 learned_addr learned my addr 172.21.15.42:0/1598811225 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd8b400640 1 --2- 172.21.15.42:0/1598811225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1ee690 0x55bb7f207830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd93e00640 1 -- 172.21.15.42:0/1598811225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1f3800 msgr2=0x55bb7f1ec530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd93e00640 1 --2- 172.21.15.42:0/1598811225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1f3800 0x55bb7f1ec530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd93e00640 1 -- 172.21.15.42:0/1598811225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1ee690 msgr2=0x55bb7f207830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd93e00640 1 --2- 172.21.15.42:0/1598811225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1ee690 0x55bb7f207830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd93e00640 1 -- 172.21.15.42:0/1598811225 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7f1e9140 con 0x55bb7f1edca0 2024-04-25T00:24:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd98a00640 1 --2- 172.21.15.42:0/1598811225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1f3800 0x55bb7f1ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.300+0000 7efd93e00640 1 --2- 172.21.15.42:0/1598811225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f1ebff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd840ee140 tx=0x7efd840f6110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1885565bd1bc6566 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.301+0000 7efd83e00640 1 -- 172.21.15.42:0/1598811225 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd840bc990 con 0x55bb7f1edca0 2024-04-25T00:24:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.301+0000 7efd83e00640 1 -- 172.21.15.42:0/1598811225 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd840fd040 con 0x55bb7f1edca0 2024-04-25T00:24:20.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.301+0000 7efd83e00640 1 -- 172.21.15.42:0/1598811225 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd840bcc20 con 0x55bb7f1edca0 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.301+0000 7efd9d829740 1 -- 172.21.15.42:0/1598811225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 msgr2=0x55bb7f1ebff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.302+0000 7efd9d829740 1 --2- 172.21.15.42:0/1598811225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f1ebff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7efd840ee140 tx=0x7efd840f6110 comp rx=0 tx=0).stop 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.302+0000 7efd9d829740 1 -- 172.21.15.42:0/1598811225 shutdown_connections 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.302+0000 7efd9d829740 1 --2- 172.21.15.42:0/1598811225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1f3800 0x55bb7f1ec530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.302+0000 7efd9d829740 1 --2- 172.21.15.42:0/1598811225 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1ee690 0x55bb7f207830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.302+0000 7efd9d829740 1 --2- 172.21.15.42:0/1598811225 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1edca0 0x55bb7f1ebff0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.302+0000 7efd9d829740 1 -- 172.21.15.42:0/1598811225 >> 172.21.15.42:0/1598811225 conn(0x55bb7f1e9310 msgr2=0x55bb7f2002f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.302+0000 7efd9d829740 1 -- 172.21.15.42:0/1598811225 shutdown_connections 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.302+0000 7efd9d829740 1 -- 172.21.15.42:0/1598811225 wait complete. 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.302+0000 7efd9d829740 1 -- start start 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.302+0000 7efd9d829740 1 Processor -- start 2024-04-25T00:24:20.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd9d829740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1edca0 0x55bb7f204e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd9d829740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1ee690 0x55bb7f205360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd9d829740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f206370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd9d829740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efd941a4f30 con 0x55bb7f1ee690 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd9d829740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efd941a63a0 con 0x55bb7f1f3800 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd9d829740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd941a5600 con 0x55bb7f1edca0 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd9d829740 10 client.0 _ll_register_callbacks cb 0x7efd7c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd98a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f206370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd93e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1edca0 0x55bb7f204e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd98a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f206370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40084/0 (socket says 172.21.15.42:40084) 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd98a00640 1 -- 172.21.15.42:0/2612813840 learned_addr learned my addr 172.21.15.42:0/2612813840 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.303+0000 7efd8b400640 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1ee690 0x55bb7f205360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.304+0000 7efd98a00640 1 -- 172.21.15.42:0/2612813840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1edca0 msgr2=0x55bb7f204e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.304+0000 7efd98a00640 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1edca0 0x55bb7f204e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.304+0000 7efd98a00640 1 -- 172.21.15.42:0/2612813840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1ee690 msgr2=0x55bb7f205360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:20.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.304+0000 7efd98a00640 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1ee690 0x55bb7f205360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.304+0000 7efd98a00640 1 -- 172.21.15.42:0/2612813840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7f206960 con 0x55bb7f1f3800 2024-04-25T00:24:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.304+0000 7efd98a00640 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f206370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009620 tx=0x7efd8c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=735b4d2690e0bf33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.304+0000 7efd81600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:24:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.304+0000 7efd81600640 1 -- 172.21.15.42:0/2612813840 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8c02f060 con 0x55bb7f1f3800 2024-04-25T00:24:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.304+0000 7efd81600640 1 -- 172.21.15.42:0/2612813840 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7efd8c01fb90 con 0x55bb7f1f3800 2024-04-25T00:24:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.304+0000 7efd9d829740 1 -- 172.21.15.42:0/2612813840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd840fd040 con 0x55bb7f1f3800 2024-04-25T00:24:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd81600640 1 -- 172.21.15.42:0/2612813840 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efd8c01fd90 con 0x55bb7f1f3800 2024-04-25T00:24:20.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd9d829740 10 client.25631 fetch_fsmap learned FSMap version 29 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd9d829740 1 -- 172.21.15.42:0/2612813840 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bb7f272630 con 0x55bb7f1f3800 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd81600640 1 -- 172.21.15.42:0/2612813840 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd840fd040 con 0x55bb7f1f3800 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd81600640 1 -- 172.21.15.42:0/2612813840 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7efd8c01e480 con 0x55bb7f1f3800 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd9d829740 10 client.25631 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd9d829740 10 client.25631 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd9d829740 20 client.25631 populate_metadata read hostname 'smithi042' 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd9d829740 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7f272fd0 0x55bb7f2753f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd9d829740 4 client.25631 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd9d829740 1 -- 172.21.15.42:0/2612813840 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb7f158340 con 0x55bb7f272fd0 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.305+0000 7efd93e00640 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7f272fd0 0x55bb7f2753f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.306+0000 7efd93e00640 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7f272fd0 0x55bb7f2753f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd3fb84a8fbd1a1e server_cookie=3ac041015cd74e7f in_seq=0 out_seq=0 2024-04-25T00:24:20.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.306+0000 7efd81600640 10 client.25631 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:20.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd81600640 1 -- 172.21.15.42:0/2612813840 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb7f158340 con 0x55bb7f272fd0 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd81600640 10 client.25631 handle_command_reply: tid=0 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 client.25631 shutdown 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 -- 172.21.15.42:0/2612813840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 msgr2=0x55bb7f206370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f206370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7efd8c009620 tx=0x7efd8c0046c0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 -- 172.21.15.42:0/2612813840 shutdown_connections 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7f272fd0 0x55bb7f2753f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7f1f3800 0x55bb7f206370 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7f1ee690 0x55bb7f205360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 --2- 172.21.15.42:0/2612813840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7f1edca0 0x55bb7f204e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 -- 172.21.15.42:0/2612813840 >> 172.21.15.42:0/2612813840 conn(0x55bb7f1e9310 msgr2=0x55bb7f1ebff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd81600640 0 client.25631 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 -- 172.21.15.42:0/2612813840 shutdown_connections 2024-04-25T00:24:20.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 1 -- 172.21.15.42:0/2612813840 wait complete. 2024-04-25T00:24:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.352+0000 7efd9d829740 20 client.25631 trim_cache size 0 max 16384 2024-04-25T00:24:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:24:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:24:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:24:20.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.353+0000 7efd9aa00640 1 -- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efd780a2c30 msgr2=0x7efd780a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.353+0000 7efd9aa00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efd780a2c30 0x7efd780a50f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55bb7f1eeef0 tx=0x7efd8c005f90 comp rx=0 tx=0).stop 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.353+0000 7efd9aa00640 1 -- 172.21.15.42:0/50625342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd941006d0 msgr2=0x7efd9410b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.353+0000 7efd9aa00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd941006d0 0x7efd9410b050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efd84000f80 tx=0x7efd840319f0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.353+0000 7efd9aa00640 1 -- 172.21.15.42:0/50625342 shutdown_connections 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.353+0000 7efd9aa00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efd780a2c30 0x7efd780a50f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.353+0000 7efd9aa00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd94102280 0x7efd94109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.353+0000 7efd9aa00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efd941018c0 0x7efd94109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.353+0000 7efd9aa00640 1 --2- 172.21.15.42:0/50625342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efd941006d0 0x7efd9410b050 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.354+0000 7efd9aa00640 1 -- 172.21.15.42:0/50625342 >> 172.21.15.42:0/50625342 conn(0x7efd940fbc40 msgr2=0x7efd94104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:20.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.354+0000 7efd9aa00640 1 -- 172.21.15.42:0/50625342 shutdown_connections 2024-04-25T00:24:20.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:20.354+0000 7efd9aa00640 1 -- 172.21.15.42:0/50625342 wait complete. 2024-04-25T00:24:21.231 INFO:tasks.workunit.client.0.smithi042.stdout: 1 309205 6.42 MB/sec execute 131 sec latency 26.326 ms 2024-04-25T00:24:21.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:21 smithi106 ceph-mon[32510]: pgmap v556: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-25T00:24:21.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:21 smithi042 ceph-mon[28130]: pgmap v556: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-25T00:24:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:21 smithi136 ceph-mon[32923]: pgmap v556: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 594 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-25T00:24:22.231 INFO:tasks.workunit.client.0.smithi042.stdout: 1 310172 6.41 MB/sec execute 132 sec latency 33.544 ms 2024-04-25T00:24:23.231 INFO:tasks.workunit.client.0.smithi042.stdout: 1 311823 6.42 MB/sec execute 133 sec latency 38.714 ms 2024-04-25T00:24:23.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:23 smithi106 ceph-mon[32510]: pgmap v557: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-04-25T00:24:23.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:23 smithi042 ceph-mon[28130]: pgmap v557: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-04-25T00:24:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:23 smithi136 ceph-mon[32923]: pgmap v557: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 767 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-04-25T00:24:24.231 INFO:tasks.workunit.client.0.smithi042.stdout: 1 312574 6.42 MB/sec execute 134 sec latency 19.643 ms 2024-04-25T00:24:25.231 INFO:tasks.workunit.client.0.smithi042.stdout: 1 313362 6.39 MB/sec execute 135 sec latency 21.253 ms 2024-04-25T00:24:25.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:25 smithi106 ceph-mon[32510]: pgmap v558: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-25T00:24:25.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:25 smithi042 ceph-mon[28130]: pgmap v558: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-25T00:24:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:25 smithi136 ceph-mon[32923]: pgmap v558: 97 pgs: 97 active+clean; 75 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-04-25T00:24:26.232 INFO:tasks.workunit.client.0.smithi042.stdout: 1 314457 6.37 MB/sec execute 136 sec latency 27.749 ms 2024-04-25T00:24:27.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 315932 6.41 MB/sec execute 137 sec latency 62.013 ms 2024-04-25T00:24:27.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:27 smithi106 ceph-mon[32510]: pgmap v559: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-04-25T00:24:27.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:27 smithi042 ceph-mon[28130]: pgmap v559: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-04-25T00:24:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:27 smithi136 ceph-mon[32923]: pgmap v559: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 1.8 MiB/s wr, 190 op/s 2024-04-25T00:24:28.232 INFO:tasks.workunit.client.0.smithi042.stdout: 1 316439 6.37 MB/sec execute 138 sec latency 24.946 ms 2024-04-25T00:24:29.232 INFO:tasks.workunit.client.0.smithi042.stdout: 1 317480 6.36 MB/sec execute 139 sec latency 34.901 ms 2024-04-25T00:24:29.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:29 smithi106 ceph-mon[32510]: pgmap v560: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-04-25T00:24:29.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:29 smithi042 ceph-mon[28130]: pgmap v560: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-04-25T00:24:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:29 smithi136 ceph-mon[32923]: pgmap v560: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-04-25T00:24:30.232 INFO:tasks.workunit.client.0.smithi042.stdout: 1 319310 6.41 MB/sec execute 140 sec latency 21.943 ms 2024-04-25T00:24:31.232 INFO:tasks.workunit.client.0.smithi042.stdout: 1 319728 6.37 MB/sec execute 141 sec latency 20.101 ms 2024-04-25T00:24:31.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:31 smithi106 ceph-mon[32510]: pgmap v561: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-04-25T00:24:31.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:31 smithi042 ceph-mon[28130]: pgmap v561: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-04-25T00:24:31.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:31 smithi136 ceph-mon[32923]: pgmap v561: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.7 MiB/s wr, 179 op/s 2024-04-25T00:24:32.232 INFO:tasks.workunit.client.0.smithi042.stdout: 1 320698 6.36 MB/sec execute 142 sec latency 45.864 ms 2024-04-25T00:24:33.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 322246 6.37 MB/sec execute 143 sec latency 25.151 ms 2024-04-25T00:24:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.509+0000 7f6790400640 1 -- 172.21.15.42:0/2448472727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67881039c0 msgr2=0x7f6788103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.509+0000 7f6790400640 1 --2- 172.21.15.42:0/2448472727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67881039c0 0x7f6788103de0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f677c009900 tx=0x7f677c02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.509+0000 7f6790400640 1 -- 172.21.15.42:0/2448472727 shutdown_connections 2024-04-25T00:24:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.509+0000 7f6790400640 1 --2- 172.21.15.42:0/2448472727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6788105570 0x7f678810be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.509+0000 7f6790400640 1 --2- 172.21.15.42:0/2448472727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6788104bb0 0x7f6788105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.509+0000 7f6790400640 1 --2- 172.21.15.42:0/2448472727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f67881039c0 0x7f6788103de0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.509+0000 7f6790400640 1 -- 172.21.15.42:0/2448472727 >> 172.21.15.42:0/2448472727 conn(0x7f67880fef30 msgr2=0x7f6788101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.509+0000 7f6790400640 1 -- 172.21.15.42:0/2448472727 shutdown_connections 2024-04-25T00:24:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.509+0000 7f6790400640 1 -- 172.21.15.42:0/2448472727 wait complete. 2024-04-25T00:24:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f6790400640 1 Processor -- start 2024-04-25T00:24:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f6790400640 1 -- start start 2024-04-25T00:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f6790400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67881039c0 0x7f6788199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f6790400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6788104bb0 0x7f678819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f6790400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6788105570 0x7f67881a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f6790400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f678810f690 con 0x7f6788105570 2024-04-25T00:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f6790400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f678810f510 con 0x7f6788104bb0 2024-04-25T00:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f6790400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f678810f810 con 0x7f67881039c0 2024-04-25T00:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f678d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6788104bb0 0x7f678819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f678da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67881039c0 0x7f6788199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f678d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6788104bb0 0x7f678819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34520/0 (socket says 172.21.15.42:34520) 2024-04-25T00:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f678d000640 1 -- 172.21.15.42:0/2560041287 learned_addr learned my addr 172.21.15.42:0/2560041287 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.511+0000 7f678e400640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6788105570 0x7f67881a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.512+0000 7f678d000640 1 -- 172.21.15.42:0/2560041287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67881039c0 msgr2=0x7f6788199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.512+0000 7f678d000640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67881039c0 0x7f6788199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.512+0000 7f678d000640 1 -- 172.21.15.42:0/2560041287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6788105570 msgr2=0x7f67881a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.512+0000 7f678d000640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6788105570 0x7f67881a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.512+0000 7f678d000640 1 -- 172.21.15.42:0/2560041287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67881a1850 con 0x7f6788104bb0 2024-04-25T00:24:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.512+0000 7f678d000640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6788104bb0 0x7f678819a180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f677800e9f0 tx=0x7f677800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.512+0000 7f6786000640 1 -- 172.21.15.42:0/2560041287 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677800cd80 con 0x7f6788104bb0 2024-04-25T00:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.512+0000 7f6786000640 1 -- 172.21.15.42:0/2560041287 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6778010910 con 0x7f6788104bb0 2024-04-25T00:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.512+0000 7f6790400640 1 -- 172.21.15.42:0/2560041287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67881a1b40 con 0x7f6788104bb0 2024-04-25T00:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.512+0000 7f6786000640 1 -- 172.21.15.42:0/2560041287 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6778018990 con 0x7f6788104bb0 2024-04-25T00:24:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.513+0000 7f6790400640 1 -- 172.21.15.42:0/2560041287 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67881a2060 con 0x7f6788104bb0 2024-04-25T00:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.516+0000 7f67931a2740 1 Processor -- start 2024-04-25T00:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.516+0000 7f67931a2740 1 -- start start 2024-04-25T00:24:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.516+0000 7f6786000640 1 -- 172.21.15.42:0/2560041287 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6778010ac0 con 0x7f6788104bb0 2024-04-25T00:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f67931a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58eca0 0x55e7cc58f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f67931a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc594190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f67931a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc594800 0x55e7cc596c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f67931a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e7cc4f1ab0 con 0x55e7cc58eca0 2024-04-25T00:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f67931a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e7cc49a640 con 0x55e7cc58f690 2024-04-25T00:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f67931a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e7cc4eda70 con 0x55e7cc594800 2024-04-25T00:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f678d000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc594800 0x55e7cc596c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f678da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc594190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f678e400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58eca0 0x55e7cc58f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f678da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc594190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34524/0 (socket says 172.21.15.42:34524) 2024-04-25T00:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f678da00640 1 -- 172.21.15.42:0/891832787 learned_addr learned my addr 172.21.15.42:0/891832787 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.517+0000 7f678d000640 1 --2- 172.21.15.42:0/891832787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc594800 0x55e7cc596c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38900/0 (socket says 172.21.15.42:38900) 2024-04-25T00:24:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.518+0000 7f678da00640 1 -- 172.21.15.42:0/891832787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc594800 msgr2=0x55e7cc596c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.518+0000 7f678da00640 1 --2- 172.21.15.42:0/891832787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc594800 0x55e7cc596c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.518+0000 7f678da00640 1 -- 172.21.15.42:0/891832787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58eca0 msgr2=0x55e7cc58f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.518+0000 7f678da00640 1 --2- 172.21.15.42:0/891832787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58eca0 0x55e7cc58f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.518+0000 7f678da00640 1 -- 172.21.15.42:0/891832787 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7cc206210 con 0x55e7cc58f690 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.518+0000 7f678da00640 1 --2- 172.21.15.42:0/891832787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc594190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f677c031440 tx=0x7f677c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b37f44af02f22ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.519+0000 7f6776a00640 1 -- 172.21.15.42:0/891832787 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c035040 con 0x55e7cc58f690 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.519+0000 7f6786000640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67700a2c30 0x7f67700a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.519+0000 7f6776a00640 1 -- 172.21.15.42:0/891832787 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f677c004430 con 0x55e7cc58f690 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.519+0000 7f6776a00640 1 -- 172.21.15.42:0/891832787 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c0316b0 con 0x55e7cc58f690 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.519+0000 7f6786000640 1 -- 172.21.15.42:0/2560041287 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f67780cf4c0 con 0x7f6788104bb0 2024-04-25T00:24:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.519+0000 7f678e400640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67700a2c30 0x7f67700a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f67931a2740 1 -- 172.21.15.42:0/891832787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 msgr2=0x55e7cc594190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f67931a2740 1 --2- 172.21.15.42:0/891832787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc594190 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f677c031440 tx=0x7f677c004040 comp rx=0 tx=0).stop 2024-04-25T00:24:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f67931a2740 1 -- 172.21.15.42:0/891832787 shutdown_connections 2024-04-25T00:24:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f67931a2740 1 --2- 172.21.15.42:0/891832787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc594800 0x55e7cc596c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f67931a2740 1 --2- 172.21.15.42:0/891832787 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc594190 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f67931a2740 1 --2- 172.21.15.42:0/891832787 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58eca0 0x55e7cc58f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f67931a2740 1 -- 172.21.15.42:0/891832787 >> 172.21.15.42:0/891832787 conn(0x55e7cc58a310 msgr2=0x55e7cc58a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f678e400640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67700a2c30 0x7f67700a50f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55e7cc58fef0 tx=0x7f6780009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f67931a2740 1 -- 172.21.15.42:0/891832787 shutdown_connections 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f67931a2740 1 -- 172.21.15.42:0/891832787 wait complete. 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.520+0000 7f67931a2740 1 -- start start 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.521+0000 7f67931a2740 1 Processor -- start 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.521+0000 7f67931a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc598eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.521+0000 7f67931a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc5993f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.521+0000 7f67931a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc594800 0x55e7cc5a0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.521+0000 7f67931a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e7cc597750 con 0x55e7cc594800 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.521+0000 7f67931a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e7cc4ad990 con 0x55e7cc58f690 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.521+0000 7f67931a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e7cc5978d0 con 0x55e7cc58eca0 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.521+0000 7f67931a2740 10 client.0 _ll_register_callbacks cb 0x55e7cc4af950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.521+0000 7f678da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc598eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.521+0000 7f678d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc5993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f678da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc598eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38912/0 (socket says 172.21.15.42:38912) 2024-04-25T00:24:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f678d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc5993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34532/0 (socket says 172.21.15.42:34532) 2024-04-25T00:24:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f678da00640 1 -- 172.21.15.42:0/1164709777 learned_addr learned my addr 172.21.15.42:0/1164709777 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f678e400640 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc594800 0x55e7cc5a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f678d000640 1 -- 172.21.15.42:0/1164709777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 msgr2=0x55e7cc598eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f678d000640 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc598eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f678d000640 1 -- 172.21.15.42:0/1164709777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc594800 msgr2=0x55e7cc5a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f678d000640 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc594800 0x55e7cc5a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f678d000640 1 -- 172.21.15.42:0/1164709777 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7cc2d2620 con 0x55e7cc58f690 2024-04-25T00:24:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f678d000640 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc5993f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f677800e9c0 tx=0x7f6778099d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c83a3dba2e6f63a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f676b400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:24:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f676b400640 1 -- 172.21.15.42:0/1164709777 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677809b650 con 0x55e7cc58f690 2024-04-25T00:24:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.522+0000 7f676b400640 1 -- 172.21.15.42:0/1164709777 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f677809b140 con 0x55e7cc58f690 2024-04-25T00:24:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f67931a2740 1 -- 172.21.15.42:0/1164709777 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f677c004430 con 0x55e7cc58f690 2024-04-25T00:24:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f676b400640 1 -- 172.21.15.42:0/1164709777 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6778099e30 con 0x55e7cc58f690 2024-04-25T00:24:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f67931a2740 10 client.25643 fetch_fsmap learned FSMap version 29 2024-04-25T00:24:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f67931a2740 1 -- 172.21.15.42:0/1164709777 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7cc5a3480 con 0x55e7cc58f690 2024-04-25T00:24:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f676b400640 1 -- 172.21.15.42:0/1164709777 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f677c004430 con 0x55e7cc58f690 2024-04-25T00:24:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f676b400640 1 -- 172.21.15.42:0/1164709777 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f67780979b0 con 0x55e7cc58f690 2024-04-25T00:24:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f67931a2740 10 client.25643 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:24:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f67931a2740 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f67931a2740 20 client.25643 populate_metadata read hostname 'smithi042' 2024-04-25T00:24:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f67931a2740 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e7cc5a4b10 0x55e7cc5a6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f67931a2740 4 client.25643 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.523+0000 7f67931a2740 1 -- 172.21.15.42:0/1164709777 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7cc4feff0 con 0x55e7cc5a4b10 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.524+0000 7f678da00640 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e7cc5a4b10 0x55e7cc5a6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.524+0000 7f678da00640 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e7cc5a4b10 0x55e7cc5a6f00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f36d8d9c6d25d092 server_cookie=8dd2c80edb4fde5b in_seq=0 out_seq=0 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.524+0000 7f676b400640 10 client.25643 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.525+0000 7f676b400640 1 -- 172.21.15.42:0/1164709777 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e7cc4feff0 con 0x55e7cc5a4b10 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.525+0000 7f676b400640 10 client.25643 handle_command_reply: tid=0 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.525+0000 7f67931a2740 1 client.25643 shutdown 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 1 -- 172.21.15.42:0/1164709777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 msgr2=0x55e7cc5993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc5993f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f677800e9c0 tx=0x7f6778099d00 comp rx=0 tx=0).stop 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 1 -- 172.21.15.42:0/1164709777 shutdown_connections 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e7cc5a4b10 0x55e7cc5a6f00 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc594800 0x55e7cc5a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc58f690 0x55e7cc5993f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 1 --2- 172.21.15.42:0/1164709777 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc598eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 1 -- 172.21.15.42:0/1164709777 >> 172.21.15.42:0/1164709777 conn(0x55e7cc58a310 msgr2=0x55e7cc589480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f676b400640 0 client.25643 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 1 -- 172.21.15.42:0/1164709777 shutdown_connections 2024-04-25T00:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 1 -- 172.21.15.42:0/1164709777 wait complete. 2024-04-25T00:24:33.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.526+0000 7f67931a2740 20 client.25643 trim_cache size 0 max 16384 2024-04-25T00:24:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.544+0000 7f67931a2740 1 Processor -- start 2024-04-25T00:24:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.544+0000 7f67931a2740 1 -- start start 2024-04-25T00:24:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.544+0000 7f67931a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc58da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.544+0000 7f67931a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58f690 0x55e7cc5a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.544+0000 7f67931a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 0x55e7cc5ab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.544+0000 7f67931a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67881ad620 con 0x55e7cc58f690 2024-04-25T00:24:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.544+0000 7f67931a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f67881aaef0 con 0x55e7cc594800 2024-04-25T00:24:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.544+0000 7f67931a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f67881a8d40 con 0x55e7cc58eca0 2024-04-25T00:24:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc58da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 0x55e7cc5ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 0x55e7cc5ab800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34548/0 (socket says 172.21.15.42:34548) 2024-04-25T00:24:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58f690 0x55e7cc5a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678e400640 1 -- 172.21.15.42:0/3298027412 learned_addr learned my addr 172.21.15.42:0/3298027412 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc58da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38916/0 (socket says 172.21.15.42:38916) 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678da00640 1 -- 172.21.15.42:0/3298027412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 msgr2=0x55e7cc5ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678da00640 1 --2- 172.21.15.42:0/3298027412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 0x55e7cc5ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678da00640 1 -- 172.21.15.42:0/3298027412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58f690 msgr2=0x55e7cc5a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678da00640 1 --2- 172.21.15.42:0/3298027412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58f690 0x55e7cc5a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678da00640 1 -- 172.21.15.42:0/3298027412 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7cc588890 con 0x55e7cc58eca0 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.545+0000 7f678e400640 1 --2- 172.21.15.42:0/3298027412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 0x55e7cc5ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.546+0000 7f678da00640 1 --2- 172.21.15.42:0/3298027412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc58da10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f677c009900 tx=0x7f677c0045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=774b9abdac0bb789 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.546+0000 7f6776000640 1 -- 172.21.15.42:0/3298027412 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c035040 con 0x55e7cc58eca0 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.546+0000 7f6776000640 1 -- 172.21.15.42:0/3298027412 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f677c004770 con 0x55e7cc58eca0 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.546+0000 7f6776000640 1 -- 172.21.15.42:0/3298027412 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c031910 con 0x55e7cc58eca0 2024-04-25T00:24:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.547+0000 7f67931a2740 1 -- 172.21.15.42:0/3298027412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 msgr2=0x55e7cc58da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.547+0000 7f67931a2740 1 --2- 172.21.15.42:0/3298027412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc58da10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f677c009900 tx=0x7f677c0045e0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.547+0000 7f67931a2740 1 -- 172.21.15.42:0/3298027412 shutdown_connections 2024-04-25T00:24:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.547+0000 7f67931a2740 1 --2- 172.21.15.42:0/3298027412 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 0x55e7cc5ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.547+0000 7f67931a2740 1 --2- 172.21.15.42:0/3298027412 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58f690 0x55e7cc5a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.547+0000 7f67931a2740 1 --2- 172.21.15.42:0/3298027412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc58da10 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.547+0000 7f67931a2740 1 -- 172.21.15.42:0/3298027412 >> 172.21.15.42:0/3298027412 conn(0x55e7cc58a310 msgr2=0x55e7cc58d300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.547+0000 7f67931a2740 1 -- 172.21.15.42:0/3298027412 shutdown_connections 2024-04-25T00:24:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.547+0000 7f67931a2740 1 -- 172.21.15.42:0/3298027412 wait complete. 2024-04-25T00:24:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.547+0000 7f67931a2740 1 -- start start 2024-04-25T00:24:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.548+0000 7f67931a2740 1 Processor -- start 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.548+0000 7f67931a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc59dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.548+0000 7f67931a2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58f690 0x55e7cc59e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.548+0000 7f67931a2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 0x55e7cc59f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.548+0000 7f67931a2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67881a93b0 con 0x55e7cc58f690 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.548+0000 7f67931a2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f67881aa820 con 0x55e7cc594800 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.548+0000 7f67931a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f67881a9a80 con 0x55e7cc58eca0 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.548+0000 7f67931a2740 10 client.0 _ll_register_callbacks cb 0x7f67780d2bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.549+0000 7f678da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc59dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.549+0000 7f678da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc59dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38928/0 (socket says 172.21.15.42:38928) 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.549+0000 7f678da00640 1 -- 172.21.15.42:0/1293954082 learned_addr learned my addr 172.21.15.42:0/1293954082 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:33.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.549+0000 7f678d000640 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58f690 0x55e7cc59e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.549+0000 7f678e400640 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 0x55e7cc59f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.549+0000 7f678da00640 1 -- 172.21.15.42:0/1293954082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 msgr2=0x55e7cc59f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.549+0000 7f678da00640 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 0x55e7cc59f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.549+0000 7f678da00640 1 -- 172.21.15.42:0/1293954082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58f690 msgr2=0x55e7cc59e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.549+0000 7f678da00640 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58f690 0x55e7cc59e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.549+0000 7f678da00640 1 -- 172.21.15.42:0/1293954082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7cc59f770 con 0x55e7cc58eca0 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.550+0000 7f678da00640 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc59dc80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f677c02a9e0 tx=0x7f677c004480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d7ac0abd61622cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.550+0000 7f676b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.550+0000 7f676b400640 1 -- 172.21.15.42:0/1293954082 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f677c035070 con 0x55e7cc58eca0 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.550+0000 7f676b400640 1 -- 172.21.15.42:0/1293954082 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f677c004db0 con 0x55e7cc58eca0 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.550+0000 7f67931a2740 1 -- 172.21.15.42:0/1293954082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f677c004770 con 0x55e7cc58eca0 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.550+0000 7f676b400640 1 -- 172.21.15.42:0/1293954082 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f677c0057b0 con 0x55e7cc58eca0 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.550+0000 7f67931a2740 10 client.25657 fetch_fsmap learned FSMap version 29 2024-04-25T00:24:33.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.550+0000 7f67931a2740 1 -- 172.21.15.42:0/1293954082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e7cc60b0e0 con 0x55e7cc58eca0 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f676b400640 1 -- 172.21.15.42:0/1293954082 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f677c004770 con 0x55e7cc58eca0 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f676b400640 1 -- 172.21.15.42:0/1293954082 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f677c0540c0 con 0x55e7cc58eca0 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f67931a2740 10 client.25657 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f67931a2740 10 client.25657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f67931a2740 20 client.25657 populate_metadata read hostname 'smithi042' 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f67931a2740 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e7cc60b610 0x55e7cc60da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f67931a2740 4 client.25657 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f67931a2740 1 -- 172.21.15.42:0/1293954082 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7cc4f9340 con 0x55e7cc60b610 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f678d000640 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e7cc60b610 0x55e7cc60da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f676b400640 10 client.25657 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:33.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.551+0000 7f678d000640 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e7cc60b610 0x55e7cc60da30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6e4c3f4ce697a8e server_cookie=7328632ff46ffa3e in_seq=0 out_seq=0 2024-04-25T00:24:33.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:33 smithi106 ceph-mon[32510]: pgmap v562: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f676b400640 1 -- 172.21.15.42:0/1293954082 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7cc4f9340 con 0x55e7cc60b610 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f676b400640 10 client.25657 handle_command_reply: tid=0 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 client.25657 shutdown 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 -- 172.21.15.42:0/1293954082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 msgr2=0x55e7cc59dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc59dc80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f677c02a9e0 tx=0x7f677c004480 comp rx=0 tx=0).stop 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 -- 172.21.15.42:0/1293954082 shutdown_connections 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e7cc60b610 0x55e7cc60da30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e7cc594800 0x55e7cc59f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e7cc58f690 0x55e7cc59e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 --2- 172.21.15.42:0/1293954082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e7cc58eca0 0x55e7cc59dc80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f676b400640 0 client.25657 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 -- 172.21.15.42:0/1293954082 >> 172.21.15.42:0/1293954082 conn(0x55e7cc58a310 msgr2=0x55e7cc588bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 -- 172.21.15.42:0/1293954082 shutdown_connections 2024-04-25T00:24:33.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.603+0000 7f67931a2740 1 -- 172.21.15.42:0/1293954082 wait complete. 2024-04-25T00:24:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f67931a2740 20 client.25657 trim_cache size 0 max 16384 2024-04-25T00:24:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:24:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:24:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:24:33.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f6790400640 1 -- 172.21.15.42:0/2560041287 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67700a2c30 msgr2=0x7f67700a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f6790400640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67700a2c30 0x7f67700a50f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x55e7cc58fef0 tx=0x7f6780009500 comp rx=0 tx=0).stop 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f6790400640 1 -- 172.21.15.42:0/2560041287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6788104bb0 msgr2=0x7f678819a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f6790400640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6788104bb0 0x7f678819a180 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f677800e9f0 tx=0x7f677800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f6790400640 1 -- 172.21.15.42:0/2560041287 shutdown_connections 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f6790400640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67700a2c30 0x7f67700a50f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f6790400640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6788105570 0x7f67881a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f6790400640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6788104bb0 0x7f678819a180 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f6790400640 1 --2- 172.21.15.42:0/2560041287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f67881039c0 0x7f6788199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.604+0000 7f6790400640 1 -- 172.21.15.42:0/2560041287 >> 172.21.15.42:0/2560041287 conn(0x7f67880fef30 msgr2=0x7f6788107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.605+0000 7f6790400640 1 -- 172.21.15.42:0/2560041287 shutdown_connections 2024-04-25T00:24:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:33.605+0000 7f6790400640 1 -- 172.21.15.42:0/2560041287 wait complete. 2024-04-25T00:24:33.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:33 smithi042 ceph-mon[28130]: pgmap v562: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-04-25T00:24:33.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:33 smithi136 ceph-mon[32923]: pgmap v562: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 881 KiB/s rd, 2.1 MiB/s wr, 232 op/s 2024-04-25T00:24:34.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 323202 6.37 MB/sec execute 144 sec latency 28.150 ms 2024-04-25T00:24:35.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 323983 6.34 MB/sec execute 145 sec latency 26.532 ms 2024-04-25T00:24:35.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:35 smithi106 ceph-mon[32510]: pgmap v563: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-04-25T00:24:35.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:35 smithi042 ceph-mon[28130]: pgmap v563: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-04-25T00:24:35.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:35 smithi136 ceph-mon[32923]: pgmap v563: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 1.5 MiB/s wr, 158 op/s 2024-04-25T00:24:36.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 325249 6.33 MB/sec execute 146 sec latency 36.706 ms 2024-04-25T00:24:37.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 326588 6.37 MB/sec execute 147 sec latency 53.696 ms 2024-04-25T00:24:37.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:37 smithi106 ceph-mon[32510]: pgmap v564: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-04-25T00:24:37.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:37 smithi042 ceph-mon[28130]: pgmap v564: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-04-25T00:24:37.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:37 smithi136 ceph-mon[32923]: pgmap v564: 97 pgs: 97 active+clean; 74 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.9 MiB/s wr, 191 op/s 2024-04-25T00:24:38.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 327082 6.33 MB/sec execute 148 sec latency 25.341 ms 2024-04-25T00:24:39.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 328032 6.32 MB/sec execute 149 sec latency 43.186 ms 2024-04-25T00:24:39.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:39 smithi106 ceph-mon[32510]: pgmap v565: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-04-25T00:24:39.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:39 smithi042 ceph-mon[28130]: pgmap v565: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-04-25T00:24:39.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:39 smithi136 ceph-mon[32923]: pgmap v565: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 719 KiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-04-25T00:24:40.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 329701 6.34 MB/sec execute 150 sec latency 30.146 ms 2024-04-25T00:24:41.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 330303 6.33 MB/sec execute 151 sec latency 19.339 ms 2024-04-25T00:24:41.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:41 smithi106 ceph-mon[32510]: pgmap v566: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-25T00:24:41.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:41 smithi042 ceph-mon[28130]: pgmap v566: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-25T00:24:41.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:41 smithi136 ceph-mon[32923]: pgmap v566: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-25T00:24:42.233 INFO:tasks.workunit.client.0.smithi042.stdout: 1 331270 6.32 MB/sec execute 152 sec latency 27.700 ms 2024-04-25T00:24:43.234 INFO:tasks.workunit.client.0.smithi042.stdout: 1 332501 6.30 MB/sec execute 153 sec latency 71.285 ms 2024-04-25T00:24:43.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:43 smithi106 ceph-mon[32510]: pgmap v567: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.8 MiB/s wr, 222 op/s 2024-04-25T00:24:43.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:43 smithi042 ceph-mon[28130]: pgmap v567: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.8 MiB/s wr, 222 op/s 2024-04-25T00:24:43.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:43 smithi136 ceph-mon[32923]: pgmap v567: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 756 KiB/s rd, 1.8 MiB/s wr, 222 op/s 2024-04-25T00:24:44.234 INFO:tasks.workunit.client.0.smithi042.stdout: 1 333724 6.33 MB/sec execute 154 sec latency 20.416 ms 2024-04-25T00:24:45.234 INFO:tasks.workunit.client.0.smithi042.stdout: 1 334229 6.29 MB/sec execute 155 sec latency 22.846 ms 2024-04-25T00:24:45.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:45 smithi106 ceph-mon[32510]: pgmap v568: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-04-25T00:24:45.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:45 smithi042 ceph-mon[28130]: pgmap v568: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-04-25T00:24:45.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:45 smithi136 ceph-mon[32923]: pgmap v568: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-04-25T00:24:46.234 INFO:tasks.workunit.client.0.smithi042.stdout: 1 335123 6.28 MB/sec execute 156 sec latency 32.879 ms 2024-04-25T00:24:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.754+0000 7fd1e2800640 1 -- 172.21.15.42:0/1795510888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc101890 msgr2=0x7fd1dc101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.754+0000 7fd1e2800640 1 --2- 172.21.15.42:0/1795510888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc101890 0x7fd1dc101d10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009900 tx=0x7fd1c802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.754+0000 7fd1e2800640 1 -- 172.21.15.42:0/1795510888 shutdown_connections 2024-04-25T00:24:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.754+0000 7fd1e2800640 1 --2- 172.21.15.42:0/1795510888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1dc102250 0x7fd1dc108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.754+0000 7fd1e2800640 1 --2- 172.21.15.42:0/1795510888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc101890 0x7fd1dc101d10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.754+0000 7fd1e2800640 1 --2- 172.21.15.42:0/1795510888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc1006a0 0x7fd1dc100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.754+0000 7fd1e2800640 1 -- 172.21.15.42:0/1795510888 >> 172.21.15.42:0/1795510888 conn(0x7fd1dc0fbc10 msgr2=0x7fd1dc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.754+0000 7fd1e2800640 1 -- 172.21.15.42:0/1795510888 shutdown_connections 2024-04-25T00:24:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.754+0000 7fd1e2800640 1 -- 172.21.15.42:0/1795510888 wait complete. 2024-04-25T00:24:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1e2800640 1 Processor -- start 2024-04-25T00:24:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1e2800640 1 -- start start 2024-04-25T00:24:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1e2800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc1006a0 0x7fd1dc199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1e2800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1dc101890 0x7fd1dc19a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1e2800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc102250 0x7fd1dc1a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1e2800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc10d4a0 con 0x7fd1dc101890 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1e2800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc10d320 con 0x7fd1dc1006a0 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1e2800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc10d620 con 0x7fd1dc102250 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc1006a0 0x7fd1dc199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc1006a0 0x7fd1dc199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49354/0 (socket says 172.21.15.42:49354) 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1db400640 1 -- 172.21.15.42:0/1060335802 learned_addr learned my addr 172.21.15.42:0/1060335802 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.756+0000 7fd1dbe00640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc102250 0x7fd1dc1a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.757+0000 7fd1daa00640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1dc101890 0x7fd1dc19a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.757+0000 7fd1db400640 1 -- 172.21.15.42:0/1060335802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc102250 msgr2=0x7fd1dc1a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.757+0000 7fd1db400640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc102250 0x7fd1dc1a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.757+0000 7fd1db400640 1 -- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1dc101890 msgr2=0x7fd1dc19a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.757+0000 7fd1db400640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1dc101890 0x7fd1dc19a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.757+0000 7fd1db400640 1 -- 172.21.15.42:0/1060335802 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1dc1a1870 con 0x7fd1dc1006a0 2024-04-25T00:24:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.757+0000 7fd1daa00640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1dc101890 0x7fd1dc19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.757+0000 7fd1db400640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc1006a0 0x7fd1dc199c60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00e9c0 tx=0x7fd1cc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.757+0000 7fd1d3e00640 1 -- 172.21.15.42:0/1060335802 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc00cd80 con 0x7fd1dc1006a0 2024-04-25T00:24:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.758+0000 7fd1d3e00640 1 -- 172.21.15.42:0/1060335802 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1cc0035f0 con 0x7fd1dc1006a0 2024-04-25T00:24:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.758+0000 7fd1e2800640 1 -- 172.21.15.42:0/1060335802 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1dc1a1b60 con 0x7fd1dc1006a0 2024-04-25T00:24:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.758+0000 7fd1d3e00640 1 -- 172.21.15.42:0/1060335802 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc010690 con 0x7fd1dc1006a0 2024-04-25T00:24:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.758+0000 7fd1e2800640 1 -- 172.21.15.42:0/1060335802 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1dc1a2100 con 0x7fd1dc1006a0 2024-04-25T00:24:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.761+0000 7fd1e5766740 1 Processor -- start 2024-04-25T00:24:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.761+0000 7fd1d3e00640 1 -- 172.21.15.42:0/1060335802 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd1cc0037a0 con 0x7fd1dc1006a0 2024-04-25T00:24:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.761+0000 7fd1e5766740 1 -- start start 2024-04-25T00:24:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad434ca0 0x5603ad4350c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad435690 0x5603ad43a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad43a800 0x5603ad43cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1e5766740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5603ad397ab0 con 0x5603ad434ca0 2024-04-25T00:24:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1e5766740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5603ad340640 con 0x5603ad43a800 2024-04-25T00:24:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1e5766740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5603ad393a70 con 0x5603ad435690 2024-04-25T00:24:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1daa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad43a800 0x5603ad43cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad434ca0 0x5603ad4350c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1daa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad43a800 0x5603ad43cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:33084/0 (socket says 172.21.15.42:33084) 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1daa00640 1 -- 172.21.15.42:0/1327172338 learned_addr learned my addr 172.21.15.42:0/1327172338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.762+0000 7fd1db400640 1 --2- 172.21.15.42:0/1327172338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad435690 0x5603ad43a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1daa00640 1 -- 172.21.15.42:0/1327172338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad435690 msgr2=0x5603ad43a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1daa00640 1 --2- 172.21.15.42:0/1327172338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad435690 0x5603ad43a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1daa00640 1 -- 172.21.15.42:0/1327172338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad434ca0 msgr2=0x5603ad4350c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1daa00640 1 --2- 172.21.15.42:0/1327172338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad434ca0 0x5603ad4350c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1daa00640 1 -- 172.21.15.42:0/1327172338 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ad0ac210 con 0x5603ad43a800 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1dbe00640 1 --2- 172.21.15.42:0/1327172338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad434ca0 0x5603ad4350c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1daa00640 1 --2- 172.21.15.42:0/1327172338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad43a800 0x5603ad43cc90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8031440 tx=0x7fd1c80040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3a5a8cf134b050 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1d3e00640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1c00a2cb0 0x7fd1c00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1d0c00640 1 -- 172.21.15.42:0/1327172338 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8035030 con 0x5603ad43a800 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1d0c00640 1 -- 172.21.15.42:0/1327172338 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c8005520 con 0x5603ad43a800 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1d0c00640 1 -- 172.21.15.42:0/1327172338 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80316b0 con 0x5603ad43a800 2024-04-25T00:24:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.763+0000 7fd1d3e00640 1 -- 172.21.15.42:0/1060335802 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd1cc0cf4c0 con 0x7fd1dc1006a0 2024-04-25T00:24:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.764+0000 7fd1dbe00640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1c00a2cb0 0x7fd1c00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.764+0000 7fd1e5766740 1 -- 172.21.15.42:0/1327172338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad43a800 msgr2=0x5603ad43cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.764+0000 7fd1e5766740 1 --2- 172.21.15.42:0/1327172338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad43a800 0x5603ad43cc90 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8031440 tx=0x7fd1c80040f0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.764+0000 7fd1e5766740 1 -- 172.21.15.42:0/1327172338 shutdown_connections 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.764+0000 7fd1e5766740 1 --2- 172.21.15.42:0/1327172338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad43a800 0x5603ad43cc90 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.764+0000 7fd1e5766740 1 --2- 172.21.15.42:0/1327172338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad435690 0x5603ad43a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.764+0000 7fd1e5766740 1 --2- 172.21.15.42:0/1327172338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad434ca0 0x5603ad4350c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.764+0000 7fd1dbe00640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1c00a2cb0 0x7fd1c00a5170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5603ad435ef0 tx=0x7fd1d40096a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.764+0000 7fd1e5766740 1 -- 172.21.15.42:0/1327172338 >> 172.21.15.42:0/1327172338 conn(0x5603ad430310 msgr2=0x5603ad430710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.765+0000 7fd1e5766740 1 -- 172.21.15.42:0/1327172338 shutdown_connections 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.765+0000 7fd1e5766740 1 -- 172.21.15.42:0/1327172338 wait complete. 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.765+0000 7fd1e5766740 1 -- start start 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.765+0000 7fd1e5766740 1 Processor -- start 2024-04-25T00:24:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad434ca0 0x5603ad43eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad435690 0x5603ad43f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad43a800 0x5603ad4463f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1e5766740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5603ad43d750 con 0x5603ad43a800 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1e5766740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5603ad353990 con 0x5603ad435690 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1e5766740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5603ad43d8d0 con 0x5603ad434ca0 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1e5766740 10 client.0 _ll_register_callbacks cb 0x5603ad355950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1daa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad435690 0x5603ad43f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1daa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad435690 0x5603ad43f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:33088/0 (socket says 172.21.15.42:33088) 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1daa00640 1 -- 172.21.15.42:0/611601049 learned_addr learned my addr 172.21.15.42:0/611601049 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1dbe00640 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad43a800 0x5603ad4463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.766+0000 7fd1db400640 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad434ca0 0x5603ad43eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1db400640 1 -- 172.21.15.42:0/611601049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad435690 msgr2=0x5603ad43f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1db400640 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad435690 0x5603ad43f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1db400640 1 -- 172.21.15.42:0/611601049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad43a800 msgr2=0x5603ad4463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1db400640 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad43a800 0x5603ad4463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1db400640 1 -- 172.21.15.42:0/611601049 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ad178620 con 0x5603ad434ca0 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1daa00640 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad435690 0x5603ad43f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1db400640 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad434ca0 0x5603ad43eee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0cd040 tx=0x7fd1cc098020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d08f04f7f8e5be1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1c5600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1c5600640 1 -- 172.21.15.42:0/611601049 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc099e90 con 0x5603ad434ca0 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1e5766740 1 -- 172.21.15.42:0/611601049 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1c8005520 con 0x5603ad434ca0 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1c5600640 1 -- 172.21.15.42:0/611601049 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd1cc0cf0f0 con 0x5603ad434ca0 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.767+0000 7fd1c5600640 1 -- 172.21.15.42:0/611601049 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd1cc0cf2f0 con 0x5603ad434ca0 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.768+0000 7fd1e5766740 10 client.25669 fetch_fsmap learned FSMap version 29 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.768+0000 7fd1e5766740 1 -- 172.21.15.42:0/611601049 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5603ad449490 con 0x5603ad434ca0 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.768+0000 7fd1c5600640 1 -- 172.21.15.42:0/611601049 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1c8005520 con 0x5603ad434ca0 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.768+0000 7fd1c5600640 1 -- 172.21.15.42:0/611601049 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fd1cc09b1a0 con 0x5603ad434ca0 2024-04-25T00:24:46.771 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.768+0000 7fd1e5766740 10 client.25669 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:24:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.768+0000 7fd1e5766740 10 client.25669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.768+0000 7fd1e5766740 20 client.25669 populate_metadata read hostname 'smithi042' 2024-04-25T00:24:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.768+0000 7fd1e5766740 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5603ad44ab30 0x5603ad44cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.768+0000 7fd1e5766740 4 client.25669 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:24:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.768+0000 7fd1e5766740 1 -- 172.21.15.42:0/611601049 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603ad3a4ff0 con 0x5603ad44ab30 2024-04-25T00:24:46.772 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.769+0000 7fd1daa00640 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5603ad44ab30 0x5603ad44cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.769+0000 7fd1daa00640 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5603ad44ab30 0x5603ad44cf80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e76752b0c3498fb server_cookie=efff0d6ce39c70d2 in_seq=0 out_seq=0 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.769+0000 7fd1c5600640 10 client.25669 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.770+0000 7fd1c5600640 1 -- 172.21.15.42:0/611601049 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5603ad3a4ff0 con 0x5603ad44ab30 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.770+0000 7fd1c5600640 10 client.25669 handle_command_reply: tid=0 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.770+0000 7fd1e5766740 1 client.25669 shutdown 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 1 -- 172.21.15.42:0/611601049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad434ca0 msgr2=0x5603ad43eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad434ca0 0x5603ad43eee0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0cd040 tx=0x7fd1cc098020 comp rx=0 tx=0).stop 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 1 -- 172.21.15.42:0/611601049 shutdown_connections 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5603ad44ab30 0x5603ad44cf80 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad43a800 0x5603ad4463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad435690 0x5603ad43f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 1 --2- 172.21.15.42:0/611601049 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad434ca0 0x5603ad43eee0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 1 -- 172.21.15.42:0/611601049 >> 172.21.15.42:0/611601049 conn(0x5603ad430310 msgr2=0x5603ad42f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1c5600640 0 client.25669 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 1 -- 172.21.15.42:0/611601049 shutdown_connections 2024-04-25T00:24:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 1 -- 172.21.15.42:0/611601049 wait complete. 2024-04-25T00:24:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.771+0000 7fd1e5766740 20 client.25669 trim_cache size 0 max 16384 2024-04-25T00:24:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.791+0000 7fd1e5766740 1 Processor -- start 2024-04-25T00:24:46.791 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.791+0000 7fd1e5766740 1 -- start start 2024-04-25T00:24:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad43eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 0x5603ad446380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad43a800 0x5603ad43f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1e5766740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1ad570 con 0x5603ad435690 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1e5766740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1aae40 con 0x5603ad434ca0 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1e5766740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1a8c90 con 0x5603ad43a800 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad43eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad43eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:33092/0 (socket says 172.21.15.42:33092) 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1db400640 1 -- 172.21.15.42:0/2471395852 learned_addr learned my addr 172.21.15.42:0/2471395852 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1daa00640 1 --2- 172.21.15.42:0/2471395852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 0x5603ad446380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1dbe00640 1 --2- 172.21.15.42:0/2471395852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad43a800 0x5603ad43f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1db400640 1 -- 172.21.15.42:0/2471395852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad43a800 msgr2=0x5603ad43f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1db400640 1 --2- 172.21.15.42:0/2471395852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad43a800 0x5603ad43f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1db400640 1 -- 172.21.15.42:0/2471395852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 msgr2=0x5603ad446380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1db400640 1 --2- 172.21.15.42:0/2471395852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 0x5603ad446380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.792+0000 7fd1db400640 1 -- 172.21.15.42:0/2471395852 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ad430140 con 0x5603ad434ca0 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.793+0000 7fd1db400640 1 --2- 172.21.15.42:0/2471395852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad43eaf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc09ba50 tx=0x7fd1cc09a260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6fa1f82626fac1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.793+0000 7fd1c7e00640 1 -- 172.21.15.42:0/2471395852 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc099e90 con 0x5603ad434ca0 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.793+0000 7fd1c7e00640 1 -- 172.21.15.42:0/2471395852 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1cc09c450 con 0x5603ad434ca0 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.793+0000 7fd1c7e00640 1 -- 172.21.15.42:0/2471395852 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc09b890 con 0x5603ad434ca0 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.794+0000 7fd1e5766740 1 -- 172.21.15.42:0/2471395852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 msgr2=0x5603ad43eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.794+0000 7fd1e5766740 1 --2- 172.21.15.42:0/2471395852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad43eaf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc09ba50 tx=0x7fd1cc09a260 comp rx=0 tx=0).stop 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.794+0000 7fd1e5766740 1 -- 172.21.15.42:0/2471395852 shutdown_connections 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.794+0000 7fd1e5766740 1 --2- 172.21.15.42:0/2471395852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad43a800 0x5603ad43f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.794+0000 7fd1e5766740 1 --2- 172.21.15.42:0/2471395852 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 0x5603ad446380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.794+0000 7fd1e5766740 1 --2- 172.21.15.42:0/2471395852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad43eaf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.794+0000 7fd1e5766740 1 -- 172.21.15.42:0/2471395852 >> 172.21.15.42:0/2471395852 conn(0x5603ad430310 msgr2=0x5603ad42afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.794+0000 7fd1e5766740 1 -- 172.21.15.42:0/2471395852 shutdown_connections 2024-04-25T00:24:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.794+0000 7fd1e5766740 1 -- 172.21.15.42:0/2471395852 wait complete. 2024-04-25T00:24:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.794+0000 7fd1e5766740 1 -- start start 2024-04-25T00:24:46.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1e5766740 1 Processor -- start 2024-04-25T00:24:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad4431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 0x5603ad4436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1e5766740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad43a800 0x5603ad4af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1e5766740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1a9300 con 0x5603ad435690 2024-04-25T00:24:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1e5766740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1aa770 con 0x5603ad434ca0 2024-04-25T00:24:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1e5766740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1dc1a99d0 con 0x5603ad43a800 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1e5766740 10 client.0 _ll_register_callbacks cb 0x7fd1cc003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad4431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad4431b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:33108/0 (socket says 172.21.15.42:33108) 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1db400640 1 -- 172.21.15.42:0/3720926456 learned_addr learned my addr 172.21.15.42:0/3720926456 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1dbe00640 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad43a800 0x5603ad4af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.795+0000 7fd1daa00640 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 0x5603ad4436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1db400640 1 -- 172.21.15.42:0/3720926456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad43a800 msgr2=0x5603ad4af0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1db400640 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad43a800 0x5603ad4af0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1db400640 1 -- 172.21.15.42:0/3720926456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 msgr2=0x5603ad4436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1db400640 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 0x5603ad4436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1db400640 1 -- 172.21.15.42:0/3720926456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ad443ce0 con 0x5603ad434ca0 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1daa00640 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 0x5603ad4436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1db400640 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad4431b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00c6a0 tx=0x7fd1cc097c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d78ad6ea5a40a19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1c5600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:24:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1c5600640 1 -- 172.21.15.42:0/3720926456 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1cc09afc0 con 0x5603ad434ca0 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1e5766740 1 -- 172.21.15.42:0/3720926456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1cc09c450 con 0x5603ad434ca0 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1c5600640 1 -- 172.21.15.42:0/3720926456 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd1cc0a14a0 con 0x5603ad434ca0 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1c5600640 1 -- 172.21.15.42:0/3720926456 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd1cc09b890 con 0x5603ad434ca0 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1e5766740 10 client.25667 fetch_fsmap learned FSMap version 29 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.796+0000 7fd1e5766740 1 -- 172.21.15.42:0/3720926456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5603ad443ce0 con 0x5603ad434ca0 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.797+0000 7fd1c5600640 1 -- 172.21.15.42:0/3720926456 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd1cc09c450 con 0x5603ad434ca0 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.797+0000 7fd1c5600640 1 -- 172.21.15.42:0/3720926456 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fd1cc082080 con 0x5603ad434ca0 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.797+0000 7fd1e5766740 10 client.25667 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.797+0000 7fd1e5766740 10 client.25667 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.797+0000 7fd1e5766740 20 client.25667 populate_metadata read hostname 'smithi042' 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.797+0000 7fd1e5766740 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5603ad4b0c00 0x5603ad4b3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.797+0000 7fd1e5766740 4 client.25667 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.797+0000 7fd1e5766740 1 -- 172.21.15.42:0/3720926456 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603ad39f340 con 0x5603ad4b0c00 2024-04-25T00:24:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.797+0000 7fd1daa00640 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5603ad4b0c00 0x5603ad4b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:24:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.798+0000 7fd1daa00640 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5603ad4b0c00 0x5603ad4b3020 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d951e881bbb6f6d server_cookie=9dccf91000b59c08 in_seq=0 out_seq=0 2024-04-25T00:24:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.798+0000 7fd1c5600640 10 client.25667 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:46.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.853+0000 7fd1c5600640 1 -- 172.21.15.42:0/3720926456 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603ad39f340 con 0x5603ad4b0c00 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.853+0000 7fd1c5600640 10 client.25667 handle_command_reply: tid=0 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.853+0000 7fd1e5766740 1 client.25667 shutdown 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 1 -- 172.21.15.42:0/3720926456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 msgr2=0x5603ad4431b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad4431b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00c6a0 tx=0x7fd1cc097c40 comp rx=0 tx=0).stop 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 1 -- 172.21.15.42:0/3720926456 shutdown_connections 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5603ad4b0c00 0x5603ad4b3020 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603ad43a800 0x5603ad4af0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5603ad435690 0x5603ad4436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 1 --2- 172.21.15.42:0/3720926456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5603ad434ca0 0x5603ad4431b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 1 -- 172.21.15.42:0/3720926456 >> 172.21.15.42:0/3720926456 conn(0x5603ad430310 msgr2=0x5603ad43ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1c5600640 0 client.25667 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 1 -- 172.21.15.42:0/3720926456 shutdown_connections 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 1 -- 172.21.15.42:0/3720926456 wait complete. 2024-04-25T00:24:46.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.854+0000 7fd1e5766740 20 client.25667 trim_cache size 0 max 16384 2024-04-25T00:24:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:24:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:24:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:24:46.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 -- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1c00a2cb0 msgr2=0x7fd1c00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1c00a2cb0 0x7fd1c00a5170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x5603ad435ef0 tx=0x7fd1d40096a0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 -- 172.21.15.42:0/1060335802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc1006a0 msgr2=0x7fd1dc199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc1006a0 0x7fd1dc199c60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc00e9c0 tx=0x7fd1cc00ee80 comp rx=0 tx=0).stop 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 -- 172.21.15.42:0/1060335802 shutdown_connections 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1c00a2cb0 0x7fd1c00a5170 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd1dc102250 0x7fd1dc1a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1dc101890 0x7fd1dc19a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 --2- 172.21.15.42:0/1060335802 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc1006a0 0x7fd1dc199c60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 -- 172.21.15.42:0/1060335802 >> 172.21.15.42:0/1060335802 conn(0x7fd1dc0fbc10 msgr2=0x7fd1dc1094a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:24:46.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 -- 172.21.15.42:0/1060335802 shutdown_connections 2024-04-25T00:24:46.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:24:46.855+0000 7fd1e2800640 1 -- 172.21.15.42:0/1060335802 wait complete. 2024-04-25T00:24:47.234 INFO:tasks.workunit.client.0.smithi042.stdout: 1 336711 6.30 MB/sec execute 157 sec latency 28.655 ms 2024-04-25T00:24:47.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:47 smithi106 ceph-mon[32510]: pgmap v569: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-25T00:24:47.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:47 smithi042 ceph-mon[28130]: pgmap v569: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-25T00:24:47.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:47 smithi136 ceph-mon[32923]: pgmap v569: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 1.8 MiB/s wr, 201 op/s 2024-04-25T00:24:48.234 INFO:tasks.workunit.client.0.smithi042.stdout: 1 337375 6.29 MB/sec execute 158 sec latency 45.760 ms 2024-04-25T00:24:49.234 INFO:tasks.workunit.client.0.smithi042.stdout: 1 338147 6.26 MB/sec execute 159 sec latency 20.842 ms 2024-04-25T00:24:49.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:49 smithi042 ceph-mon[28130]: pgmap v570: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-04-25T00:24:49.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:49 smithi136 ceph-mon[32923]: pgmap v570: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-04-25T00:24:49.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:49 smithi106 ceph-mon[32510]: pgmap v570: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 696 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-04-25T00:24:50.234 INFO:tasks.workunit.client.0.smithi042.stdout: 1 339347 6.25 MB/sec execute 160 sec latency 67.790 ms 2024-04-25T00:24:51.235 INFO:tasks.workunit.client.0.smithi042.stdout: 1 340742 6.29 MB/sec execute 161 sec latency 19.533 ms 2024-04-25T00:24:51.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:51 smithi042 ceph-mon[28130]: pgmap v571: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-04-25T00:24:51.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:51 smithi136 ceph-mon[32923]: pgmap v571: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-04-25T00:24:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:51 smithi106 ceph-mon[32510]: pgmap v571: 97 pgs: 97 active+clean; 79 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 566 KiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-04-25T00:24:52.235 INFO:tasks.workunit.client.0.smithi042.stdout: 1 341200 6.25 MB/sec execute 162 sec latency 27.560 ms 2024-04-25T00:24:53.235 INFO:tasks.workunit.client.0.smithi042.stdout: 1 342210 6.25 MB/sec execute 163 sec latency 41.897 ms 2024-04-25T00:24:53.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:53 smithi042 ceph-mon[28130]: pgmap v572: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-04-25T00:24:53.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:53 smithi136 ceph-mon[32923]: pgmap v572: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-04-25T00:24:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:53 smithi106 ceph-mon[32510]: pgmap v572: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 771 KiB/s rd, 1.9 MiB/s wr, 210 op/s 2024-04-25T00:24:54.235 INFO:tasks.workunit.client.0.smithi042.stdout: 1 343856 6.26 MB/sec execute 164 sec latency 33.315 ms 2024-04-25T00:24:55.235 INFO:tasks.workunit.client.0.smithi042.stdout: 1 344467 6.25 MB/sec execute 165 sec latency 24.434 ms 2024-04-25T00:24:55.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:55 smithi042 ceph-mon[28130]: pgmap v573: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-04-25T00:24:55.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:55 smithi136 ceph-mon[32923]: pgmap v573: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-04-25T00:24:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:55 smithi106 ceph-mon[32510]: pgmap v573: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 554 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-04-25T00:24:56.235 INFO:tasks.workunit.client.0.smithi042.stdout: 1 345057 6.22 MB/sec execute 166 sec latency 22.995 ms 2024-04-25T00:24:57.235 INFO:tasks.workunit.client.0.smithi042.stdout: 1 346046 6.21 MB/sec execute 167 sec latency 30.898 ms 2024-04-25T00:24:57.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:57 smithi042 ceph-mon[28130]: pgmap v574: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-04-25T00:24:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:57 smithi136 ceph-mon[32923]: pgmap v574: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-04-25T00:24:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:57 smithi106 ceph-mon[32510]: pgmap v574: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 1.8 MiB/s wr, 195 op/s 2024-04-25T00:24:58.235 INFO:tasks.workunit.client.0.smithi042.stdout: 1 347829 6.26 MB/sec execute 168 sec latency 23.591 ms 2024-04-25T00:24:59.236 INFO:tasks.workunit.client.0.smithi042.stdout: 1 348552 6.22 MB/sec execute 169 sec latency 24.952 ms 2024-04-25T00:24:59.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:24:59 smithi042 ceph-mon[28130]: pgmap v575: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-25T00:24:59.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:24:59 smithi136 ceph-mon[32923]: pgmap v575: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-25T00:24:59.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:24:59 smithi106 ceph-mon[32510]: pgmap v575: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 606 KiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-25T00:25:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.021+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1708218131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f980ffe30 msgr2=0x7f5f98102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.021+0000 7f5f9ce00640 1 --2- 172.21.15.42:0/1708218131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f980ffe30 0x7f5f98102260 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009900 tx=0x7f5f8802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.021+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1708218131 shutdown_connections 2024-04-25T00:25:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.021+0000 7f5f9ce00640 1 --2- 172.21.15.42:0/1708218131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f98105150 0x7f5f981055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.021+0000 7f5f9ce00640 1 --2- 172.21.15.42:0/1708218131 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f981027a0 0x7f5f98104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.022+0000 7f5f9ce00640 1 --2- 172.21.15.42:0/1708218131 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f980ffe30 0x7f5f98102260 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.022+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1708218131 >> 172.21.15.42:0/1708218131 conn(0x7f5f980fba30 msgr2=0x7f5f980fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.022+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1708218131 shutdown_connections 2024-04-25T00:25:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.022+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1708218131 wait complete. 2024-04-25T00:25:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.023+0000 7f5f9ce00640 1 Processor -- start 2024-04-25T00:25:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.023+0000 7f5f9ce00640 1 -- start start 2024-04-25T00:25:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.023+0000 7f5f9ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f980ffe30 0x7f5f9819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.023+0000 7f5f9ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f981027a0 0x7f5f9819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.023+0000 7f5f9ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f98105150 0x7f5f981a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.023+0000 7f5f9ce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98113810 con 0x7f5f980ffe30 2024-04-25T00:25:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.023+0000 7f5f9ce00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98113690 con 0x7f5f981027a0 2024-04-25T00:25:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.023+0000 7f5f9ce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5f98113990 con 0x7f5f98105150 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f96a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f981027a0 0x7f5f9819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f96a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f981027a0 0x7f5f9819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38814/0 (socket says 172.21.15.42:38814) 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f96a00640 1 -- 172.21.15.42:0/1546232357 learned_addr learned my addr 172.21.15.42:0/1546232357 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f97400640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f980ffe30 0x7f5f9819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f97e00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f98105150 0x7f5f981a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f96a00640 1 -- 172.21.15.42:0/1546232357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f98105150 msgr2=0x7f5f981a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f96a00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f98105150 0x7f5f981a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f96a00640 1 -- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f980ffe30 msgr2=0x7f5f9819dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f96a00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f980ffe30 0x7f5f9819dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f96a00640 1 -- 172.21.15.42:0/1546232357 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f981a5ac0 con 0x7f5f981027a0 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f97400640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f980ffe30 0x7f5f9819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.024+0000 7f5f96a00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f981027a0 0x7f5f9819e3e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400d950 tx=0x7f5f8400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.025+0000 7f5f8fe00640 1 -- 172.21.15.42:0/1546232357 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84017070 con 0x7f5f981027a0 2024-04-25T00:25:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.025+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1546232357 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f981a5db0 con 0x7f5f981027a0 2024-04-25T00:25:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.025+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1546232357 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f981a6280 con 0x7f5f981027a0 2024-04-25T00:25:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.025+0000 7f5f8fe00640 1 -- 172.21.15.42:0/1546232357 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f84009c30 con 0x7f5f981027a0 2024-04-25T00:25:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.025+0000 7f5f8fe00640 1 -- 172.21.15.42:0/1546232357 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f84020060 con 0x7f5f981027a0 2024-04-25T00:25:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.029+0000 7f5fa126b740 1 Processor -- start 2024-04-25T00:25:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5fa126b740 1 -- start start 2024-04-25T00:25:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad02e50 0x56047ad03270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad08340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad089b0 0x56047ad0ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5fa126b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56047ac65ab0 con 0x56047ad02e50 2024-04-25T00:25:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5fa126b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56047ac0e640 con 0x56047ad089b0 2024-04-25T00:25:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5fa126b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56047ac61a70 con 0x56047ad03840 2024-04-25T00:25:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5f97400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad08340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5f97e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad02e50 0x56047ad03270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5f97400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad08340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44686/0 (socket says 172.21.15.42:44686) 2024-04-25T00:25:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.030+0000 7f5f97400640 1 -- 172.21.15.42:0/168737966 learned_addr learned my addr 172.21.15.42:0/168737966 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.031+0000 7f5f8fe00640 1 -- 172.21.15.42:0/1546232357 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5f84010460 con 0x7f5f981027a0 2024-04-25T00:25:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.031+0000 7f5f97400640 1 -- 172.21.15.42:0/168737966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad089b0 msgr2=0x56047ad0ae40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:25:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.031+0000 7f5f97400640 1 --2- 172.21.15.42:0/168737966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad089b0 0x56047ad0ae40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.031+0000 7f5f97400640 1 -- 172.21.15.42:0/168737966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad02e50 msgr2=0x56047ad03270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.031+0000 7f5f97400640 1 --2- 172.21.15.42:0/168737966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad02e50 0x56047ad03270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.031+0000 7f5f97400640 1 -- 172.21.15.42:0/168737966 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56047a97a210 con 0x56047ad03840 2024-04-25T00:25:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.032+0000 7f5f97400640 1 --2- 172.21.15.42:0/168737966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad08340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f88031610 tx=0x7f5f88032cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2540ffe087ba81e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.032+0000 7f5f8cc00640 1 -- 172.21.15.42:0/168737966 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8803f070 con 0x56047ad03840 2024-04-25T00:25:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.032+0000 7f5f8cc00640 1 -- 172.21.15.42:0/168737966 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f880318b0 con 0x56047ad03840 2024-04-25T00:25:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.032+0000 7f5f8cc00640 1 -- 172.21.15.42:0/168737966 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88035070 con 0x56047ad03840 2024-04-25T00:25:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.033+0000 7f5fa126b740 1 -- 172.21.15.42:0/168737966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 msgr2=0x56047ad08340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.033+0000 7f5fa126b740 1 --2- 172.21.15.42:0/168737966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad08340 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f5f88031610 tx=0x7f5f88032cb0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.033+0000 7f5fa126b740 1 -- 172.21.15.42:0/168737966 shutdown_connections 2024-04-25T00:25:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.033+0000 7f5fa126b740 1 --2- 172.21.15.42:0/168737966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad089b0 0x56047ad0ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.033+0000 7f5fa126b740 1 --2- 172.21.15.42:0/168737966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad08340 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.033+0000 7f5fa126b740 1 --2- 172.21.15.42:0/168737966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad02e50 0x56047ad03270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.033+0000 7f5fa126b740 1 -- 172.21.15.42:0/168737966 >> 172.21.15.42:0/168737966 conn(0x56047acfe4c0 msgr2=0x56047acfe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.033+0000 7f5fa126b740 1 -- 172.21.15.42:0/168737966 shutdown_connections 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.033+0000 7f5fa126b740 1 -- 172.21.15.42:0/168737966 wait complete. 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.033+0000 7f5f8fe00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5f7c0a2bb0 0x7f5f7c0a5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.034+0000 7f5f8fe00640 1 -- 172.21.15.42:0/1546232357 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f5f840ceb70 con 0x7f5f981027a0 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.034+0000 7f5f97400640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5f7c0a2bb0 0x7f5f7c0a5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.034+0000 7f5fa126b740 1 -- start start 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.034+0000 7f5fa126b740 1 Processor -- start 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5f97400640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5f7c0a2bb0 0x7f5f7c0a5070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f880096c0 tx=0x7f5f88000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad0d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5fa126b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56047ad0b900 con 0x56047ad089b0 2024-04-25T00:25:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5fa126b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56047ac21990 con 0x56047ad02e50 2024-04-25T00:25:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5fa126b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56047ad0ba80 con 0x56047ad03840 2024-04-25T00:25:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5f97400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5fa126b740 10 client.0 _ll_register_callbacks cb 0x56047ac23950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:00.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5f97400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38820/0 (socket says 172.21.15.42:38820) 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5f97400640 1 -- 172.21.15.42:0/2980574056 learned_addr learned my addr 172.21.15.42:0/2980574056 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.035+0000 7f5f96a00640 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad0d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.036+0000 7f5f97400640 1 -- 172.21.15.42:0/2980574056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 msgr2=0x56047ad0d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.036+0000 7f5f97400640 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad0d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.036+0000 7f5f97400640 1 -- 172.21.15.42:0/2980574056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 msgr2=0x56047ad145d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.036+0000 7f5f97400640 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.036+0000 7f5f97400640 1 -- 172.21.15.42:0/2980574056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56047aa46620 con 0x56047ad02e50 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.036+0000 7f5f97400640 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0d050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5f880057c0 tx=0x7f5f88005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=661f25daf561b34d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.036+0000 7f5f81600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.036+0000 7f5f81600640 1 -- 172.21.15.42:0/2980574056 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88055070 con 0x56047ad02e50 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.037+0000 7f5fa126b740 1 -- 172.21.15.42:0/2980574056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f880318b0 con 0x56047ad02e50 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.037+0000 7f5f81600640 1 -- 172.21.15.42:0/2980574056 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f5f8804c470 con 0x56047ad02e50 2024-04-25T00:25:00.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.037+0000 7f5f81600640 1 -- 172.21.15.42:0/2980574056 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f88061050 con 0x56047ad02e50 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.037+0000 7f5fa126b740 10 client.25676 fetch_fsmap learned FSMap version 29 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.037+0000 7f5fa126b740 1 -- 172.21.15.42:0/2980574056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56047ad175b0 con 0x56047ad02e50 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.037+0000 7f5f81600640 1 -- 172.21.15.42:0/2980574056 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f880318b0 con 0x56047ad02e50 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.038+0000 7f5f81600640 1 -- 172.21.15.42:0/2980574056 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f5f8803b690 con 0x56047ad02e50 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.038+0000 7f5fa126b740 10 client.25676 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.038+0000 7f5fa126b740 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.038+0000 7f5fa126b740 20 client.25676 populate_metadata read hostname 'smithi042' 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.038+0000 7f5fa126b740 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56047ad18950 0x56047ad1adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.038+0000 7f5fa126b740 4 client.25676 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.038+0000 7f5fa126b740 1 -- 172.21.15.42:0/2980574056 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56047ac72ff0 con 0x56047ad18950 2024-04-25T00:25:00.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.038+0000 7f5f96a00640 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56047ad18950 0x56047ad1adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.039+0000 7f5f96a00640 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56047ad18950 0x56047ad1adc0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e73c2916a3b536 server_cookie=e44e67961db1be9 in_seq=0 out_seq=0 2024-04-25T00:25:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.039+0000 7f5f81600640 10 client.25676 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5f81600640 1 -- 172.21.15.42:0/2980574056 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56047ac72ff0 con 0x56047ad18950 2024-04-25T00:25:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5f81600640 10 client.25676 handle_command_reply: tid=0 2024-04-25T00:25:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5fa126b740 1 client.25676 shutdown 2024-04-25T00:25:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5fa126b740 1 -- 172.21.15.42:0/2980574056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 msgr2=0x56047ad0d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5fa126b740 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0d050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5f880057c0 tx=0x7f5f88005bd0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5fa126b740 1 -- 172.21.15.42:0/2980574056 shutdown_connections 2024-04-25T00:25:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5fa126b740 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56047ad18950 0x56047ad1adc0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5fa126b740 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad145d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5fa126b740 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad0d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5fa126b740 1 --2- 172.21.15.42:0/2980574056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0d050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5f81600640 0 client.25676 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.040+0000 7f5fa126b740 1 -- 172.21.15.42:0/2980574056 >> 172.21.15.42:0/2980574056 conn(0x56047acfe4c0 msgr2=0x56047acfd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.041+0000 7f5fa126b740 1 -- 172.21.15.42:0/2980574056 shutdown_connections 2024-04-25T00:25:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.041+0000 7f5fa126b740 1 -- 172.21.15.42:0/2980574056 wait complete. 2024-04-25T00:25:00.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.041+0000 7f5fa126b740 20 client.25676 trim_cache size 0 max 16384 2024-04-25T00:25:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.065+0000 7f5fa126b740 1 Processor -- start 2024-04-25T00:25:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.065+0000 7f5fa126b740 1 -- start start 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.065+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.065+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad14590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.065+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad0d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.065+0000 7f5fa126b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5f78010cb0 con 0x56047ad089b0 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.065+0000 7f5fa126b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7800e580 con 0x56047ad02e50 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.065+0000 7f5fa126b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7800c390 con 0x56047ad03840 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f97400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f97400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38832/0 (socket says 172.21.15.42:38832) 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f97400640 1 -- 172.21.15.42:0/255985304 learned_addr learned my addr 172.21.15.42:0/255985304 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f97e00640 1 --2- 172.21.15.42:0/255985304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad0d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f96a00640 1 --2- 172.21.15.42:0/255985304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad14590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f97400640 1 -- 172.21.15.42:0/255985304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 msgr2=0x56047ad14590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f97400640 1 --2- 172.21.15.42:0/255985304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad14590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f97400640 1 -- 172.21.15.42:0/255985304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 msgr2=0x56047ad0d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f97400640 1 --2- 172.21.15.42:0/255985304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad0d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f97400640 1 -- 172.21.15.42:0/255985304 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56047ad03c20 con 0x56047ad02e50 2024-04-25T00:25:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.066+0000 7f5f96a00640 1 --2- 172.21.15.42:0/255985304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad14590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.067+0000 7f5f97400640 1 --2- 172.21.15.42:0/255985304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0cc60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5f8803bab0 tx=0x7f5f8802a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8b3846340a5f9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.067+0000 7f5f83e00640 1 -- 172.21.15.42:0/255985304 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f88061050 con 0x56047ad02e50 2024-04-25T00:25:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.067+0000 7f5f83e00640 1 -- 172.21.15.42:0/255985304 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f88031890 con 0x56047ad02e50 2024-04-25T00:25:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.067+0000 7f5f83e00640 1 -- 172.21.15.42:0/255985304 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8803f040 con 0x56047ad02e50 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 -- 172.21.15.42:0/255985304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 msgr2=0x56047ad0cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 --2- 172.21.15.42:0/255985304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0cc60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f5f8803bab0 tx=0x7f5f8802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 -- 172.21.15.42:0/255985304 shutdown_connections 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 --2- 172.21.15.42:0/255985304 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad0d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 --2- 172.21.15.42:0/255985304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad03840 0x56047ad14590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 --2- 172.21.15.42:0/255985304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad02e50 0x56047ad0cc60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 -- 172.21.15.42:0/255985304 >> 172.21.15.42:0/255985304 conn(0x56047acfe4c0 msgr2=0x56047ad04ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 -- 172.21.15.42:0/255985304 shutdown_connections 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 -- 172.21.15.42:0/255985304 wait complete. 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 -- start start 2024-04-25T00:25:00.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.068+0000 7f5fa126b740 1 Processor -- start 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad02e50 0x56047ad117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad03840 0x56047ad11ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5fa126b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad12c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5fa126b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7800ca40 con 0x56047ad089b0 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5fa126b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7800deb0 con 0x56047ad03840 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5fa126b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5f7800d110 con 0x56047ad02e50 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5fa126b740 10 client.0 _ll_register_callbacks cb 0x7f5f90009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5f97400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad02e50 0x56047ad117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5f96a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad03840 0x56047ad11ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5f97400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad02e50 0x56047ad117a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44718/0 (socket says 172.21.15.42:44718) 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5f96a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad03840 0x56047ad11ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38838/0 (socket says 172.21.15.42:38838) 2024-04-25T00:25:00.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5f97400640 1 -- 172.21.15.42:0/3425027869 learned_addr learned my addr 172.21.15.42:0/3425027869 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.069+0000 7f5f97e00640 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad12c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5f97400640 1 -- 172.21.15.42:0/3425027869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad03840 msgr2=0x56047ad11ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5f97400640 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad03840 0x56047ad11ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5f97400640 1 -- 172.21.15.42:0/3425027869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 msgr2=0x56047ad12c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5f97400640 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad12c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5f97400640 1 -- 172.21.15.42:0/3425027869 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56047ad13270 con 0x56047ad02e50 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5f96a00640 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad03840 0x56047ad11ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5f97400640 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad02e50 0x56047ad117a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009270 tx=0x7f5f88003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fddbcebedc72b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5f81600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5f81600640 1 -- 172.21.15.42:0/3425027869 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f8803f030 con 0x56047ad02e50 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5f81600640 1 -- 172.21.15.42:0/3425027869 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f5f8803b840 con 0x56047ad02e50 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.070+0000 7f5fa126b740 1 -- 172.21.15.42:0/3425027869 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5f88031890 con 0x56047ad02e50 2024-04-25T00:25:00.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5f81600640 1 -- 172.21.15.42:0/3425027869 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5f88061070 con 0x56047ad02e50 2024-04-25T00:25:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5fa126b740 10 client.25693 fetch_fsmap learned FSMap version 29 2024-04-25T00:25:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5fa126b740 1 -- 172.21.15.42:0/3425027869 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56047ad14580 con 0x56047ad02e50 2024-04-25T00:25:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5f81600640 1 -- 172.21.15.42:0/3425027869 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5f88031890 con 0x56047ad02e50 2024-04-25T00:25:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5f81600640 1 -- 172.21.15.42:0/3425027869 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f5f8805a630 con 0x56047ad02e50 2024-04-25T00:25:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5fa126b740 10 client.25693 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:25:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5fa126b740 10 client.25693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5fa126b740 20 client.25693 populate_metadata read hostname 'smithi042' 2024-04-25T00:25:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5fa126b740 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56047ad7f1f0 0x56047ad815e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5fa126b740 4 client.25693 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:25:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.071+0000 7f5fa126b740 1 -- 172.21.15.42:0/3425027869 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56047ac6d340 con 0x56047ad7f1f0 2024-04-25T00:25:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.072+0000 7f5f96a00640 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56047ad7f1f0 0x56047ad815e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.072+0000 7f5f81600640 10 client.25693 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.072+0000 7f5f96a00640 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56047ad7f1f0 0x56047ad815e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b93719a2bc0e9d4b server_cookie=6f3f7c08a4d3ee50 in_seq=0 out_seq=0 2024-04-25T00:25:00.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.118+0000 7f5f81600640 1 -- 172.21.15.42:0/3425027869 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56047ac6d340 con 0x56047ad7f1f0 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.118+0000 7f5f81600640 10 client.25693 handle_command_reply: tid=0 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 client.25693 shutdown 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 -- 172.21.15.42:0/3425027869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad02e50 msgr2=0x56047ad117a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad02e50 0x56047ad117a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5f88009270 tx=0x7f5f88003f60 comp rx=0 tx=0).stop 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 -- 172.21.15.42:0/3425027869 shutdown_connections 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56047ad7f1f0 0x56047ad815e0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56047ad089b0 0x56047ad12c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56047ad03840 0x56047ad11ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 --2- 172.21.15.42:0/3425027869 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56047ad02e50 0x56047ad117a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5f81600640 0 client.25693 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 -- 172.21.15.42:0/3425027869 >> 172.21.15.42:0/3425027869 conn(0x56047acfe4c0 msgr2=0x56047ad0cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:00.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 -- 172.21.15.42:0/3425027869 shutdown_connections 2024-04-25T00:25:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 1 -- 172.21.15.42:0/3425027869 wait complete. 2024-04-25T00:25:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.119+0000 7f5fa126b740 20 client.25693 trim_cache size 0 max 16384 2024-04-25T00:25:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:25:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:25:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:25:00.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:25:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5f7c0a2bb0 msgr2=0x7f5f7c0a5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5f7c0a2bb0 0x7f5f7c0a5070 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f880096c0 tx=0x7f5f88000bc0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1546232357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f981027a0 msgr2=0x7f5f9819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f981027a0 0x7f5f9819e3e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f5f8400d950 tx=0x7f5f8400de10 comp rx=0 tx=0).stop 2024-04-25T00:25:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1546232357 shutdown_connections 2024-04-25T00:25:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5f7c0a2bb0 0x7f5f7c0a5070 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5f98105150 0x7f5f981a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f981027a0 0x7f5f9819e3e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 --2- 172.21.15.42:0/1546232357 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5f980ffe30 0x7f5f9819dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1546232357 >> 172.21.15.42:0/1546232357 conn(0x7f5f980fba30 msgr2=0x7f5f98103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1546232357 shutdown_connections 2024-04-25T00:25:00.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:00.120+0000 7f5f9ce00640 1 -- 172.21.15.42:0/1546232357 wait complete. 2024-04-25T00:25:00.236 INFO:tasks.workunit.client.0.smithi042.stdout: 1 350089 6.23 MB/sec execute 170 sec latency 24.186 ms 2024-04-25T00:25:01.236 INFO:tasks.workunit.client.0.smithi042.stdout: 1 351565 6.26 MB/sec execute 171 sec latency 11.725 ms 2024-04-25T00:25:01.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:01 smithi042 ceph-mon[28130]: pgmap v576: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-25T00:25:01.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:01 smithi136 ceph-mon[32923]: pgmap v576: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-25T00:25:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:01 smithi106 ceph-mon[32510]: pgmap v576: 97 pgs: 97 active+clean; 78 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 573 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-25T00:25:02.236 INFO:tasks.workunit.client.0.smithi042.stdout: 1 352611 6.25 MB/sec execute 172 sec latency 30.887 ms 2024-04-25T00:25:03.236 INFO:tasks.workunit.client.0.smithi042.stdout: 1 354078 6.26 MB/sec execute 173 sec latency 48.492 ms 2024-04-25T00:25:03.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:03 smithi136 ceph-mon[32923]: pgmap v577: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-04-25T00:25:03.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:03 smithi106 ceph-mon[32510]: pgmap v577: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-04-25T00:25:03.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:03 smithi042 ceph-mon[28130]: pgmap v577: 97 pgs: 97 active+clean; 77 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 2.0 MiB/s wr, 234 op/s 2024-04-25T00:25:04.236 INFO:tasks.workunit.client.0.smithi042.stdout: 1 355187 6.26 MB/sec execute 174 sec latency 22.736 ms 2024-04-25T00:25:05.236 INFO:tasks.workunit.client.0.smithi042.stdout: 1 356309 6.25 MB/sec execute 175 sec latency 24.682 ms 2024-04-25T00:25:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:05 smithi136 ceph-mon[32923]: pgmap v578: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-04-25T00:25:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:05 smithi106 ceph-mon[32510]: pgmap v578: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-04-25T00:25:05.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:05.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:05 smithi042 ceph-mon[28130]: pgmap v578: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-04-25T00:25:05.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:25:05.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:05.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.237 INFO:tasks.workunit.client.0.smithi042.stdout: 1 358356 6.29 MB/sec execute 176 sec latency 18.605 ms 2024-04-25T00:25:06.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:06 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-25T00:25:06.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:06 smithi136 ceph-mon[32923]: Unable to set osd_memory_target on smithi106 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T00:25:06.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:06.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:06.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:25:06.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:06 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-25T00:25:06.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:06 smithi106 ceph-mon[32510]: Unable to set osd_memory_target on smithi106 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T00:25:06.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:06.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:06.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:25:06.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:06 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-25T00:25:06.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:06 smithi042 ceph-mon[28130]: Unable to set osd_memory_target on smithi106 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T00:25:06.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:25:06.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:25:06.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:25:06.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:25:07.237 INFO:tasks.workunit.client.0.smithi042.stdout: 1 359040 6.27 MB/sec execute 177 sec latency 28.323 ms 2024-04-25T00:25:07.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:07 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-25T00:25:07.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:07 smithi136 ceph-mon[32923]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T00:25:07.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:07 smithi136 ceph-mon[32923]: pgmap v579: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:25:07.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:07 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-25T00:25:07.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:07 smithi106 ceph-mon[32510]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T00:25:07.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:07 smithi106 ceph-mon[32510]: pgmap v579: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:25:07.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:07 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-25T00:25:07.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:07 smithi042 ceph-mon[28130]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T00:25:07.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:07 smithi042 ceph-mon[28130]: pgmap v579: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 762 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:25:08.237 INFO:tasks.workunit.client.0.smithi042.stdout: 1 360313 6.26 MB/sec execute 178 sec latency 47.403 ms 2024-04-25T00:25:09.237 INFO:tasks.workunit.client.0.smithi042.stdout: 1 362075 6.30 MB/sec execute 179 sec latency 17.135 ms 2024-04-25T00:25:09.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:09 smithi136 ceph-mon[32923]: pgmap v580: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-04-25T00:25:09.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:09 smithi106 ceph-mon[32510]: pgmap v580: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-04-25T00:25:09.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:09 smithi042 ceph-mon[28130]: pgmap v580: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 714 KiB/s rd, 1.5 MiB/s wr, 197 op/s 2024-04-25T00:25:10.237 INFO:tasks.workunit.client.0.smithi042.stdout: 1 362961 6.28 MB/sec execute 180 sec latency 33.068 ms 2024-04-25T00:25:11.237 INFO:tasks.workunit.client.0.smithi042.stdout: 1 364345 6.27 MB/sec execute 181 sec latency 24.530 ms 2024-04-25T00:25:11.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:11 smithi136 ceph-mon[32923]: pgmap v581: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-04-25T00:25:11.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:11 smithi106 ceph-mon[32510]: pgmap v581: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-04-25T00:25:11.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:11 smithi042 ceph-mon[28130]: pgmap v581: 97 pgs: 97 active+clean; 79 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 663 KiB/s rd, 1.5 MiB/s wr, 178 op/s 2024-04-25T00:25:12.237 INFO:tasks.workunit.client.0.smithi042.stdout: 1 365755 6.30 MB/sec execute 182 sec latency 12.596 ms 2024-04-25T00:25:13.238 INFO:tasks.workunit.client.0.smithi042.stdout: 1 366828 6.29 MB/sec execute 183 sec latency 20.496 ms 2024-04-25T00:25:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.276+0000 7ff586800640 1 -- 172.21.15.42:0/1166938965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff580105580 msgr2=0x7ff58010be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.276+0000 7ff586800640 1 --2- 172.21.15.42:0/1166938965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff580105580 0x7ff58010be10 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff578009ca0 tx=0x7ff57802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.276+0000 7ff586800640 1 -- 172.21.15.42:0/1166938965 shutdown_connections 2024-04-25T00:25:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.276+0000 7ff586800640 1 --2- 172.21.15.42:0/1166938965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff580105580 0x7ff58010be10 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.276+0000 7ff586800640 1 --2- 172.21.15.42:0/1166938965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff580104bc0 0x7ff580105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.276+0000 7ff586800640 1 --2- 172.21.15.42:0/1166938965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff5801039d0 0x7ff580103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.276+0000 7ff586800640 1 -- 172.21.15.42:0/1166938965 >> 172.21.15.42:0/1166938965 conn(0x7ff5800fef60 msgr2=0x7ff580101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.277+0000 7ff586800640 1 -- 172.21.15.42:0/1166938965 shutdown_connections 2024-04-25T00:25:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.277+0000 7ff586800640 1 -- 172.21.15.42:0/1166938965 wait complete. 2024-04-25T00:25:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.278+0000 7ff586800640 1 Processor -- start 2024-04-25T00:25:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.278+0000 7ff586800640 1 -- start start 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.278+0000 7ff586800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5801039d0 0x7ff580199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.278+0000 7ff586800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff580104bc0 0x7ff58019a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.278+0000 7ff586800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff580105580 0x7ff5801a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.278+0000 7ff586800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010f680 con 0x7ff580104bc0 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.278+0000 7ff586800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010f500 con 0x7ff5801039d0 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.278+0000 7ff586800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010f800 con 0x7ff580105580 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff580104bc0 0x7ff58019a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff580105580 0x7ff5801a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5801039d0 0x7ff580199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff580104bc0 0x7ff58019a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43226/0 (socket says 172.21.15.42:43226) 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57ea00640 1 -- 172.21.15.42:0/2259773622 learned_addr learned my addr 172.21.15.42:0/2259773622 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:13.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57f400640 1 -- 172.21.15.42:0/2259773622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff580105580 msgr2=0x7ff5801a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57f400640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff580105580 0x7ff5801a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57f400640 1 -- 172.21.15.42:0/2259773622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff580104bc0 msgr2=0x7ff58019a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57f400640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff580104bc0 0x7ff58019a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57f400640 1 -- 172.21.15.42:0/2259773622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5801a17f0 con 0x7ff5801039d0 2024-04-25T00:25:13.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.279+0000 7ff57f400640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5801039d0 0x7ff580199bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff57000e9c0 tx=0x7ff57000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.280+0000 7ff577e00640 1 -- 172.21.15.42:0/2259773622 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57000cd80 con 0x7ff5801039d0 2024-04-25T00:25:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.280+0000 7ff577e00640 1 -- 172.21.15.42:0/2259773622 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff570010910 con 0x7ff5801039d0 2024-04-25T00:25:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.280+0000 7ff577e00640 1 -- 172.21.15.42:0/2259773622 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570018920 con 0x7ff5801039d0 2024-04-25T00:25:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.280+0000 7ff586800640 1 -- 172.21.15.42:0/2259773622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5801a1ae0 con 0x7ff5801039d0 2024-04-25T00:25:13.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.280+0000 7ff586800640 1 -- 172.21.15.42:0/2259773622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5801a2080 con 0x7ff5801039d0 2024-04-25T00:25:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.285+0000 7ff589767740 1 Processor -- start 2024-04-25T00:25:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.285+0000 7ff577e00640 1 -- 172.21.15.42:0/2259773622 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff57001f020 con 0x7ff5801039d0 2024-04-25T00:25:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.285+0000 7ff589767740 1 -- start start 2024-04-25T00:25:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.285+0000 7ff589767740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca030c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.285+0000 7ff589767740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca03690 0x560b6ca08190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.285+0000 7ff589767740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 0x560b6ca0ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.285+0000 7ff589767740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560b6c965ab0 con 0x560b6ca03690 2024-04-25T00:25:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.285+0000 7ff589767740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560b6c90e640 con 0x560b6ca02ca0 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.285+0000 7ff589767740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b6c961a70 con 0x560b6ca08800 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca030c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca030c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56874/0 (socket says 172.21.15.42:56874) 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57fe00640 1 -- 172.21.15.42:0/3433259930 learned_addr learned my addr 172.21.15.42:0/3433259930 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57f400640 1 --2- 172.21.15.42:0/3433259930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca03690 0x560b6ca08190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57ea00640 1 --2- 172.21.15.42:0/3433259930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 0x560b6ca0ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57fe00640 1 -- 172.21.15.42:0/3433259930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 msgr2=0x560b6ca0ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57fe00640 1 --2- 172.21.15.42:0/3433259930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 0x560b6ca0ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57fe00640 1 -- 172.21.15.42:0/3433259930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca03690 msgr2=0x560b6ca08190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57fe00640 1 --2- 172.21.15.42:0/3433259930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca03690 0x560b6ca08190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57fe00640 1 -- 172.21.15.42:0/3433259930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b6c67a210 con 0x560b6ca02ca0 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57ea00640 1 --2- 172.21.15.42:0/3433259930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 0x560b6ca0ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff57fe00640 1 --2- 172.21.15.42:0/3433259930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca030c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff578009ca0 tx=0x7ff5780093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55e306f50030c084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.286+0000 7ff577e00640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5640a2cb0 0x7ff5640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.287+0000 7ff574c00640 1 -- 172.21.15.42:0/3433259930 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578031650 con 0x560b6ca02ca0 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.287+0000 7ff574c00640 1 -- 172.21.15.42:0/3433259930 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff578036030 con 0x560b6ca02ca0 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.287+0000 7ff574c00640 1 -- 172.21.15.42:0/3433259930 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578004740 con 0x560b6ca02ca0 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.287+0000 7ff577e00640 1 -- 172.21.15.42:0/2259773622 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff5700cef80 con 0x7ff5801039d0 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.287+0000 7ff57f400640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5640a2cb0 0x7ff5640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.287+0000 7ff589767740 1 -- 172.21.15.42:0/3433259930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 msgr2=0x560b6ca030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.287+0000 7ff589767740 1 --2- 172.21.15.42:0/3433259930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca030c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff578009ca0 tx=0x7ff5780093c0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.288+0000 7ff57f400640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5640a2cb0 0x7ff5640a5170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x560b6ca03a90 tx=0x7ff570000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.288+0000 7ff589767740 1 -- 172.21.15.42:0/3433259930 shutdown_connections 2024-04-25T00:25:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.288+0000 7ff589767740 1 --2- 172.21.15.42:0/3433259930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 0x560b6ca0ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.288+0000 7ff589767740 1 --2- 172.21.15.42:0/3433259930 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca03690 0x560b6ca08190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.288+0000 7ff589767740 1 --2- 172.21.15.42:0/3433259930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca030c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.288+0000 7ff589767740 1 -- 172.21.15.42:0/3433259930 >> 172.21.15.42:0/3433259930 conn(0x560b6c9fe310 msgr2=0x560b6c9fe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.288+0000 7ff589767740 1 -- 172.21.15.42:0/3433259930 shutdown_connections 2024-04-25T00:25:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.288+0000 7ff589767740 1 -- 172.21.15.42:0/3433259930 wait complete. 2024-04-25T00:25:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.288+0000 7ff589767740 1 -- start start 2024-04-25T00:25:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.288+0000 7ff589767740 1 Processor -- start 2024-04-25T00:25:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff589767740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca0cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff589767740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca03690 0x560b6ca0d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff589767740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 0x560b6ca142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff589767740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560b6ca0b5d0 con 0x560b6ca03690 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff589767740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560b6c921990 con 0x560b6ca02ca0 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff589767740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b6ca0b750 con 0x560b6ca08800 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff589767740 10 client.0 _ll_register_callbacks cb 0x560b6c923950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca0cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff57ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca03690 0x560b6ca0d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca0cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56878/0 (socket says 172.21.15.42:56878) 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff57fe00640 1 -- 172.21.15.42:0/3311644518 learned_addr learned my addr 172.21.15.42:0/3311644518 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.289+0000 7ff57fe00640 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 0x560b6ca142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.290+0000 7ff57fe00640 1 -- 172.21.15.42:0/3311644518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 msgr2=0x560b6ca142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.290+0000 7ff57fe00640 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 0x560b6ca142e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.290+0000 7ff57fe00640 1 -- 172.21.15.42:0/3311644518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca03690 msgr2=0x560b6ca0d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.290+0000 7ff57fe00640 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca03690 0x560b6ca0d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.290+0000 7ff57fe00640 1 -- 172.21.15.42:0/3311644518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b6c746620 con 0x560b6ca02ca0 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.290+0000 7ff57fe00640 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca0cd60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff578031ae0 tx=0x7ff578004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b97ee1038bd4a0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.290+0000 7ff569600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.290+0000 7ff569600640 1 -- 172.21.15.42:0/3311644518 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578003b70 con 0x560b6ca02ca0 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.290+0000 7ff569600640 1 -- 172.21.15.42:0/3311644518 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff578003d10 con 0x560b6ca02ca0 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff589767740 1 -- 172.21.15.42:0/3311644518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff578036030 con 0x560b6ca02ca0 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff569600640 1 -- 172.21.15.42:0/3311644518 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff578004b40 con 0x560b6ca02ca0 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff589767740 10 client.25694 fetch_fsmap learned FSMap version 29 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff589767740 1 -- 172.21.15.42:0/3311644518 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b6ca0b5d0 con 0x560b6ca02ca0 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff569600640 1 -- 172.21.15.42:0/3311644518 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff578036030 con 0x560b6ca02ca0 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff569600640 1 -- 172.21.15.42:0/3311644518 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff578043070 con 0x560b6ca02ca0 2024-04-25T00:25:13.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff589767740 10 client.25694 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff589767740 10 client.25694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff589767740 20 client.25694 populate_metadata read hostname 'smithi042' 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff589767740 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560b6ca186c0 0x560b6ca1aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff589767740 4 client.25694 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.291+0000 7ff589767740 1 -- 172.21.15.42:0/3311644518 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b6c972ff0 con 0x560b6ca186c0 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.292+0000 7ff57ea00640 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560b6ca186c0 0x560b6ca1aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.292+0000 7ff57ea00640 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560b6ca186c0 0x560b6ca1aab0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b1e4709d51c0af4 server_cookie=1413c9586dc0d354 in_seq=0 out_seq=0 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.292+0000 7ff569600640 10 client.25694 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff569600640 1 -- 172.21.15.42:0/3311644518 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b6c972ff0 con 0x560b6ca186c0 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff569600640 10 client.25694 handle_command_reply: tid=0 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff589767740 1 client.25694 shutdown 2024-04-25T00:25:13.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff589767740 1 -- 172.21.15.42:0/3311644518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 msgr2=0x560b6ca0cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff589767740 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca0cd60 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff578031ae0 tx=0x7ff578004040 comp rx=0 tx=0).stop 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff589767740 1 -- 172.21.15.42:0/3311644518 shutdown_connections 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff589767740 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560b6ca186c0 0x560b6ca1aab0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff589767740 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca08800 0x560b6ca142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff589767740 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca03690 0x560b6ca0d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff589767740 1 --2- 172.21.15.42:0/3311644518 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca0cd60 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff589767740 1 -- 172.21.15.42:0/3311644518 >> 172.21.15.42:0/3311644518 conn(0x560b6c9fe310 msgr2=0x560b6c9fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.293+0000 7ff569600640 0 client.25694 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.294+0000 7ff589767740 1 -- 172.21.15.42:0/3311644518 shutdown_connections 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.294+0000 7ff589767740 1 -- 172.21.15.42:0/3311644518 wait complete. 2024-04-25T00:25:13.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.294+0000 7ff589767740 20 client.25694 trim_cache size 0 max 16384 2024-04-25T00:25:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.316+0000 7ff589767740 1 Processor -- start 2024-04-25T00:25:13.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff589767740 1 -- start start 2024-04-25T00:25:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff589767740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca01b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff589767740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca03690 0x560b6ca142a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff589767740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca08800 0x560b6c9fdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff589767740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801ad580 con 0x560b6ca08800 2024-04-25T00:25:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff589767740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801aae50 con 0x560b6ca02ca0 2024-04-25T00:25:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff589767740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801a8ca0 con 0x560b6ca03690 2024-04-25T00:25:13.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca01b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca01b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56884/0 (socket says 172.21.15.42:56884) 2024-04-25T00:25:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff57fe00640 1 -- 172.21.15.42:0/2091569480 learned_addr learned my addr 172.21.15.42:0/2091569480 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.317+0000 7ff57fe00640 1 --2- 172.21.15.42:0/2091569480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca08800 0x560b6c9fdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.318+0000 7ff57ea00640 1 --2- 172.21.15.42:0/2091569480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca03690 0x560b6ca142a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.318+0000 7ff57fe00640 1 -- 172.21.15.42:0/2091569480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca03690 msgr2=0x560b6ca142a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.318+0000 7ff57fe00640 1 --2- 172.21.15.42:0/2091569480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca03690 0x560b6ca142a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.318+0000 7ff57fe00640 1 -- 172.21.15.42:0/2091569480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca08800 msgr2=0x560b6c9fdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.318+0000 7ff57fe00640 1 --2- 172.21.15.42:0/2091569480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca08800 0x560b6c9fdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.318+0000 7ff57fe00640 1 -- 172.21.15.42:0/2091569480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b6ca1ae20 con 0x560b6ca02ca0 2024-04-25T00:25:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.318+0000 7ff57fe00640 1 --2- 172.21.15.42:0/2091569480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca01b10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff578006010 tx=0x7ff57802a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74a57001e0252679 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.318+0000 7ff56be00640 1 -- 172.21.15.42:0/2091569480 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5780305c0 con 0x560b6ca02ca0 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.319+0000 7ff56be00640 1 -- 172.21.15.42:0/2091569480 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff578036070 con 0x560b6ca02ca0 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.319+0000 7ff56be00640 1 -- 172.21.15.42:0/2091569480 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff57802faf0 con 0x560b6ca02ca0 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.319+0000 7ff589767740 1 -- 172.21.15.42:0/2091569480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 msgr2=0x560b6ca01b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.319+0000 7ff589767740 1 --2- 172.21.15.42:0/2091569480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca01b10 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff578006010 tx=0x7ff57802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.320+0000 7ff589767740 1 -- 172.21.15.42:0/2091569480 shutdown_connections 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.320+0000 7ff589767740 1 --2- 172.21.15.42:0/2091569480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca08800 0x560b6c9fdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.320+0000 7ff589767740 1 --2- 172.21.15.42:0/2091569480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca03690 0x560b6ca142a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.320+0000 7ff589767740 1 --2- 172.21.15.42:0/2091569480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca02ca0 0x560b6ca01b10 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.320+0000 7ff589767740 1 -- 172.21.15.42:0/2091569480 >> 172.21.15.42:0/2091569480 conn(0x560b6c9fe310 msgr2=0x560b6ca0cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.320+0000 7ff589767740 1 -- 172.21.15.42:0/2091569480 shutdown_connections 2024-04-25T00:25:13.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.320+0000 7ff589767740 1 -- 172.21.15.42:0/2091569480 wait complete. 2024-04-25T00:25:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.320+0000 7ff589767740 1 -- start start 2024-04-25T00:25:13.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.320+0000 7ff589767740 1 Processor -- start 2024-04-25T00:25:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff589767740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca02ca0 0x560b6ca11800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff589767740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca03690 0x560b6ca11d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff589767740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca08800 0x560b6ca12d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff589767740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801a9310 con 0x560b6ca08800 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff589767740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801aa780 con 0x560b6ca03690 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff589767740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801a99e0 con 0x560b6ca02ca0 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff589767740 10 client.0 _ll_register_callbacks cb 0x7ff578009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca08800 0x560b6ca12d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff57ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca03690 0x560b6ca11d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca02ca0 0x560b6ca11800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca02ca0 0x560b6ca11800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:56382/0 (socket says 172.21.15.42:56382) 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.321+0000 7ff57fe00640 1 -- 172.21.15.42:0/2829282782 learned_addr learned my addr 172.21.15.42:0/2829282782 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.322+0000 7ff57fe00640 1 -- 172.21.15.42:0/2829282782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca03690 msgr2=0x560b6ca11d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.322+0000 7ff57fe00640 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca03690 0x560b6ca11d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.322+0000 7ff57fe00640 1 -- 172.21.15.42:0/2829282782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca08800 msgr2=0x560b6ca12d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.322+0000 7ff57fe00640 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca08800 0x560b6ca12d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.322+0000 7ff57fe00640 1 -- 172.21.15.42:0/2829282782 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b6ca13340 con 0x560b6ca02ca0 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.322+0000 7ff57ea00640 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca03690 0x560b6ca11d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.322+0000 7ff57fe00640 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca02ca0 0x560b6ca11800 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff578030960 tx=0x7ff578000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d5f6913305a6e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.322+0000 7ff569600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.323+0000 7ff569600640 1 -- 172.21.15.42:0/2829282782 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff578030ea0 con 0x560b6ca02ca0 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.323+0000 7ff569600640 1 -- 172.21.15.42:0/2829282782 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff578003e00 con 0x560b6ca02ca0 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.323+0000 7ff589767740 1 -- 172.21.15.42:0/2829282782 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff578036070 con 0x560b6ca02ca0 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.323+0000 7ff569600640 1 -- 172.21.15.42:0/2829282782 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff5780305c0 con 0x560b6ca02ca0 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.323+0000 7ff589767740 10 client.25711 fetch_fsmap learned FSMap version 29 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.323+0000 7ff589767740 1 -- 172.21.15.42:0/2829282782 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b6ca14650 con 0x560b6ca02ca0 2024-04-25T00:25:13.327 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.323+0000 7ff569600640 1 -- 172.21.15.42:0/2829282782 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff578036070 con 0x560b6ca02ca0 2024-04-25T00:25:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.323+0000 7ff569600640 1 -- 172.21.15.42:0/2829282782 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7ff578043040 con 0x560b6ca02ca0 2024-04-25T00:25:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.324+0000 7ff589767740 10 client.25711 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:25:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.324+0000 7ff589767740 10 client.25711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.324+0000 7ff589767740 20 client.25711 populate_metadata read hostname 'smithi042' 2024-04-25T00:25:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.324+0000 7ff589767740 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560b6ca7f270 0x560b6ca81690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.324+0000 7ff589767740 4 client.25711 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:25:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.324+0000 7ff589767740 1 -- 172.21.15.42:0/2829282782 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b6c96d340 con 0x560b6ca7f270 2024-04-25T00:25:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.324+0000 7ff57ea00640 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560b6ca7f270 0x560b6ca81690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:13.328 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.325+0000 7ff57ea00640 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560b6ca7f270 0x560b6ca81690 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e4140ff231fe90b server_cookie=2dd81ed8d842a80d in_seq=0 out_seq=0 2024-04-25T00:25:13.329 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.325+0000 7ff569600640 10 client.25711 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.377+0000 7ff569600640 1 -- 172.21.15.42:0/2829282782 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560b6c96d340 con 0x560b6ca7f270 2024-04-25T00:25:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.377+0000 7ff569600640 10 client.25711 handle_command_reply: tid=0 2024-04-25T00:25:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.377+0000 7ff589767740 1 client.25711 shutdown 2024-04-25T00:25:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.378+0000 7ff589767740 1 -- 172.21.15.42:0/2829282782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca02ca0 msgr2=0x560b6ca11800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.378+0000 7ff589767740 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca02ca0 0x560b6ca11800 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff578030960 tx=0x7ff578000f40 comp rx=0 tx=0).stop 2024-04-25T00:25:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.379+0000 7ff589767740 1 -- 172.21.15.42:0/2829282782 shutdown_connections 2024-04-25T00:25:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.379+0000 7ff589767740 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560b6ca7f270 0x560b6ca81690 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.379+0000 7ff589767740 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560b6ca08800 0x560b6ca12d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.379+0000 7ff589767740 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560b6ca03690 0x560b6ca11d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.379+0000 7ff589767740 1 --2- 172.21.15.42:0/2829282782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b6ca02ca0 0x560b6ca11800 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.379+0000 7ff589767740 1 -- 172.21.15.42:0/2829282782 >> 172.21.15.42:0/2829282782 conn(0x560b6c9fe310 msgr2=0x560b6c9fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.379+0000 7ff569600640 0 client.25711 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.380+0000 7ff589767740 1 -- 172.21.15.42:0/2829282782 shutdown_connections 2024-04-25T00:25:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.380+0000 7ff589767740 1 -- 172.21.15.42:0/2829282782 wait complete. 2024-04-25T00:25:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.380+0000 7ff589767740 20 client.25711 trim_cache size 0 max 16384 2024-04-25T00:25:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:25:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:25:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:25:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:25:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 -- 172.21.15.42:0/2259773622 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5640a2cb0 msgr2=0x7ff5640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5640a2cb0 0x7ff5640a5170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x560b6ca03a90 tx=0x7ff570000bc0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 -- 172.21.15.42:0/2259773622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5801039d0 msgr2=0x7ff580199bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5801039d0 0x7ff580199bd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff57000e9c0 tx=0x7ff57000ee80 comp rx=0 tx=0).stop 2024-04-25T00:25:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 -- 172.21.15.42:0/2259773622 shutdown_connections 2024-04-25T00:25:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5640a2cb0 0x7ff5640a5170 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff580105580 0x7ff5801a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff580104bc0 0x7ff58019a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 --2- 172.21.15.42:0/2259773622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5801039d0 0x7ff580199bd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 -- 172.21.15.42:0/2259773622 >> 172.21.15.42:0/2259773622 conn(0x7ff5800fef60 msgr2=0x7ff580107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 -- 172.21.15.42:0/2259773622 shutdown_connections 2024-04-25T00:25:13.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:13.382+0000 7ff586800640 1 -- 172.21.15.42:0/2259773622 wait complete. 2024-04-25T00:25:13.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:13 smithi136 ceph-mon[32923]: pgmap v582: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-04-25T00:25:13.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:13 smithi106 ceph-mon[32510]: pgmap v582: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-04-25T00:25:13.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:13 smithi042 ceph-mon[28130]: pgmap v582: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 871 KiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-04-25T00:25:14.238 INFO:tasks.workunit.client.0.smithi042.stdout: 1 368676 6.31 MB/sec execute 184 sec latency 25.158 ms 2024-04-25T00:25:15.238 INFO:tasks.workunit.client.0.smithi042.stdout: 1 369357 6.30 MB/sec execute 185 sec latency 15.527 ms 2024-04-25T00:25:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:15 smithi136 ceph-mon[32923]: pgmap v583: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-25T00:25:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:15 smithi106 ceph-mon[32510]: pgmap v583: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-25T00:25:15.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:15 smithi042 ceph-mon[28130]: pgmap v583: 97 pgs: 97 active+clean; 81 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-04-25T00:25:16.238 INFO:tasks.workunit.client.0.smithi042.stdout: 1 370403 6.30 MB/sec execute 186 sec latency 32.878 ms 2024-04-25T00:25:17.238 INFO:tasks.workunit.client.0.smithi042.stdout: 1 371815 6.30 MB/sec execute 187 sec latency 30.375 ms 2024-04-25T00:25:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:17 smithi136 ceph-mon[32923]: pgmap v584: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-04-25T00:25:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:17 smithi106 ceph-mon[32510]: pgmap v584: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-04-25T00:25:17.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:17 smithi042 ceph-mon[28130]: pgmap v584: 97 pgs: 97 active+clean; 82 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.7 MiB/s wr, 216 op/s 2024-04-25T00:25:18.238 INFO:tasks.workunit.client.0.smithi042.stdout: 1 372820 6.30 MB/sec execute 188 sec latency 49.950 ms 2024-04-25T00:25:19.238 INFO:tasks.workunit.client.0.smithi042.stdout: 1 373841 6.30 MB/sec execute 189 sec latency 22.621 ms 2024-04-25T00:25:19.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:19 smithi136 ceph-mon[32923]: pgmap v585: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-04-25T00:25:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:19 smithi106 ceph-mon[32510]: pgmap v585: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-04-25T00:25:19.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:19 smithi042 ceph-mon[28130]: pgmap v585: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 740 KiB/s rd, 1.7 MiB/s wr, 222 op/s 2024-04-25T00:25:20.238 INFO:tasks.workunit.client.0.smithi042.stdout: 1 375198 6.29 MB/sec execute 190 sec latency 26.580 ms 2024-04-25T00:25:21.239 INFO:tasks.workunit.client.0.smithi042.stdout: 1 376362 6.30 MB/sec execute 191 sec latency 14.799 ms 2024-04-25T00:25:21.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:21 smithi136 ceph-mon[32923]: pgmap v586: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-04-25T00:25:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:21 smithi106 ceph-mon[32510]: pgmap v586: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-04-25T00:25:21.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:21 smithi042 ceph-mon[28130]: pgmap v586: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 662 KiB/s rd, 1.5 MiB/s wr, 187 op/s 2024-04-25T00:25:22.239 INFO:tasks.workunit.client.0.smithi042.stdout: 1 377195 6.28 MB/sec execute 192 sec latency 25.499 ms 2024-04-25T00:25:23.239 INFO:tasks.workunit.client.0.smithi042.stdout: 1 378383 6.28 MB/sec execute 193 sec latency 21.054 ms 2024-04-25T00:25:23.679 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:23 smithi042 ceph-mon[28130]: pgmap v587: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-04-25T00:25:23.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:23 smithi136 ceph-mon[32923]: pgmap v587: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-04-25T00:25:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:23 smithi106 ceph-mon[32510]: pgmap v587: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-04-25T00:25:24.239 INFO:tasks.workunit.client.0.smithi042.stdout: 1 379814 6.30 MB/sec execute 194 sec latency 27.612 ms 2024-04-25T00:25:25.239 INFO:tasks.workunit.client.0.smithi042.stdout: 1 380446 6.28 MB/sec execute 195 sec latency 25.607 ms 2024-04-25T00:25:25.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:25 smithi136 ceph-mon[32923]: pgmap v588: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-04-25T00:25:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:25 smithi106 ceph-mon[32510]: pgmap v588: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-04-25T00:25:25.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:25 smithi042 ceph-mon[28130]: pgmap v588: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-04-25T00:25:26.239 INFO:tasks.workunit.client.0.smithi042.stdout: 1 381483 6.27 MB/sec execute 196 sec latency 24.776 ms 2024-04-25T00:25:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.544+0000 7fa457400640 1 -- 172.21.15.42:0/1265257645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa450105570 msgr2=0x7fa45010be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.544+0000 7fa457400640 1 --2- 172.21.15.42:0/1265257645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa450105570 0x7fa45010be00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa448009ca0 tx=0x7fa44802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.544+0000 7fa457400640 1 -- 172.21.15.42:0/1265257645 shutdown_connections 2024-04-25T00:25:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.544+0000 7fa457400640 1 --2- 172.21.15.42:0/1265257645 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa450105570 0x7fa45010be00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.544+0000 7fa457400640 1 --2- 172.21.15.42:0/1265257645 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa450104bb0 0x7fa450105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.544+0000 7fa457400640 1 --2- 172.21.15.42:0/1265257645 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa4501039c0 0x7fa450103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.544+0000 7fa457400640 1 -- 172.21.15.42:0/1265257645 >> 172.21.15.42:0/1265257645 conn(0x7fa4500fef30 msgr2=0x7fa450101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.544+0000 7fa457400640 1 -- 172.21.15.42:0/1265257645 shutdown_connections 2024-04-25T00:25:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.544+0000 7fa457400640 1 -- 172.21.15.42:0/1265257645 wait complete. 2024-04-25T00:25:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.545+0000 7fa457400640 1 Processor -- start 2024-04-25T00:25:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.545+0000 7fa457400640 1 -- start start 2024-04-25T00:25:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa457400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4501039c0 0x7fa450199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa457400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa450104bb0 0x7fa45019a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa457400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa450105570 0x7fa4501a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa457400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa45010f670 con 0x7fa4501039c0 2024-04-25T00:25:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa457400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa45010f4f0 con 0x7fa450105570 2024-04-25T00:25:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa457400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa45010f7f0 con 0x7fa450104bb0 2024-04-25T00:25:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa454a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4501039c0 0x7fa450199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa455400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa450105570 0x7fa4501a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa455400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa450105570 0x7fa4501a1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:55004/0 (socket says 172.21.15.42:55004) 2024-04-25T00:25:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa455400640 1 -- 172.21.15.42:0/425836190 learned_addr learned my addr 172.21.15.42:0/425836190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa454a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4501039c0 0x7fa450199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58326/0 (socket says 172.21.15.42:58326) 2024-04-25T00:25:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.546+0000 7fa44fe00640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa450104bb0 0x7fa45019a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.547+0000 7fa455400640 1 -- 172.21.15.42:0/425836190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa450104bb0 msgr2=0x7fa45019a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.547+0000 7fa455400640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa450104bb0 0x7fa45019a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.547+0000 7fa455400640 1 -- 172.21.15.42:0/425836190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4501039c0 msgr2=0x7fa450199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.547+0000 7fa455400640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4501039c0 0x7fa450199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.547+0000 7fa455400640 1 -- 172.21.15.42:0/425836190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4501a17b0 con 0x7fa450105570 2024-04-25T00:25:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.547+0000 7fa455400640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa450105570 0x7fa4501a1140 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa44802f540 tx=0x7fa448031750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.548+0000 7fa44d600640 1 -- 172.21.15.42:0/425836190 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448003b70 con 0x7fa450105570 2024-04-25T00:25:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.548+0000 7fa457400640 1 -- 172.21.15.42:0/425836190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4501a1a40 con 0x7fa450105570 2024-04-25T00:25:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.548+0000 7fa44d600640 1 -- 172.21.15.42:0/425836190 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa448003d10 con 0x7fa450105570 2024-04-25T00:25:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.548+0000 7fa44d600640 1 -- 172.21.15.42:0/425836190 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44802fb50 con 0x7fa450105570 2024-04-25T00:25:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.548+0000 7fa457400640 1 -- 172.21.15.42:0/425836190 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4501a1fe0 con 0x7fa450105570 2024-04-25T00:25:26.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.551+0000 7fa45a1e7740 1 Processor -- start 2024-04-25T00:25:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa44d600640 1 -- 172.21.15.42:0/425836190 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa44802fcf0 con 0x7fa450105570 2024-04-25T00:25:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa45a1e7740 1 -- start start 2024-04-25T00:25:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc2ca0 0x5588a5fc30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fc8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc8800 0x5588a5fcac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5588a5f25ab0 con 0x5588a5fc8800 2024-04-25T00:25:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5588a5ece640 con 0x5588a5fc2ca0 2024-04-25T00:25:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5588a5f21a70 con 0x5588a5fc3690 2024-04-25T00:25:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa454a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fc8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa454a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fc8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:37806/0 (socket says 172.21.15.42:37806) 2024-04-25T00:25:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.552+0000 7fa454a00640 1 -- 172.21.15.42:0/4151747107 learned_addr learned my addr 172.21.15.42:0/4151747107 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.553+0000 7fa44fe00640 1 --2- 172.21.15.42:0/4151747107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc2ca0 0x5588a5fc30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.553+0000 7fa44fe00640 1 --2- 172.21.15.42:0/4151747107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc8800 0x5588a5fcac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.553+0000 7fa454a00640 1 -- 172.21.15.42:0/4151747107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc2ca0 msgr2=0x5588a5fc30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.553+0000 7fa454a00640 1 --2- 172.21.15.42:0/4151747107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc2ca0 0x5588a5fc30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.553+0000 7fa454a00640 1 -- 172.21.15.42:0/4151747107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc8800 msgr2=0x5588a5fcac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.553+0000 7fa454a00640 1 --2- 172.21.15.42:0/4151747107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc8800 0x5588a5fcac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.553+0000 7fa454a00640 1 -- 172.21.15.42:0/4151747107 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a5c3a210 con 0x5588a5fc3690 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.553+0000 7fa454a00640 1 --2- 172.21.15.42:0/4151747107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fc8190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa444009c80 tx=0x7fa444011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b836c74c47d48bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.554+0000 7fa441c00640 1 -- 172.21.15.42:0/4151747107 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44401e070 con 0x5588a5fc3690 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.554+0000 7fa441c00640 1 -- 172.21.15.42:0/4151747107 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa444009e30 con 0x5588a5fc3690 2024-04-25T00:25:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.554+0000 7fa441c00640 1 -- 172.21.15.42:0/4151747107 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44400ea80 con 0x5588a5fc3690 2024-04-25T00:25:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa45a1e7740 1 -- 172.21.15.42:0/4151747107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 msgr2=0x5588a5fc8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/4151747107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fc8190 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa444009c80 tx=0x7fa444011810 comp rx=0 tx=0).stop 2024-04-25T00:25:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa45a1e7740 1 -- 172.21.15.42:0/4151747107 shutdown_connections 2024-04-25T00:25:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/4151747107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc8800 0x5588a5fcac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/4151747107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fc8190 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/4151747107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc2ca0 0x5588a5fc30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa45a1e7740 1 -- 172.21.15.42:0/4151747107 >> 172.21.15.42:0/4151747107 conn(0x5588a5fbe310 msgr2=0x5588a5fbe710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa44d600640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4380a2c30 0x7fa4380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa45a1e7740 1 -- 172.21.15.42:0/4151747107 shutdown_connections 2024-04-25T00:25:26.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa44d600640 1 -- 172.21.15.42:0/425836190 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa4480f0450 con 0x7fa450105570 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa45a1e7740 1 -- 172.21.15.42:0/4151747107 wait complete. 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.555+0000 7fa454a00640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4380a2c30 0x7fa4380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.556+0000 7fa45a1e7740 1 -- start start 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.556+0000 7fa45a1e7740 1 Processor -- start 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.556+0000 7fa454a00640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4380a2c30 0x7fa4380a50f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa444011d10 tx=0x7fa444009770 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.556+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 0x5588a5fcced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fcd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fd4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5588a5fcb7b0 con 0x5588a5fc2ca0 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5588a5ee1990 con 0x5588a5fc8800 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5588a5fcb930 con 0x5588a5fc3690 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa45a1e7740 10 client.0 _ll_register_callbacks cb 0x5588a5ee3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:26.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa44fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fcd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa455400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fd4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa44fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fcd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:37822/0 (socket says 172.21.15.42:37822) 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa44fe00640 1 -- 172.21.15.42:0/3032653902 learned_addr learned my addr 172.21.15.42:0/3032653902 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa454a00640 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 0x5588a5fcced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa44fe00640 1 -- 172.21.15.42:0/3032653902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 msgr2=0x5588a5fd4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa44fe00640 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fd4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa44fe00640 1 -- 172.21.15.42:0/3032653902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 msgr2=0x5588a5fcced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa44fe00640 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 0x5588a5fcced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.557+0000 7fa44fe00640 1 -- 172.21.15.42:0/3032653902 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a5d06620 con 0x5588a5fc3690 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.558+0000 7fa44fe00640 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fcd410 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa43c010720 tx=0x7fa43c010be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa4d1392ab4e5bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.558+0000 7fa436000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.558+0000 7fa436000640 1 -- 172.21.15.42:0/3032653902 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c021060 con 0x5588a5fc3690 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.558+0000 7fa436000640 1 -- 172.21.15.42:0/3032653902 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa43c004460 con 0x5588a5fc3690 2024-04-25T00:25:26.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.558+0000 7fa45a1e7740 1 -- 172.21.15.42:0/3032653902 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa444009e30 con 0x5588a5fc3690 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.558+0000 7fa436000640 1 -- 172.21.15.42:0/3032653902 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa43c00f940 con 0x5588a5fc3690 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.558+0000 7fa45a1e7740 10 client.25723 fetch_fsmap learned FSMap version 29 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.558+0000 7fa45a1e7740 1 -- 172.21.15.42:0/3032653902 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5588a5fd7550 con 0x5588a5fc3690 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.558+0000 7fa436000640 1 -- 172.21.15.42:0/3032653902 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa444009e30 con 0x5588a5fc3690 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.559+0000 7fa436000640 1 -- 172.21.15.42:0/3032653902 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa43c01c400 con 0x5588a5fc3690 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.559+0000 7fa45a1e7740 10 client.25723 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.559+0000 7fa45a1e7740 10 client.25723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.559+0000 7fa45a1e7740 20 client.25723 populate_metadata read hostname 'smithi042' 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.559+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5588a5fd8c90 0x5588a5fdb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.559+0000 7fa45a1e7740 4 client.25723 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.559+0000 7fa45a1e7740 1 -- 172.21.15.42:0/3032653902 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588a5f32ff0 con 0x5588a5fd8c90 2024-04-25T00:25:26.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.559+0000 7fa454a00640 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5588a5fd8c90 0x5588a5fdb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.560+0000 7fa454a00640 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5588a5fd8c90 0x5588a5fdb0e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a15f5c96720ddcf server_cookie=41667ecfda862029 in_seq=0 out_seq=0 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.560+0000 7fa436000640 10 client.25723 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa436000640 1 -- 172.21.15.42:0/3032653902 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5588a5f32ff0 con 0x5588a5fd8c90 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa436000640 10 client.25723 handle_command_reply: tid=0 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 client.25723 shutdown 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 -- 172.21.15.42:0/3032653902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 msgr2=0x5588a5fcd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fcd410 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa43c010720 tx=0x7fa43c010be0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 -- 172.21.15.42:0/3032653902 shutdown_connections 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5588a5fd8c90 0x5588a5fdb0e0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fd4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fcd410 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa436000640 0 client.25723 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/3032653902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 0x5588a5fcced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 -- 172.21.15.42:0/3032653902 >> 172.21.15.42:0/3032653902 conn(0x5588a5fbe310 msgr2=0x5588a5fbd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 -- 172.21.15.42:0/3032653902 shutdown_connections 2024-04-25T00:25:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.561+0000 7fa45a1e7740 1 -- 172.21.15.42:0/3032653902 wait complete. 2024-04-25T00:25:26.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.562+0000 7fa45a1e7740 20 client.25723 trim_cache size 0 max 16384 2024-04-25T00:25:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.580+0000 7fa45a1e7740 1 Processor -- start 2024-04-25T00:25:26.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.580+0000 7fa45a1e7740 1 -- start start 2024-04-25T00:25:26.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 0x5588a5fcc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fd4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fcc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501ad510 con 0x5588a5fc2ca0 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501aade0 con 0x5588a5fc8800 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501a8c30 con 0x5588a5fc3690 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa455400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fcc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa455400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fcc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:55036/0 (socket says 172.21.15.42:55036) 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa455400640 1 -- 172.21.15.42:0/1368053528 learned_addr learned my addr 172.21.15.42:0/1368053528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa454a00640 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 0x5588a5fcc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa44fe00640 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fd4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa455400640 1 -- 172.21.15.42:0/1368053528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 msgr2=0x5588a5fd4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa455400640 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fd4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.581+0000 7fa455400640 1 -- 172.21.15.42:0/1368053528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 msgr2=0x5588a5fcc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:26.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.582+0000 7fa455400640 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 0x5588a5fcc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.582+0000 7fa455400640 1 -- 172.21.15.42:0/1368053528 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588a5fd7370 con 0x5588a5fc8800 2024-04-25T00:25:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.582+0000 7fa44fe00640 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fd4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.582+0000 7fa454a00640 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 0x5588a5fcc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.582+0000 7fa455400640 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fcc720 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa448005e30 tx=0x7fa4480eddf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75182b9b374eedce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.582+0000 7fa441200640 1 -- 172.21.15.42:0/1368053528 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4480da070 con 0x5588a5fc8800 2024-04-25T00:25:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.582+0000 7fa441200640 1 -- 172.21.15.42:0/1368053528 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa448036040 con 0x5588a5fc8800 2024-04-25T00:25:26.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.582+0000 7fa441200640 1 -- 172.21.15.42:0/1368053528 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa448004640 con 0x5588a5fc8800 2024-04-25T00:25:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.583+0000 7fa45a1e7740 1 -- 172.21.15.42:0/1368053528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 msgr2=0x5588a5fcc720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.583+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fcc720 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa448005e30 tx=0x7fa4480eddf0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.583+0000 7fa45a1e7740 1 -- 172.21.15.42:0/1368053528 shutdown_connections 2024-04-25T00:25:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.583+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fcc720 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.583+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc3690 0x5588a5fd4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.583+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/1368053528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc2ca0 0x5588a5fcc1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.583+0000 7fa45a1e7740 1 -- 172.21.15.42:0/1368053528 >> 172.21.15.42:0/1368053528 conn(0x5588a5fbe310 msgr2=0x5588a5fb8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.583+0000 7fa45a1e7740 1 -- 172.21.15.42:0/1368053528 shutdown_connections 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.583+0000 7fa45a1e7740 1 -- 172.21.15.42:0/1368053528 wait complete. 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.584+0000 7fa45a1e7740 1 -- start start 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.584+0000 7fa45a1e7740 1 Processor -- start 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.584+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc2ca0 0x5588a5fd20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.584+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc3690 0x5588a5fd2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.584+0000 7fa45a1e7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fd3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.584+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501a92a0 con 0x5588a5fc3690 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.584+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501aa710 con 0x5588a5fc8800 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.584+0000 7fa45a1e7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa4501a9970 con 0x5588a5fc2ca0 2024-04-25T00:25:26.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.584+0000 7fa45a1e7740 10 client.0 _ll_register_callbacks cb 0x7fa43c009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa454a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc2ca0 0x5588a5fd20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa455400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fd3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa454a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc2ca0 0x5588a5fd20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:37834/0 (socket says 172.21.15.42:37834) 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa454a00640 1 -- 172.21.15.42:0/2777300377 learned_addr learned my addr 172.21.15.42:0/2777300377 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa44fe00640 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc3690 0x5588a5fd2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa454a00640 1 -- 172.21.15.42:0/2777300377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 msgr2=0x5588a5fd3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa454a00640 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fd3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa454a00640 1 -- 172.21.15.42:0/2777300377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc3690 msgr2=0x5588a5fd2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa454a00640 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc3690 0x5588a5fd2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa454a00640 1 -- 172.21.15.42:0/2777300377 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588a5fd3b30 con 0x5588a5fc2ca0 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa455400640 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fd3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa454a00640 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc2ca0 0x5588a5fd20d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa444005490 tx=0x7fa444011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e7a3b7d9bc76fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.585+0000 7fa436000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:25:26.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.586+0000 7fa436000640 1 -- 172.21.15.42:0/2777300377 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44402c040 con 0x5588a5fc2ca0 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.586+0000 7fa436000640 1 -- 172.21.15.42:0/2777300377 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa44400e510 con 0x5588a5fc2ca0 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.586+0000 7fa45a1e7740 1 -- 172.21.15.42:0/2777300377 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa448036040 con 0x5588a5fc2ca0 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.586+0000 7fa436000640 1 -- 172.21.15.42:0/2777300377 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa44401e050 con 0x5588a5fc2ca0 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.586+0000 7fa45a1e7740 10 client.25729 fetch_fsmap learned FSMap version 29 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.586+0000 7fa45a1e7740 1 -- 172.21.15.42:0/2777300377 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5588a603f750 con 0x5588a5fc2ca0 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.586+0000 7fa436000640 1 -- 172.21.15.42:0/2777300377 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa448036040 con 0x5588a5fc2ca0 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.587+0000 7fa436000640 1 -- 172.21.15.42:0/2777300377 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fa4440049c0 con 0x5588a5fc2ca0 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.587+0000 7fa45a1e7740 10 client.25729 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.587+0000 7fa45a1e7740 10 client.25729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.587+0000 7fa45a1e7740 20 client.25729 populate_metadata read hostname 'smithi042' 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.587+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5588a603fff0 0x5588a6042410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.587+0000 7fa45a1e7740 4 client.25729 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.587+0000 7fa45a1e7740 1 -- 172.21.15.42:0/2777300377 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5588a5f2d340 con 0x5588a603fff0 2024-04-25T00:25:26.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.587+0000 7fa44fe00640 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5588a603fff0 0x5588a6042410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.588+0000 7fa44fe00640 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5588a603fff0 0x5588a6042410 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb67187c6d941bf server_cookie=f643798536b43fd5 in_seq=0 out_seq=0 2024-04-25T00:25:26.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.588+0000 7fa436000640 10 client.25729 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:26.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa436000640 1 -- 172.21.15.42:0/2777300377 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5588a5f2d340 con 0x5588a603fff0 2024-04-25T00:25:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa436000640 10 client.25729 handle_command_reply: tid=0 2024-04-25T00:25:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 client.25729 shutdown 2024-04-25T00:25:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 -- 172.21.15.42:0/2777300377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc2ca0 msgr2=0x5588a5fd20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc2ca0 0x5588a5fd20d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa444005490 tx=0x7fa444011810 comp rx=0 tx=0).stop 2024-04-25T00:25:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 -- 172.21.15.42:0/2777300377 shutdown_connections 2024-04-25T00:25:26.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5588a603fff0 0x5588a6042410 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5588a5fc8800 0x5588a5fd3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5588a5fc3690 0x5588a5fd2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 --2- 172.21.15.42:0/2777300377 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5588a5fc2ca0 0x5588a5fd20d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa436000640 0 client.25729 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 -- 172.21.15.42:0/2777300377 >> 172.21.15.42:0/2777300377 conn(0x5588a5fbe310 msgr2=0x5588a5fcc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 -- 172.21.15.42:0/2777300377 shutdown_connections 2024-04-25T00:25:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 1 -- 172.21.15.42:0/2777300377 wait complete. 2024-04-25T00:25:26.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.637+0000 7fa45a1e7740 20 client.25729 trim_cache size 0 max 16384 2024-04-25T00:25:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:25:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:25:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:25:26.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:25:26.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.638+0000 7fa457400640 1 -- 172.21.15.42:0/425836190 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4380a2c30 msgr2=0x7fa4380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.638+0000 7fa457400640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4380a2c30 0x7fa4380a50f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa444011d10 tx=0x7fa444009770 comp rx=0 tx=0).stop 2024-04-25T00:25:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.638+0000 7fa457400640 1 -- 172.21.15.42:0/425836190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa450105570 msgr2=0x7fa4501a1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.638+0000 7fa457400640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa450105570 0x7fa4501a1140 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa44802f540 tx=0x7fa448031750 comp rx=0 tx=0).stop 2024-04-25T00:25:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.638+0000 7fa457400640 1 -- 172.21.15.42:0/425836190 shutdown_connections 2024-04-25T00:25:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.638+0000 7fa457400640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa4380a2c30 0x7fa4380a50f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.638+0000 7fa457400640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa450105570 0x7fa4501a1140 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.638+0000 7fa457400640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa450104bb0 0x7fa45019a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.638+0000 7fa457400640 1 --2- 172.21.15.42:0/425836190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa4501039c0 0x7fa450199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.638+0000 7fa457400640 1 -- 172.21.15.42:0/425836190 >> 172.21.15.42:0/425836190 conn(0x7fa4500fef30 msgr2=0x7fa450107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.639+0000 7fa457400640 1 -- 172.21.15.42:0/425836190 shutdown_connections 2024-04-25T00:25:26.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:26.639+0000 7fa457400640 1 -- 172.21.15.42:0/425836190 wait complete. 2024-04-25T00:25:27.239 INFO:tasks.workunit.client.0.smithi042.stdout: 1 383215 6.30 MB/sec execute 197 sec latency 26.009 ms 2024-04-25T00:25:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:27 smithi136 ceph-mon[32923]: pgmap v589: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-04-25T00:25:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:27 smithi106 ceph-mon[32510]: pgmap v589: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-04-25T00:25:27.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:27 smithi042 ceph-mon[28130]: pgmap v589: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 1.7 MiB/s wr, 200 op/s 2024-04-25T00:25:28.239 INFO:tasks.workunit.client.0.smithi042.stdout: 1 383783 6.28 MB/sec execute 198 sec latency 19.176 ms 2024-04-25T00:25:29.239 INFO:tasks.workunit.client.0.smithi042.stdout: 1 384930 6.27 MB/sec execute 199 sec latency 40.653 ms 2024-04-25T00:25:29.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:29 smithi136 ceph-mon[32923]: pgmap v590: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-04-25T00:25:29.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:29 smithi106 ceph-mon[32510]: pgmap v590: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-04-25T00:25:29.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:29 smithi042 ceph-mon[28130]: pgmap v590: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 684 KiB/s rd, 1.6 MiB/s wr, 179 op/s 2024-04-25T00:25:30.240 INFO:tasks.workunit.client.0.smithi042.stdout: 1 386498 6.28 MB/sec execute 200 sec latency 19.435 ms 2024-04-25T00:25:31.240 INFO:tasks.workunit.client.0.smithi042.stdout: 1 387135 6.28 MB/sec execute 201 sec latency 18.829 ms 2024-04-25T00:25:31.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:31 smithi136 ceph-mon[32923]: pgmap v591: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-04-25T00:25:31.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:31 smithi106 ceph-mon[32510]: pgmap v591: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-04-25T00:25:31.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:31 smithi042 ceph-mon[28130]: pgmap v591: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 645 KiB/s rd, 1.5 MiB/s wr, 147 op/s 2024-04-25T00:25:32.240 INFO:tasks.workunit.client.0.smithi042.stdout: 1 388101 6.27 MB/sec execute 202 sec latency 31.815 ms 2024-04-25T00:25:33.240 INFO:tasks.workunit.client.0.smithi042.stdout: 1 389551 6.27 MB/sec execute 203 sec latency 22.573 ms 2024-04-25T00:25:33.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:33 smithi136 ceph-mon[32923]: pgmap v592: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-04-25T00:25:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:33 smithi106 ceph-mon[32510]: pgmap v592: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-04-25T00:25:33.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:33 smithi042 ceph-mon[28130]: pgmap v592: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 926 KiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-04-25T00:25:34.240 INFO:tasks.workunit.client.0.smithi042.stdout: 1 390636 6.28 MB/sec execute 204 sec latency 29.151 ms 2024-04-25T00:25:35.240 INFO:tasks.workunit.client.0.smithi042.stdout: 1 391644 6.27 MB/sec execute 205 sec latency 46.219 ms 2024-04-25T00:25:35.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:35 smithi136 ceph-mon[32923]: pgmap v593: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-04-25T00:25:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:35 smithi106 ceph-mon[32510]: pgmap v593: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-04-25T00:25:35.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:35 smithi042 ceph-mon[28130]: pgmap v593: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 677 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-04-25T00:25:36.240 INFO:tasks.workunit.client.0.smithi042.stdout: 1 393413 6.28 MB/sec execute 206 sec latency 18.894 ms 2024-04-25T00:25:37.240 INFO:tasks.workunit.client.0.smithi042.stdout: 1 394271 6.28 MB/sec execute 207 sec latency 16.863 ms 2024-04-25T00:25:37.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:37 smithi136 ceph-mon[32923]: pgmap v594: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-04-25T00:25:37.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:37 smithi106 ceph-mon[32510]: pgmap v594: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-04-25T00:25:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:37 smithi042 ceph-mon[28130]: pgmap v594: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 1.7 MiB/s wr, 207 op/s 2024-04-25T00:25:38.240 INFO:tasks.workunit.client.0.smithi042.stdout: 1 395459 6.27 MB/sec execute 208 sec latency 29.800 ms 2024-04-25T00:25:39.240 INFO:tasks.workunit.client.0.smithi042.stdout: 1 397384 6.30 MB/sec execute 209 sec latency 20.594 ms 2024-04-25T00:25:39.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:39 smithi136 ceph-mon[32923]: pgmap v595: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-04-25T00:25:39.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.793+0000 7f5ff1400640 1 -- 172.21.15.42:0/2546050104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fec0741a0 msgr2=0x7f5fec074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.793+0000 7f5ff1400640 1 --2- 172.21.15.42:0/2546050104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fec0741a0 0x7f5fec074620 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009900 tx=0x7f5fd402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.793+0000 7f5ff1400640 1 -- 172.21.15.42:0/2546050104 shutdown_connections 2024-04-25T00:25:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.793+0000 7f5ff1400640 1 --2- 172.21.15.42:0/2546050104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec109490 0x7f5fec10b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.793+0000 7f5ff1400640 1 --2- 172.21.15.42:0/2546050104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fec0741a0 0x7f5fec074620 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.793+0000 7f5ff1400640 1 --2- 172.21.15.42:0/2546050104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fec075780 0x7f5fec073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.793+0000 7f5ff1400640 1 -- 172.21.15.42:0/2546050104 >> 172.21.15.42:0/2546050104 conn(0x7f5fec0fcb40 msgr2=0x7f5fec0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.793+0000 7f5ff1400640 1 -- 172.21.15.42:0/2546050104 shutdown_connections 2024-04-25T00:25:39.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.793+0000 7f5ff1400640 1 -- 172.21.15.42:0/2546050104 wait complete. 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.794+0000 7f5ff1400640 1 Processor -- start 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.794+0000 7f5ff1400640 1 -- start start 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5ff1400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec0741a0 0x7f5fec199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5ff1400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fec075780 0x7f5fec19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5ff1400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fec109490 0x7f5fec1a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5ff1400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec10f480 con 0x7f5fec075780 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5ff1400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec10f300 con 0x7f5fec109490 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5ff1400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec10f600 con 0x7f5fec0741a0 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5fe3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fec075780 0x7f5fec19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5febe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec0741a0 0x7f5fec199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5febe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec0741a0 0x7f5fec199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45968/0 (socket says 172.21.15.42:45968) 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5febe00640 1 -- 172.21.15.42:0/3487483221 learned_addr learned my addr 172.21.15.42:0/3487483221 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:39.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fec109490 0x7f5fec1a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5febe00640 1 -- 172.21.15.42:0/3487483221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fec109490 msgr2=0x7f5fec1a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5febe00640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fec109490 0x7f5fec1a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5febe00640 1 -- 172.21.15.42:0/3487483221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fec075780 msgr2=0x7f5fec19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5febe00640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fec075780 0x7f5fec19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5febe00640 1 -- 172.21.15.42:0/3487483221 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fec1a1720 con 0x7f5fec0741a0 2024-04-25T00:25:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.795+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fec109490 0x7f5fec1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:39.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.796+0000 7f5febe00640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec0741a0 0x7f5fec199b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e9f0 tx=0x7f5fdc00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.796+0000 7f5fe9600640 1 -- 172.21.15.42:0/3487483221 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc00cd80 con 0x7f5fec0741a0 2024-04-25T00:25:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.796+0000 7f5fe9600640 1 -- 172.21.15.42:0/3487483221 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fdc010910 con 0x7f5fec0741a0 2024-04-25T00:25:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.796+0000 7f5ff1400640 1 -- 172.21.15.42:0/3487483221 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fec1a1a10 con 0x7f5fec0741a0 2024-04-25T00:25:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.796+0000 7f5fe9600640 1 -- 172.21.15.42:0/3487483221 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc018990 con 0x7f5fec0741a0 2024-04-25T00:25:39.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.796+0000 7f5ff1400640 1 -- 172.21.15.42:0/3487483221 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fec1a1f80 con 0x7f5fec0741a0 2024-04-25T00:25:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.799+0000 7f5fe9600640 1 -- 172.21.15.42:0/3487483221 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5fdc010ac0 con 0x7f5fec0741a0 2024-04-25T00:25:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.799+0000 7f5ff5769740 1 Processor -- start 2024-04-25T00:25:39.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.800+0000 7f5ff5769740 1 -- start start 2024-04-25T00:25:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.800+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 0x55a235397270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.800+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235397840 0x55a23539c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.800+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 0x55a23539ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.800+0000 7f5ff5769740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a2352f9ab0 con 0x55a235397840 2024-04-25T00:25:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.800+0000 7f5ff5769740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a2352a2640 con 0x55a235396e50 2024-04-25T00:25:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.800+0000 7f5ff5769740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a2352f5a70 con 0x55a23539c9b0 2024-04-25T00:25:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.800+0000 7f5ff0a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 0x55a235397270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.800+0000 7f5ff0a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 0x55a235397270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:54682/0 (socket says 172.21.15.42:54682) 2024-04-25T00:25:39.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.800+0000 7f5ff0a00640 1 -- 172.21.15.42:0/3315721265 learned_addr learned my addr 172.21.15.42:0/3315721265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.801+0000 7f5fe3e00640 1 --2- 172.21.15.42:0/3315721265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 0x55a23539ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.801+0000 7f5febe00640 1 --2- 172.21.15.42:0/3315721265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235397840 0x55a23539c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.801+0000 7f5ff0a00640 1 -- 172.21.15.42:0/3315721265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 msgr2=0x55a23539ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.801+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/3315721265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 0x55a23539ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.801+0000 7f5ff0a00640 1 -- 172.21.15.42:0/3315721265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235397840 msgr2=0x55a23539c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.801+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/3315721265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235397840 0x55a23539c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.801+0000 7f5ff0a00640 1 -- 172.21.15.42:0/3315721265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a23500e210 con 0x55a235396e50 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.801+0000 7f5febe00640 1 --2- 172.21.15.42:0/3315721265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235397840 0x55a23539c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.801+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/3315721265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 0x55a235397270 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009f90 tx=0x7f5fe400f7f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4d08b8f5bb6412d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.801+0000 7f5fe9600640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5fd00a2c00 0x7f5fd00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.802+0000 7f5fe1200640 1 -- 172.21.15.42:0/3315721265 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4003b70 con 0x55a235396e50 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.802+0000 7f5fe1200640 1 -- 172.21.15.42:0/3315721265 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe4003d10 con 0x55a235396e50 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.802+0000 7f5fe9600640 1 -- 172.21.15.42:0/3487483221 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f5fdc0cf390 con 0x7f5fec0741a0 2024-04-25T00:25:39.804 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.802+0000 7f5febe00640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5fd00a2c00 0x7f5fd00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.802+0000 7f5fe1200640 1 -- 172.21.15.42:0/3315721265 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4011a30 con 0x55a235396e50 2024-04-25T00:25:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5ff5769740 1 -- 172.21.15.42:0/3315721265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 msgr2=0x55a235397270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5febe00640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5fd00a2c00 0x7f5fd00a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc09bb50 tx=0x7f5fdc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5ff5769740 1 --2- 172.21.15.42:0/3315721265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 0x55a235397270 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5fe4009f90 tx=0x7f5fe400f7f0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5ff5769740 1 -- 172.21.15.42:0/3315721265 shutdown_connections 2024-04-25T00:25:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5ff5769740 1 --2- 172.21.15.42:0/3315721265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 0x55a23539ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5ff5769740 1 --2- 172.21.15.42:0/3315721265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235397840 0x55a23539c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5ff5769740 1 --2- 172.21.15.42:0/3315721265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 0x55a235397270 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.808 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5ff5769740 1 -- 172.21.15.42:0/3315721265 >> 172.21.15.42:0/3315721265 conn(0x55a2353924c0 msgr2=0x55a2353928c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5ff5769740 1 -- 172.21.15.42:0/3315721265 shutdown_connections 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5ff5769740 1 -- 172.21.15.42:0/3315721265 wait complete. 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.803+0000 7f5ff5769740 1 -- start start 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5ff5769740 1 Processor -- start 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235396e50 0x55a2353a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a23539c9b0 0x55a2353a8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5ff5769740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a23539f960 con 0x55a235396e50 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5ff5769740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a2352b5990 con 0x55a23539c9b0 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5ff5769740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a23539fae0 con 0x55a235397840 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5ff5769740 10 client.0 _ll_register_callbacks cb 0x55a2352b7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5ff0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235396e50 0x55a2353a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5fe3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.809 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5fe3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45998/0 (socket says 172.21.15.42:45998) 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5fe3e00640 1 -- 172.21.15.42:0/3087200179 learned_addr learned my addr 172.21.15.42:0/3087200179 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.804+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a23539c9b0 0x55a2353a8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.805+0000 7f5fe3e00640 1 -- 172.21.15.42:0/3087200179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a23539c9b0 msgr2=0x55a2353a8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.805+0000 7f5fe3e00640 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a23539c9b0 0x55a2353a8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.805+0000 7f5fe3e00640 1 -- 172.21.15.42:0/3087200179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235396e50 msgr2=0x55a2353a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.805+0000 7f5fe3e00640 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235396e50 0x55a2353a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.805+0000 7f5fe3e00640 1 -- 172.21.15.42:0/3087200179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2350da620 con 0x55a235397840 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.805+0000 7f5fe3e00640 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a15e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009900 tx=0x7f5fd40043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8095d819d91a99c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.805+0000 7f5fd9600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.805+0000 7f5fd9600640 1 -- 172.21.15.42:0/3087200179 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4035040 con 0x55a235397840 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.805+0000 7f5fd9600640 1 -- 172.21.15.42:0/3087200179 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f5fd4005450 con 0x55a235397840 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.805+0000 7f5ff5769740 1 -- 172.21.15.42:0/3087200179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fe4003d10 con 0x55a235397840 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5fd9600640 1 -- 172.21.15.42:0/3087200179 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5fd4005650 con 0x55a235397840 2024-04-25T00:25:39.810 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5ff5769740 10 client.25741 fetch_fsmap learned FSMap version 29 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5ff5769740 1 -- 172.21.15.42:0/3087200179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a2353ab5e0 con 0x55a235397840 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5fd9600640 1 -- 172.21.15.42:0/3087200179 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fe4003d10 con 0x55a235397840 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5fd9600640 1 -- 172.21.15.42:0/3087200179 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f5fd4031c20 con 0x55a235397840 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5ff5769740 10 client.25741 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5ff5769740 10 client.25741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5ff5769740 20 client.25741 populate_metadata read hostname 'smithi042' 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5ff5769740 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a2353accb0 0x55a2353af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5ff5769740 4 client.25741 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.806+0000 7f5ff5769740 1 -- 172.21.15.42:0/3087200179 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a235306ff0 con 0x55a2353accb0 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.807+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a2353accb0 0x55a2353af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.807+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a2353accb0 0x55a2353af0c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd69624c6dc70337 server_cookie=114c670018cb1965 in_seq=0 out_seq=0 2024-04-25T00:25:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.807+0000 7f5fd9600640 10 client.25741 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5fd9600640 1 -- 172.21.15.42:0/3087200179 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a235306ff0 con 0x55a2353accb0 2024-04-25T00:25:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5fd9600640 10 client.25741 handle_command_reply: tid=0 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 client.25741 shutdown 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 -- 172.21.15.42:0/3087200179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 msgr2=0x55a2353a15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a15e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4009900 tx=0x7f5fd40043b0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 -- 172.21.15.42:0/3087200179 shutdown_connections 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a2353accb0 0x55a2353af0c0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a23539c9b0 0x55a2353a8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a15e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 --2- 172.21.15.42:0/3087200179 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235396e50 0x55a2353a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 -- 172.21.15.42:0/3087200179 >> 172.21.15.42:0/3087200179 conn(0x55a2353924c0 msgr2=0x55a235391620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5fd9600640 0 client.25741 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 -- 172.21.15.42:0/3087200179 shutdown_connections 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 1 -- 172.21.15.42:0/3087200179 wait complete. 2024-04-25T00:25:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.808+0000 7f5ff5769740 20 client.25741 trim_cache size 0 max 16384 2024-04-25T00:25:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.825+0000 7f5ff5769740 1 Processor -- start 2024-04-25T00:25:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.825+0000 7f5ff5769740 1 -- start start 2024-04-25T00:25:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.825+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 0x55a23538d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.825+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.825+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a23539c9b0 0x55a2353954b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.825+0000 7f5ff5769740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1ad440 con 0x55a23539c9b0 2024-04-25T00:25:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.825+0000 7f5ff5769740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1aad10 con 0x55a235396e50 2024-04-25T00:25:39.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.825+0000 7f5ff5769740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1a8b60 con 0x55a235397840 2024-04-25T00:25:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5fe3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5fe3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:46000/0 (socket says 172.21.15.42:46000) 2024-04-25T00:25:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5fe3e00640 1 -- 172.21.15.42:0/1860999434 learned_addr learned my addr 172.21.15.42:0/1860999434 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/1860999434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 0x55a23538d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/1860999434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a23539c9b0 0x55a2353954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5fe3e00640 1 -- 172.21.15.42:0/1860999434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 msgr2=0x55a23538d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5fe3e00640 1 --2- 172.21.15.42:0/1860999434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 0x55a23538d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5fe3e00640 1 -- 172.21.15.42:0/1860999434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a23539c9b0 msgr2=0x55a2353954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:39.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5fe3e00640 1 --2- 172.21.15.42:0/1860999434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a23539c9b0 0x55a2353954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5fe3e00640 1 -- 172.21.15.42:0/1860999434 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a235396a90 con 0x55a235397840 2024-04-25T00:25:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/1860999434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a23539c9b0 0x55a2353954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5fe3e00640 1 --2- 172.21.15.42:0/1860999434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a85e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5fd402a910 tx=0x7f5fd4003a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53e95ce02bfcd56b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.826+0000 7f5fdbe00640 1 -- 172.21.15.42:0/1860999434 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4035030 con 0x55a235397840 2024-04-25T00:25:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.827+0000 7f5fdbe00640 1 -- 172.21.15.42:0/1860999434 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd4055070 con 0x55a235397840 2024-04-25T00:25:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.827+0000 7f5fdbe00640 1 -- 172.21.15.42:0/1860999434 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4003ee0 con 0x55a235397840 2024-04-25T00:25:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.827+0000 7f5ff5769740 1 -- 172.21.15.42:0/1860999434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 msgr2=0x55a2353a85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.827+0000 7f5ff5769740 1 --2- 172.21.15.42:0/1860999434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a85e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5fd402a910 tx=0x7f5fd4003a20 comp rx=0 tx=0).stop 2024-04-25T00:25:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.827+0000 7f5ff5769740 1 -- 172.21.15.42:0/1860999434 shutdown_connections 2024-04-25T00:25:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.827+0000 7f5ff5769740 1 --2- 172.21.15.42:0/1860999434 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a23539c9b0 0x55a2353954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.827+0000 7f5ff5769740 1 --2- 172.21.15.42:0/1860999434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a235397840 0x55a2353a85e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.827+0000 7f5ff5769740 1 --2- 172.21.15.42:0/1860999434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235396e50 0x55a23538d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.827+0000 7f5ff5769740 1 -- 172.21.15.42:0/1860999434 >> 172.21.15.42:0/1860999434 conn(0x55a2353924c0 msgr2=0x55a23539c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.827+0000 7f5ff5769740 1 -- 172.21.15.42:0/1860999434 shutdown_connections 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.828+0000 7f5ff5769740 1 -- 172.21.15.42:0/1860999434 wait complete. 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.828+0000 7f5ff5769740 1 -- start start 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.828+0000 7f5ff5769740 1 Processor -- start 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235396e50 0x55a2353a5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235397840 0x55a2353a6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5ff5769740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 0x55a2353a73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5ff5769740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1a91d0 con 0x55a235396e50 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5ff5769740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1aa640 con 0x55a235397840 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5ff5769740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5fec1a98a0 con 0x55a23539c9b0 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5ff5769740 10 client.0 _ll_register_callbacks cb 0x7f5fd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5fe3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235397840 0x55a2353a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5fe3e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235397840 0x55a2353a6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:54704/0 (socket says 172.21.15.42:54704) 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5fe3e00640 1 -- 172.21.15.42:0/1284763534 learned_addr learned my addr 172.21.15.42:0/1284763534 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235396e50 0x55a2353a5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 0x55a2353a73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5fe3e00640 1 -- 172.21.15.42:0/1284763534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 msgr2=0x55a2353a73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5fe3e00640 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 0x55a2353a73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5fe3e00640 1 -- 172.21.15.42:0/1284763534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235396e50 msgr2=0x55a2353a5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5fe3e00640 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235396e50 0x55a2353a5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5fe3e00640 1 -- 172.21.15.42:0/1284763534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2353922f0 con 0x55a235397840 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.829+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 0x55a2353a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.830+0000 7f5fe3e00640 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235397840 0x55a2353a6430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40035f0 tx=0x7f5fd4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff148f36c13e62af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.830+0000 7f5fd9600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:25:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.830+0000 7f5fd9600640 1 -- 172.21.15.42:0/1284763534 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4035040 con 0x55a235397840 2024-04-25T00:25:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.830+0000 7f5fd9600640 1 -- 172.21.15.42:0/1284763534 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f5fd4004980 con 0x55a235397840 2024-04-25T00:25:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.830+0000 7f5ff5769740 1 -- 172.21.15.42:0/1284763534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fd4055070 con 0x55a235397840 2024-04-25T00:25:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.830+0000 7f5fd9600640 1 -- 172.21.15.42:0/1284763534 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5fd4004390 con 0x55a235397840 2024-04-25T00:25:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.830+0000 7f5ff5769740 10 client.25733 fetch_fsmap learned FSMap version 29 2024-04-25T00:25:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.830+0000 7f5ff5769740 1 -- 172.21.15.42:0/1284763534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a235413240 con 0x55a235397840 2024-04-25T00:25:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.830+0000 7f5fd9600640 1 -- 172.21.15.42:0/1284763534 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5fd4055070 con 0x55a235397840 2024-04-25T00:25:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.831+0000 7f5fd9600640 1 -- 172.21.15.42:0/1284763534 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f5fd405b470 con 0x55a235397840 2024-04-25T00:25:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.831+0000 7f5ff5769740 10 client.25733 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:25:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.831+0000 7f5ff5769740 10 client.25733 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.831+0000 7f5ff5769740 20 client.25733 populate_metadata read hostname 'smithi042' 2024-04-25T00:25:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.831+0000 7f5ff5769740 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a235413ad0 0x55a235415ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.831+0000 7f5ff5769740 4 client.25733 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:25:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.831+0000 7f5ff5769740 1 -- 172.21.15.42:0/1284763534 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a235301340 con 0x55a235413ad0 2024-04-25T00:25:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.831+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a235413ad0 0x55a235415ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.832+0000 7f5ff0a00640 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a235413ad0 0x55a235415ef0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d3ae83af7dfae77 server_cookie=88bd7880fb713018 in_seq=0 out_seq=0 2024-04-25T00:25:39.836 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.832+0000 7f5fd9600640 10 client.25733 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:39.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:39 smithi106 ceph-mon[32510]: pgmap v595: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-04-25T00:25:39.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:39 smithi042 ceph-mon[28130]: pgmap v595: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.7 MiB/s wr, 199 op/s 2024-04-25T00:25:39.883 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5fd9600640 1 -- 172.21.15.42:0/1284763534 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a235301340 con 0x55a235413ad0 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5fd9600640 10 client.25733 handle_command_reply: tid=0 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 client.25733 shutdown 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 -- 172.21.15.42:0/1284763534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235397840 msgr2=0x55a2353a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235397840 0x55a2353a6430 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5fd40035f0 tx=0x7f5fd4005fd0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 -- 172.21.15.42:0/1284763534 shutdown_connections 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a235413ad0 0x55a235415ef0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a23539c9b0 0x55a2353a73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a235397840 0x55a2353a6430 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 --2- 172.21.15.42:0/1284763534 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a235396e50 0x55a2353a5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5fd9600640 0 client.25733 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 -- 172.21.15.42:0/1284763534 >> 172.21.15.42:0/1284763534 conn(0x55a2353924c0 msgr2=0x55a23539bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 -- 172.21.15.42:0/1284763534 shutdown_connections 2024-04-25T00:25:39.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.882+0000 7f5ff5769740 1 -- 172.21.15.42:0/1284763534 wait complete. 2024-04-25T00:25:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.883+0000 7f5ff5769740 20 client.25733 trim_cache size 0 max 16384 2024-04-25T00:25:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:25:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:25:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:25:39.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:25:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.883+0000 7f5ff1400640 1 -- 172.21.15.42:0/3487483221 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5fd00a2c00 msgr2=0x7f5fd00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.883+0000 7f5ff1400640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5fd00a2c00 0x7f5fd00a50c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc09bb50 tx=0x7f5fdc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.883+0000 7f5ff1400640 1 -- 172.21.15.42:0/3487483221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec0741a0 msgr2=0x7f5fec199b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.883+0000 7f5ff1400640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec0741a0 0x7f5fec199b50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00e9f0 tx=0x7f5fdc00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.884+0000 7f5ff1400640 1 -- 172.21.15.42:0/3487483221 shutdown_connections 2024-04-25T00:25:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.884+0000 7f5ff1400640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5fd00a2c00 0x7f5fd00a50c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.884+0000 7f5ff1400640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5fec109490 0x7f5fec1a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.884+0000 7f5ff1400640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5fec075780 0x7f5fec19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.884+0000 7f5ff1400640 1 --2- 172.21.15.42:0/3487483221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5fec0741a0 0x7f5fec199b50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.884+0000 7f5ff1400640 1 -- 172.21.15.42:0/3487483221 >> 172.21.15.42:0/3487483221 conn(0x7f5fec0fcb40 msgr2=0x7f5fec105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.884+0000 7f5ff1400640 1 -- 172.21.15.42:0/3487483221 shutdown_connections 2024-04-25T00:25:39.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:39.884+0000 7f5ff1400640 1 -- 172.21.15.42:0/3487483221 wait complete. 2024-04-25T00:25:40.241 INFO:tasks.workunit.client.0.smithi042.stdout: 1 398031 6.28 MB/sec execute 210 sec latency 36.969 ms 2024-04-25T00:25:41.241 INFO:tasks.workunit.client.0.smithi042.stdout: 1 399373 6.28 MB/sec execute 211 sec latency 28.516 ms 2024-04-25T00:25:41.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:41 smithi106 ceph-mon[32510]: pgmap v596: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-25T00:25:41.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:41 smithi042 ceph-mon[28130]: pgmap v596: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-25T00:25:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:41 smithi136 ceph-mon[32923]: pgmap v596: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 699 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-04-25T00:25:42.241 INFO:tasks.workunit.client.0.smithi042.stdout: 1 401106 6.31 MB/sec execute 212 sec latency 14.116 ms 2024-04-25T00:25:43.241 INFO:tasks.workunit.client.0.smithi042.stdout: 1 401802 6.29 MB/sec execute 213 sec latency 15.502 ms 2024-04-25T00:25:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:43 smithi106 ceph-mon[32510]: pgmap v597: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.0 MiB/s wr, 249 op/s 2024-04-25T00:25:43.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:43 smithi042 ceph-mon[28130]: pgmap v597: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.0 MiB/s wr, 249 op/s 2024-04-25T00:25:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:43 smithi136 ceph-mon[32923]: pgmap v597: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 894 KiB/s rd, 2.0 MiB/s wr, 249 op/s 2024-04-25T00:25:44.241 INFO:tasks.workunit.client.0.smithi042.stdout: 1 403205 6.28 MB/sec execute 214 sec latency 45.962 ms 2024-04-25T00:25:45.241 INFO:tasks.workunit.client.0.smithi042.stdout: 1 404688 6.31 MB/sec execute 215 sec latency 15.749 ms 2024-04-25T00:25:45.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:45 smithi106 ceph-mon[32510]: pgmap v598: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-25T00:25:45.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:45 smithi042 ceph-mon[28130]: pgmap v598: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-25T00:25:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:45 smithi136 ceph-mon[32923]: pgmap v598: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 570 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-25T00:25:46.241 INFO:tasks.workunit.client.0.smithi042.stdout: 1 405379 6.29 MB/sec execute 216 sec latency 23.949 ms 2024-04-25T00:25:47.242 INFO:tasks.workunit.client.0.smithi042.stdout: 1 406600 6.28 MB/sec execute 217 sec latency 23.646 ms 2024-04-25T00:25:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:47 smithi106 ceph-mon[32510]: pgmap v599: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-04-25T00:25:47.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:47 smithi042 ceph-mon[28130]: pgmap v599: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-04-25T00:25:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:47 smithi136 ceph-mon[32923]: pgmap v599: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 822 KiB/s rd, 2.0 MiB/s wr, 229 op/s 2024-04-25T00:25:48.242 INFO:tasks.workunit.client.0.smithi042.stdout: 1 408224 6.31 MB/sec execute 218 sec latency 13.852 ms 2024-04-25T00:25:49.242 INFO:tasks.workunit.client.0.smithi042.stdout: 1 408872 6.29 MB/sec execute 219 sec latency 37.858 ms 2024-04-25T00:25:49.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:49 smithi106 ceph-mon[32510]: pgmap v600: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T00:25:49.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:49 smithi042 ceph-mon[28130]: pgmap v600: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T00:25:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:49 smithi136 ceph-mon[32923]: pgmap v600: 97 pgs: 97 active+clean; 88 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T00:25:50.242 INFO:tasks.workunit.client.0.smithi042.stdout: 1 410500 6.29 MB/sec execute 220 sec latency 26.343 ms 2024-04-25T00:25:51.242 INFO:tasks.workunit.client.0.smithi042.stdout: 1 411873 6.31 MB/sec execute 221 sec latency 15.353 ms 2024-04-25T00:25:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:51 smithi106 ceph-mon[32510]: pgmap v601: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-04-25T00:25:51.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:51 smithi042 ceph-mon[28130]: pgmap v601: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-04-25T00:25:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:51 smithi136 ceph-mon[32923]: pgmap v601: 97 pgs: 97 active+clean; 87 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-04-25T00:25:52.242 INFO:tasks.workunit.client.0.smithi042.stdout: 1 412655 6.29 MB/sec execute 222 sec latency 23.518 ms 2024-04-25T00:25:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.050+0000 7fd3b9600640 1 -- 172.21.15.42:0/1195096119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b41020a0 msgr2=0x7fd3b4104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.050+0000 7fd3b9600640 1 --2- 172.21.15.42:0/1195096119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b41020a0 0x7fd3b4104490 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009900 tx=0x7fd3a402a410 comp rx=0 tx=0).stop 2024-04-25T00:25:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.050+0000 7fd3b9600640 1 -- 172.21.15.42:0/1195096119 shutdown_connections 2024-04-25T00:25:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.050+0000 7fd3b9600640 1 --2- 172.21.15.42:0/1195096119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b4107300 0x7fd3b41096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.050+0000 7fd3b9600640 1 --2- 172.21.15.42:0/1195096119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b41049d0 0x7fd3b4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.050+0000 7fd3b9600640 1 --2- 172.21.15.42:0/1195096119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b41020a0 0x7fd3b4104490 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.050+0000 7fd3b9600640 1 -- 172.21.15.42:0/1195096119 >> 172.21.15.42:0/1195096119 conn(0x7fd3b40fba30 msgr2=0x7fd3b40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.050+0000 7fd3b9600640 1 -- 172.21.15.42:0/1195096119 shutdown_connections 2024-04-25T00:25:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.050+0000 7fd3b9600640 1 -- 172.21.15.42:0/1195096119 wait complete. 2024-04-25T00:25:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.052+0000 7fd3b9600640 1 Processor -- start 2024-04-25T00:25:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.052+0000 7fd3b9600640 1 -- start start 2024-04-25T00:25:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.052+0000 7fd3b9600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b41020a0 0x7fd3b4199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.052+0000 7fd3b9600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b41049d0 0x7fd3b419a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.052+0000 7fd3b9600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b4107300 0x7fd3b41a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.052+0000 7fd3b9600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b410d280 con 0x7fd3b41020a0 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.052+0000 7fd3b9600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b410d100 con 0x7fd3b4107300 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.052+0000 7fd3b9600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b410d400 con 0x7fd3b41049d0 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b8c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b4107300 0x7fd3b41a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b41049d0 0x7fd3b419a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b8c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b4107300 0x7fd3b41a10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58548/0 (socket says 172.21.15.42:58548) 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b8c00640 1 -- 172.21.15.42:0/1111101257 learned_addr learned my addr 172.21.15.42:0/1111101257 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b3e00640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b41020a0 0x7fd3b4199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b8c00640 1 -- 172.21.15.42:0/1111101257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b41049d0 msgr2=0x7fd3b419a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b8c00640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b41049d0 0x7fd3b419a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b8c00640 1 -- 172.21.15.42:0/1111101257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b41020a0 msgr2=0x7fd3b4199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:53.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b8c00640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b41020a0 0x7fd3b4199b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b8c00640 1 -- 172.21.15.42:0/1111101257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b41a1750 con 0x7fd3b4107300 2024-04-25T00:25:53.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b8c00640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b4107300 0x7fd3b41a10e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009fd0 tx=0x7fd3ac00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:53.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.053+0000 7fd3b0c00640 1 -- 172.21.15.42:0/1111101257 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac017070 con 0x7fd3b4107300 2024-04-25T00:25:53.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.054+0000 7fd3b0c00640 1 -- 172.21.15.42:0/1111101257 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3ac003e50 con 0x7fd3b4107300 2024-04-25T00:25:53.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.054+0000 7fd3b9600640 1 -- 172.21.15.42:0/1111101257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b41a1a40 con 0x7fd3b4107300 2024-04-25T00:25:53.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.054+0000 7fd3b0c00640 1 -- 172.21.15.42:0/1111101257 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0106c0 con 0x7fd3b4107300 2024-04-25T00:25:53.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.054+0000 7fd3b9600640 1 -- 172.21.15.42:0/1111101257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b41a1f40 con 0x7fd3b4107300 2024-04-25T00:25:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.057+0000 7fd3b0c00640 1 -- 172.21.15.42:0/1111101257 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd3ac003970 con 0x7fd3b4107300 2024-04-25T00:25:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.057+0000 7fd3bda3a740 1 Processor -- start 2024-04-25T00:25:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.057+0000 7fd3bda3a740 1 -- start start 2024-04-25T00:25:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11cee50 0x5594b11cf270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11d4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11d49b0 0x5594b11d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5594b1131ab0 con 0x5594b11d49b0 2024-04-25T00:25:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5594b10da640 con 0x5594b11cee50 2024-04-25T00:25:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5594b112da70 con 0x5594b11cf840 2024-04-25T00:25:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3b3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11d4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3b3400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11d49b0 0x5594b11d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3b3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11d4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45794/0 (socket says 172.21.15.42:45794) 2024-04-25T00:25:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3b3400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11cee50 0x5594b11cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.058+0000 7fd3b3e00640 1 -- 172.21.15.42:0/4169225335 learned_addr learned my addr 172.21.15.42:0/4169225335 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.059+0000 7fd3b3e00640 1 -- 172.21.15.42:0/4169225335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11cee50 msgr2=0x5594b11cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.059+0000 7fd3b3e00640 1 --2- 172.21.15.42:0/4169225335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11cee50 0x5594b11cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.059+0000 7fd3b3e00640 1 -- 172.21.15.42:0/4169225335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11d49b0 msgr2=0x5594b11d6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.059+0000 7fd3b3e00640 1 --2- 172.21.15.42:0/4169225335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11d49b0 0x5594b11d6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.059+0000 7fd3b3e00640 1 -- 172.21.15.42:0/4169225335 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b0e46210 con 0x5594b11cf840 2024-04-25T00:25:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.059+0000 7fd3b0c00640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3980a2b80 0x7fd3980a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.059+0000 7fd3b0c00640 1 -- 172.21.15.42:0/1111101257 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd3ac0cee30 con 0x7fd3b4107300 2024-04-25T00:25:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.059+0000 7fd3b3e00640 1 --2- 172.21.15.42:0/4169225335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11d4340 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4005ab0 tx=0x7fd3a4004170 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a91d34c2d484b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.059+0000 7fd3b3400640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3980a2b80 0x7fd3980a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.060+0000 7fd3a9200640 1 -- 172.21.15.42:0/4169225335 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4035030 con 0x5594b11cf840 2024-04-25T00:25:53.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.060+0000 7fd3a9200640 1 -- 172.21.15.42:0/4169225335 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a4005520 con 0x5594b11cf840 2024-04-25T00:25:53.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.060+0000 7fd3a9200640 1 -- 172.21.15.42:0/4169225335 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40316b0 con 0x5594b11cf840 2024-04-25T00:25:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.061+0000 7fd3bda3a740 1 -- 172.21.15.42:0/4169225335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 msgr2=0x5594b11d4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.061+0000 7fd3b3400640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3980a2b80 0x7fd3980a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5594b11d00a0 tx=0x7fd3a0004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:53.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.061+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/4169225335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11d4340 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4005ab0 tx=0x7fd3a4004170 comp rx=0 tx=0).stop 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.061+0000 7fd3bda3a740 1 -- 172.21.15.42:0/4169225335 shutdown_connections 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.061+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/4169225335 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11d49b0 0x5594b11d6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.061+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/4169225335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11d4340 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.061+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/4169225335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11cee50 0x5594b11cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.061+0000 7fd3bda3a740 1 -- 172.21.15.42:0/4169225335 >> 172.21.15.42:0/4169225335 conn(0x5594b11ca4c0 msgr2=0x5594b11ca8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.061+0000 7fd3bda3a740 1 -- 172.21.15.42:0/4169225335 shutdown_connections 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.061+0000 7fd3bda3a740 1 -- 172.21.15.42:0/4169225335 wait complete. 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.062+0000 7fd3bda3a740 1 -- start start 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.062+0000 7fd3bda3a740 1 Processor -- start 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cee50 0x5594b11d9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cf840 0x5594b11d95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11e05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5594b11d7900 con 0x5594b11cf840 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5594b10ed990 con 0x5594b11d49b0 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5594b11d7a80 con 0x5594b11cee50 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3bda3a740 10 client.0 _ll_register_callbacks cb 0x5594b10ef950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3b3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cee50 0x5594b11d9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3b3e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cee50 0x5594b11d9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45810/0 (socket says 172.21.15.42:45810) 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3b3e00640 1 -- 172.21.15.42:0/897722105 learned_addr learned my addr 172.21.15.42:0/897722105 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3b8c00640 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11e05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3b3e00640 1 -- 172.21.15.42:0/897722105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 msgr2=0x5594b11e05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3b3e00640 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11e05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3b3e00640 1 -- 172.21.15.42:0/897722105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cf840 msgr2=0x5594b11d95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3b3e00640 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cf840 0x5594b11d95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.063+0000 7fd3b3e00640 1 -- 172.21.15.42:0/897722105 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b0f12620 con 0x5594b11cee50 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.064+0000 7fd3b3e00640 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cee50 0x5594b11d9070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4000c00 tx=0x7fd3a40315b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc05656c1c79ab83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.064+0000 7fd39d600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:25:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.064+0000 7fd39d600640 1 -- 172.21.15.42:0/897722105 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4035040 con 0x5594b11cee50 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.064+0000 7fd39d600640 1 -- 172.21.15.42:0/897722105 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd3a40051b0 con 0x5594b11cee50 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.064+0000 7fd3bda3a740 1 -- 172.21.15.42:0/897722105 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a4005520 con 0x5594b11cee50 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.064+0000 7fd39d600640 1 -- 172.21.15.42:0/897722105 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3a40316b0 con 0x5594b11cee50 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.064+0000 7fd3bda3a740 10 client.25756 fetch_fsmap learned FSMap version 29 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.064+0000 7fd3bda3a740 1 -- 172.21.15.42:0/897722105 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5594b11d7900 con 0x5594b11cee50 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.065+0000 7fd39d600640 1 -- 172.21.15.42:0/897722105 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a4005520 con 0x5594b11cee50 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.065+0000 7fd39d600640 1 -- 172.21.15.42:0/897722105 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fd3a4054030 con 0x5594b11cee50 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.065+0000 7fd3bda3a740 10 client.25756 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.065+0000 7fd3bda3a740 10 client.25756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.065+0000 7fd3bda3a740 20 client.25756 populate_metadata read hostname 'smithi042' 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.065+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5594b11e48d0 0x5594b11e6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.065+0000 7fd3bda3a740 4 client.25756 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:25:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.065+0000 7fd3bda3a740 1 -- 172.21.15.42:0/897722105 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594b113eff0 con 0x5594b11e48d0 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.065+0000 7fd3b3400640 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5594b11e48d0 0x5594b11e6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.066+0000 7fd3b3400640 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5594b11e48d0 0x5594b11e6cc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1785c50ac07feff6 server_cookie=339e003bc5cb7d97 in_seq=0 out_seq=0 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.066+0000 7fd39d600640 10 client.25756 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd39d600640 1 -- 172.21.15.42:0/897722105 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5594b113eff0 con 0x5594b11e48d0 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd39d600640 10 client.25756 handle_command_reply: tid=0 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 client.25756 shutdown 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 -- 172.21.15.42:0/897722105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cee50 msgr2=0x5594b11d9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cee50 0x5594b11d9070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4000c00 tx=0x7fd3a40315b0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 -- 172.21.15.42:0/897722105 shutdown_connections 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5594b11e48d0 0x5594b11e6cc0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11e05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cf840 0x5594b11d95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/897722105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cee50 0x5594b11d9070 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd39d600640 0 client.25756 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 -- 172.21.15.42:0/897722105 >> 172.21.15.42:0/897722105 conn(0x5594b11ca4c0 msgr2=0x5594b11c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 -- 172.21.15.42:0/897722105 shutdown_connections 2024-04-25T00:25:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 1 -- 172.21.15.42:0/897722105 wait complete. 2024-04-25T00:25:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.067+0000 7fd3bda3a740 20 client.25756 trim_cache size 0 max 16384 2024-04-25T00:25:53.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.085+0000 7fd3bda3a740 1 Processor -- start 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.085+0000 7fd3bda3a740 1 -- start start 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.085+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cee50 0x5594b11c9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.085+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11eb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.085+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11ebb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.085+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41ad500 con 0x5594b11cee50 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.085+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41aadd0 con 0x5594b11d49b0 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.085+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41a8c20 con 0x5594b11cf840 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.085+0000 7fd3b3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11eb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.085+0000 7fd3b3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11eb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45820/0 (socket says 172.21.15.42:45820) 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.086+0000 7fd3b3400640 1 -- 172.21.15.42:0/2364145843 learned_addr learned my addr 172.21.15.42:0/2364145843 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.086+0000 7fd3b3e00640 1 --2- 172.21.15.42:0/2364145843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cee50 0x5594b11c9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.086+0000 7fd3b8c00640 1 --2- 172.21.15.42:0/2364145843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11ebb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.086+0000 7fd3b3400640 1 -- 172.21.15.42:0/2364145843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 msgr2=0x5594b11ebb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.086+0000 7fd3b3400640 1 --2- 172.21.15.42:0/2364145843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11ebb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.086+0000 7fd3b3400640 1 -- 172.21.15.42:0/2364145843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cee50 msgr2=0x5594b11c9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.086+0000 7fd3b3400640 1 --2- 172.21.15.42:0/2364145843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cee50 0x5594b11c9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.086+0000 7fd3b3400640 1 -- 172.21.15.42:0/2364145843 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5594b11e7340 con 0x5594b11cf840 2024-04-25T00:25:53.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.086+0000 7fd3b8c00640 1 --2- 172.21.15.42:0/2364145843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.086+0000 7fd3b3400640 1 --2- 172.21.15.42:0/2364145843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11eb5c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd3b419b040 tx=0x7fd3a0014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afb9418b1819d49e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.087+0000 7fd39fe00640 1 -- 172.21.15.42:0/2364145843 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0010e10 con 0x5594b11cf840 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.087+0000 7fd39fe00640 1 -- 172.21.15.42:0/2364145843 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a0030070 con 0x5594b11cf840 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.087+0000 7fd39fe00640 1 -- 172.21.15.42:0/2364145843 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a002b640 con 0x5594b11cf840 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.087+0000 7fd3bda3a740 1 -- 172.21.15.42:0/2364145843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 msgr2=0x5594b11eb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.087+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/2364145843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11eb5c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd3b419b040 tx=0x7fd3a0014040 comp rx=0 tx=0).stop 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.087+0000 7fd3bda3a740 1 -- 172.21.15.42:0/2364145843 shutdown_connections 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.087+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/2364145843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11ebb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.087+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/2364145843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11eb5c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.087+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/2364145843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cee50 0x5594b11c9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.087+0000 7fd3bda3a740 1 -- 172.21.15.42:0/2364145843 >> 172.21.15.42:0/2364145843 conn(0x5594b11ca4c0 msgr2=0x5594b11cd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.088+0000 7fd3bda3a740 1 -- 172.21.15.42:0/2364145843 shutdown_connections 2024-04-25T00:25:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.088+0000 7fd3bda3a740 1 -- 172.21.15.42:0/2364145843 wait complete. 2024-04-25T00:25:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.088+0000 7fd3bda3a740 1 -- start start 2024-04-25T00:25:53.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.088+0000 7fd3bda3a740 1 Processor -- start 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cee50 0x5594b11ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11de320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3bda3a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11df330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41a9290 con 0x5594b11cee50 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41aa700 con 0x5594b11d49b0 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3bda3a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b41a9960 con 0x5594b11cf840 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3bda3a740 10 client.0 _ll_register_callbacks cb 0x7fd3a000bde0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b8c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11df330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b3e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cee50 0x5594b11ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b3400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11de320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45826/0 (socket says 172.21.15.42:45826) 2024-04-25T00:25:53.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b8c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11df330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58574/0 (socket says 172.21.15.42:58574) 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b3400640 1 -- 172.21.15.42:0/272357169 learned_addr learned my addr 172.21.15.42:0/272357169 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b3400640 1 -- 172.21.15.42:0/272357169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 msgr2=0x5594b11df330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b3400640 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11df330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b3400640 1 -- 172.21.15.42:0/272357169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cee50 msgr2=0x5594b11ddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b3400640 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cee50 0x5594b11ddde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.089+0000 7fd3b3400640 1 -- 172.21.15.42:0/272357169 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594b11ca2f0 con 0x5594b11cf840 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.090+0000 7fd3b3400640 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11de320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0004700 tx=0x7fd3a0010f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c352fa0febf58fd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.090+0000 7fd39d600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.090+0000 7fd39d600640 1 -- 172.21.15.42:0/272357169 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a0038070 con 0x5594b11cf840 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.090+0000 7fd39d600640 1 -- 172.21.15.42:0/272357169 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd3a0004250 con 0x5594b11cf840 2024-04-25T00:25:53.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.090+0000 7fd3bda3a740 1 -- 172.21.15.42:0/272357169 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3a0030070 con 0x5594b11cf840 2024-04-25T00:25:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.090+0000 7fd39d600640 1 -- 172.21.15.42:0/272357169 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3a0044070 con 0x5594b11cf840 2024-04-25T00:25:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd3bda3a740 10 client.25762 fetch_fsmap learned FSMap version 29 2024-04-25T00:25:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd3bda3a740 1 -- 172.21.15.42:0/272357169 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5594b124b040 con 0x5594b11cf840 2024-04-25T00:25:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd39d600640 1 -- 172.21.15.42:0/272357169 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3a0030070 con 0x5594b11cf840 2024-04-25T00:25:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd39d600640 1 -- 172.21.15.42:0/272357169 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fd3a002b620 con 0x5594b11cf840 2024-04-25T00:25:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd3bda3a740 10 client.25762 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:25:53.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd3bda3a740 10 client.25762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:25:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd3bda3a740 20 client.25762 populate_metadata read hostname 'smithi042' 2024-04-25T00:25:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5594b124ba20 0x5594b124de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:25:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd3bda3a740 4 client.25762 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:25:53.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd3bda3a740 1 -- 172.21.15.42:0/272357169 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5594b1139340 con 0x5594b124ba20 2024-04-25T00:25:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.091+0000 7fd3b3e00640 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5594b124ba20 0x5594b124de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:25:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.092+0000 7fd39d600640 10 client.25762 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:53.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.092+0000 7fd3b3e00640 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5594b124ba20 0x5594b124de40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e279e56fd7aec7 server_cookie=dbb2c1747aadfb05 in_seq=0 out_seq=0 2024-04-25T00:25:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd39d600640 1 -- 172.21.15.42:0/272357169 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5594b1139340 con 0x5594b124ba20 2024-04-25T00:25:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd39d600640 10 client.25762 handle_command_reply: tid=0 2024-04-25T00:25:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd3bda3a740 1 client.25762 shutdown 2024-04-25T00:25:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd3bda3a740 1 -- 172.21.15.42:0/272357169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 msgr2=0x5594b11de320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11de320 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0004700 tx=0x7fd3a0010f70 comp rx=0 tx=0).stop 2024-04-25T00:25:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd3bda3a740 1 -- 172.21.15.42:0/272357169 shutdown_connections 2024-04-25T00:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5594b124ba20 0x5594b124de40 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5594b11d49b0 0x5594b11df330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5594b11cf840 0x5594b11de320 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd39d600640 0 client.25762 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd3bda3a740 1 --2- 172.21.15.42:0/272357169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5594b11cee50 0x5594b11ddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd3bda3a740 1 -- 172.21.15.42:0/272357169 >> 172.21.15.42:0/272357169 conn(0x5594b11ca4c0 msgr2=0x5594b11c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.145+0000 7fd3bda3a740 1 -- 172.21.15.42:0/272357169 shutdown_connections 2024-04-25T00:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3bda3a740 1 -- 172.21.15.42:0/272357169 wait complete. 2024-04-25T00:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3bda3a740 20 client.25762 trim_cache size 0 max 16384 2024-04-25T00:25:53.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3b9600640 1 -- 172.21.15.42:0/1111101257 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3980a2b80 msgr2=0x7fd3980a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3b9600640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3980a2b80 0x7fd3980a5040 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5594b11d00a0 tx=0x7fd3a0004730 comp rx=0 tx=0).stop 2024-04-25T00:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3b9600640 1 -- 172.21.15.42:0/1111101257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b4107300 msgr2=0x7fd3b41a10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:25:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3b9600640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b4107300 0x7fd3b41a10e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac009fd0 tx=0x7fd3ac00b260 comp rx=0 tx=0).stop 2024-04-25T00:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3b9600640 1 -- 172.21.15.42:0/1111101257 shutdown_connections 2024-04-25T00:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3b9600640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3980a2b80 0x7fd3980a5040 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3b9600640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd3b4107300 0x7fd3b41a10e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3b9600640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b41049d0 0x7fd3b419a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.146+0000 7fd3b9600640 1 --2- 172.21.15.42:0/1111101257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3b41020a0 0x7fd3b4199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.147+0000 7fd3b9600640 1 -- 172.21.15.42:0/1111101257 >> 172.21.15.42:0/1111101257 conn(0x7fd3b40fba30 msgr2=0x7fd3b41017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.147+0000 7fd3b9600640 1 -- 172.21.15.42:0/1111101257 shutdown_connections 2024-04-25T00:25:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:25:53.147+0000 7fd3b9600640 1 -- 172.21.15.42:0/1111101257 wait complete. 2024-04-25T00:25:53.242 INFO:tasks.workunit.client.0.smithi042.stdout: 1 413948 6.29 MB/sec execute 223 sec latency 19.134 ms 2024-04-25T00:25:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:53 smithi106 ceph-mon[32510]: pgmap v602: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-04-25T00:25:53.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:53 smithi042 ceph-mon[28130]: pgmap v602: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-04-25T00:25:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:53 smithi136 ceph-mon[32923]: pgmap v602: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 905 KiB/s rd, 2.0 MiB/s wr, 244 op/s 2024-04-25T00:25:54.242 INFO:tasks.workunit.client.0.smithi042.stdout: 1 415325 6.31 MB/sec execute 224 sec latency 50.783 ms 2024-04-25T00:25:55.243 INFO:tasks.workunit.client.0.smithi042.stdout: 1 416019 6.29 MB/sec execute 225 sec latency 25.731 ms 2024-04-25T00:25:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:55 smithi106 ceph-mon[32510]: pgmap v603: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-04-25T00:25:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:55 smithi042 ceph-mon[28130]: pgmap v603: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-04-25T00:25:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:55 smithi136 ceph-mon[32923]: pgmap v603: 97 pgs: 97 active+clean; 89 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 625 KiB/s rd, 1.5 MiB/s wr, 155 op/s 2024-04-25T00:25:56.243 INFO:tasks.workunit.client.0.smithi042.stdout: 1 417538 6.29 MB/sec execute 226 sec latency 36.606 ms 2024-04-25T00:25:57.243 INFO:tasks.workunit.client.0.smithi042.stdout: 1 418910 6.31 MB/sec execute 227 sec latency 15.534 ms 2024-04-25T00:25:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:57 smithi106 ceph-mon[32510]: pgmap v604: 97 pgs: 97 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-04-25T00:25:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:57 smithi042 ceph-mon[28130]: pgmap v604: 97 pgs: 97 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-04-25T00:25:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:57 smithi136 ceph-mon[32923]: pgmap v604: 97 pgs: 97 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-04-25T00:25:58.243 INFO:tasks.workunit.client.0.smithi042.stdout: 1 419583 6.29 MB/sec execute 228 sec latency 32.710 ms 2024-04-25T00:25:59.243 INFO:tasks.workunit.client.0.smithi042.stdout: 1 421098 6.29 MB/sec execute 229 sec latency 61.045 ms 2024-04-25T00:25:59.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:25:59 smithi106 ceph-mon[32510]: pgmap v605: 97 pgs: 97 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-04-25T00:25:59.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:25:59 smithi042 ceph-mon[28130]: pgmap v605: 97 pgs: 97 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-04-25T00:26:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:25:59 smithi136 ceph-mon[32923]: pgmap v605: 97 pgs: 97 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 654 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-04-25T00:26:00.243 INFO:tasks.workunit.client.0.smithi042.stdout: 1 422498 6.32 MB/sec execute 230 sec latency 23.259 ms 2024-04-25T00:26:01.243 INFO:tasks.workunit.client.0.smithi042.stdout: 1 423225 6.30 MB/sec execute 231 sec latency 69.388 ms 2024-04-25T00:26:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:01 smithi106 ceph-mon[32510]: pgmap v606: 97 pgs: 97 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-04-25T00:26:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:01 smithi042 ceph-mon[28130]: pgmap v606: 97 pgs: 97 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-04-25T00:26:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:01 smithi136 ceph-mon[32923]: pgmap v606: 97 pgs: 97 active+clean; 87 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 617 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-04-25T00:26:02.243 INFO:tasks.workunit.client.0.smithi042.stdout: 1 424732 6.30 MB/sec execute 232 sec latency 21.954 ms 2024-04-25T00:26:03.244 INFO:tasks.workunit.client.0.smithi042.stdout: 1 426044 6.32 MB/sec execute 233 sec latency 15.064 ms 2024-04-25T00:26:03.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:03 smithi106 ceph-mon[32510]: pgmap v607: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 2.0 MiB/s wr, 239 op/s 2024-04-25T00:26:03.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:03 smithi042 ceph-mon[28130]: pgmap v607: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 2.0 MiB/s wr, 239 op/s 2024-04-25T00:26:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:03 smithi136 ceph-mon[32923]: pgmap v607: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 2.0 MiB/s wr, 239 op/s 2024-04-25T00:26:04.244 INFO:tasks.workunit.client.0.smithi042.stdout: 1 426713 6.30 MB/sec execute 234 sec latency 44.450 ms 2024-04-25T00:26:05.244 INFO:tasks.workunit.client.0.smithi042.stdout: 1 428441 6.31 MB/sec execute 235 sec latency 25.699 ms 2024-04-25T00:26:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:05 smithi106 ceph-mon[32510]: pgmap v608: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-25T00:26:05.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:05 smithi042 ceph-mon[28130]: pgmap v608: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-25T00:26:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:05 smithi136 ceph-mon[32923]: pgmap v608: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-04-25T00:26:06.244 INFO:tasks.workunit.client.0.smithi042.stdout: 1 429602 6.32 MB/sec execute 236 sec latency 15.925 ms 2024-04-25T00:26:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.302+0000 7f6b9f200640 1 -- 172.21.15.42:0/4092821112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba0101600 msgr2=0x7f6ba0101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.302+0000 7f6b9f200640 1 --2- 172.21.15.42:0/4092821112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba0101600 0x7f6ba0101a20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6b90009900 tx=0x7f6b9002a410 comp rx=0 tx=0).stop 2024-04-25T00:26:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.302+0000 7f6b9f200640 1 -- 172.21.15.42:0/4092821112 shutdown_connections 2024-04-25T00:26:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.302+0000 7f6b9f200640 1 --2- 172.21.15.42:0/4092821112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba01031b0 0x7f6ba0109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.302+0000 7f6b9f200640 1 --2- 172.21.15.42:0/4092821112 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba01027f0 0x7f6ba0102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.302+0000 7f6b9f200640 1 --2- 172.21.15.42:0/4092821112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba0101600 0x7f6ba0101a20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.302+0000 7f6b9f200640 1 -- 172.21.15.42:0/4092821112 >> 172.21.15.42:0/4092821112 conn(0x7f6ba00fcb70 msgr2=0x7f6ba00fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.302+0000 7f6b9f200640 1 -- 172.21.15.42:0/4092821112 shutdown_connections 2024-04-25T00:26:06.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.302+0000 7f6b9f200640 1 -- 172.21.15.42:0/4092821112 wait complete. 2024-04-25T00:26:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.304+0000 7f6b9f200640 1 Processor -- start 2024-04-25T00:26:06.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.304+0000 7f6b9f200640 1 -- start start 2024-04-25T00:26:06.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.305+0000 7f6b9f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba0101600 0x7f6ba0197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.305+0000 7f6b9f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba01027f0 0x7f6ba0197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.305+0000 7f6b9f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba01031b0 0x7f6ba019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.305+0000 7f6b9f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba010d2c0 con 0x7f6ba01031b0 2024-04-25T00:26:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.305+0000 7f6b9f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba010d140 con 0x7f6ba0101600 2024-04-25T00:26:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.305+0000 7f6b9f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba010d440 con 0x7f6ba01027f0 2024-04-25T00:26:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.305+0000 7f6b9de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba0101600 0x7f6ba0197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.305+0000 7f6b9de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba0101600 0x7f6ba0197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:33190/0 (socket says 172.21.15.42:33190) 2024-04-25T00:26:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.305+0000 7f6b9de00640 1 -- 172.21.15.42:0/3667334353 learned_addr learned my addr 172.21.15.42:0/3667334353 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:06.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.306+0000 7f6b9d400640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba01027f0 0x7f6ba0197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.306+0000 7f6b9e800640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba01031b0 0x7f6ba019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.306+0000 7f6b9de00640 1 -- 172.21.15.42:0/3667334353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba01027f0 msgr2=0x7f6ba0197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.306+0000 7f6b9de00640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba01027f0 0x7f6ba0197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.306+0000 7f6b9de00640 1 -- 172.21.15.42:0/3667334353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba01031b0 msgr2=0x7f6ba019eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.306+0000 7f6b9de00640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba01031b0 0x7f6ba019eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.306+0000 7f6b9de00640 1 -- 172.21.15.42:0/3667334353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ba019f520 con 0x7f6ba0101600 2024-04-25T00:26:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.307+0000 7f6b9de00640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba0101600 0x7f6ba0197900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6b90031440 tx=0x7f6b90032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.307+0000 7f6b96a00640 1 -- 172.21.15.42:0/3667334353 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b9003f030 con 0x7f6ba0101600 2024-04-25T00:26:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.307+0000 7f6b96a00640 1 -- 172.21.15.42:0/3667334353 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b90003ce0 con 0x7f6ba0101600 2024-04-25T00:26:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.307+0000 7f6b9f200640 1 -- 172.21.15.42:0/3667334353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ba019f7b0 con 0x7f6ba0101600 2024-04-25T00:26:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.307+0000 7f6b96a00640 1 -- 172.21.15.42:0/3667334353 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b90035070 con 0x7f6ba0101600 2024-04-25T00:26:06.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.307+0000 7f6b9f200640 1 -- 172.21.15.42:0/3667334353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ba019fcf0 con 0x7f6ba0101600 2024-04-25T00:26:06.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.311+0000 7f6b96a00640 1 -- 172.21.15.42:0/3667334353 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b90042030 con 0x7f6ba0101600 2024-04-25T00:26:06.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.311+0000 7f6ba7e38740 1 Processor -- start 2024-04-25T00:26:06.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.311+0000 7f6ba7e38740 1 -- start start 2024-04-25T00:26:06.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e70270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 0x55ffa6e75340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e759b0 0x55ffa6e77e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ffa6dd2ab0 con 0x55ffa6e759b0 2024-04-25T00:26:06.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ffa6d7b640 con 0x55ffa6e70840 2024-04-25T00:26:06.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ffa6dcea70 con 0x55ffa6e6fe50 2024-04-25T00:26:06.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6b9e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e70270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6b9de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 0x55ffa6e75340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6b9e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e70270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:46754/0 (socket says 172.21.15.42:46754) 2024-04-25T00:26:06.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6b9e800640 1 -- 172.21.15.42:0/4221432118 learned_addr learned my addr 172.21.15.42:0/4221432118 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.312+0000 7f6b9d400640 1 --2- 172.21.15.42:0/4221432118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e759b0 0x55ffa6e77e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b9e800640 1 -- 172.21.15.42:0/4221432118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 msgr2=0x55ffa6e75340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b9e800640 1 --2- 172.21.15.42:0/4221432118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 0x55ffa6e75340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b9e800640 1 -- 172.21.15.42:0/4221432118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e759b0 msgr2=0x55ffa6e77e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b9e800640 1 --2- 172.21.15.42:0/4221432118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e759b0 0x55ffa6e77e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b9e800640 1 -- 172.21.15.42:0/4221432118 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa6ae7210 con 0x55ffa6e6fe50 2024-04-25T00:26:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b96a00640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b840a2c00 0x7f6b840a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b9e800640 1 --2- 172.21.15.42:0/4221432118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e70270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009ee0 tx=0x7f6b98010740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0b531b3ccd23c7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b96a00640 1 -- 172.21.15.42:0/3667334353 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6b900ef590 con 0x7f6ba0101600 2024-04-25T00:26:06.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b8b400640 1 -- 172.21.15.42:0/4221432118 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98016070 con 0x55ffa6e6fe50 2024-04-25T00:26:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b8b400640 1 -- 172.21.15.42:0/4221432118 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b98004d20 con 0x55ffa6e6fe50 2024-04-25T00:26:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b8b400640 1 -- 172.21.15.42:0/4221432118 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b980127d0 con 0x55ffa6e6fe50 2024-04-25T00:26:06.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.313+0000 7f6b9de00640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b840a2c00 0x7f6b840a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.314+0000 7f6ba7e38740 1 -- 172.21.15.42:0/4221432118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e6fe50 msgr2=0x55ffa6e70270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:06.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.314+0000 7f6b9de00640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b840a2c00 0x7f6b840a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55ffa6e70c40 tx=0x7f6b900f9470 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.314+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/4221432118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e70270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009ee0 tx=0x7f6b98010740 comp rx=0 tx=0).stop 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.314+0000 7f6ba7e38740 1 -- 172.21.15.42:0/4221432118 shutdown_connections 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.314+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/4221432118 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e759b0 0x55ffa6e77e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.314+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/4221432118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 0x55ffa6e75340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.314+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/4221432118 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e70270 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.314+0000 7f6ba7e38740 1 -- 172.21.15.42:0/4221432118 >> 172.21.15.42:0/4221432118 conn(0x55ffa6e6b4c0 msgr2=0x55ffa6e6b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.315+0000 7f6ba7e38740 1 -- 172.21.15.42:0/4221432118 shutdown_connections 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.315+0000 7f6ba7e38740 1 -- 172.21.15.42:0/4221432118 wait complete. 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.315+0000 7f6ba7e38740 1 -- start start 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.315+0000 7f6ba7e38740 1 Processor -- start 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e7a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e70840 0x55ffa6e7a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 0x55ffa6e81650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55ffa6e78960 con 0x55ffa6e70840 2024-04-25T00:26:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55ffa6d8e990 con 0x55ffa6e6fe50 2024-04-25T00:26:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ffa6e78ae0 con 0x55ffa6e759b0 2024-04-25T00:26:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6ba7e38740 10 client.0 _ll_register_callbacks cb 0x55ffa6d90950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6b9e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 0x55ffa6e81650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6b9e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 0x55ffa6e81650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:46766/0 (socket says 172.21.15.42:46766) 2024-04-25T00:26:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6b9e800640 1 -- 172.21.15.42:0/78717102 learned_addr learned my addr 172.21.15.42:0/78717102 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6b9d400640 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e70840 0x55ffa6e7a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.316+0000 7f6b9e800640 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e7a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.317+0000 7f6b9e800640 1 -- 172.21.15.42:0/78717102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 msgr2=0x55ffa6e7a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.317+0000 7f6b9e800640 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e7a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.317+0000 7f6b9e800640 1 -- 172.21.15.42:0/78717102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e70840 msgr2=0x55ffa6e7a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.317+0000 7f6b9e800640 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e70840 0x55ffa6e7a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.317+0000 7f6b9e800640 1 -- 172.21.15.42:0/78717102 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa6bb3620 con 0x55ffa6e759b0 2024-04-25T00:26:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.317+0000 7f6b9e800640 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 0x55ffa6e81650 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b98005cc0 tx=0x7f6b98005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2af5415feb5d4c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.317+0000 7f6b7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:26:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.317+0000 7f6b7fe00640 1 -- 172.21.15.42:0/78717102 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98016040 con 0x55ffa6e759b0 2024-04-25T00:26:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.317+0000 7f6b7fe00640 1 -- 172.21.15.42:0/78717102 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6b980128c0 con 0x55ffa6e759b0 2024-04-25T00:26:06.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.317+0000 7f6ba7e38740 1 -- 172.21.15.42:0/78717102 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b98004d20 con 0x55ffa6e759b0 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.318+0000 7f6b7fe00640 1 -- 172.21.15.42:0/78717102 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6b98012ac0 con 0x55ffa6e759b0 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.318+0000 7f6ba7e38740 10 client.25774 fetch_fsmap learned FSMap version 29 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.318+0000 7f6ba7e38740 1 -- 172.21.15.42:0/78717102 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffa6e84630 con 0x55ffa6e759b0 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.318+0000 7f6b7fe00640 1 -- 172.21.15.42:0/78717102 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b98004d20 con 0x55ffa6e759b0 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.318+0000 7f6b7fe00640 1 -- 172.21.15.42:0/78717102 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f6b9800f040 con 0x55ffa6e759b0 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.318+0000 7f6ba7e38740 10 client.25774 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.318+0000 7f6ba7e38740 10 client.25774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.318+0000 7f6ba7e38740 20 client.25774 populate_metadata read hostname 'smithi042' 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.318+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ffa6e85bc0 0x55ffa6e87fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.319+0000 7f6ba7e38740 4 client.25774 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.319+0000 7f6ba7e38740 1 -- 172.21.15.42:0/78717102 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffa6ddfff0 con 0x55ffa6e85bc0 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.319+0000 7f6b9d400640 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ffa6e85bc0 0x55ffa6e87fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.319+0000 7f6b9d400640 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ffa6e85bc0 0x55ffa6e87fb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caad54fb489a61f0 server_cookie=185f755664a38928 in_seq=0 out_seq=0 2024-04-25T00:26:06.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.320+0000 7f6b7fe00640 10 client.25774 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6b7fe00640 1 -- 172.21.15.42:0/78717102 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ffa6ddfff0 con 0x55ffa6e85bc0 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6b7fe00640 10 client.25774 handle_command_reply: tid=0 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6ba7e38740 1 client.25774 shutdown 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6ba7e38740 1 -- 172.21.15.42:0/78717102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 msgr2=0x55ffa6e81650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 0x55ffa6e81650 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6b98005cc0 tx=0x7f6b98005760 comp rx=0 tx=0).stop 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6ba7e38740 1 -- 172.21.15.42:0/78717102 shutdown_connections 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ffa6e85bc0 0x55ffa6e87fb0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 0x55ffa6e81650 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e70840 0x55ffa6e7a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/78717102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e7a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6ba7e38740 1 -- 172.21.15.42:0/78717102 >> 172.21.15.42:0/78717102 conn(0x55ffa6e6b4c0 msgr2=0x55ffa6e6a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.321+0000 7f6b7fe00640 0 client.25774 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:06.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.322+0000 7f6ba7e38740 1 -- 172.21.15.42:0/78717102 shutdown_connections 2024-04-25T00:26:06.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.322+0000 7f6ba7e38740 1 -- 172.21.15.42:0/78717102 wait complete. 2024-04-25T00:26:06.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.323+0000 7f6ba7e38740 20 client.25774 trim_cache size 0 max 16384 2024-04-25T00:26:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.344+0000 7f6ba7e38740 1 Processor -- start 2024-04-25T00:26:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.344+0000 7f6ba7e38740 1 -- start start 2024-04-25T00:26:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.344+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.344+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 0x55ffa6e81610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.344+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 0x55ffa6e6e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.344+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01a9110 con 0x55ffa6e6fe50 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.344+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01a69e0 con 0x55ffa6e70840 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.344+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01a4830 con 0x55ffa6e759b0 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 0x55ffa6e81610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 0x55ffa6e81610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:33228/0 (socket says 172.21.15.42:33228) 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9d400640 1 -- 172.21.15.42:0/3046251501 learned_addr learned my addr 172.21.15.42:0/3046251501 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9e800640 1 --2- 172.21.15.42:0/3046251501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 0x55ffa6e6e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9d400640 1 -- 172.21.15.42:0/3046251501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 msgr2=0x55ffa6e6e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9d400640 1 --2- 172.21.15.42:0/3046251501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 0x55ffa6e6e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9d400640 1 -- 172.21.15.42:0/3046251501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e6fe50 msgr2=0x55ffa6e750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9d400640 1 --2- 172.21.15.42:0/3046251501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e750d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9d400640 1 -- 172.21.15.42:0/3046251501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffa6e88320 con 0x55ffa6e70840 2024-04-25T00:26:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.345+0000 7f6b9e800640 1 --2- 172.21.15.42:0/3046251501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.346+0000 7f6b9d400640 1 --2- 172.21.15.42:0/3046251501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 0x55ffa6e81610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55ffa6e88810 tx=0x7f6b8c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=828e75052311b3e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.346+0000 7f6b8aa00640 1 -- 172.21.15.42:0/3046251501 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c023030 con 0x55ffa6e70840 2024-04-25T00:26:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.346+0000 7f6b8aa00640 1 -- 172.21.15.42:0/3046251501 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c01f910 con 0x55ffa6e70840 2024-04-25T00:26:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.346+0000 7f6b8aa00640 1 -- 172.21.15.42:0/3046251501 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c010b50 con 0x55ffa6e70840 2024-04-25T00:26:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.347+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3046251501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 msgr2=0x55ffa6e81610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.347+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/3046251501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 0x55ffa6e81610 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x55ffa6e88810 tx=0x7f6b8c005f90 comp rx=0 tx=0).stop 2024-04-25T00:26:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.347+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3046251501 shutdown_connections 2024-04-25T00:26:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.347+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/3046251501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e759b0 0x55ffa6e6e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.347+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/3046251501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e70840 0x55ffa6e81610 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.347+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/3046251501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.347+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3046251501 >> 172.21.15.42:0/3046251501 conn(0x55ffa6e6b4c0 msgr2=0x55ffa6e66110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.347+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3046251501 shutdown_connections 2024-04-25T00:26:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.347+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3046251501 wait complete. 2024-04-25T00:26:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.348+0000 7f6ba7e38740 1 -- start start 2024-04-25T00:26:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.348+0000 7f6ba7e38740 1 Processor -- start 2024-04-25T00:26:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e7eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e70840 0x55ffa6e7f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6ba7e38740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e759b0 0x55ffa6e7ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01a4ea0 con 0x55ffa6e759b0 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01a6310 con 0x55ffa6e6fe50 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6ba7e38740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6ba01a5570 con 0x55ffa6e70840 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6ba7e38740 10 client.0 _ll_register_callbacks cb 0x7f6b900f2ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6b9e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e7eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6b9d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e70840 0x55ffa6e7f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6b9e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e7eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:33232/0 (socket says 172.21.15.42:33232) 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6b9e800640 1 -- 172.21.15.42:0/3507683948 learned_addr learned my addr 172.21.15.42:0/3507683948 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.349+0000 7f6b9e800640 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e759b0 0x55ffa6e7ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.350+0000 7f6b9e800640 1 -- 172.21.15.42:0/3507683948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e70840 msgr2=0x55ffa6e7f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.350+0000 7f6b9e800640 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e70840 0x55ffa6e7f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.350+0000 7f6b9e800640 1 -- 172.21.15.42:0/3507683948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e759b0 msgr2=0x55ffa6e7ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.350+0000 7f6b9e800640 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e759b0 0x55ffa6e7ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.350+0000 7f6b9e800640 1 -- 172.21.15.42:0/3507683948 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffa6e80580 con 0x55ffa6e6fe50 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.350+0000 7f6b9e800640 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e7eb00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009700 tx=0x7f6b98005c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f92af32700db0ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.350+0000 7f6b7fe00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.350+0000 7f6b7fe00640 1 -- 172.21.15.42:0/3507683948 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b98016040 con 0x55ffa6e6fe50 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.350+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3507683948 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b8c01f910 con 0x55ffa6e6fe50 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.350+0000 7f6b7fe00640 1 -- 172.21.15.42:0/3507683948 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6b98025040 con 0x55ffa6e6fe50 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6b7fe00640 1 -- 172.21.15.42:0/3507683948 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6b980124d0 con 0x55ffa6e6fe50 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6ba7e38740 10 client.25769 fetch_fsmap learned FSMap version 29 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3507683948 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ffa6e81890 con 0x55ffa6e6fe50 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6b7fe00640 1 -- 172.21.15.42:0/3507683948 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b8c01f910 con 0x55ffa6e6fe50 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6b7fe00640 1 -- 172.21.15.42:0/3507683948 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7f6b98044b00 con 0x55ffa6e6fe50 2024-04-25T00:26:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6ba7e38740 10 client.25769 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:26:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6ba7e38740 10 client.25769 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6ba7e38740 20 client.25769 populate_metadata read hostname 'smithi042' 2024-04-25T00:26:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ffa6eec690 0x55ffa6eeeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6ba7e38740 4 client.25769 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:26:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.351+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3507683948 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ffa6dda340 con 0x55ffa6eec690 2024-04-25T00:26:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.352+0000 7f6b9d400640 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ffa6eec690 0x55ffa6eeeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.352+0000 7f6b9d400640 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ffa6eec690 0x55ffa6eeeab0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57bdd30f7fe15194 server_cookie=9f5c7c1ee4422cf8 in_seq=0 out_seq=0 2024-04-25T00:26:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.352+0000 7f6b7fe00640 10 client.25769 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.407+0000 7f6b7fe00640 1 -- 172.21.15.42:0/3507683948 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ffa6dda340 con 0x55ffa6eec690 2024-04-25T00:26:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.407+0000 7f6b7fe00640 10 client.25769 handle_command_reply: tid=0 2024-04-25T00:26:06.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.407+0000 7f6ba7e38740 1 client.25769 shutdown 2024-04-25T00:26:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.407+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3507683948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 msgr2=0x55ffa6e7eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.407+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e7eb00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6b98009700 tx=0x7f6b98005c50 comp rx=0 tx=0).stop 2024-04-25T00:26:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.407+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3507683948 shutdown_connections 2024-04-25T00:26:06.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.407+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55ffa6eec690 0x55ffa6eeeab0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.408+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55ffa6e759b0 0x55ffa6e7ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.408+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffa6e70840 0x55ffa6e7f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.408+0000 7f6ba7e38740 1 --2- 172.21.15.42:0/3507683948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ffa6e6fe50 0x55ffa6e7eb00 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.408+0000 7f6b7fe00640 0 client.25769 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.408+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3507683948 >> 172.21.15.42:0/3507683948 conn(0x55ffa6e6b4c0 msgr2=0x55ffa6e6e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.408+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3507683948 shutdown_connections 2024-04-25T00:26:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.408+0000 7f6ba7e38740 1 -- 172.21.15.42:0/3507683948 wait complete. 2024-04-25T00:26:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.408+0000 7f6ba7e38740 20 client.25769 trim_cache size 0 max 16384 2024-04-25T00:26:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:26:06.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:26:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:26:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:26:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 -- 172.21.15.42:0/3667334353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b840a2c00 msgr2=0x7f6b840a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:06.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b840a2c00 0x7f6b840a50c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x55ffa6e70c40 tx=0x7f6b900f9470 comp rx=0 tx=0).stop 2024-04-25T00:26:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 -- 172.21.15.42:0/3667334353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba0101600 msgr2=0x7f6ba0197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba0101600 0x7f6ba0197900 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6b90031440 tx=0x7f6b90032ce0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 -- 172.21.15.42:0/3667334353 shutdown_connections 2024-04-25T00:26:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b840a2c00 0x7f6b840a50c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6ba01031b0 0x7f6ba019eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6ba01027f0 0x7f6ba0197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 --2- 172.21.15.42:0/3667334353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ba0101600 0x7f6ba0197900 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 -- 172.21.15.42:0/3667334353 >> 172.21.15.42:0/3667334353 conn(0x7f6ba00fcb70 msgr2=0x7f6ba0105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 -- 172.21.15.42:0/3667334353 shutdown_connections 2024-04-25T00:26:06.411 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:06.409+0000 7f6b9f200640 1 -- 172.21.15.42:0/3667334353 wait complete. 2024-04-25T00:26:06.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:06.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:07.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:26:07.244 INFO:tasks.workunit.client.0.smithi042.stdout: 1 430625 6.31 MB/sec execute 237 sec latency 49.347 ms 2024-04-25T00:26:07.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:07 smithi042 ceph-mon[28130]: pgmap v609: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-04-25T00:26:07.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:07.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:07.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:07.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:07 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:07 smithi136 ceph-mon[32923]: pgmap v609: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-04-25T00:26:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:08.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:08.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:07 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:07 smithi106 ceph-mon[32510]: pgmap v609: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 893 KiB/s rd, 2.1 MiB/s wr, 231 op/s 2024-04-25T00:26:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:07 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:26:08.244 INFO:tasks.workunit.client.0.smithi042.stdout: 1 431981 6.31 MB/sec execute 238 sec latency 22.395 ms 2024-04-25T00:26:08.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:08 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T00:26:08.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:08 smithi042 ceph-mon[28130]: Unable to set osd_memory_target on smithi136 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T00:26:08.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:08 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:08.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:08 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:08.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:08 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:26:09.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:08 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T00:26:09.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:08 smithi136 ceph-mon[32923]: Unable to set osd_memory_target on smithi136 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T00:26:09.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:08 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:09.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:08 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:09.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:08 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:26:09.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:08 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T00:26:09.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:08 smithi106 ceph-mon[32510]: Unable to set osd_memory_target on smithi136 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T00:26:09.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:08 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:26:09.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:08 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:26:09.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:08 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:26:09.244 INFO:tasks.workunit.client.0.smithi042.stdout: 1 433152 6.32 MB/sec execute 239 sec latency 16.023 ms 2024-04-25T00:26:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:09 smithi136 ceph-mon[32923]: pgmap v610: 97 pgs: 97 active+clean; 92 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-04-25T00:26:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:09 smithi106 ceph-mon[32510]: pgmap v610: 97 pgs: 97 active+clean; 92 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-04-25T00:26:10.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:09 smithi042 ceph-mon[28130]: pgmap v610: 97 pgs: 97 active+clean; 92 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 721 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-04-25T00:26:10.244 INFO:tasks.workunit.client.0.smithi042.stdout: 1 434150 6.31 MB/sec execute 240 sec latency 45.323 ms 2024-04-25T00:26:11.244 INFO:tasks.workunit.client.0.smithi042.stdout: 1 435931 6.32 MB/sec execute 241 sec latency 23.843 ms 2024-04-25T00:26:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:11 smithi136 ceph-mon[32923]: pgmap v611: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-04-25T00:26:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:11 smithi106 ceph-mon[32510]: pgmap v611: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-04-25T00:26:12.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:11 smithi042 ceph-mon[28130]: pgmap v611: 97 pgs: 97 active+clean; 91 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-04-25T00:26:12.245 INFO:tasks.workunit.client.0.smithi042.stdout: 1 436752 6.32 MB/sec execute 242 sec latency 15.965 ms 2024-04-25T00:26:13.245 INFO:tasks.workunit.client.0.smithi042.stdout: 1 437766 6.31 MB/sec execute 243 sec latency 22.492 ms 2024-04-25T00:26:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:13 smithi136 ceph-mon[32923]: pgmap v612: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.0 MiB/s wr, 237 op/s 2024-04-25T00:26:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:13 smithi106 ceph-mon[32510]: pgmap v612: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.0 MiB/s wr, 237 op/s 2024-04-25T00:26:14.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:13 smithi042 ceph-mon[28130]: pgmap v612: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 2.0 MiB/s wr, 237 op/s 2024-04-25T00:26:14.245 INFO:tasks.workunit.client.0.smithi042.stdout: 1 439108 6.31 MB/sec execute 244 sec latency 22.172 ms 2024-04-25T00:26:15.245 INFO:tasks.workunit.client.0.smithi042.stdout: 1 440243 6.32 MB/sec execute 245 sec latency 61.627 ms 2024-04-25T00:26:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:15 smithi136 ceph-mon[32923]: pgmap v613: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-25T00:26:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:15 smithi106 ceph-mon[32510]: pgmap v613: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-25T00:26:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:15 smithi042 ceph-mon[28130]: pgmap v613: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-25T00:26:16.245 INFO:tasks.workunit.client.0.smithi042.stdout: 1 441240 6.31 MB/sec execute 246 sec latency 17.272 ms 2024-04-25T00:26:17.245 INFO:tasks.workunit.client.0.smithi042.stdout: 1 442868 6.32 MB/sec execute 247 sec latency 29.470 ms 2024-04-25T00:26:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:17 smithi136 ceph-mon[32923]: pgmap v614: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-04-25T00:26:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:17 smithi106 ceph-mon[32510]: pgmap v614: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-04-25T00:26:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:17 smithi042 ceph-mon[28130]: pgmap v614: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 903 KiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-04-25T00:26:18.245 INFO:tasks.workunit.client.0.smithi042.stdout: 1 443843 6.32 MB/sec execute 248 sec latency 56.786 ms 2024-04-25T00:26:19.246 INFO:tasks.workunit.client.0.smithi042.stdout: 1 444875 6.32 MB/sec execute 249 sec latency 23.243 ms 2024-04-25T00:26:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.569+0000 7fd390000640 1 -- 172.21.15.42:0/3913818495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd388104bc0 msgr2=0x7fd388105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:19.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.569+0000 7fd390000640 1 --2- 172.21.15.42:0/3913818495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd388104bc0 0x7fd388105040 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd374009900 tx=0x7fd37402a400 comp rx=0 tx=0).stop 2024-04-25T00:26:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.569+0000 7fd390000640 1 -- 172.21.15.42:0/3913818495 shutdown_connections 2024-04-25T00:26:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.569+0000 7fd390000640 1 --2- 172.21.15.42:0/3913818495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd388105580 0x7fd38810be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.569+0000 7fd390000640 1 --2- 172.21.15.42:0/3913818495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd388104bc0 0x7fd388105040 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.569+0000 7fd390000640 1 --2- 172.21.15.42:0/3913818495 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd3881039d0 0x7fd388103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.569+0000 7fd390000640 1 -- 172.21.15.42:0/3913818495 >> 172.21.15.42:0/3913818495 conn(0x7fd3880fef60 msgr2=0x7fd388101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.569+0000 7fd390000640 1 -- 172.21.15.42:0/3913818495 shutdown_connections 2024-04-25T00:26:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.569+0000 7fd390000640 1 -- 172.21.15.42:0/3913818495 wait complete. 2024-04-25T00:26:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.577+0000 7fd390000640 1 Processor -- start 2024-04-25T00:26:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.578+0000 7fd390000640 1 -- start start 2024-04-25T00:26:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.578+0000 7fd390000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3881039d0 0x7fd38819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.579+0000 7fd390000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd388104bc0 0x7fd38819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.579+0000 7fd390000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd388105580 0x7fd3881a5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.579+0000 7fd390000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd388076ee0 con 0x7fd388104bc0 2024-04-25T00:26:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.579+0000 7fd390000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd388076d60 con 0x7fd388105580 2024-04-25T00:26:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.579+0000 7fd390000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd388077060 con 0x7fd3881039d0 2024-04-25T00:26:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.579+0000 7fd38e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd388105580 0x7fd3881a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.579+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd388104bc0 0x7fd38819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.580+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd388104bc0 0x7fd38819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:54902/0 (socket says 172.21.15.42:54902) 2024-04-25T00:26:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.580+0000 7fd38cc00640 1 -- 172.21.15.42:0/1322198165 learned_addr learned my addr 172.21.15.42:0/1322198165 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.580+0000 7fd38e000640 1 -- 172.21.15.42:0/1322198165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3881039d0 msgr2=0x7fd38819e060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:26:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.580+0000 7fd38e000640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3881039d0 0x7fd38819e060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.580+0000 7fd38e000640 1 -- 172.21.15.42:0/1322198165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd388104bc0 msgr2=0x7fd38819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.580+0000 7fd38e000640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd388104bc0 0x7fd38819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.580+0000 7fd38e000640 1 -- 172.21.15.42:0/1322198165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3881a5c80 con 0x7fd388105580 2024-04-25T00:26:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.581+0000 7fd38e000640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd388105580 0x7fd3881a5610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd380009fd0 tx=0x7fd38000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.581+0000 7fd386000640 1 -- 172.21.15.42:0/1322198165 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38000ce20 con 0x7fd388105580 2024-04-25T00:26:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.581+0000 7fd386000640 1 -- 172.21.15.42:0/1322198165 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd380005020 con 0x7fd388105580 2024-04-25T00:26:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.581+0000 7fd386000640 1 -- 172.21.15.42:0/1322198165 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380010720 con 0x7fd388105580 2024-04-25T00:26:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.581+0000 7fd390000640 1 -- 172.21.15.42:0/1322198165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3881a5f70 con 0x7fd388105580 2024-04-25T00:26:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.582+0000 7fd390000640 1 -- 172.21.15.42:0/1322198165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881a6460 con 0x7fd388105580 2024-04-25T00:26:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.590+0000 7fd386000640 1 -- 172.21.15.42:0/1322198165 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd380005540 con 0x7fd388105580 2024-04-25T00:26:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.590+0000 7fd392e65740 1 Processor -- start 2024-04-25T00:26:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.590+0000 7fd392e65740 1 -- start start 2024-04-25T00:26:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.591+0000 7fd392e65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca8ca0 0x55e43cca90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.591+0000 7fd392e65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca9690 0x55e43ccae190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.591+0000 7fd392e65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43ccae800 0x55e43ccb0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.591+0000 7fd392e65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e43cc0bab0 con 0x55e43cca9690 2024-04-25T00:26:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.591+0000 7fd392e65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e43cbb4640 con 0x55e43cca8ca0 2024-04-25T00:26:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.591+0000 7fd392e65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e43cc07a70 con 0x55e43ccae800 2024-04-25T00:26:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.592+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca8ca0 0x55e43cca90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.592+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca8ca0 0x55e43cca90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:54038/0 (socket says 172.21.15.42:54038) 2024-04-25T00:26:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.592+0000 7fd38cc00640 1 -- 172.21.15.42:0/346365622 learned_addr learned my addr 172.21.15.42:0/346365622 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.592+0000 7fd38cc00640 1 --2- 172.21.15.42:0/346365622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43ccae800 0x55e43ccb0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.592+0000 7fd38cc00640 1 -- 172.21.15.42:0/346365622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43ccae800 msgr2=0x55e43ccb0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.592+0000 7fd38cc00640 1 --2- 172.21.15.42:0/346365622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43ccae800 0x55e43ccb0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.592+0000 7fd38cc00640 1 -- 172.21.15.42:0/346365622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca9690 msgr2=0x55e43ccae190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:26:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.593+0000 7fd386000640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3700a2da0 0x7fd3700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.593+0000 7fd386000640 1 -- 172.21.15.42:0/1322198165 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd3800cf4b0 con 0x7fd388105580 2024-04-25T00:26:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.593+0000 7fd38d600640 1 --2- 172.21.15.42:0/346365622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca9690 0x55e43ccae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.594+0000 7fd38cc00640 1 --2- 172.21.15.42:0/346365622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca9690 0x55e43ccae190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.594+0000 7fd38cc00640 1 -- 172.21.15.42:0/346365622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43c920210 con 0x55e43cca8ca0 2024-04-25T00:26:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.594+0000 7fd38d600640 1 --2- 172.21.15.42:0/346365622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca9690 0x55e43ccae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:26:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.595+0000 7fd38cc00640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3700a2da0 0x7fd3700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.595+0000 7fd38cc00640 1 --2- 172.21.15.42:0/346365622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca8ca0 0x55e43cca90c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd374005ec0 tx=0x7fd3740056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca568d4e47572295 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:19.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.596+0000 7fd37a600640 1 -- 172.21.15.42:0/346365622 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374035040 con 0x55e43cca8ca0 2024-04-25T00:26:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.596+0000 7fd37a600640 1 -- 172.21.15.42:0/346365622 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd374032c60 con 0x55e43cca8ca0 2024-04-25T00:26:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.596+0000 7fd37a600640 1 -- 172.21.15.42:0/346365622 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37403d940 con 0x55e43cca8ca0 2024-04-25T00:26:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.598+0000 7fd392e65740 1 -- 172.21.15.42:0/346365622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca8ca0 msgr2=0x55e43cca90c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.598+0000 7fd392e65740 1 --2- 172.21.15.42:0/346365622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca8ca0 0x55e43cca90c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd374005ec0 tx=0x7fd3740056d0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.598+0000 7fd38cc00640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3700a2da0 0x7fd3700a5260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd374004650 tx=0x7fd37403dba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.598+0000 7fd392e65740 1 -- 172.21.15.42:0/346365622 shutdown_connections 2024-04-25T00:26:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.598+0000 7fd392e65740 1 --2- 172.21.15.42:0/346365622 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43ccae800 0x55e43ccb0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.598+0000 7fd392e65740 1 --2- 172.21.15.42:0/346365622 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca9690 0x55e43ccae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.598+0000 7fd392e65740 1 --2- 172.21.15.42:0/346365622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca8ca0 0x55e43cca90c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.598+0000 7fd392e65740 1 -- 172.21.15.42:0/346365622 >> 172.21.15.42:0/346365622 conn(0x55e43cca4310 msgr2=0x55e43cca4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.598+0000 7fd392e65740 1 -- 172.21.15.42:0/346365622 shutdown_connections 2024-04-25T00:26:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.598+0000 7fd392e65740 1 -- 172.21.15.42:0/346365622 wait complete. 2024-04-25T00:26:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.599+0000 7fd392e65740 1 -- start start 2024-04-25T00:26:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.599+0000 7fd392e65740 1 Processor -- start 2024-04-25T00:26:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd392e65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca8ca0 0x55e43ccb2f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd392e65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca9690 0x55e43ccb3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd392e65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43ccae800 0x55e43ccba490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd392e65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e43ccb17b0 con 0x55e43ccae800 2024-04-25T00:26:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd392e65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e43cbc7990 con 0x55e43cca9690 2024-04-25T00:26:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd392e65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e43ccb1930 con 0x55e43cca8ca0 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd392e65740 10 client.0 _ll_register_callbacks cb 0x55e43cbc9950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca9690 0x55e43ccb3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd38d600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca8ca0 0x55e43ccb2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca9690 0x55e43ccb3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:54052/0 (socket says 172.21.15.42:54052) 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd38cc00640 1 -- 172.21.15.42:0/1782666074 learned_addr learned my addr 172.21.15.42:0/1782666074 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd38e000640 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43ccae800 0x55e43ccba490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd38cc00640 1 -- 172.21.15.42:0/1782666074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca8ca0 msgr2=0x55e43ccb2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd38cc00640 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca8ca0 0x55e43ccb2f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd38cc00640 1 -- 172.21.15.42:0/1782666074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43ccae800 msgr2=0x55e43ccba490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd38cc00640 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43ccae800 0x55e43ccba490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.600+0000 7fd38cc00640 1 -- 172.21.15.42:0/1782666074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43c9ec620 con 0x55e43cca9690 2024-04-25T00:26:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.601+0000 7fd38e000640 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43ccae800 0x55e43ccba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.601+0000 7fd38cc00640 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca9690 0x55e43ccb3450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd374000c00 tx=0x7fd374042000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9836dba41242d115 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.601+0000 7fd36ea00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:26:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.601+0000 7fd36ea00640 1 -- 172.21.15.42:0/1782666074 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd374035070 con 0x55e43cca9690 2024-04-25T00:26:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.601+0000 7fd36ea00640 1 -- 172.21.15.42:0/1782666074 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd374032960 con 0x55e43cca9690 2024-04-25T00:26:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.601+0000 7fd392e65740 1 -- 172.21.15.42:0/1782666074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd374032c60 con 0x55e43cca9690 2024-04-25T00:26:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd36ea00640 1 -- 172.21.15.42:0/1782666074 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd374032e10 con 0x55e43cca9690 2024-04-25T00:26:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd392e65740 10 client.25781 fetch_fsmap learned FSMap version 29 2024-04-25T00:26:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd392e65740 1 -- 172.21.15.42:0/1782666074 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e43ccbd3e0 con 0x55e43cca9690 2024-04-25T00:26:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd36ea00640 1 -- 172.21.15.42:0/1782666074 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd374032c60 con 0x55e43cca9690 2024-04-25T00:26:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd36ea00640 1 -- 172.21.15.42:0/1782666074 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fd37403a040 con 0x55e43cca9690 2024-04-25T00:26:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd392e65740 10 client.25781 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:26:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd392e65740 10 client.25781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:19.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd392e65740 20 client.25781 populate_metadata read hostname 'smithi042' 2024-04-25T00:26:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd392e65740 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e43ccbe730 0x55e43ccc0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd392e65740 4 client.25781 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.602+0000 7fd392e65740 1 -- 172.21.15.42:0/1782666074 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e43cc18ff0 con 0x55e43ccbe730 2024-04-25T00:26:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.603+0000 7fd38d600640 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e43ccbe730 0x55e43ccc0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.603+0000 7fd38d600640 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e43ccbe730 0x55e43ccc0b20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fae59cf005c9da3 server_cookie=4b353ad4fa7f5450 in_seq=0 out_seq=0 2024-04-25T00:26:19.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.603+0000 7fd36ea00640 10 client.25781 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd36ea00640 1 -- 172.21.15.42:0/1782666074 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e43cc18ff0 con 0x55e43ccbe730 2024-04-25T00:26:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd36ea00640 10 client.25781 handle_command_reply: tid=0 2024-04-25T00:26:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 client.25781 shutdown 2024-04-25T00:26:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 -- 172.21.15.42:0/1782666074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca9690 msgr2=0x55e43ccb3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca9690 0x55e43ccb3450 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd374000c00 tx=0x7fd374042000 comp rx=0 tx=0).stop 2024-04-25T00:26:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 -- 172.21.15.42:0/1782666074 shutdown_connections 2024-04-25T00:26:19.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e43ccbe730 0x55e43ccc0b20 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43ccae800 0x55e43ccba490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43cca9690 0x55e43ccb3450 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 --2- 172.21.15.42:0/1782666074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca8ca0 0x55e43ccb2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 -- 172.21.15.42:0/1782666074 >> 172.21.15.42:0/1782666074 conn(0x55e43cca4310 msgr2=0x55e43cca3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd36ea00640 0 client.25781 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 -- 172.21.15.42:0/1782666074 shutdown_connections 2024-04-25T00:26:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.605+0000 7fd392e65740 1 -- 172.21.15.42:0/1782666074 wait complete. 2024-04-25T00:26:19.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.606+0000 7fd392e65740 20 client.25781 trim_cache size 0 max 16384 2024-04-25T00:26:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.624+0000 7fd392e65740 1 Processor -- start 2024-04-25T00:26:19.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd392e65740 1 -- start start 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd392e65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca8ca0 0x55e43cca7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd392e65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 0x55e43ccba450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd392e65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43cca7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd392e65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881b1aa0 con 0x55e43cca8ca0 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd392e65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881af370 con 0x55e43ccae800 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd392e65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881ad1c0 con 0x55e43cca9690 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd38e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43cca7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 0x55e43ccba450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd38e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43cca7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:54054/0 (socket says 172.21.15.42:54054) 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd38e000640 1 -- 172.21.15.42:0/3650108681 learned_addr learned my addr 172.21.15.42:0/3650108681 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.625+0000 7fd38d600640 1 --2- 172.21.15.42:0/3650108681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca8ca0 0x55e43cca7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.626+0000 7fd38e000640 1 -- 172.21.15.42:0/3650108681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 msgr2=0x55e43ccba450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:19.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.626+0000 7fd38e000640 1 --2- 172.21.15.42:0/3650108681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 0x55e43ccba450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.626+0000 7fd38e000640 1 -- 172.21.15.42:0/3650108681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca8ca0 msgr2=0x55e43cca7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.626+0000 7fd38e000640 1 --2- 172.21.15.42:0/3650108681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca8ca0 0x55e43cca7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.626+0000 7fd38e000640 1 -- 172.21.15.42:0/3650108681 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43cca4140 con 0x55e43ccae800 2024-04-25T00:26:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.626+0000 7fd38e000640 1 --2- 172.21.15.42:0/3650108681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43cca7760 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd38000c760 tx=0x7fd38009c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=291b3147cc1482a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.626+0000 7fd379c00640 1 -- 172.21.15.42:0/3650108681 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3800a49b0 con 0x55e43ccae800 2024-04-25T00:26:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.626+0000 7fd379c00640 1 -- 172.21.15.42:0/3650108681 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd380099de0 con 0x55e43ccae800 2024-04-25T00:26:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.626+0000 7fd379c00640 1 -- 172.21.15.42:0/3650108681 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3800cf310 con 0x55e43ccae800 2024-04-25T00:26:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.627+0000 7fd392e65740 1 -- 172.21.15.42:0/3650108681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 msgr2=0x55e43cca7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.627+0000 7fd392e65740 1 --2- 172.21.15.42:0/3650108681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43cca7760 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd38000c760 tx=0x7fd38009c440 comp rx=0 tx=0).stop 2024-04-25T00:26:19.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.627+0000 7fd392e65740 1 -- 172.21.15.42:0/3650108681 shutdown_connections 2024-04-25T00:26:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.627+0000 7fd392e65740 1 --2- 172.21.15.42:0/3650108681 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43cca7760 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.627+0000 7fd392e65740 1 --2- 172.21.15.42:0/3650108681 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 0x55e43ccba450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.627+0000 7fd392e65740 1 --2- 172.21.15.42:0/3650108681 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca8ca0 0x55e43cca7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.627+0000 7fd392e65740 1 -- 172.21.15.42:0/3650108681 >> 172.21.15.42:0/3650108681 conn(0x55e43cca4310 msgr2=0x55e43ccc0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.627+0000 7fd392e65740 1 -- 172.21.15.42:0/3650108681 shutdown_connections 2024-04-25T00:26:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.628+0000 7fd392e65740 1 -- 172.21.15.42:0/3650108681 wait complete. 2024-04-25T00:26:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.628+0000 7fd392e65740 1 -- start start 2024-04-25T00:26:19.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.628+0000 7fd392e65740 1 Processor -- start 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd392e65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca8ca0 0x55e43ccb7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd392e65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 0x55e43ccb7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd392e65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43ccb8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd392e65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881ad830 con 0x55e43cca8ca0 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd392e65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881aeca0 con 0x55e43ccae800 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd392e65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881adf00 con 0x55e43cca9690 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd392e65740 10 client.0 _ll_register_callbacks cb 0x7fd374009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 0x55e43ccb7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38d600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca8ca0 0x55e43ccb7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 0x55e43ccb7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:54852/0 (socket says 172.21.15.42:54852) 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38cc00640 1 -- 172.21.15.42:0/513330817 learned_addr learned my addr 172.21.15.42:0/513330817 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38e000640 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43ccb8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38cc00640 1 -- 172.21.15.42:0/513330817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 msgr2=0x55e43ccb8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:19.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38cc00640 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43ccb8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38cc00640 1 -- 172.21.15.42:0/513330817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca8ca0 msgr2=0x55e43ccb7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38cc00640 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca8ca0 0x55e43ccb7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38cc00640 1 -- 172.21.15.42:0/513330817 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43ccb9400 con 0x55e43cca9690 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.629+0000 7fd38e000640 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43ccb8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.630+0000 7fd38cc00640 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 0x55e43ccb7ec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd374006010 tx=0x7fd37403d990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9513c93fb7b650f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.630+0000 7fd36ea00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.630+0000 7fd36ea00640 1 -- 172.21.15.42:0/513330817 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37406e030 con 0x55e43cca9690 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.630+0000 7fd36ea00640 1 -- 172.21.15.42:0/513330817 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd374005540 con 0x55e43cca9690 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.630+0000 7fd392e65740 1 -- 172.21.15.42:0/513330817 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd380099de0 con 0x55e43cca9690 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.630+0000 7fd36ea00640 1 -- 172.21.15.42:0/513330817 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd374035070 con 0x55e43cca9690 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.630+0000 7fd392e65740 10 client.25795 fetch_fsmap learned FSMap version 29 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.630+0000 7fd392e65740 1 -- 172.21.15.42:0/513330817 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e43ccba710 con 0x55e43cca9690 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.630+0000 7fd36ea00640 1 -- 172.21.15.42:0/513330817 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd380099de0 con 0x55e43cca9690 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.631+0000 7fd36ea00640 1 -- 172.21.15.42:0/513330817 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fd374003710 con 0x55e43cca9690 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.631+0000 7fd392e65740 10 client.25795 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:26:19.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.631+0000 7fd392e65740 10 client.25795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.631+0000 7fd392e65740 20 client.25795 populate_metadata read hostname 'smithi042' 2024-04-25T00:26:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.631+0000 7fd392e65740 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e43cd25350 0x55e43cd27770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.631+0000 7fd392e65740 4 client.25795 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:26:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.631+0000 7fd392e65740 1 -- 172.21.15.42:0/513330817 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e43cc13340 con 0x55e43cd25350 2024-04-25T00:26:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.631+0000 7fd38d600640 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e43cd25350 0x55e43cd27770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.632+0000 7fd38d600640 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e43cd25350 0x55e43cd27770 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=912f2ddd1c1db959 server_cookie=2c2c2c085f4dc491 in_seq=0 out_seq=0 2024-04-25T00:26:19.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.632+0000 7fd36ea00640 10 client.25795 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd36ea00640 1 -- 172.21.15.42:0/513330817 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e43cc13340 con 0x55e43cd25350 2024-04-25T00:26:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd36ea00640 10 client.25795 handle_command_reply: tid=0 2024-04-25T00:26:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 client.25795 shutdown 2024-04-25T00:26:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 -- 172.21.15.42:0/513330817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 msgr2=0x55e43ccb7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:19.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 0x55e43ccb7ec0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd374006010 tx=0x7fd37403d990 comp rx=0 tx=0).stop 2024-04-25T00:26:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 -- 172.21.15.42:0/513330817 shutdown_connections 2024-04-25T00:26:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e43cd25350 0x55e43cd27770 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e43ccae800 0x55e43ccb8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e43cca9690 0x55e43ccb7ec0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd36ea00640 0 client.25795 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 --2- 172.21.15.42:0/513330817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e43cca8ca0 0x55e43ccb7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 -- 172.21.15.42:0/513330817 >> 172.21.15.42:0/513330817 conn(0x55e43cca4310 msgr2=0x55e43cca7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 -- 172.21.15.42:0/513330817 shutdown_connections 2024-04-25T00:26:19.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 1 -- 172.21.15.42:0/513330817 wait complete. 2024-04-25T00:26:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.680+0000 7fd392e65740 20 client.25795 trim_cache size 0 max 16384 2024-04-25T00:26:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:26:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:26:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:26:19.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:26:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.681+0000 7fd390000640 1 -- 172.21.15.42:0/1322198165 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3700a2da0 msgr2=0x7fd3700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.681+0000 7fd390000640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3700a2da0 0x7fd3700a5260 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd374004650 tx=0x7fd37403dba0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.681+0000 7fd390000640 1 -- 172.21.15.42:0/1322198165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd388105580 msgr2=0x7fd3881a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.681+0000 7fd390000640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd388105580 0x7fd3881a5610 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd380009fd0 tx=0x7fd38000ef10 comp rx=0 tx=0).stop 2024-04-25T00:26:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.681+0000 7fd390000640 1 -- 172.21.15.42:0/1322198165 shutdown_connections 2024-04-25T00:26:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.681+0000 7fd390000640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd3700a2da0 0x7fd3700a5260 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.681+0000 7fd390000640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd388105580 0x7fd3881a5610 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.681+0000 7fd390000640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd388104bc0 0x7fd38819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.681+0000 7fd390000640 1 --2- 172.21.15.42:0/1322198165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3881039d0 0x7fd38819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.681+0000 7fd390000640 1 -- 172.21.15.42:0/1322198165 >> 172.21.15.42:0/1322198165 conn(0x7fd3880fef60 msgr2=0x7fd388105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.682+0000 7fd390000640 1 -- 172.21.15.42:0/1322198165 shutdown_connections 2024-04-25T00:26:19.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:19.682+0000 7fd390000640 1 -- 172.21.15.42:0/1322198165 wait complete. 2024-04-25T00:26:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:19 smithi136 ceph-mon[32923]: pgmap v615: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-04-25T00:26:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:19 smithi106 ceph-mon[32510]: pgmap v615: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-04-25T00:26:20.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:19 smithi042 ceph-mon[28130]: pgmap v615: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 651 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-04-25T00:26:20.246 INFO:tasks.workunit.client.0.smithi042.stdout: 1 446715 6.33 MB/sec execute 250 sec latency 81.798 ms 2024-04-25T00:26:21.246 INFO:tasks.workunit.client.0.smithi042.stdout: 1 447424 6.32 MB/sec execute 251 sec latency 18.819 ms 2024-04-25T00:26:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:21 smithi136 ceph-mon[32923]: pgmap v616: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-25T00:26:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:21 smithi106 ceph-mon[32510]: pgmap v616: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-25T00:26:22.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:21 smithi042 ceph-mon[28130]: pgmap v616: 97 pgs: 97 active+clean; 95 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 655 KiB/s rd, 1.6 MiB/s wr, 181 op/s 2024-04-25T00:26:22.246 INFO:tasks.workunit.client.0.smithi042.stdout: 1 448442 6.32 MB/sec execute 252 sec latency 58.649 ms 2024-04-25T00:26:23.246 INFO:tasks.workunit.client.0.smithi042.stdout: 1 450296 6.33 MB/sec execute 253 sec latency 18.670 ms 2024-04-25T00:26:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:23 smithi136 ceph-mon[32923]: pgmap v617: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.0 MiB/s wr, 260 op/s 2024-04-25T00:26:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:23 smithi106 ceph-mon[32510]: pgmap v617: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.0 MiB/s wr, 260 op/s 2024-04-25T00:26:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:23 smithi042 ceph-mon[28130]: pgmap v617: 97 pgs: 97 active+clean; 93 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 2.0 MiB/s wr, 260 op/s 2024-04-25T00:26:24.246 INFO:tasks.workunit.client.0.smithi042.stdout: 1 450969 6.32 MB/sec execute 254 sec latency 16.225 ms 2024-04-25T00:26:25.246 INFO:tasks.workunit.client.0.smithi042.stdout: 1 452018 6.32 MB/sec execute 255 sec latency 79.301 ms 2024-04-25T00:26:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:25 smithi136 ceph-mon[32923]: pgmap v618: 97 pgs: 97 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-25T00:26:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:25 smithi106 ceph-mon[32510]: pgmap v618: 97 pgs: 97 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-25T00:26:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:25 smithi042 ceph-mon[28130]: pgmap v618: 97 pgs: 97 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 619 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-25T00:26:26.246 INFO:tasks.workunit.client.0.smithi042.stdout: 1 453856 6.33 MB/sec execute 256 sec latency 18.495 ms 2024-04-25T00:26:27.246 INFO:tasks.workunit.client.0.smithi042.stdout: 1 454479 6.32 MB/sec execute 257 sec latency 22.108 ms 2024-04-25T00:26:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:27 smithi136 ceph-mon[32923]: pgmap v619: 97 pgs: 97 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-04-25T00:26:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:27 smithi106 ceph-mon[32510]: pgmap v619: 97 pgs: 97 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-04-25T00:26:28.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:27 smithi042 ceph-mon[28130]: pgmap v619: 97 pgs: 97 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 2.0 MiB/s wr, 242 op/s 2024-04-25T00:26:28.247 INFO:tasks.workunit.client.0.smithi042.stdout: 1 455438 6.32 MB/sec execute 258 sec latency 26.431 ms 2024-04-25T00:26:29.247 INFO:tasks.workunit.client.0.smithi042.stdout: 1 456440 6.30 MB/sec execute 259 sec latency 23.404 ms 2024-04-25T00:26:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:29 smithi136 ceph-mon[32923]: pgmap v620: 97 pgs: 97 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.8 MiB/s wr, 188 op/s 2024-04-25T00:26:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:29 smithi106 ceph-mon[32510]: pgmap v620: 97 pgs: 97 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.8 MiB/s wr, 188 op/s 2024-04-25T00:26:30.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:29 smithi042 ceph-mon[28130]: pgmap v620: 97 pgs: 97 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 1.8 MiB/s wr, 188 op/s 2024-04-25T00:26:30.247 INFO:tasks.workunit.client.0.smithi042.stdout: 1 457830 6.32 MB/sec execute 260 sec latency 34.422 ms 2024-04-25T00:26:31.247 INFO:tasks.workunit.client.0.smithi042.stdout: 1 458274 6.30 MB/sec execute 261 sec latency 97.688 ms 2024-04-25T00:26:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:31 smithi136 ceph-mon[32923]: pgmap v621: 97 pgs: 97 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-04-25T00:26:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:31 smithi106 ceph-mon[32510]: pgmap v621: 97 pgs: 97 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-04-25T00:26:32.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:31 smithi042 ceph-mon[28130]: pgmap v621: 97 pgs: 97 active+clean; 96 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 644 KiB/s rd, 1.7 MiB/s wr, 181 op/s 2024-04-25T00:26:32.247 INFO:tasks.workunit.client.0.smithi042.stdout: 1 458765 6.33 MB/sec execute 262 sec latency 42.988 ms 2024-04-25T00:26:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.850+0000 7fdbbe400640 1 -- 172.21.15.42:0/1878667598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbb8100050 msgr2=0x7fdbb8100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.850+0000 7fdbbe400640 1 --2- 172.21.15.42:0/1878667598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbb8100050 0x7fdbb8100470 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009be0 tx=0x7fdba802a6b0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.850+0000 7fdbbe400640 1 -- 172.21.15.42:0/1878667598 shutdown_connections 2024-04-25T00:26:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.850+0000 7fdbbe400640 1 --2- 172.21.15.42:0/1878667598 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb8101350 0x7fdbb81017d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.850+0000 7fdbbe400640 1 --2- 172.21.15.42:0/1878667598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbb81009b0 0x7fdbb8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.850+0000 7fdbbe400640 1 --2- 172.21.15.42:0/1878667598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbb8100050 0x7fdbb8100470 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.850+0000 7fdbbe400640 1 -- 172.21.15.42:0/1878667598 >> 172.21.15.42:0/1878667598 conn(0x7fdbb80fbc10 msgr2=0x7fdbb80fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.850+0000 7fdbbe400640 1 -- 172.21.15.42:0/1878667598 shutdown_connections 2024-04-25T00:26:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.850+0000 7fdbbe400640 1 -- 172.21.15.42:0/1878667598 wait complete. 2024-04-25T00:26:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.851+0000 7fdbbe400640 1 Processor -- start 2024-04-25T00:26:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbbe400640 1 -- start start 2024-04-25T00:26:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbbe400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbb8100050 0x7fdbb8199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbbe400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbb81009b0 0x7fdbb819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbbe400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb8101350 0x7fdbb81a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbbe400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb810f610 con 0x7fdbb8101350 2024-04-25T00:26:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbbe400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb810f490 con 0x7fdbb81009b0 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbbe400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb810f790 con 0x7fdbb8100050 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbb7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbb8100050 0x7fdbb8199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbb7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbb8100050 0x7fdbb8199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38340/0 (socket says 172.21.15.42:38340) 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbb7400640 1 -- 172.21.15.42:0/226956062 learned_addr learned my addr 172.21.15.42:0/226956062 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.852+0000 7fdbaea00640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbb81009b0 0x7fdbb819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.853+0000 7fdbb7e00640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb8101350 0x7fdbb81a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.853+0000 7fdbb7400640 1 -- 172.21.15.42:0/226956062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbb81009b0 msgr2=0x7fdbb819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.853+0000 7fdbb7400640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbb81009b0 0x7fdbb819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.853+0000 7fdbb7400640 1 -- 172.21.15.42:0/226956062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb8101350 msgr2=0x7fdbb81a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.853+0000 7fdbb7400640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb8101350 0x7fdbb81a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.853+0000 7fdbb7400640 1 -- 172.21.15.42:0/226956062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbb81a1850 con 0x7fdbb8100050 2024-04-25T00:26:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.853+0000 7fdbb7400640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbb8100050 0x7fdbb8199c40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdba802aca0 tx=0x7fdba80305d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.853+0000 7fdbb4c00640 1 -- 172.21.15.42:0/226956062 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8041020 con 0x7fdbb8100050 2024-04-25T00:26:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.853+0000 7fdbb4c00640 1 -- 172.21.15.42:0/226956062 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba8031920 con 0x7fdbb8100050 2024-04-25T00:26:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.854+0000 7fdbb4c00640 1 -- 172.21.15.42:0/226956062 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba8036040 con 0x7fdbb8100050 2024-04-25T00:26:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.854+0000 7fdbbe400640 1 -- 172.21.15.42:0/226956062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbb81a1ae0 con 0x7fdbb8100050 2024-04-25T00:26:32.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.854+0000 7fdbbe400640 1 -- 172.21.15.42:0/226956062 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbb81a1f80 con 0x7fdbb8100050 2024-04-25T00:26:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.857+0000 7fdbc1393740 1 Processor -- start 2024-04-25T00:26:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.857+0000 7fdbb4c00640 1 -- 172.21.15.42:0/226956062 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdba8044020 con 0x7fdbb8100050 2024-04-25T00:26:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbc1393740 1 -- start start 2024-04-25T00:26:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d309fca0 0x55c0d30a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30a5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d30a7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbc1393740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c0d3002ab0 con 0x55c0d309fca0 2024-04-25T00:26:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbc1393740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c0d2fab640 con 0x55c0d30a5800 2024-04-25T00:26:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbc1393740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c0d2ffea70 con 0x55c0d30a0690 2024-04-25T00:26:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbb7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbaea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d30a7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbaea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d30a7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56850/0 (socket says 172.21.15.42:56850) 2024-04-25T00:26:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbb7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d309fca0 0x55c0d30a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbb7400640 1 --2- 172.21.15.42:0/2353300975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30a5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38350/0 (socket says 172.21.15.42:38350) 2024-04-25T00:26:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.858+0000 7fdbaea00640 1 -- 172.21.15.42:0/2353300975 learned_addr learned my addr 172.21.15.42:0/2353300975 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.859+0000 7fdbaea00640 1 -- 172.21.15.42:0/2353300975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 msgr2=0x55c0d30a5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.859+0000 7fdbaea00640 1 --2- 172.21.15.42:0/2353300975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30a5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.859+0000 7fdbaea00640 1 -- 172.21.15.42:0/2353300975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d309fca0 msgr2=0x55c0d30a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.859+0000 7fdbaea00640 1 --2- 172.21.15.42:0/2353300975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d309fca0 0x55c0d30a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.859+0000 7fdbaea00640 1 -- 172.21.15.42:0/2353300975 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d2d17210 con 0x55c0d30a5800 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.859+0000 7fdbaea00640 1 --2- 172.21.15.42:0/2353300975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d30a7c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009ce0 tx=0x7fdba0011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=501078b39288171c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.859+0000 7fdbb4c00640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdb9c0a2cb0 0x7fdb9c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.860+0000 7fdbacc00640 1 -- 172.21.15.42:0/2353300975 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba001e070 con 0x55c0d30a5800 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.860+0000 7fdbacc00640 1 -- 172.21.15.42:0/2353300975 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba0019b10 con 0x55c0d30a5800 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.860+0000 7fdbacc00640 1 -- 172.21.15.42:0/2353300975 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba000eaa0 con 0x55c0d30a5800 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.860+0000 7fdbb4c00640 1 -- 172.21.15.42:0/226956062 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fdba80efe90 con 0x7fdbb8100050 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.860+0000 7fdbb7e00640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdb9c0a2cb0 0x7fdb9c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.860+0000 7fdbc1393740 1 -- 172.21.15.42:0/2353300975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 msgr2=0x55c0d30a7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.860+0000 7fdbc1393740 1 --2- 172.21.15.42:0/2353300975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d30a7c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdba0009ce0 tx=0x7fdba0011810 comp rx=0 tx=0).stop 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.861+0000 7fdbb7e00640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdb9c0a2cb0 0x7fdb9c0a5170 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55c0d30a0ef0 tx=0x7fdbb0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.861+0000 7fdbc1393740 1 -- 172.21.15.42:0/2353300975 shutdown_connections 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.861+0000 7fdbc1393740 1 --2- 172.21.15.42:0/2353300975 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d30a7c90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.861+0000 7fdbc1393740 1 --2- 172.21.15.42:0/2353300975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30a5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.861+0000 7fdbc1393740 1 --2- 172.21.15.42:0/2353300975 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d309fca0 0x55c0d30a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.861+0000 7fdbc1393740 1 -- 172.21.15.42:0/2353300975 >> 172.21.15.42:0/2353300975 conn(0x55c0d309b310 msgr2=0x55c0d309b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.861+0000 7fdbc1393740 1 -- 172.21.15.42:0/2353300975 shutdown_connections 2024-04-25T00:26:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.861+0000 7fdbc1393740 1 -- 172.21.15.42:0/2353300975 wait complete. 2024-04-25T00:26:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.861+0000 7fdbc1393740 1 -- start start 2024-04-25T00:26:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.861+0000 7fdbc1393740 1 Processor -- start 2024-04-25T00:26:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d309fca0 0x55c0d30a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30aa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d30b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbc1393740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c0d30a87b0 con 0x55c0d309fca0 2024-04-25T00:26:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbc1393740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c0d2fbe990 con 0x55c0d30a5800 2024-04-25T00:26:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbc1393740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c0d30a8930 con 0x55c0d30a0690 2024-04-25T00:26:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbc1393740 10 client.0 _ll_register_callbacks cb 0x55c0d2fc0950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:32.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbb7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d309fca0 0x55c0d30a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbaea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbaea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38358/0 (socket says 172.21.15.42:38358) 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbaea00640 1 -- 172.21.15.42:0/364953838 learned_addr learned my addr 172.21.15.42:0/364953838 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.862+0000 7fdbb7e00640 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d30b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.863+0000 7fdbaea00640 1 -- 172.21.15.42:0/364953838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 msgr2=0x55c0d30b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.863+0000 7fdbaea00640 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d30b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.863+0000 7fdbaea00640 1 -- 172.21.15.42:0/364953838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d309fca0 msgr2=0x55c0d30a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.863+0000 7fdbaea00640 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d309fca0 0x55c0d30a9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.863+0000 7fdbaea00640 1 -- 172.21.15.42:0/364953838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d2de3620 con 0x55c0d30a0690 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.863+0000 7fdbaea00640 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30aa450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdba0011d10 tx=0x7fdba0009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d371c0cbd48d8859 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.863+0000 7fdba5600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.863+0000 7fdba5600640 1 -- 172.21.15.42:0/364953838 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba001e050 con 0x55c0d30a0690 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.864+0000 7fdbc1393740 1 -- 172.21.15.42:0/364953838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba0019b10 con 0x55c0d30a0690 2024-04-25T00:26:32.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.864+0000 7fdba5600640 1 -- 172.21.15.42:0/364953838 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fdba0005250 con 0x55c0d30a0690 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.864+0000 7fdba5600640 1 -- 172.21.15.42:0/364953838 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdba0005450 con 0x55c0d30a0690 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.864+0000 7fdbc1393740 10 client.25807 fetch_fsmap learned FSMap version 29 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.864+0000 7fdbc1393740 1 -- 172.21.15.42:0/364953838 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0d30b4470 con 0x55c0d30a0690 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.864+0000 7fdba5600640 1 -- 172.21.15.42:0/364953838 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba0019b10 con 0x55c0d30a0690 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.864+0000 7fdba5600640 1 -- 172.21.15.42:0/364953838 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fdba00046c0 con 0x55c0d30a0690 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.864+0000 7fdbc1393740 10 client.25807 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.864+0000 7fdbc1393740 10 client.25807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.864+0000 7fdbc1393740 20 client.25807 populate_metadata read hostname 'smithi042' 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.865+0000 7fdbc1393740 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c0d30b59a0 0x55c0d30b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.865+0000 7fdbc1393740 4 client.25807 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.865+0000 7fdbc1393740 1 -- 172.21.15.42:0/364953838 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0d300fff0 con 0x55c0d30b59a0 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.865+0000 7fdbb7400640 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c0d30b59a0 0x55c0d30b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.865+0000 7fdbb7400640 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c0d30b59a0 0x55c0d30b7d90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d607c9c85a85597d server_cookie=b32b9e1b1c86cd2 in_seq=0 out_seq=0 2024-04-25T00:26:32.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.865+0000 7fdba5600640 10 client.25807 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdba5600640 1 -- 172.21.15.42:0/364953838 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c0d300fff0 con 0x55c0d30b59a0 2024-04-25T00:26:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdba5600640 10 client.25807 handle_command_reply: tid=0 2024-04-25T00:26:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdbc1393740 1 client.25807 shutdown 2024-04-25T00:26:32.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdbc1393740 1 -- 172.21.15.42:0/364953838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 msgr2=0x55c0d30aa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdbc1393740 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30aa450 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdba0011d10 tx=0x7fdba0009910 comp rx=0 tx=0).stop 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdbc1393740 1 -- 172.21.15.42:0/364953838 shutdown_connections 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdbc1393740 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c0d30b59a0 0x55c0d30b7d90 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdbc1393740 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d30b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdbc1393740 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a0690 0x55c0d30aa450 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdbc1393740 1 --2- 172.21.15.42:0/364953838 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d309fca0 0x55c0d30a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdbc1393740 1 -- 172.21.15.42:0/364953838 >> 172.21.15.42:0/364953838 conn(0x55c0d309b310 msgr2=0x55c0d309a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.866+0000 7fdba5600640 0 client.25807 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.867+0000 7fdbc1393740 1 -- 172.21.15.42:0/364953838 shutdown_connections 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.867+0000 7fdbc1393740 1 -- 172.21.15.42:0/364953838 wait complete. 2024-04-25T00:26:32.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.867+0000 7fdbc1393740 20 client.25807 trim_cache size 0 max 16384 2024-04-25T00:26:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.885+0000 7fdbc1393740 1 Processor -- start 2024-04-25T00:26:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.885+0000 7fdbc1393740 1 -- start start 2024-04-25T00:26:32.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.885+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d309fca0 0x55c0d309dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.885+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d30a0690 0x55c0d30b1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.885+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a5800 0x55c0d309e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.885+0000 7fdbc1393740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81ad5a0 con 0x55c0d30a0690 2024-04-25T00:26:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.885+0000 7fdbc1393740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81aae70 con 0x55c0d309fca0 2024-04-25T00:26:32.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.885+0000 7fdbc1393740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81a8cc0 con 0x55c0d30a5800 2024-04-25T00:26:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbb7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d309fca0 0x55c0d309dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbb7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d309fca0 0x55c0d309dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56858/0 (socket says 172.21.15.42:56858) 2024-04-25T00:26:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbb7400640 1 -- 172.21.15.42:0/2178029735 learned_addr learned my addr 172.21.15.42:0/2178029735 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:32.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbaea00640 1 --2- 172.21.15.42:0/2178029735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d30a0690 0x55c0d30b1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbb7e00640 1 --2- 172.21.15.42:0/2178029735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a5800 0x55c0d309e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbb7400640 1 -- 172.21.15.42:0/2178029735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a5800 msgr2=0x55c0d309e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbb7400640 1 --2- 172.21.15.42:0/2178029735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a5800 0x55c0d309e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbb7400640 1 -- 172.21.15.42:0/2178029735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d30a0690 msgr2=0x55c0d30b1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbb7400640 1 --2- 172.21.15.42:0/2178029735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d30a0690 0x55c0d30b1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbb7400640 1 -- 172.21.15.42:0/2178029735 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d309b140 con 0x55c0d309fca0 2024-04-25T00:26:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdbb7400640 1 --2- 172.21.15.42:0/2178029735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d309fca0 0x55c0d309dff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdba80bc420 tx=0x7fdba80ed890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb9cd1dafabb8e88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdba7e00640 1 -- 172.21.15.42:0/2178029735 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80b8930 con 0x55c0d309fca0 2024-04-25T00:26:32.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.886+0000 7fdba7e00640 1 -- 172.21.15.42:0/2178029735 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba80a70b0 con 0x55c0d309fca0 2024-04-25T00:26:32.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.887+0000 7fdba7e00640 1 -- 172.21.15.42:0/2178029735 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80ba860 con 0x55c0d309fca0 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.887+0000 7fdbc1393740 1 -- 172.21.15.42:0/2178029735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d309fca0 msgr2=0x55c0d309dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.887+0000 7fdbc1393740 1 --2- 172.21.15.42:0/2178029735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d309fca0 0x55c0d309dff0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdba80bc420 tx=0x7fdba80ed890 comp rx=0 tx=0).stop 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.887+0000 7fdbc1393740 1 -- 172.21.15.42:0/2178029735 shutdown_connections 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.887+0000 7fdbc1393740 1 --2- 172.21.15.42:0/2178029735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d30a5800 0x55c0d309e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.887+0000 7fdbc1393740 1 --2- 172.21.15.42:0/2178029735 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d30a0690 0x55c0d30b1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.887+0000 7fdbc1393740 1 --2- 172.21.15.42:0/2178029735 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d309fca0 0x55c0d309dff0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.887+0000 7fdbc1393740 1 -- 172.21.15.42:0/2178029735 >> 172.21.15.42:0/2178029735 conn(0x55c0d309b310 msgr2=0x55c0d30a1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.888+0000 7fdbc1393740 1 -- 172.21.15.42:0/2178029735 shutdown_connections 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.888+0000 7fdbc1393740 1 -- 172.21.15.42:0/2178029735 wait complete. 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.888+0000 7fdbc1393740 1 -- start start 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.888+0000 7fdbc1393740 1 Processor -- start 2024-04-25T00:26:32.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d309fca0 0x55c0d30ae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d30a0690 0x55c0d30ae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbc1393740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d3119ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbc1393740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81a9330 con 0x55c0d30a0690 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbc1393740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81aa7a0 con 0x55c0d30a5800 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbc1393740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdbb81a9a00 con 0x55c0d309fca0 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbc1393740 10 client.0 _ll_register_callbacks cb 0x7fdba001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbb7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d309fca0 0x55c0d30ae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbaea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d30a0690 0x55c0d30ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbb7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d309fca0 0x55c0d30ae410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38368/0 (socket says 172.21.15.42:38368) 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbb7400640 1 -- 172.21.15.42:0/282192038 learned_addr learned my addr 172.21.15.42:0/282192038 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.889+0000 7fdbb7e00640 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d3119ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdbb7400640 1 -- 172.21.15.42:0/282192038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 msgr2=0x55c0d3119ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdbb7400640 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d3119ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdbb7400640 1 -- 172.21.15.42:0/282192038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d30a0690 msgr2=0x55c0d30ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdbb7400640 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d30a0690 0x55c0d30ae950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdbb7400640 1 -- 172.21.15.42:0/282192038 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d311a480 con 0x55c0d309fca0 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdbb7400640 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d309fca0 0x55c0d30ae410 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdba80eda10 tx=0x7fdba8004040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc9b386f72c333fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdba5600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdba5600640 1 -- 172.21.15.42:0/282192038 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdba80bc080 con 0x55c0d309fca0 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdba5600640 1 -- 172.21.15.42:0/282192038 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fdba8003c90 con 0x55c0d309fca0 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdbc1393740 1 -- 172.21.15.42:0/282192038 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdba80a70b0 con 0x55c0d309fca0 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdba5600640 1 -- 172.21.15.42:0/282192038 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdba80bc260 con 0x55c0d309fca0 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdbc1393740 10 client.25813 fetch_fsmap learned FSMap version 29 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.890+0000 7fdbc1393740 1 -- 172.21.15.42:0/282192038 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c0d311b790 con 0x55c0d309fca0 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.891+0000 7fdba5600640 1 -- 172.21.15.42:0/282192038 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdba80a70b0 con 0x55c0d309fca0 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.891+0000 7fdba5600640 1 -- 172.21.15.42:0/282192038 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 29) ==== 8338+0+0 (secure 0 0 0) 0x7fdba8004600 con 0x55c0d309fca0 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.891+0000 7fdbc1393740 10 client.25813 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.891+0000 7fdbc1393740 10 client.25813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.891+0000 7fdbc1393740 20 client.25813 populate_metadata read hostname 'smithi042' 2024-04-25T00:26:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.891+0000 7fdbc1393740 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c0d311c120 0x55c0d311e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.891+0000 7fdbc1393740 4 client.25813 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:26:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.891+0000 7fdbc1393740 1 -- 172.21.15.42:0/282192038 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0d300a340 con 0x55c0d311c120 2024-04-25T00:26:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.891+0000 7fdbaea00640 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c0d311c120 0x55c0d311e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.892+0000 7fdbaea00640 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c0d311c120 0x55c0d311e540 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=530fb4690bdfcdb server_cookie=7be7859cca4dbdb9 in_seq=0 out_seq=0 2024-04-25T00:26:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.892+0000 7fdba5600640 10 client.25813 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdba5600640 1 -- 172.21.15.42:0/282192038 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0d300a340 con 0x55c0d311c120 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdba5600640 10 client.25813 handle_command_reply: tid=0 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 client.25813 shutdown 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 -- 172.21.15.42:0/282192038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d309fca0 msgr2=0x55c0d30ae410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d309fca0 0x55c0d30ae410 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdba80eda10 tx=0x7fdba8004040 comp rx=0 tx=0).stop 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 -- 172.21.15.42:0/282192038 shutdown_connections 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c0d311c120 0x55c0d311e540 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c0d30a5800 0x55c0d3119ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c0d30a0690 0x55c0d30ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 --2- 172.21.15.42:0/282192038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0d309fca0 0x55c0d30ae410 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdba5600640 0 client.25813 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 -- 172.21.15.42:0/282192038 >> 172.21.15.42:0/282192038 conn(0x55c0d309b310 msgr2=0x55c0d309dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 -- 172.21.15.42:0/282192038 shutdown_connections 2024-04-25T00:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.940+0000 7fdbc1393740 1 -- 172.21.15.42:0/282192038 wait complete. 2024-04-25T00:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.941+0000 7fdbc1393740 20 client.25813 trim_cache size 0 max 16384 2024-04-25T00:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:26:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.941+0000 7fdbbe400640 1 -- 172.21.15.42:0/226956062 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdb9c0a2cb0 msgr2=0x7fdb9c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.941+0000 7fdbbe400640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdb9c0a2cb0 0x7fdb9c0a5170 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55c0d30a0ef0 tx=0x7fdbb0005f90 comp rx=0 tx=0).stop 2024-04-25T00:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.941+0000 7fdbbe400640 1 -- 172.21.15.42:0/226956062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbb8100050 msgr2=0x7fdbb8199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.941+0000 7fdbbe400640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbb8100050 0x7fdbb8199c40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdba802aca0 tx=0x7fdba80305d0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.941+0000 7fdbbe400640 1 -- 172.21.15.42:0/226956062 shutdown_connections 2024-04-25T00:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.941+0000 7fdbbe400640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdb9c0a2cb0 0x7fdb9c0a5170 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.941+0000 7fdbbe400640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdbb8101350 0x7fdbb81a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.941+0000 7fdbbe400640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdbb81009b0 0x7fdbb819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.941+0000 7fdbbe400640 1 --2- 172.21.15.42:0/226956062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdbb8100050 0x7fdbb8199c40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.942+0000 7fdbbe400640 1 -- 172.21.15.42:0/226956062 >> 172.21.15.42:0/226956062 conn(0x7fdbb80fbc10 msgr2=0x7fdbb80fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.942+0000 7fdbbe400640 1 -- 172.21.15.42:0/226956062 shutdown_connections 2024-04-25T00:26:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:32.942+0000 7fdbbe400640 1 -- 172.21.15.42:0/226956062 wait complete. 2024-04-25T00:26:33.247 INFO:tasks.workunit.client.0.smithi042.stdout: 1 459856 6.35 MB/sec execute 263 sec latency 23.462 ms 2024-04-25T00:26:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:33 smithi136 ceph-mon[32923]: pgmap v622: 97 pgs: 97 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-04-25T00:26:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:33 smithi106 ceph-mon[32510]: pgmap v622: 97 pgs: 97 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-04-25T00:26:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:33 smithi042 ceph-mon[28130]: pgmap v622: 97 pgs: 97 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 947 KiB/s rd, 2.1 MiB/s wr, 229 op/s 2024-04-25T00:26:34.247 INFO:tasks.workunit.client.0.smithi042.stdout: 1 461605 6.36 MB/sec execute 264 sec latency 20.005 ms 2024-04-25T00:26:35.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:34 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:26:35.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:34 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:26:35.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:34 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:26:35.247 INFO:tasks.workunit.client.0.smithi042.stdout: 1 461837 6.35 MB/sec execute 265 sec latency 894.061 ms 2024-04-25T00:26:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:35 smithi136 ceph-mon[32923]: pgmap v623: 97 pgs: 97 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-04-25T00:26:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:35 smithi106 ceph-mon[32510]: pgmap v623: 97 pgs: 97 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-04-25T00:26:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:35 smithi042 ceph-mon[28130]: pgmap v623: 97 pgs: 97 active+clean; 99 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 1.6 MiB/s wr, 139 op/s 2024-04-25T00:26:36.248 INFO:tasks.workunit.client.0.smithi042.stdout: 1 461837 6.33 MB/sec execute 266 sec latency 1894.169 ms 2024-04-25T00:26:37.248 INFO:tasks.workunit.client.0.smithi042.stdout: 1 461837 6.30 MB/sec execute 267 sec latency 2894.276 ms 2024-04-25T00:26:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:37 smithi136 ceph-mon[32923]: pgmap v624: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-04-25T00:26:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:37 smithi106 ceph-mon[32510]: pgmap v624: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-04-25T00:26:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:37 smithi042 ceph-mon[28130]: pgmap v624: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 1.9 MiB/s wr, 206 op/s 2024-04-25T00:26:38.248 INFO:tasks.workunit.client.0.smithi042.stdout: 1 461837 6.28 MB/sec execute 268 sec latency 3894.419 ms 2024-04-25T00:26:39.248 INFO:tasks.workunit.client.0.smithi042.stdout: 1 462271 6.26 MB/sec execute 269 sec latency 4169.464 ms 2024-04-25T00:26:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:39 smithi106 ceph-mon[32510]: pgmap v625: 97 pgs: 97 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-04-25T00:26:40.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:39 smithi042 ceph-mon[28130]: pgmap v625: 97 pgs: 97 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-04-25T00:26:40.248 INFO:tasks.workunit.client.0.smithi042.stdout: 1 462953 6.25 MB/sec execute 270 sec latency 27.701 ms 2024-04-25T00:26:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:39 smithi136 ceph-mon[32923]: pgmap v625: 97 pgs: 97 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 524 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-04-25T00:26:41.248 INFO:tasks.workunit.client.0.smithi042.stdout: 1 464357 6.24 MB/sec execute 271 sec latency 43.255 ms 2024-04-25T00:26:42.065 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:41 smithi136 ceph-mon[32923]: pgmap v626: 97 pgs: 97 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-04-25T00:26:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:41 smithi106 ceph-mon[32510]: pgmap v626: 97 pgs: 97 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-04-25T00:26:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:41 smithi042 ceph-mon[28130]: pgmap v626: 97 pgs: 97 active+clean; 82 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.3 MiB/s wr, 150 op/s 2024-04-25T00:26:42.248 INFO:tasks.workunit.client.0.smithi042.stdout: 1 465756 6.26 MB/sec execute 272 sec latency 18.106 ms 2024-04-25T00:26:43.248 INFO:tasks.workunit.client.0.smithi042.stdout: 1 466491 6.25 MB/sec execute 273 sec latency 19.508 ms 2024-04-25T00:26:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:43 smithi106 ceph-mon[32510]: pgmap v627: 97 pgs: 97 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-04-25T00:26:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:43 smithi042 ceph-mon[28130]: pgmap v627: 97 pgs: 97 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-04-25T00:26:44.248 INFO:tasks.workunit.client.0.smithi042.stdout: 1 467932 6.24 MB/sec execute 274 sec latency 26.762 ms 2024-04-25T00:26:44.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:43 smithi136 ceph-mon[32923]: pgmap v627: 97 pgs: 97 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-04-25T00:26:45.249 INFO:tasks.workunit.client.0.smithi042.stdout: 1 469282 6.26 MB/sec execute 275 sec latency 41.562 ms 2024-04-25T00:26:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:45 smithi106 ceph-mon[32510]: pgmap v628: 97 pgs: 97 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-04-25T00:26:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.098+0000 7f7223800640 1 -- 172.21.15.42:0/291597108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f721c101890 msgr2=0x7f721c101d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.098+0000 7f7223800640 1 --2- 172.21.15.42:0/291597108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f721c101890 0x7f721c101d10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7208009900 tx=0x7f720802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.099+0000 7f7223800640 1 -- 172.21.15.42:0/291597108 shutdown_connections 2024-04-25T00:26:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.099+0000 7f7223800640 1 --2- 172.21.15.42:0/291597108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f721c102250 0x7f721c108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.099+0000 7f7223800640 1 --2- 172.21.15.42:0/291597108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f721c101890 0x7f721c101d10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.099+0000 7f7223800640 1 --2- 172.21.15.42:0/291597108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c1006a0 0x7f721c100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.099+0000 7f7223800640 1 -- 172.21.15.42:0/291597108 >> 172.21.15.42:0/291597108 conn(0x7f721c0fbc10 msgr2=0x7f721c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.099+0000 7f7223800640 1 -- 172.21.15.42:0/291597108 shutdown_connections 2024-04-25T00:26:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.099+0000 7f7223800640 1 -- 172.21.15.42:0/291597108 wait complete. 2024-04-25T00:26:46.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.100+0000 7f7223800640 1 Processor -- start 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.100+0000 7f7223800640 1 -- start start 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7223800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f721c1006a0 0x7f721c197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7223800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f721c101890 0x7f721c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7223800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c102250 0x7f721c19f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7223800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10c370 con 0x7f721c102250 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7223800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10c1f0 con 0x7f721c101890 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7223800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f721c10c4f0 con 0x7f721c1006a0 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f721be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f721c101890 0x7f721c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7221800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c102250 0x7f721c19f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7220e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f721c1006a0 0x7f721c197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f721be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f721c101890 0x7f721c197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53498/0 (socket says 172.21.15.42:53498) 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f721be00640 1 -- 172.21.15.42:0/1290645142 learned_addr learned my addr 172.21.15.42:0/1290645142 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7220e00640 1 -- 172.21.15.42:0/1290645142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f721c101890 msgr2=0x7f721c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7220e00640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f721c101890 0x7f721c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7220e00640 1 -- 172.21.15.42:0/1290645142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c102250 msgr2=0x7f721c19f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.101+0000 7f7220e00640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c102250 0x7f721c19f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.102+0000 7f7220e00640 1 -- 172.21.15.42:0/1290645142 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f721c19f670 con 0x7f721c1006a0 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.102+0000 7f7221800640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c102250 0x7f721c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.102+0000 7f721be00640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f721c101890 0x7f721c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.102+0000 7f7220e00640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f721c1006a0 0x7f721c197aa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f721000e9c0 tx=0x7f721000ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.102+0000 7f7219600640 1 -- 172.21.15.42:0/1290645142 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721000ccc0 con 0x7f721c1006a0 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.102+0000 7f7219600640 1 -- 172.21.15.42:0/1290645142 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72100035f0 con 0x7f721c1006a0 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.102+0000 7f7219600640 1 -- 172.21.15.42:0/1290645142 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f721000ce60 con 0x7f721c1006a0 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.102+0000 7f7223800640 1 -- 172.21.15.42:0/1290645142 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f721c19f960 con 0x7f721c1006a0 2024-04-25T00:26:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.102+0000 7f7223800640 1 -- 172.21.15.42:0/1290645142 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f721c19ff00 con 0x7f721c1006a0 2024-04-25T00:26:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.106+0000 7f7219600640 1 -- 172.21.15.42:0/1290645142 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72100037a0 con 0x7f721c1006a0 2024-04-25T00:26:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f722660a740 1 Processor -- start 2024-04-25T00:26:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f722660a740 1 -- start start 2024-04-25T00:26:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f722660a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 0x55e66f49c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f722660a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49caa0 0x55e66f4a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f722660a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f4a1c10 0x55e66f4a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f722660a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e66f3faab0 con 0x55e66f49c0b0 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f722660a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e66f3a3640 con 0x55e66f4a1c10 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f722660a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e66f3f6a70 con 0x55e66f49caa0 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f7221800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 0x55e66f49c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f7220e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49caa0 0x55e66f4a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.107+0000 7f721be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f4a1c10 0x55e66f4a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f7221800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 0x55e66f49c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39356/0 (socket says 172.21.15.42:39356) 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f7221800640 1 -- 172.21.15.42:0/548533002 learned_addr learned my addr 172.21.15.42:0/548533002 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f721be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f4a1c10 0x55e66f4a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53512/0 (socket says 172.21.15.42:53512) 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f7219600640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72040a2cb0 0x7f72040a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f721be00640 1 -- 172.21.15.42:0/548533002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49caa0 msgr2=0x55e66f4a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f721be00640 1 --2- 172.21.15.42:0/548533002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49caa0 0x55e66f4a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f721be00640 1 -- 172.21.15.42:0/548533002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 msgr2=0x55e66f49c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f721be00640 1 --2- 172.21.15.42:0/548533002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 0x55e66f49c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f721be00640 1 -- 172.21.15.42:0/548533002 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e66f10f210 con 0x55e66f4a1c10 2024-04-25T00:26:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f7219600640 1 -- 172.21.15.42:0/1290645142 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f72100cf380 con 0x7f721c1006a0 2024-04-25T00:26:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f7221800640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72040a2cb0 0x7f72040a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.108+0000 7f721be00640 1 --2- 172.21.15.42:0/548533002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f4a1c10 0x55e66f4a40a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7208006010 tx=0x7f72080043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfdbf71505083eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.109+0000 7f720e000640 1 -- 172.21.15.42:0/548533002 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208040070 con 0x55e66f4a1c10 2024-04-25T00:26:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.109+0000 7f720e000640 1 -- 172.21.15.42:0/548533002 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72080047a0 con 0x55e66f4a1c10 2024-04-25T00:26:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.109+0000 7f720e000640 1 -- 172.21.15.42:0/548533002 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7208035040 con 0x55e66f4a1c10 2024-04-25T00:26:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f722660a740 1 -- 172.21.15.42:0/548533002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f4a1c10 msgr2=0x55e66f4a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f722660a740 1 --2- 172.21.15.42:0/548533002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f4a1c10 0x55e66f4a40a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7208006010 tx=0x7f72080043f0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f7221800640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72040a2cb0 0x7f72040a5170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55e66f49d300 tx=0x7f7214004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f722660a740 1 -- 172.21.15.42:0/548533002 shutdown_connections 2024-04-25T00:26:46.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f722660a740 1 --2- 172.21.15.42:0/548533002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f4a1c10 0x55e66f4a40a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f722660a740 1 --2- 172.21.15.42:0/548533002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49caa0 0x55e66f4a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f722660a740 1 --2- 172.21.15.42:0/548533002 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 0x55e66f49c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f722660a740 1 -- 172.21.15.42:0/548533002 >> 172.21.15.42:0/548533002 conn(0x55e66f493310 msgr2=0x55e66f493710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f722660a740 1 -- 172.21.15.42:0/548533002 shutdown_connections 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f722660a740 1 -- 172.21.15.42:0/548533002 wait complete. 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.110+0000 7f722660a740 1 -- start start 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f722660a740 1 Processor -- start 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f722660a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49c0b0 0x55e66f4a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f722660a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4a6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f722660a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f4a1c10 0x55e66f4ad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f722660a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e66f4a4bb0 con 0x55e66f4a1c10 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f722660a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e66f3b6990 con 0x55e66f49caa0 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f722660a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e66f4a4d30 con 0x55e66f49c0b0 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f722660a740 10 client.0 _ll_register_callbacks cb 0x55e66f3b8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:46.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f7220e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49c0b0 0x55e66f4a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f721be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f7221800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f4a1c10 0x55e66f4ad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f7220e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49c0b0 0x55e66f4a62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:40706/0 (socket says 172.21.15.42:40706) 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.111+0000 7f7220e00640 1 -- 172.21.15.42:0/4102490292 learned_addr learned my addr 172.21.15.42:0/4102490292 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.112+0000 7f721be00640 1 -- 172.21.15.42:0/4102490292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49c0b0 msgr2=0x55e66f4a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.112+0000 7f721be00640 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49c0b0 0x55e66f4a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.112+0000 7f721be00640 1 -- 172.21.15.42:0/4102490292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f4a1c10 msgr2=0x55e66f4ad860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.112+0000 7f721be00640 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f4a1c10 0x55e66f4ad860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.112+0000 7f721be00640 1 -- 172.21.15.42:0/4102490292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e66f1db620 con 0x55e66f49caa0 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.112+0000 7f7220e00640 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49c0b0 0x55e66f4a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.112+0000 7f721be00640 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4a6820 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7208031440 tx=0x7f7208031a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4fda9501ef9a7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.112+0000 7f7202a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.112+0000 7f7202a00640 1 -- 172.21.15.42:0/4102490292 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720804c020 con 0x55e66f49caa0 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.113+0000 7f722660a740 1 -- 172.21.15.42:0/4102490292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72080047a0 con 0x55e66f49caa0 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.113+0000 7f7202a00640 1 -- 172.21.15.42:0/4102490292 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f72080037f0 con 0x55e66f49caa0 2024-04-25T00:26:46.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.113+0000 7f7202a00640 1 -- 172.21.15.42:0/4102490292 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7208040070 con 0x55e66f49caa0 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.113+0000 7f722660a740 10 client.25817 fetch_fsmap learned FSMap version 30 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.113+0000 7f722660a740 1 -- 172.21.15.42:0/4102490292 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e66f4b0840 con 0x55e66f49caa0 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.113+0000 7f7202a00640 1 -- 172.21.15.42:0/4102490292 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f72080047a0 con 0x55e66f49caa0 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.113+0000 7f7202a00640 1 -- 172.21.15.42:0/4102490292 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f720803c9b0 con 0x55e66f49caa0 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.113+0000 7f722660a740 10 client.25817 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.113+0000 7f722660a740 10 client.25817 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.114+0000 7f722660a740 20 client.25817 populate_metadata read hostname 'smithi042' 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.114+0000 7f722660a740 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e66f4b1b90 0x55e66f4b3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.114+0000 7f722660a740 4 client.25817 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.114+0000 7f722660a740 1 -- 172.21.15.42:0/4102490292 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e66f407ff0 con 0x55e66f4b1b90 2024-04-25T00:26:46.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.114+0000 7f7220e00640 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e66f4b1b90 0x55e66f4b3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.114+0000 7f7202a00640 10 client.25817 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.114+0000 7f7220e00640 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e66f4b1b90 0x55e66f4b3f80 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf5b689936f5ec69 server_cookie=47ab5ebb6904a4d0 in_seq=0 out_seq=0 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.115+0000 7f7202a00640 1 -- 172.21.15.42:0/4102490292 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e66f407ff0 con 0x55e66f4b1b90 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.115+0000 7f7202a00640 10 client.25817 handle_command_reply: tid=0 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.115+0000 7f722660a740 1 client.25817 shutdown 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.115+0000 7f722660a740 1 -- 172.21.15.42:0/4102490292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 msgr2=0x55e66f4a6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.115+0000 7f722660a740 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4a6820 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f7208031440 tx=0x7f7208031a80 comp rx=0 tx=0).stop 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.116+0000 7f722660a740 1 -- 172.21.15.42:0/4102490292 shutdown_connections 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.116+0000 7f722660a740 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e66f4b1b90 0x55e66f4b3f80 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.116+0000 7f7202a00640 0 client.25817 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.116+0000 7f722660a740 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f4a1c10 0x55e66f4ad860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.116+0000 7f722660a740 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4a6820 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.116+0000 7f722660a740 1 --2- 172.21.15.42:0/4102490292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f49c0b0 0x55e66f4a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.116+0000 7f722660a740 1 -- 172.21.15.42:0/4102490292 >> 172.21.15.42:0/4102490292 conn(0x55e66f493310 msgr2=0x55e66f492480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.116+0000 7f722660a740 1 -- 172.21.15.42:0/4102490292 shutdown_connections 2024-04-25T00:26:46.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.116+0000 7f722660a740 1 -- 172.21.15.42:0/4102490292 wait complete. 2024-04-25T00:26:46.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.116+0000 7f722660a740 20 client.25817 trim_cache size 0 max 16384 2024-04-25T00:26:46.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:45 smithi042 ceph-mon[28130]: pgmap v628: 97 pgs: 97 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-04-25T00:26:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.135+0000 7f722660a740 1 Processor -- start 2024-04-25T00:26:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.135+0000 7f722660a740 1 -- start start 2024-04-25T00:26:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.135+0000 7f722660a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49c0b0 0x55e66f496900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.135+0000 7f722660a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49caa0 0x55e66f4ad820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.135+0000 7f722660a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 0x55e66f4b8690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.135+0000 7f722660a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1a91e0 con 0x55e66f49caa0 2024-04-25T00:26:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.135+0000 7f722660a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1a6a80 con 0x55e66f49c0b0 2024-04-25T00:26:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.135+0000 7f722660a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1a48d0 con 0x55e66f4a1c10 2024-04-25T00:26:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f7220e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49c0b0 0x55e66f496900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f7220e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49c0b0 0x55e66f496900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53528/0 (socket says 172.21.15.42:53528) 2024-04-25T00:26:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f7220e00640 1 -- 172.21.15.42:0/1763926071 learned_addr learned my addr 172.21.15.42:0/1763926071 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f7221800640 1 --2- 172.21.15.42:0/1763926071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 0x55e66f4b8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f721be00640 1 --2- 172.21.15.42:0/1763926071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49caa0 0x55e66f4ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f7220e00640 1 -- 172.21.15.42:0/1763926071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 msgr2=0x55e66f4b8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f7220e00640 1 --2- 172.21.15.42:0/1763926071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 0x55e66f4b8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f7220e00640 1 -- 172.21.15.42:0/1763926071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49caa0 msgr2=0x55e66f4ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f7220e00640 1 --2- 172.21.15.42:0/1763926071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49caa0 0x55e66f4ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f7220e00640 1 -- 172.21.15.42:0/1763926071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e66f493140 con 0x55e66f49c0b0 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.136+0000 7f7220e00640 1 --2- 172.21.15.42:0/1763926071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49c0b0 0x55e66f496900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7210098580 tx=0x7f7210005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f9b6103e590e24b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.137+0000 7f720d600640 1 -- 172.21.15.42:0/1763926071 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72100b2070 con 0x55e66f49c0b0 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.137+0000 7f720d600640 1 -- 172.21.15.42:0/1763926071 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f721009c2b0 con 0x55e66f49c0b0 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.137+0000 7f720d600640 1 -- 172.21.15.42:0/1763926071 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7210099f80 con 0x55e66f49c0b0 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.138+0000 7f722660a740 1 -- 172.21.15.42:0/1763926071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49c0b0 msgr2=0x55e66f496900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.138+0000 7f722660a740 1 --2- 172.21.15.42:0/1763926071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49c0b0 0x55e66f496900 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7210098580 tx=0x7f7210005f10 comp rx=0 tx=0).stop 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.138+0000 7f722660a740 1 -- 172.21.15.42:0/1763926071 shutdown_connections 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.138+0000 7f722660a740 1 --2- 172.21.15.42:0/1763926071 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 0x55e66f4b8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.138+0000 7f722660a740 1 --2- 172.21.15.42:0/1763926071 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49caa0 0x55e66f4ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.138+0000 7f722660a740 1 --2- 172.21.15.42:0/1763926071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49c0b0 0x55e66f496900 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.138+0000 7f722660a740 1 -- 172.21.15.42:0/1763926071 >> 172.21.15.42:0/1763926071 conn(0x55e66f493310 msgr2=0x55e66f4961f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.138+0000 7f722660a740 1 -- 172.21.15.42:0/1763926071 shutdown_connections 2024-04-25T00:26:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.138+0000 7f722660a740 1 -- 172.21.15.42:0/1763926071 wait complete. 2024-04-25T00:26:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.138+0000 7f722660a740 1 -- start start 2024-04-25T00:26:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f722660a740 1 Processor -- start 2024-04-25T00:26:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f722660a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 0x55e66f4ab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f722660a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4abc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f722660a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 0x55e66f4acc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f722660a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1a4f40 con 0x55e66f49c0b0 2024-04-25T00:26:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f722660a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1a63b0 con 0x55e66f49caa0 2024-04-25T00:26:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f722660a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f721c1a5610 con 0x55e66f4a1c10 2024-04-25T00:26:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f722660a740 10 client.0 _ll_register_callbacks cb 0x7f7214009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f721be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f721be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4abc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53534/0 (socket says 172.21.15.42:53534) 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f721be00640 1 -- 172.21.15.42:0/2979054900 learned_addr learned my addr 172.21.15.42:0/2979054900 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.139+0000 7f7221800640 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 0x55e66f4acc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f7220e00640 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 0x55e66f4ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f721be00640 1 -- 172.21.15.42:0/2979054900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 msgr2=0x55e66f4acc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f721be00640 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 0x55e66f4acc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f721be00640 1 -- 172.21.15.42:0/2979054900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 msgr2=0x55e66f4ab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f721be00640 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 0x55e66f4ab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f721be00640 1 -- 172.21.15.42:0/2979054900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e66f48e620 con 0x55e66f49caa0 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f7221800640 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 0x55e66f4acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f721be00640 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4abc80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f72080443e0 tx=0x7f720803cab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=728a2cadd0cfb208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f7202a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f7202a00640 1 -- 172.21.15.42:0/2979054900 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720804c040 con 0x55e66f49caa0 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.140+0000 7f7202a00640 1 -- 172.21.15.42:0/2979054900 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f7208005220 con 0x55e66f49caa0 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f722660a740 1 -- 172.21.15.42:0/2979054900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f721009c2b0 con 0x55e66f49caa0 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f7202a00640 1 -- 172.21.15.42:0/2979054900 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7208000e60 con 0x55e66f49caa0 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f722660a740 10 client.25823 fetch_fsmap learned FSMap version 30 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f722660a740 1 -- 172.21.15.42:0/2979054900 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55e66f4a6b90 con 0x55e66f49caa0 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f7202a00640 1 -- 172.21.15.42:0/2979054900 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f721009c2b0 con 0x55e66f49caa0 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f7202a00640 1 -- 172.21.15.42:0/2979054900 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f720805a030 con 0x55e66f49caa0 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f722660a740 10 client.25823 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f722660a740 10 client.25823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f722660a740 20 client.25823 populate_metadata read hostname 'smithi042' 2024-04-25T00:26:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f722660a740 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e66f4b1b90 0x55e66f51acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f722660a740 4 client.25823 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:26:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.141+0000 7f722660a740 1 -- 172.21.15.42:0/2979054900 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e66f402340 con 0x55e66f4b1b90 2024-04-25T00:26:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.142+0000 7f7220e00640 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e66f4b1b90 0x55e66f51acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.142+0000 7f7220e00640 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e66f4b1b90 0x55e66f51acb0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1b28ab71acf449 server_cookie=a3887f741eafe877 in_seq=0 out_seq=0 2024-04-25T00:26:46.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.142+0000 7f7202a00640 10 client.25823 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:46.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.180+0000 7f7202a00640 1 -- 172.21.15.42:0/2979054900 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e66f402340 con 0x55e66f4b1b90 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.180+0000 7f7202a00640 10 client.25823 handle_command_reply: tid=0 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 client.25823 shutdown 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 -- 172.21.15.42:0/2979054900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 msgr2=0x55e66f4abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4abc80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f72080443e0 tx=0x7f720803cab0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 -- 172.21.15.42:0/2979054900 shutdown_connections 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e66f4b1b90 0x55e66f51acb0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e66f4a1c10 0x55e66f4acc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e66f49caa0 0x55e66f4abc80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f7202a00640 0 client.25823 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 --2- 172.21.15.42:0/2979054900 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e66f49c0b0 0x55e66f4ab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 -- 172.21.15.42:0/2979054900 >> 172.21.15.42:0/2979054900 conn(0x55e66f493310 msgr2=0x55e66f491bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:46.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 -- 172.21.15.42:0/2979054900 shutdown_connections 2024-04-25T00:26:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 1 -- 172.21.15.42:0/2979054900 wait complete. 2024-04-25T00:26:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.181+0000 7f722660a740 20 client.25823 trim_cache size 0 max 16384 2024-04-25T00:26:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:26:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:26:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:26:46.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:26:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.182+0000 7f7223800640 1 -- 172.21.15.42:0/1290645142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72040a2cb0 msgr2=0x7f72040a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.182+0000 7f7223800640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72040a2cb0 0x7f72040a5170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x55e66f49d300 tx=0x7f7214004a40 comp rx=0 tx=0).stop 2024-04-25T00:26:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.182+0000 7f7223800640 1 -- 172.21.15.42:0/1290645142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f721c1006a0 msgr2=0x7f721c197aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.182+0000 7f7223800640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f721c1006a0 0x7f721c197aa0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f721000e9c0 tx=0x7f721000ee80 comp rx=0 tx=0).stop 2024-04-25T00:26:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.182+0000 7f7223800640 1 -- 172.21.15.42:0/1290645142 shutdown_connections 2024-04-25T00:26:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.182+0000 7f7223800640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f72040a2cb0 0x7f72040a5170 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.182+0000 7f7223800640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f721c102250 0x7f721c19f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.182+0000 7f7223800640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f721c101890 0x7f721c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.182+0000 7f7223800640 1 --2- 172.21.15.42:0/1290645142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f721c1006a0 0x7f721c197aa0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:46.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.182+0000 7f7223800640 1 -- 172.21.15.42:0/1290645142 >> 172.21.15.42:0/1290645142 conn(0x7f721c0fbc10 msgr2=0x7f721c0fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.183+0000 7f7223800640 1 -- 172.21.15.42:0/1290645142 shutdown_connections 2024-04-25T00:26:46.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:46.183+0000 7f7223800640 1 -- 172.21.15.42:0/1290645142 wait complete. 2024-04-25T00:26:46.249 INFO:tasks.workunit.client.0.smithi042.stdout: 1 470100 6.25 MB/sec execute 276 sec latency 34.368 ms 2024-04-25T00:26:46.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:45 smithi136 ceph-mon[32923]: pgmap v628: 97 pgs: 97 active+clean; 83 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-04-25T00:26:47.249 INFO:tasks.workunit.client.0.smithi042.stdout: 1 471445 6.24 MB/sec execute 277 sec latency 30.583 ms 2024-04-25T00:26:48.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:47 smithi106 ceph-mon[32510]: pgmap v629: 97 pgs: 97 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-04-25T00:26:48.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:47 smithi042 ceph-mon[28130]: pgmap v629: 97 pgs: 97 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-04-25T00:26:48.249 INFO:tasks.workunit.client.0.smithi042.stdout: 1 472843 6.26 MB/sec execute 278 sec latency 16.235 ms 2024-04-25T00:26:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:47 smithi136 ceph-mon[32923]: pgmap v629: 97 pgs: 97 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-04-25T00:26:49.249 INFO:tasks.workunit.client.0.smithi042.stdout: 1 473518 6.25 MB/sec execute 279 sec latency 18.218 ms 2024-04-25T00:26:50.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:49 smithi106 ceph-mon[32510]: pgmap v630: 97 pgs: 97 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-04-25T00:26:50.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:49 smithi042 ceph-mon[28130]: pgmap v630: 97 pgs: 97 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-04-25T00:26:50.250 INFO:tasks.workunit.client.0.smithi042.stdout: 1 474884 6.24 MB/sec execute 280 sec latency 29.661 ms 2024-04-25T00:26:50.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:49 smithi136 ceph-mon[32923]: pgmap v630: 97 pgs: 97 active+clean; 84 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-04-25T00:26:51.249 INFO:tasks.workunit.client.0.smithi042.stdout: 1 476319 6.26 MB/sec execute 281 sec latency 64.676 ms 2024-04-25T00:26:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:51 smithi106 ceph-mon[32510]: pgmap v631: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-04-25T00:26:52.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:51 smithi042 ceph-mon[28130]: pgmap v631: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-04-25T00:26:52.249 INFO:tasks.workunit.client.0.smithi042.stdout: 1 476888 6.25 MB/sec execute 282 sec latency 26.735 ms 2024-04-25T00:26:52.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:51 smithi136 ceph-mon[32923]: pgmap v631: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-04-25T00:26:53.249 INFO:tasks.workunit.client.0.smithi042.stdout: 1 478181 6.24 MB/sec execute 283 sec latency 25.247 ms 2024-04-25T00:26:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:53 smithi106 ceph-mon[32510]: pgmap v632: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-04-25T00:26:54.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:53 smithi042 ceph-mon[28130]: pgmap v632: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-04-25T00:26:54.250 INFO:tasks.workunit.client.0.smithi042.stdout: 1 479829 6.26 MB/sec execute 284 sec latency 17.887 ms 2024-04-25T00:26:54.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:53 smithi136 ceph-mon[32923]: pgmap v632: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-04-25T00:26:55.250 INFO:tasks.workunit.client.0.smithi042.stdout: 1 480392 6.25 MB/sec execute 285 sec latency 29.861 ms 2024-04-25T00:26:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:55 smithi106 ceph-mon[32510]: pgmap v633: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-04-25T00:26:56.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:55 smithi042 ceph-mon[28130]: pgmap v633: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-04-25T00:26:56.250 INFO:tasks.workunit.client.0.smithi042.stdout: 1 481624 6.24 MB/sec execute 286 sec latency 47.965 ms 2024-04-25T00:26:56.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:55 smithi136 ceph-mon[32923]: pgmap v633: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-04-25T00:26:57.250 INFO:tasks.workunit.client.0.smithi042.stdout: 1 483333 6.26 MB/sec execute 287 sec latency 18.303 ms 2024-04-25T00:26:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:57 smithi106 ceph-mon[32510]: pgmap v634: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-04-25T00:26:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:57 smithi042 ceph-mon[28130]: pgmap v634: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-04-25T00:26:58.250 INFO:tasks.workunit.client.0.smithi042.stdout: 1 483915 6.25 MB/sec execute 288 sec latency 16.634 ms 2024-04-25T00:26:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:57 smithi136 ceph-mon[32923]: pgmap v634: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 470 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-04-25T00:26:59.250 INFO:tasks.workunit.client.0.smithi042.stdout: 1 485080 6.24 MB/sec execute 289 sec latency 24.739 ms 2024-04-25T00:26:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.340+0000 7f80a6400640 1 -- 172.21.15.42:0/2813320086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a01017d0 msgr2=0x7f80a0101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.340+0000 7f80a6400640 1 --2- 172.21.15.42:0/2813320086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a01017d0 0x7f80a0101bf0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8090009900 tx=0x7f809002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.341+0000 7f80a6400640 1 -- 172.21.15.42:0/2813320086 shutdown_connections 2024-04-25T00:26:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.341+0000 7f80a6400640 1 --2- 172.21.15.42:0/2813320086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80a0103380 0x7f80a0109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.341+0000 7f80a6400640 1 --2- 172.21.15.42:0/2813320086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a01029c0 0x7f80a0102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.341+0000 7f80a6400640 1 --2- 172.21.15.42:0/2813320086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a01017d0 0x7f80a0101bf0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.341+0000 7f80a6400640 1 -- 172.21.15.42:0/2813320086 >> 172.21.15.42:0/2813320086 conn(0x7f80a00fcd60 msgr2=0x7f80a00ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.341+0000 7f80a6400640 1 -- 172.21.15.42:0/2813320086 shutdown_connections 2024-04-25T00:26:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.341+0000 7f80a6400640 1 -- 172.21.15.42:0/2813320086 wait complete. 2024-04-25T00:26:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.342+0000 7f80a6400640 1 Processor -- start 2024-04-25T00:26:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.342+0000 7f80a6400640 1 -- start start 2024-04-25T00:26:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.342+0000 7f80a6400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a01017d0 0x7f80a0199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.342+0000 7f80a6400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a01029c0 0x7f80a019a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.342+0000 7f80a6400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80a0103380 0x7f80a01a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.342+0000 7f80a6400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80a010d420 con 0x7f80a01029c0 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.342+0000 7f80a6400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80a010d2a0 con 0x7f80a0103380 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.342+0000 7f80a6400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80a010d5a0 con 0x7f80a01017d0 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a01029c0 0x7f80a019a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a01017d0 0x7f80a0199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a01017d0 0x7f80a0199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45120/0 (socket says 172.21.15.42:45120) 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809f400640 1 -- 172.21.15.42:0/1092162830 learned_addr learned my addr 172.21.15.42:0/1092162830 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a01029c0 0x7f80a019a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49742/0 (socket says 172.21.15.42:49742) 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809fe00640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80a0103380 0x7f80a01a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809f400640 1 -- 172.21.15.42:0/1092162830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80a0103380 msgr2=0x7f80a01a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809f400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80a0103380 0x7f80a01a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809f400640 1 -- 172.21.15.42:0/1092162830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a01029c0 msgr2=0x7f80a019a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809f400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a01029c0 0x7f80a019a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.343+0000 7f809f400640 1 -- 172.21.15.42:0/1092162830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a01a17c0 con 0x7f80a01017d0 2024-04-25T00:26:59.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.344+0000 7f809f400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a01017d0 0x7f80a0199ba0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8090031610 tx=0x7f8090004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.344+0000 7f8097e00640 1 -- 172.21.15.42:0/1092162830 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809003f030 con 0x7f80a01017d0 2024-04-25T00:26:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.344+0000 7f8097e00640 1 -- 172.21.15.42:0/1092162830 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8090003ad0 con 0x7f80a01017d0 2024-04-25T00:26:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.344+0000 7f8097e00640 1 -- 172.21.15.42:0/1092162830 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8090035070 con 0x7f80a01017d0 2024-04-25T00:26:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.344+0000 7f80a6400640 1 -- 172.21.15.42:0/1092162830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a01a1a50 con 0x7f80a01017d0 2024-04-25T00:26:59.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.344+0000 7f80a6400640 1 -- 172.21.15.42:0/1092162830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a01a1f90 con 0x7f80a01017d0 2024-04-25T00:26:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.348+0000 7f80a93db740 1 Processor -- start 2024-04-25T00:26:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.348+0000 7f8097e00640 1 -- 172.21.15.42:0/1092162830 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8090042030 con 0x7f80a01017d0 2024-04-25T00:26:59.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.348+0000 7f80a93db740 1 -- start start 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.348+0000 7f80a93db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c8343e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.348+0000 7f80a93db740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343e690 0x563c83443190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.348+0000 7f80a93db740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c83443800 0x563c83445c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.348+0000 7f80a93db740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563c833a0ab0 con 0x563c83443800 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.348+0000 7f80a93db740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563c83349640 con 0x563c8343e690 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.348+0000 7f80a93db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563c8339ca70 con 0x563c8343dca0 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.349+0000 7f809fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c8343e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.349+0000 7f809fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c8343e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45136/0 (socket says 172.21.15.42:45136) 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.349+0000 7f809fe00640 1 -- 172.21.15.42:0/360144719 learned_addr learned my addr 172.21.15.42:0/360144719 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.349+0000 7f809ea00640 1 --2- 172.21.15.42:0/360144719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c83443800 0x563c83445c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.349+0000 7f809f400640 1 --2- 172.21.15.42:0/360144719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343e690 0x563c83443190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.349+0000 7f809fe00640 1 -- 172.21.15.42:0/360144719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343e690 msgr2=0x563c83443190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.349+0000 7f809fe00640 1 --2- 172.21.15.42:0/360144719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343e690 0x563c83443190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.349+0000 7f809fe00640 1 -- 172.21.15.42:0/360144719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c83443800 msgr2=0x563c83445c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.349+0000 7f809fe00640 1 --2- 172.21.15.42:0/360144719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c83443800 0x563c83445c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.349+0000 7f809fe00640 1 -- 172.21.15.42:0/360144719 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c830b5210 con 0x563c8343dca0 2024-04-25T00:26:59.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.350+0000 7f809fe00640 1 --2- 172.21.15.42:0/360144719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c8343e0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8098009fd0 tx=0x7f80980118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1041b844544aa3a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.350+0000 7f8097e00640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80840a2cb0 0x7f80840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.350+0000 7f8094c00640 1 -- 172.21.15.42:0/360144719 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098004d20 con 0x563c8343dca0 2024-04-25T00:26:59.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.350+0000 7f8094c00640 1 -- 172.21.15.42:0/360144719 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8098004050 con 0x563c8343dca0 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.350+0000 7f8094c00640 1 -- 172.21.15.42:0/360144719 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098005260 con 0x563c8343dca0 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.350+0000 7f8097e00640 1 -- 172.21.15.42:0/1092162830 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f80900ef640 con 0x7f80a01017d0 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.350+0000 7f809f400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80840a2cb0 0x7f80840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.351+0000 7f80a93db740 1 -- 172.21.15.42:0/360144719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 msgr2=0x563c8343e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.351+0000 7f80a93db740 1 --2- 172.21.15.42:0/360144719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c8343e0c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8098009fd0 tx=0x7f80980118d0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.351+0000 7f809f400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80840a2cb0 0x7f80840a5170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f80900ed4e0 tx=0x7f80900f5550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.351+0000 7f80a93db740 1 -- 172.21.15.42:0/360144719 shutdown_connections 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.351+0000 7f80a93db740 1 --2- 172.21.15.42:0/360144719 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c83443800 0x563c83445c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.351+0000 7f80a93db740 1 --2- 172.21.15.42:0/360144719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343e690 0x563c83443190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.351+0000 7f80a93db740 1 --2- 172.21.15.42:0/360144719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c8343e0c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.351+0000 7f80a93db740 1 -- 172.21.15.42:0/360144719 >> 172.21.15.42:0/360144719 conn(0x563c83439310 msgr2=0x563c83439710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.351+0000 7f80a93db740 1 -- 172.21.15.42:0/360144719 shutdown_connections 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.351+0000 7f80a93db740 1 -- 172.21.15.42:0/360144719 wait complete. 2024-04-25T00:26:59.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.352+0000 7f80a93db740 1 -- start start 2024-04-25T00:26:59.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.352+0000 7f80a93db740 1 Processor -- start 2024-04-25T00:26:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f80a93db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c83447f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f80a93db740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 0x563c83448480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f80a93db740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 0x563c8344f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f80a93db740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563c834467b0 con 0x563c8343e690 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f80a93db740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x563c8335c990 con 0x563c83443800 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f80a93db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563c83446930 con 0x563c8343dca0 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f80a93db740 10 client.0 _ll_register_callbacks cb 0x563c8335e950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f809fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c83447f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f809fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c83447f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45144/0 (socket says 172.21.15.42:45144) 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f809fe00640 1 -- 172.21.15.42:0/4135246187 learned_addr learned my addr 172.21.15.42:0/4135246187 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f809fe00640 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 0x563c8344f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.353+0000 7f809ea00640 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 0x563c83448480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f809fe00640 1 -- 172.21.15.42:0/4135246187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 msgr2=0x563c8344f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f809fe00640 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 0x563c8344f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f809fe00640 1 -- 172.21.15.42:0/4135246187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 msgr2=0x563c83448480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f809fe00640 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 0x563c83448480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f809fe00640 1 -- 172.21.15.42:0/4135246187 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c83181620 con 0x563c8343dca0 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f809ea00640 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 0x563c83448480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f809fe00640 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c83447f40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8098009de0 tx=0x7f8098000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d98682bcb5e69cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f8089600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f8089600640 1 -- 172.21.15.42:0/4135246187 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809800f4e0 con 0x563c8343dca0 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f8089600640 1 -- 172.21.15.42:0/4135246187 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f809800f680 con 0x563c8343dca0 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.354+0000 7f80a93db740 1 -- 172.21.15.42:0/4135246187 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8098004050 con 0x563c8343dca0 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f8089600640 1 -- 172.21.15.42:0/4135246187 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f809800f880 con 0x563c8343dca0 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f80a93db740 10 client.25843 fetch_fsmap learned FSMap version 30 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f80a93db740 1 -- 172.21.15.42:0/4135246187 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c834524a0 con 0x563c8343dca0 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f8089600640 1 -- 172.21.15.42:0/4135246187 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8098004050 con 0x563c8343dca0 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f8089600640 1 -- 172.21.15.42:0/4135246187 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f8098025040 con 0x563c8343dca0 2024-04-25T00:26:59.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f80a93db740 10 client.25843 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:26:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f80a93db740 10 client.25843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f80a93db740 20 client.25843 populate_metadata read hostname 'smithi042' 2024-04-25T00:26:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f80a93db740 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563c834539c0 0x563c83455db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f80a93db740 4 client.25843 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:26:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.355+0000 7f80a93db740 1 -- 172.21.15.42:0/4135246187 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c833adff0 con 0x563c834539c0 2024-04-25T00:26:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.356+0000 7f809ea00640 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563c834539c0 0x563c83455db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.356+0000 7f809ea00640 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563c834539c0 0x563c83455db0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a0e9079dc2583d4 server_cookie=2eb1b64c6fa88acf in_seq=0 out_seq=0 2024-04-25T00:26:59.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.356+0000 7f8089600640 10 client.25843 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f8089600640 1 -- 172.21.15.42:0/4135246187 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563c833adff0 con 0x563c834539c0 2024-04-25T00:26:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f8089600640 10 client.25843 handle_command_reply: tid=0 2024-04-25T00:26:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f80a93db740 1 client.25843 shutdown 2024-04-25T00:26:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f80a93db740 1 -- 172.21.15.42:0/4135246187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 msgr2=0x563c83447f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:59.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f80a93db740 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c83447f40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8098009de0 tx=0x7f8098000f40 comp rx=0 tx=0).stop 2024-04-25T00:26:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f80a93db740 1 -- 172.21.15.42:0/4135246187 shutdown_connections 2024-04-25T00:26:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f80a93db740 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563c834539c0 0x563c83455db0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f80a93db740 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 0x563c8344f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f80a93db740 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 0x563c83448480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f80a93db740 1 --2- 172.21.15.42:0/4135246187 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343dca0 0x563c83447f40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f8089600640 0 client.25843 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.358+0000 7f80a93db740 1 -- 172.21.15.42:0/4135246187 >> 172.21.15.42:0/4135246187 conn(0x563c83439310 msgr2=0x563c83438480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.359+0000 7f80a93db740 1 -- 172.21.15.42:0/4135246187 shutdown_connections 2024-04-25T00:26:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.359+0000 7f80a93db740 1 -- 172.21.15.42:0/4135246187 wait complete. 2024-04-25T00:26:59.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.359+0000 7f80a93db740 20 client.25843 trim_cache size 0 max 16384 2024-04-25T00:26:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.377+0000 7f80a93db740 1 Processor -- start 2024-04-25T00:26:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.377+0000 7f80a93db740 1 -- start start 2024-04-25T00:26:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.377+0000 7f80a93db740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343dca0 0x563c83433fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f80a93db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343e690 0x563c8344f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f80a93db740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 0x563c83447b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f80a93db740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01ad5b0 con 0x563c8343dca0 2024-04-25T00:26:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f80a93db740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01aae50 con 0x563c83443800 2024-04-25T00:26:59.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f80a93db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01a8ca0 con 0x563c8343e690 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343e690 0x563c8344f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343e690 0x563c8344f480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45146/0 (socket says 172.21.15.42:45146) 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809ea00640 1 -- 172.21.15.42:0/418072915 learned_addr learned my addr 172.21.15.42:0/418072915 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809fe00640 1 --2- 172.21.15.42:0/418072915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343dca0 0x563c83433fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809fe00640 1 --2- 172.21.15.42:0/418072915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 0x563c83447b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809ea00640 1 -- 172.21.15.42:0/418072915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 msgr2=0x563c83447b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809ea00640 1 --2- 172.21.15.42:0/418072915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 0x563c83447b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809ea00640 1 -- 172.21.15.42:0/418072915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343dca0 msgr2=0x563c83433fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809ea00640 1 --2- 172.21.15.42:0/418072915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343dca0 0x563c83433fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809ea00640 1 -- 172.21.15.42:0/418072915 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c8343d8e0 con 0x563c8343e690 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.378+0000 7f809fe00640 1 --2- 172.21.15.42:0/418072915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 0x563c83447b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.379+0000 7f809ea00640 1 --2- 172.21.15.42:0/418072915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343e690 0x563c8344f480 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f808c000c00 tx=0x7f808c010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5899fbe0686ae41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:59.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.379+0000 7f808be00640 1 -- 172.21.15.42:0/418072915 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c01f270 con 0x563c8343e690 2024-04-25T00:26:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.379+0000 7f808be00640 1 -- 172.21.15.42:0/418072915 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f808c023030 con 0x563c8343e690 2024-04-25T00:26:59.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.379+0000 7f808be00640 1 -- 172.21.15.42:0/418072915 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c01edc0 con 0x563c8343e690 2024-04-25T00:26:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.380+0000 7f80a93db740 1 -- 172.21.15.42:0/418072915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343e690 msgr2=0x563c8344f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.380+0000 7f80a93db740 1 --2- 172.21.15.42:0/418072915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343e690 0x563c8344f480 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f808c000c00 tx=0x7f808c010800 comp rx=0 tx=0).stop 2024-04-25T00:26:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.380+0000 7f80a93db740 1 -- 172.21.15.42:0/418072915 shutdown_connections 2024-04-25T00:26:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.380+0000 7f80a93db740 1 --2- 172.21.15.42:0/418072915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c83443800 0x563c83447b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.380+0000 7f80a93db740 1 --2- 172.21.15.42:0/418072915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c8343e690 0x563c8344f480 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.380+0000 7f80a93db740 1 --2- 172.21.15.42:0/418072915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343dca0 0x563c83433fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.380+0000 7f80a93db740 1 -- 172.21.15.42:0/418072915 >> 172.21.15.42:0/418072915 conn(0x563c83439310 msgr2=0x563c83442f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.380+0000 7f80a93db740 1 -- 172.21.15.42:0/418072915 shutdown_connections 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.380+0000 7f80a93db740 1 -- 172.21.15.42:0/418072915 wait complete. 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.381+0000 7f80a93db740 1 -- start start 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.381+0000 7f80a93db740 1 Processor -- start 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.381+0000 7f80a93db740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343dca0 0x563c8344ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.381+0000 7f80a93db740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 0x563c8344d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.381+0000 7f80a93db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c83443800 0x563c8344e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.381+0000 7f80a93db740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01a9310 con 0x563c8343e690 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.381+0000 7f80a93db740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01aa780 con 0x563c8343dca0 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.381+0000 7f80a93db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01a99e0 con 0x563c83443800 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.381+0000 7f80a93db740 10 client.0 _ll_register_callbacks cb 0x7f808c00a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343dca0 0x563c8344ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343dca0 0x563c8344ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40432/0 (socket says 172.21.15.42:40432) 2024-04-25T00:26:59.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809fe00640 1 -- 172.21.15.42:0/2505034324 learned_addr learned my addr 172.21.15.42:0/2505034324 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809fe00640 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c83443800 0x563c8344e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809ea00640 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 0x563c8344d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809fe00640 1 -- 172.21.15.42:0/2505034324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c83443800 msgr2=0x563c8344e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809fe00640 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c83443800 0x563c8344e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809fe00640 1 -- 172.21.15.42:0/2505034324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 msgr2=0x563c8344d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809fe00640 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 0x563c8344d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809fe00640 1 -- 172.21.15.42:0/2505034324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c83447000 con 0x563c8343dca0 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f809fe00640 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343dca0 0x563c8344ccd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8098003f50 tx=0x7f8098003ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b278515ba86c1b34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.382+0000 7f8089600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.383+0000 7f8089600640 1 -- 172.21.15.42:0/2505034324 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098028680 con 0x563c8343dca0 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.383+0000 7f8089600640 1 -- 172.21.15.42:0/2505034324 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f8098028820 con 0x563c8343dca0 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.383+0000 7f80a93db740 1 -- 172.21.15.42:0/2505034324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f808c023030 con 0x563c8343dca0 2024-04-25T00:26:59.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.383+0000 7f8089600640 1 -- 172.21.15.42:0/2505034324 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8098045050 con 0x563c8343dca0 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.383+0000 7f80a93db740 10 client.25841 fetch_fsmap learned FSMap version 30 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.383+0000 7f80a93db740 1 -- 172.21.15.42:0/2505034324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563c83439140 con 0x563c8343dca0 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.383+0000 7f8089600640 1 -- 172.21.15.42:0/2505034324 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f808c023030 con 0x563c8343dca0 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.383+0000 7f8089600640 1 -- 172.21.15.42:0/2505034324 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f8098019810 con 0x563c8343dca0 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.384+0000 7f80a93db740 10 client.25841 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.384+0000 7f80a93db740 10 client.25841 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.384+0000 7f80a93db740 20 client.25841 populate_metadata read hostname 'smithi042' 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.384+0000 7f80a93db740 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563c834ba530 0x563c834bc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.384+0000 7f80a93db740 4 client.25841 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.384+0000 7f80a93db740 1 -- 172.21.15.42:0/2505034324 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563c833a8340 con 0x563c834ba530 2024-04-25T00:26:59.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.384+0000 7f809ea00640 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563c834ba530 0x563c834bc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:26:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.384+0000 7f809ea00640 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563c834ba530 0x563c834bc950 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b3a6cab448196d8 server_cookie=134605ace725806e in_seq=0 out_seq=0 2024-04-25T00:26:59.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.384+0000 7f8089600640 10 client.25841 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f8089600640 1 -- 172.21.15.42:0/2505034324 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563c833a8340 con 0x563c834ba530 2024-04-25T00:26:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f8089600640 10 client.25841 handle_command_reply: tid=0 2024-04-25T00:26:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f80a93db740 1 client.25841 shutdown 2024-04-25T00:26:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f80a93db740 1 -- 172.21.15.42:0/2505034324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343dca0 msgr2=0x563c8344ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f80a93db740 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343dca0 0x563c8344ccd0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8098003f50 tx=0x7f8098003ec0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f80a93db740 1 -- 172.21.15.42:0/2505034324 shutdown_connections 2024-04-25T00:26:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f80a93db740 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x563c834ba530 0x563c834bc950 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f80a93db740 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563c83443800 0x563c8344e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f80a93db740 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x563c8343e690 0x563c8344d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f80a93db740 1 --2- 172.21.15.42:0/2505034324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x563c8343dca0 0x563c8344ccd0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f80a93db740 1 -- 172.21.15.42:0/2505034324 >> 172.21.15.42:0/2505034324 conn(0x563c83439310 msgr2=0x563c8343c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.439+0000 7f8089600640 0 client.25841 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:26:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.440+0000 7f80a93db740 1 -- 172.21.15.42:0/2505034324 shutdown_connections 2024-04-25T00:26:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.440+0000 7f80a93db740 1 -- 172.21.15.42:0/2505034324 wait complete. 2024-04-25T00:26:59.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.440+0000 7f80a93db740 20 client.25841 trim_cache size 0 max 16384 2024-04-25T00:26:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:26:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:26:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:26:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:26:59.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 -- 172.21.15.42:0/1092162830 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80840a2cb0 msgr2=0x7f80840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80840a2cb0 0x7f80840a5170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f80900ed4e0 tx=0x7f80900f5550 comp rx=0 tx=0).stop 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 -- 172.21.15.42:0/1092162830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a01017d0 msgr2=0x7f80a0199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a01017d0 0x7f80a0199ba0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8090031610 tx=0x7f8090004270 comp rx=0 tx=0).stop 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 -- 172.21.15.42:0/1092162830 shutdown_connections 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80840a2cb0 0x7f80840a5170 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80a0103380 0x7f80a01a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80a01029c0 0x7f80a019a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 --2- 172.21.15.42:0/1092162830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80a01017d0 0x7f80a0199ba0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 -- 172.21.15.42:0/1092162830 >> 172.21.15.42:0/1092162830 conn(0x7f80a00fcd60 msgr2=0x7f80a0073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 -- 172.21.15.42:0/1092162830 shutdown_connections 2024-04-25T00:26:59.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:26:59.441+0000 7f80a6400640 1 -- 172.21.15.42:0/1092162830 wait complete. 2024-04-25T00:27:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:26:59 smithi106 ceph-mon[32510]: pgmap v635: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-04-25T00:27:00.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:26:59 smithi042 ceph-mon[28130]: pgmap v635: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-04-25T00:27:00.251 INFO:tasks.workunit.client.0.smithi042.stdout: 1 486893 6.26 MB/sec execute 290 sec latency 30.228 ms 2024-04-25T00:27:00.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:26:59 smithi136 ceph-mon[32923]: pgmap v635: 97 pgs: 97 active+clean; 83 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.4 MiB/s wr, 160 op/s 2024-04-25T00:27:01.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:00 smithi042 ceph-mon[28130]: pgmap v636: 97 pgs: 97 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-04-25T00:27:01.251 INFO:tasks.workunit.client.0.smithi042.stdout: 1 487444 6.25 MB/sec execute 291 sec latency 54.559 ms 2024-04-25T00:27:01.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:00 smithi136 ceph-mon[32923]: pgmap v636: 97 pgs: 97 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-04-25T00:27:01.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:00 smithi106 ceph-mon[32510]: pgmap v636: 97 pgs: 97 active+clean; 82 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-04-25T00:27:02.251 INFO:tasks.workunit.client.0.smithi042.stdout: 1 488622 6.24 MB/sec execute 292 sec latency 24.996 ms 2024-04-25T00:27:03.251 INFO:tasks.workunit.client.0.smithi042.stdout: 1 490414 6.27 MB/sec execute 293 sec latency 15.665 ms 2024-04-25T00:27:03.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:03 smithi106 ceph-mon[32510]: pgmap v637: 97 pgs: 97 active+clean; 78 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-04-25T00:27:03.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:03 smithi042 ceph-mon[28130]: pgmap v637: 97 pgs: 97 active+clean; 78 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-04-25T00:27:03.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:03 smithi136 ceph-mon[32923]: pgmap v637: 97 pgs: 97 active+clean; 78 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 491 KiB/s rd, 1.8 MiB/s wr, 235 op/s 2024-04-25T00:27:04.251 INFO:tasks.workunit.client.0.smithi042.stdout: 1 490976 6.25 MB/sec execute 294 sec latency 22.465 ms 2024-04-25T00:27:05.251 INFO:tasks.workunit.client.0.smithi042.stdout: 1 492134 6.25 MB/sec execute 295 sec latency 39.479 ms 2024-04-25T00:27:05.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:05 smithi106 ceph-mon[32510]: pgmap v638: 97 pgs: 97 active+clean; 78 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-25T00:27:05.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:05 smithi042 ceph-mon[28130]: pgmap v638: 97 pgs: 97 active+clean; 78 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-25T00:27:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:05 smithi136 ceph-mon[32923]: pgmap v638: 97 pgs: 97 active+clean; 78 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 333 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-04-25T00:27:06.251 INFO:tasks.workunit.client.0.smithi042.stdout: 1 493817 6.27 MB/sec execute 296 sec latency 70.065 ms 2024-04-25T00:27:07.252 INFO:tasks.workunit.client.0.smithi042.stdout: 1 494404 6.25 MB/sec execute 297 sec latency 22.246 ms 2024-04-25T00:27:07.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:07 smithi106 ceph-mon[32510]: pgmap v639: 97 pgs: 97 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-04-25T00:27:07.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:07 smithi042 ceph-mon[28130]: pgmap v639: 97 pgs: 97 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-04-25T00:27:07.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:07 smithi136 ceph-mon[32923]: pgmap v639: 97 pgs: 97 active+clean; 85 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 2.4 MiB/s wr, 235 op/s 2024-04-25T00:27:08.252 INFO:tasks.workunit.client.0.smithi042.stdout: 1 495678 6.25 MB/sec execute 298 sec latency 38.155 ms 2024-04-25T00:27:08.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:08 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:08.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:08 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:08.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:08 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:27:09.253 INFO:tasks.workunit.client.0.smithi042.stdout: 1 497623 6.27 MB/sec execute 299 sec latency 18.033 ms 2024-04-25T00:27:09.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:09 smithi106 ceph-mon[32510]: pgmap v640: 97 pgs: 97 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-04-25T00:27:09.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:09 smithi042 ceph-mon[28130]: pgmap v640: 97 pgs: 97 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-04-25T00:27:09.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:09 smithi136 ceph-mon[32923]: pgmap v640: 97 pgs: 97 active+clean; 86 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 2.0 MiB/s wr, 171 op/s 2024-04-25T00:27:10.253 INFO:tasks.workunit.client.0.smithi042.stdout: 1 498283 6.25 MB/sec execute 300 sec latency 25.606 ms 2024-04-25T00:27:10.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:10 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:10.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:10 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:10.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:10 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:27:10.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:10 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:10.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:10 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:10.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:10 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:27:10.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:10 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:27:10.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:10 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:27:10.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:10 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:27:11.253 INFO:tasks.workunit.client.0.smithi042.stdout: 1 499839 6.25 MB/sec execute 301 sec latency 27.897 ms 2024-04-25T00:27:11.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:11 smithi042 ceph-mon[28130]: pgmap v641: 97 pgs: 97 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-04-25T00:27:11.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:11 smithi136 ceph-mon[32923]: pgmap v641: 97 pgs: 97 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-04-25T00:27:11.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:11 smithi106 ceph-mon[32510]: pgmap v641: 97 pgs: 97 active+clean; 95 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 2.7 MiB/s wr, 174 op/s 2024-04-25T00:27:12.253 INFO:tasks.workunit.client.0.smithi042.stdout: 1 501306 6.27 MB/sec execute 302 sec latency 20.370 ms 2024-04-25T00:27:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.593+0000 7f8574a00640 1 -- 172.21.15.42:0/391221120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85701053b0 msgr2=0x7f857010bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.593+0000 7f8574a00640 1 --2- 172.21.15.42:0/391221120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85701053b0 0x7f857010bc40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8568009ca0 tx=0x7f856802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.593+0000 7f8574a00640 1 -- 172.21.15.42:0/391221120 shutdown_connections 2024-04-25T00:27:12.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.593+0000 7f8574a00640 1 --2- 172.21.15.42:0/391221120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85701053b0 0x7f857010bc40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.593+0000 7f8574a00640 1 --2- 172.21.15.42:0/391221120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85701049f0 0x7f8570104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.593+0000 7f8574a00640 1 --2- 172.21.15.42:0/391221120 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8570103800 0x7f8570103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.593+0000 7f8574a00640 1 -- 172.21.15.42:0/391221120 >> 172.21.15.42:0/391221120 conn(0x7f85700fed70 msgr2=0x7f85701011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.594+0000 7f8574a00640 1 -- 172.21.15.42:0/391221120 shutdown_connections 2024-04-25T00:27:12.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.594+0000 7f8574a00640 1 -- 172.21.15.42:0/391221120 wait complete. 2024-04-25T00:27:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f8574a00640 1 Processor -- start 2024-04-25T00:27:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f8574a00640 1 -- start start 2024-04-25T00:27:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f8574a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8570103800 0x7f8570199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f8574a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85701049f0 0x7f8570199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f8574a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85701053b0 0x7f85701a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f8574a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f857010f460 con 0x7f85701049f0 2024-04-25T00:27:12.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f8574a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f857010f2e0 con 0x7f8570103800 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f8574a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f857010f5e0 con 0x7f85701053b0 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f856e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8570103800 0x7f8570199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f856e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8570103800 0x7f8570199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40170/0 (socket says 172.21.15.42:40170) 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f856e800640 1 -- 172.21.15.42:0/296061765 learned_addr learned my addr 172.21.15.42:0/296061765 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f856f200640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85701053b0 0x7f85701a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.595+0000 7f8567e00640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85701049f0 0x7f8570199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f856e800640 1 -- 172.21.15.42:0/296061765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85701053b0 msgr2=0x7f85701a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f856e800640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85701053b0 0x7f85701a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f856e800640 1 -- 172.21.15.42:0/296061765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85701049f0 msgr2=0x7f8570199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f856e800640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85701049f0 0x7f8570199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f856e800640 1 -- 172.21.15.42:0/296061765 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85701a1690 con 0x7f8570103800 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f856e800640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8570103800 0x7f8570199a70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f856000e9c0 tx=0x7f856000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:12.599 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f8567400640 1 -- 172.21.15.42:0/296061765 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856000cd80 con 0x7f8570103800 2024-04-25T00:27:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f8567400640 1 -- 172.21.15.42:0/296061765 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85600035f0 con 0x7f8570103800 2024-04-25T00:27:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f8574a00640 1 -- 172.21.15.42:0/296061765 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85701a1980 con 0x7f8570103800 2024-04-25T00:27:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f8567400640 1 -- 172.21.15.42:0/296061765 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560010690 con 0x7f8570103800 2024-04-25T00:27:12.600 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.596+0000 7f8574a00640 1 -- 172.21.15.42:0/296061765 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85701a1f20 con 0x7f8570103800 2024-04-25T00:27:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.600+0000 7f8567400640 1 -- 172.21.15.42:0/296061765 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f85600037a0 con 0x7f8570103800 2024-04-25T00:27:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.600+0000 7f8578371740 1 Processor -- start 2024-04-25T00:27:12.601 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.600+0000 7f8578371740 1 -- start start 2024-04-25T00:27:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.600+0000 7f8578371740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 0x560dd98cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f8578371740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 0x560dd98d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.602 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f8578371740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f8578371740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560dd982dab0 con 0x560dd98d09b0 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f8578371740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560dd97d6640 con 0x560dd98cb840 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f8578371740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560dd9829a70 con 0x560dd98cae50 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f856e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 0x560dd98d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f856f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 0x560dd98cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f856f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 0x560dd98cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:34864/0 (socket says 172.21.15.42:34864) 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f856f200640 1 -- 172.21.15.42:0/542151650 learned_addr learned my addr 172.21.15.42:0/542151650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f8567e00640 1 --2- 172.21.15.42:0/542151650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f856f200640 1 -- 172.21.15.42:0/542151650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 msgr2=0x560dd98d0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f856f200640 1 --2- 172.21.15.42:0/542151650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 0x560dd98d0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f856f200640 1 -- 172.21.15.42:0/542151650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 msgr2=0x560dd98d2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f856f200640 1 --2- 172.21.15.42:0/542151650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98d2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.603 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.601+0000 7f856f200640 1 -- 172.21.15.42:0/542151650 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd9542210 con 0x560dd98cae50 2024-04-25T00:27:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.602+0000 7f8567400640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f85540a2c00 0x7f85540a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.602+0000 7f856f200640 1 --2- 172.21.15.42:0/542151650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 0x560dd98cb270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8568009ca0 tx=0x7f8568004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f926324110c59b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.602+0000 7f8567400640 1 -- 172.21.15.42:0/296061765 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f85600cf480 con 0x7f8570103800 2024-04-25T00:27:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.602+0000 7f855f400640 1 -- 172.21.15.42:0/542151650 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85680098c0 con 0x560dd98cae50 2024-04-25T00:27:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.602+0000 7f855f400640 1 -- 172.21.15.42:0/542151650 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8568036040 con 0x560dd98cae50 2024-04-25T00:27:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.602+0000 7f855f400640 1 -- 172.21.15.42:0/542151650 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568004290 con 0x560dd98cae50 2024-04-25T00:27:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.602+0000 7f856e800640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f85540a2c00 0x7f85540a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.603+0000 7f8578371740 1 -- 172.21.15.42:0/542151650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 msgr2=0x560dd98cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.603+0000 7f8578371740 1 --2- 172.21.15.42:0/542151650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 0x560dd98cb270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f8568009ca0 tx=0x7f8568004a40 comp rx=0 tx=0).stop 2024-04-25T00:27:12.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.603+0000 7f856e800640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f85540a2c00 0x7f85540a50c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x560dd98cbc40 tx=0x7f85600d2ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.603+0000 7f8578371740 1 -- 172.21.15.42:0/542151650 shutdown_connections 2024-04-25T00:27:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.603+0000 7f8578371740 1 --2- 172.21.15.42:0/542151650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98d2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.603+0000 7f8578371740 1 --2- 172.21.15.42:0/542151650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 0x560dd98d0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.603+0000 7f8578371740 1 --2- 172.21.15.42:0/542151650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 0x560dd98cb270 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.603+0000 7f8578371740 1 -- 172.21.15.42:0/542151650 >> 172.21.15.42:0/542151650 conn(0x560dd98c64c0 msgr2=0x560dd98c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.603+0000 7f8578371740 1 -- 172.21.15.42:0/542151650 shutdown_connections 2024-04-25T00:27:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.603+0000 7f8578371740 1 -- 172.21.15.42:0/542151650 wait complete. 2024-04-25T00:27:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.604+0000 7f8578371740 1 -- start start 2024-04-25T00:27:12.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.604+0000 7f8578371740 1 Processor -- start 2024-04-25T00:27:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.604+0000 7f8578371740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 0x560dd98d4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.604+0000 7f8578371740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 0x560dd98d53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.604+0000 7f8578371740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98dc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.604+0000 7f8578371740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560dd98d3720 con 0x560dd98d09b0 2024-04-25T00:27:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.604+0000 7f8578371740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560dd97e9990 con 0x560dd98cb840 2024-04-25T00:27:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.604+0000 7f8578371740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560dd98d38a0 con 0x560dd98cae50 2024-04-25T00:27:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.604+0000 7f8578371740 10 client.0 _ll_register_callbacks cb 0x560dd97eb950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:12.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f8567e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 0x560dd98d53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f8567e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 0x560dd98d53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40188/0 (socket says 172.21.15.42:40188) 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f8567e00640 1 -- 172.21.15.42:0/3576866468 learned_addr learned my addr 172.21.15.42:0/3576866468 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f856f200640 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 0x560dd98d4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f856f200640 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98dc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f8567e00640 1 -- 172.21.15.42:0/3576866468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 msgr2=0x560dd98d4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f8567e00640 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 0x560dd98d4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f8567e00640 1 -- 172.21.15.42:0/3576866468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 msgr2=0x560dd98dc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f8567e00640 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98dc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f8567e00640 1 -- 172.21.15.42:0/3576866468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd960e620 con 0x560dd98cb840 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.605+0000 7f8567e00640 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 0x560dd98d53a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8558009d20 tx=0x7f855800b690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d455fc57621d11 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.606+0000 7f8553e00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.606+0000 7f8553e00640 1 -- 172.21.15.42:0/3576866468 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558027060 con 0x560dd98cb840 2024-04-25T00:27:12.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.606+0000 7f8578371740 1 -- 172.21.15.42:0/3576866468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8568036040 con 0x560dd98cb840 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.606+0000 7f8553e00640 1 -- 172.21.15.42:0/3576866468 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f8558015d50 con 0x560dd98cb840 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.606+0000 7f8553e00640 1 -- 172.21.15.42:0/3576866468 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8558004050 con 0x560dd98cb840 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.606+0000 7f8578371740 10 client.25853 fetch_fsmap learned FSMap version 30 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.606+0000 7f8578371740 1 -- 172.21.15.42:0/3576866468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560dd98df480 con 0x560dd98cb840 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.606+0000 7f8553e00640 1 -- 172.21.15.42:0/3576866468 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8568036040 con 0x560dd98cb840 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.607+0000 7f8553e00640 1 -- 172.21.15.42:0/3576866468 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f85580170d0 con 0x560dd98cb840 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.607+0000 7f8578371740 10 client.25853 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.607+0000 7f8578371740 10 client.25853 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.607+0000 7f8578371740 20 client.25853 populate_metadata read hostname 'smithi042' 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.607+0000 7f8578371740 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560dd98e0b60 0x560dd98e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.607+0000 7f8578371740 4 client.25853 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.607+0000 7f8578371740 1 -- 172.21.15.42:0/3576866468 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560dd983aff0 con 0x560dd98e0b60 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.607+0000 7f856f200640 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560dd98e0b60 0x560dd98e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.607+0000 7f856f200640 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560dd98e0b60 0x560dd98e2f80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6dc02b210fd39ec server_cookie=8fc20e71db7ee1f5 in_seq=0 out_seq=0 2024-04-25T00:27:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.607+0000 7f8553e00640 10 client.25853 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8553e00640 1 -- 172.21.15.42:0/3576866468 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560dd983aff0 con 0x560dd98e0b60 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8553e00640 10 client.25853 handle_command_reply: tid=0 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8578371740 1 client.25853 shutdown 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8578371740 1 -- 172.21.15.42:0/3576866468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 msgr2=0x560dd98d53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8578371740 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 0x560dd98d53a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f8558009d20 tx=0x7f855800b690 comp rx=0 tx=0).stop 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8578371740 1 -- 172.21.15.42:0/3576866468 shutdown_connections 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8578371740 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560dd98e0b60 0x560dd98e2f80 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8578371740 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98dc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8578371740 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cb840 0x560dd98d53a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8578371740 1 --2- 172.21.15.42:0/3576866468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cae50 0x560dd98d4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8553e00640 0 client.25853 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.609+0000 7f8578371740 1 -- 172.21.15.42:0/3576866468 >> 172.21.15.42:0/3576866468 conn(0x560dd98c64c0 msgr2=0x560dd98c5620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.610+0000 7f8578371740 1 -- 172.21.15.42:0/3576866468 shutdown_connections 2024-04-25T00:27:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.610+0000 7f8578371740 1 -- 172.21.15.42:0/3576866468 wait complete. 2024-04-25T00:27:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.610+0000 7f8578371740 20 client.25853 trim_cache size 0 max 16384 2024-04-25T00:27:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.626+0000 7f8578371740 1 Processor -- start 2024-04-25T00:27:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.626+0000 7f8578371740 1 -- start start 2024-04-25T00:27:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.626+0000 7f8578371740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 0x560dd98cca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.626+0000 7f8578371740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cb840 0x560dd98dc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.627+0000 7f8578371740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.627+0000 7f8578371740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85701a9020 con 0x560dd98d09b0 2024-04-25T00:27:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.627+0000 7f8578371740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f85701a68c0 con 0x560dd98cae50 2024-04-25T00:27:12.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.627+0000 7f8578371740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85701a4710 con 0x560dd98cb840 2024-04-25T00:27:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.627+0000 7f8567e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cb840 0x560dd98dc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.627+0000 7f856f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 0x560dd98cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.627+0000 7f8567e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cb840 0x560dd98dc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:34888/0 (socket says 172.21.15.42:34888) 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.627+0000 7f8567e00640 1 -- 172.21.15.42:0/4097854523 learned_addr learned my addr 172.21.15.42:0/4097854523 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.627+0000 7f856f200640 1 --2- 172.21.15.42:0/4097854523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.627+0000 7f8567e00640 1 -- 172.21.15.42:0/4097854523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 msgr2=0x560dd98cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.628+0000 7f8567e00640 1 --2- 172.21.15.42:0/4097854523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 0x560dd98cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.628+0000 7f8567e00640 1 -- 172.21.15.42:0/4097854523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 msgr2=0x560dd98ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.628+0000 7f8567e00640 1 --2- 172.21.15.42:0/4097854523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.628+0000 7f8567e00640 1 -- 172.21.15.42:0/4097854523 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dd98caa90 con 0x560dd98cb840 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.628+0000 7f8567e00640 1 --2- 172.21.15.42:0/4097854523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cb840 0x560dd98dc3a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8558001eb0 tx=0x7f855800bc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a5baa8ac6c632ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.628+0000 7f855ea00640 1 -- 172.21.15.42:0/4097854523 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558027070 con 0x560dd98cb840 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.628+0000 7f855ea00640 1 -- 172.21.15.42:0/4097854523 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85580170d0 con 0x560dd98cb840 2024-04-25T00:27:12.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.628+0000 7f855ea00640 1 -- 172.21.15.42:0/4097854523 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8558032040 con 0x560dd98cb840 2024-04-25T00:27:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.629+0000 7f8578371740 1 -- 172.21.15.42:0/4097854523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cb840 msgr2=0x560dd98dc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.629+0000 7f8578371740 1 --2- 172.21.15.42:0/4097854523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cb840 0x560dd98dc3a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f8558001eb0 tx=0x7f855800bc50 comp rx=0 tx=0).stop 2024-04-25T00:27:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.629+0000 7f8578371740 1 -- 172.21.15.42:0/4097854523 shutdown_connections 2024-04-25T00:27:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.629+0000 7f8578371740 1 --2- 172.21.15.42:0/4097854523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98d09b0 0x560dd98ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.629+0000 7f8578371740 1 --2- 172.21.15.42:0/4097854523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98cb840 0x560dd98dc3a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.629+0000 7f8578371740 1 --2- 172.21.15.42:0/4097854523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 0x560dd98cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.629+0000 7f8578371740 1 -- 172.21.15.42:0/4097854523 >> 172.21.15.42:0/4097854523 conn(0x560dd98c64c0 msgr2=0x560dd98d0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.629+0000 7f8578371740 1 -- 172.21.15.42:0/4097854523 shutdown_connections 2024-04-25T00:27:12.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.629+0000 7f8578371740 1 -- 172.21.15.42:0/4097854523 wait complete. 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.629+0000 7f8578371740 1 -- start start 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f8578371740 1 Processor -- start 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f8578371740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 0x560dd98dbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f8578371740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98cb840 0x560dd98dc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f8578371740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98d09b0 0x560dd98da440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f8578371740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f85701a4d80 con 0x560dd98cb840 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f8578371740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f85701a61f0 con 0x560dd98cae50 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f8578371740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85701a5450 con 0x560dd98d09b0 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f8578371740 10 client.0 _ll_register_callbacks cb 0x7f8558017de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f856f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 0x560dd98dbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f8567e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98cb840 0x560dd98dc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f856f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 0x560dd98dbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40210/0 (socket says 172.21.15.42:40210) 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.630+0000 7f856f200640 1 -- 172.21.15.42:0/3920681017 learned_addr learned my addr 172.21.15.42:0/3920681017 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:12.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.631+0000 7f856f200640 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98d09b0 0x560dd98da440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.631+0000 7f856f200640 1 -- 172.21.15.42:0/3920681017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98d09b0 msgr2=0x560dd98da440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.631+0000 7f856f200640 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98d09b0 0x560dd98da440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.631+0000 7f856f200640 1 -- 172.21.15.42:0/3920681017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98cb840 msgr2=0x560dd98dc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.631+0000 7f856f200640 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98cb840 0x560dd98dc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.631+0000 7f856f200640 1 -- 172.21.15.42:0/3920681017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dd98cc510 con 0x560dd98cae50 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.631+0000 7f856f200640 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 0x560dd98dbda0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8568005e30 tx=0x7f856803d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8addeaf9b399bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.631+0000 7f8553e00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.631+0000 7f8553e00640 1 -- 172.21.15.42:0/3920681017 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85680542d0 con 0x560dd98cae50 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8553e00640 1 -- 172.21.15.42:0/3920681017 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f8568054470 con 0x560dd98cae50 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8578371740 1 -- 172.21.15.42:0/3920681017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85580170d0 con 0x560dd98cae50 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8553e00640 1 -- 172.21.15.42:0/3920681017 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8568054670 con 0x560dd98cae50 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8578371740 10 client.25859 fetch_fsmap learned FSMap version 30 2024-04-25T00:27:12.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8578371740 1 -- 172.21.15.42:0/3920681017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560dd98c7190 con 0x560dd98cae50 2024-04-25T00:27:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8553e00640 1 -- 172.21.15.42:0/3920681017 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f85580170d0 con 0x560dd98cae50 2024-04-25T00:27:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8553e00640 1 -- 172.21.15.42:0/3920681017 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f85680305c0 con 0x560dd98cae50 2024-04-25T00:27:12.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8578371740 10 client.25859 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:27:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8578371740 10 client.25859 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8578371740 20 client.25859 populate_metadata read hostname 'smithi042' 2024-04-25T00:27:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8578371740 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560dd9947f80 0x560dd994a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8578371740 4 client.25859 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:27:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.632+0000 7f8578371740 1 -- 172.21.15.42:0/3920681017 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560dd9835340 con 0x560dd9947f80 2024-04-25T00:27:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.633+0000 7f8567e00640 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560dd9947f80 0x560dd994a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.633+0000 7f8567e00640 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560dd9947f80 0x560dd994a3a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baf5584497199fe9 server_cookie=4fd38800d1bf2bc5 in_seq=0 out_seq=0 2024-04-25T00:27:12.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.633+0000 7f8553e00640 10 client.25859 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8553e00640 1 -- 172.21.15.42:0/3920681017 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560dd9835340 con 0x560dd9947f80 2024-04-25T00:27:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8553e00640 10 client.25859 handle_command_reply: tid=0 2024-04-25T00:27:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 client.25859 shutdown 2024-04-25T00:27:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 -- 172.21.15.42:0/3920681017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 msgr2=0x560dd98dbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 0x560dd98dbda0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8568005e30 tx=0x7f856803d040 comp rx=0 tx=0).stop 2024-04-25T00:27:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 -- 172.21.15.42:0/3920681017 shutdown_connections 2024-04-25T00:27:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560dd9947f80 0x560dd994a3a0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560dd98d09b0 0x560dd98da440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560dd98cb840 0x560dd98dc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 --2- 172.21.15.42:0/3920681017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560dd98cae50 0x560dd98dbda0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8553e00640 0 client.25859 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 -- 172.21.15.42:0/3920681017 >> 172.21.15.42:0/3920681017 conn(0x560dd98c64c0 msgr2=0x560dd98d4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 -- 172.21.15.42:0/3920681017 shutdown_connections 2024-04-25T00:27:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 1 -- 172.21.15.42:0/3920681017 wait complete. 2024-04-25T00:27:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.692+0000 7f8578371740 20 client.25859 trim_cache size 0 max 16384 2024-04-25T00:27:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:27:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:27:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:27:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.693+0000 7f8574a00640 1 -- 172.21.15.42:0/296061765 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f85540a2c00 msgr2=0x7f85540a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.693+0000 7f8574a00640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f85540a2c00 0x7f85540a50c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x560dd98cbc40 tx=0x7f85600d2ec0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.693+0000 7f8574a00640 1 -- 172.21.15.42:0/296061765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8570103800 msgr2=0x7f8570199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.693+0000 7f8574a00640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8570103800 0x7f8570199a70 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f856000e9c0 tx=0x7f856000ee80 comp rx=0 tx=0).stop 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.693+0000 7f8574a00640 1 -- 172.21.15.42:0/296061765 shutdown_connections 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.693+0000 7f8574a00640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f85540a2c00 0x7f85540a50c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.693+0000 7f8574a00640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85701053b0 0x7f85701a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.693+0000 7f8574a00640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f85701049f0 0x7f8570199fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.693+0000 7f8574a00640 1 --2- 172.21.15.42:0/296061765 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8570103800 0x7f8570199a70 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.693+0000 7f8574a00640 1 -- 172.21.15.42:0/296061765 >> 172.21.15.42:0/296061765 conn(0x7f85700fed70 msgr2=0x7f8570107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.694+0000 7f8574a00640 1 -- 172.21.15.42:0/296061765 shutdown_connections 2024-04-25T00:27:12.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:12.694+0000 7f8574a00640 1 -- 172.21.15.42:0/296061765 wait complete. 2024-04-25T00:27:13.253 INFO:tasks.workunit.client.0.smithi042.stdout: 1 502429 6.27 MB/sec execute 303 sec latency 28.721 ms 2024-04-25T00:27:13.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:13 smithi042 ceph-mon[28130]: pgmap v642: 97 pgs: 97 active+clean; 102 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.2 MiB/s wr, 220 op/s 2024-04-25T00:27:13.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:13 smithi136 ceph-mon[32923]: pgmap v642: 97 pgs: 97 active+clean; 102 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.2 MiB/s wr, 220 op/s 2024-04-25T00:27:13.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:13 smithi106 ceph-mon[32510]: pgmap v642: 97 pgs: 97 active+clean; 102 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 3.2 MiB/s wr, 220 op/s 2024-04-25T00:27:14.253 INFO:tasks.workunit.client.0.smithi042.stdout: 1 504304 6.28 MB/sec execute 304 sec latency 19.479 ms 2024-04-25T00:27:15.253 INFO:tasks.workunit.client.0.smithi042.stdout: 1 505093 6.27 MB/sec execute 305 sec latency 16.133 ms 2024-04-25T00:27:15.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:15 smithi042 ceph-mon[28130]: pgmap v643: 97 pgs: 97 active+clean; 102 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 2.7 MiB/s wr, 140 op/s 2024-04-25T00:27:15.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:15 smithi136 ceph-mon[32923]: pgmap v643: 97 pgs: 97 active+clean; 102 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 2.7 MiB/s wr, 140 op/s 2024-04-25T00:27:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:15 smithi106 ceph-mon[32510]: pgmap v643: 97 pgs: 97 active+clean; 102 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 2.7 MiB/s wr, 140 op/s 2024-04-25T00:27:16.253 INFO:tasks.workunit.client.0.smithi042.stdout: 1 506352 6.27 MB/sec execute 306 sec latency 36.412 ms 2024-04-25T00:27:17.254 INFO:tasks.workunit.client.0.smithi042.stdout: 1 508178 6.29 MB/sec execute 307 sec latency 21.749 ms 2024-04-25T00:27:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:17 smithi042 ceph-mon[28130]: pgmap v644: 97 pgs: 97 active+clean; 102 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 3.2 MiB/s wr, 229 op/s 2024-04-25T00:27:17.640 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:17 smithi106 ceph-mon[32510]: pgmap v644: 97 pgs: 97 active+clean; 102 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 3.2 MiB/s wr, 229 op/s 2024-04-25T00:27:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:17 smithi136 ceph-mon[32923]: pgmap v644: 97 pgs: 97 active+clean; 102 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 3.2 MiB/s wr, 229 op/s 2024-04-25T00:27:18.254 INFO:tasks.workunit.client.0.smithi042.stdout: 1 508823 6.28 MB/sec execute 308 sec latency 16.384 ms 2024-04-25T00:27:19.254 INFO:tasks.workunit.client.0.smithi042.stdout: 1 510362 6.28 MB/sec execute 309 sec latency 23.923 ms 2024-04-25T00:27:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:19 smithi136 ceph-mon[32923]: pgmap v645: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-04-25T00:27:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:19 smithi106 ceph-mon[32510]: pgmap v645: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-04-25T00:27:19.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:19 smithi042 ceph-mon[28130]: pgmap v645: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 2.3 MiB/s wr, 172 op/s 2024-04-25T00:27:20.254 INFO:tasks.workunit.client.0.smithi042.stdout: 1 511877 6.30 MB/sec execute 310 sec latency 14.321 ms 2024-04-25T00:27:21.254 INFO:tasks.workunit.client.0.smithi042.stdout: 1 512560 6.28 MB/sec execute 311 sec latency 78.116 ms 2024-04-25T00:27:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:21 smithi106 ceph-mon[32510]: pgmap v646: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-04-25T00:27:21.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:21 smithi042 ceph-mon[28130]: pgmap v646: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-04-25T00:27:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:21 smithi136 ceph-mon[32923]: pgmap v646: 97 pgs: 97 active+clean; 104 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 2.3 MiB/s wr, 178 op/s 2024-04-25T00:27:22.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 514341 6.29 MB/sec execute 312 sec latency 24.459 ms 2024-04-25T00:27:23.255 INFO:tasks.workunit.client.0.smithi042.stdout: 1 515523 6.30 MB/sec execute 313 sec latency 21.545 ms 2024-04-25T00:27:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:23 smithi106 ceph-mon[32510]: pgmap v647: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-04-25T00:27:23.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:23 smithi042 ceph-mon[28130]: pgmap v647: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-04-25T00:27:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:23 smithi136 ceph-mon[32923]: pgmap v647: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 569 KiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-04-25T00:27:24.255 INFO:tasks.workunit.client.0.smithi042.stdout: 1 516550 6.29 MB/sec execute 314 sec latency 23.908 ms 2024-04-25T00:27:25.255 INFO:tasks.workunit.client.0.smithi042.stdout: 1 518291 6.30 MB/sec execute 315 sec latency 21.525 ms 2024-04-25T00:27:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:25 smithi106 ceph-mon[32510]: pgmap v648: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-04-25T00:27:25.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.848+0000 7fcfe3600640 1 -- 172.21.15.42:0/44938215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfdc107540 msgr2=0x7fcfdc109930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.848+0000 7fcfe3600640 1 --2- 172.21.15.42:0/44938215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfdc107540 0x7fcfdc109930 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc009ca0 tx=0x7fcfcc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.849+0000 7fcfe3600640 1 -- 172.21.15.42:0/44938215 shutdown_connections 2024-04-25T00:27:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.849+0000 7fcfe3600640 1 --2- 172.21.15.42:0/44938215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfdc107540 0x7fcfdc109930 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.849+0000 7fcfe3600640 1 --2- 172.21.15.42:0/44938215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfdc104c10 0x7fcfdc107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.849+0000 7fcfe3600640 1 --2- 172.21.15.42:0/44938215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfdc1022e0 0x7fcfdc1046d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.849+0000 7fcfe3600640 1 -- 172.21.15.42:0/44938215 >> 172.21.15.42:0/44938215 conn(0x7fcfdc0fbc10 msgr2=0x7fcfdc0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.849+0000 7fcfe3600640 1 -- 172.21.15.42:0/44938215 shutdown_connections 2024-04-25T00:27:25.851 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.849+0000 7fcfe3600640 1 -- 172.21.15.42:0/44938215 wait complete. 2024-04-25T00:27:25.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.850+0000 7fcfe3600640 1 Processor -- start 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.850+0000 7fcfe3600640 1 -- start start 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.850+0000 7fcfe3600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfdc1022e0 0x7fcfdc101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.850+0000 7fcfe3600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfdc104c10 0x7fcfdc100290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.850+0000 7fcfe3600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfdc107540 0x7fcfdc1007d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.850+0000 7fcfe3600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10d4a0 con 0x7fcfdc104c10 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.850+0000 7fcfe3600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10d320 con 0x7fcfdc1022e0 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.850+0000 7fcfe3600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc10d620 con 0x7fcfdc107540 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.851+0000 7fcfe1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfdc107540 0x7fcfdc1007d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.851+0000 7fcfe1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfdc107540 0x7fcfdc1007d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57316/0 (socket says 172.21.15.42:57316) 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.851+0000 7fcfe1600640 1 -- 172.21.15.42:0/3963117010 learned_addr learned my addr 172.21.15.42:0/3963117010 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.851+0000 7fcfe0c00640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfdc1022e0 0x7fcfdc101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.851+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfdc104c10 0x7fcfdc100290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.851+0000 7fcfe0c00640 1 -- 172.21.15.42:0/3963117010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfdc107540 msgr2=0x7fcfdc1007d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.851+0000 7fcfe0c00640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfdc107540 0x7fcfdc1007d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.851+0000 7fcfe0c00640 1 -- 172.21.15.42:0/3963117010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfdc104c10 msgr2=0x7fcfdc100290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.851+0000 7fcfe0c00640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfdc104c10 0x7fcfdc100290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.851+0000 7fcfe0c00640 1 -- 172.21.15.42:0/3963117010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc101060 con 0x7fcfdc1022e0 2024-04-25T00:27:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.852+0000 7fcfe0c00640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfdc1022e0 0x7fcfdc101c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400e9f0 tx=0x7fcfd400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.852+0000 7fcfd9600640 1 -- 172.21.15.42:0/3963117010 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd400ccc0 con 0x7fcfdc1022e0 2024-04-25T00:27:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.852+0000 7fcfd9600640 1 -- 172.21.15.42:0/3963117010 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfd4010910 con 0x7fcfdc1022e0 2024-04-25T00:27:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.852+0000 7fcfd9600640 1 -- 172.21.15.42:0/3963117010 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4018960 con 0x7fcfdc1022e0 2024-04-25T00:27:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.853+0000 7fcfe3600640 1 -- 172.21.15.42:0/3963117010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc1a1b30 con 0x7fcfdc1022e0 2024-04-25T00:27:25.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.853+0000 7fcfe3600640 1 -- 172.21.15.42:0/3963117010 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1a2000 con 0x7fcfdc1022e0 2024-04-25T00:27:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.856+0000 7fcfe65aa740 1 Processor -- start 2024-04-25T00:27:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.856+0000 7fcfd9600640 1 -- 172.21.15.42:0/3963117010 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcfd401f020 con 0x7fcfdc1022e0 2024-04-25T00:27:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.856+0000 7fcfe65aa740 1 -- start start 2024-04-25T00:27:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f19210c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1921690 0x5576f1926190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 0x5576f1928c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5576f1883ab0 con 0x5576f1920ca0 2024-04-25T00:27:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5576f182c640 con 0x5576f1926800 2024-04-25T00:27:25.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5576f187fa70 con 0x5576f1921690 2024-04-25T00:27:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 0x5576f1928c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 0x5576f1928c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53296/0 (socket says 172.21.15.42:53296) 2024-04-25T00:27:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfe0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1921690 0x5576f1926190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.860 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfdbe00640 1 -- 172.21.15.42:0/812158197 learned_addr learned my addr 172.21.15.42:0/812158197 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfe1600640 1 --2- 172.21.15.42:0/812158197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f19210c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfdbe00640 1 -- 172.21.15.42:0/812158197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1921690 msgr2=0x5576f1926190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/812158197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1921690 0x5576f1926190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfdbe00640 1 -- 172.21.15.42:0/812158197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 msgr2=0x5576f19210c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/812158197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f19210c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.857+0000 7fcfdbe00640 1 -- 172.21.15.42:0/812158197 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f1598210 con 0x5576f1926800 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.858+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/812158197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 0x5576f1928c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009c60 tx=0x7fcfc4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8bc462b7e57f9e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.858+0000 7fcfd1c00640 1 -- 172.21.15.42:0/812158197 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4004230 con 0x5576f1926800 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.858+0000 7fcfd1c00640 1 -- 172.21.15.42:0/812158197 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc40043d0 con 0x5576f1926800 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.858+0000 7fcfd1c00640 1 -- 172.21.15.42:0/812158197 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4019820 con 0x5576f1926800 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.858+0000 7fcfd9600640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfc00a2da0 0x7fcfc00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.859+0000 7fcfd9600640 1 -- 172.21.15.42:0/3963117010 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fcfd40cf200 con 0x7fcfdc1022e0 2024-04-25T00:27:25.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.859+0000 7fcfe1600640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfc00a2da0 0x7fcfc00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.859+0000 7fcfe65aa740 1 -- 172.21.15.42:0/812158197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 msgr2=0x5576f1928c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.859+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/812158197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 0x5576f1928c90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009c60 tx=0x7fcfc4011810 comp rx=0 tx=0).stop 2024-04-25T00:27:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.859+0000 7fcfe65aa740 1 -- 172.21.15.42:0/812158197 shutdown_connections 2024-04-25T00:27:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.859+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/812158197 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 0x5576f1928c90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.859+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/812158197 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1921690 0x5576f1926190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.859+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/812158197 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f19210c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.859+0000 7fcfe65aa740 1 -- 172.21.15.42:0/812158197 >> 172.21.15.42:0/812158197 conn(0x5576f191c310 msgr2=0x5576f191c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.860+0000 7fcfe65aa740 1 -- 172.21.15.42:0/812158197 shutdown_connections 2024-04-25T00:27:25.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.860+0000 7fcfe65aa740 1 -- 172.21.15.42:0/812158197 wait complete. 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.860+0000 7fcfe1600640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfc00a2da0 0x7fcfc00a5260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5576f1921ef0 tx=0x7fcfcc031bb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.860+0000 7fcfe65aa740 1 -- start start 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.860+0000 7fcfe65aa740 1 Processor -- start 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f192aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f192b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f1932470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5576f19297b0 con 0x5576f1920ca0 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5576f183f990 con 0x5576f1921690 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5576f1929930 con 0x5576f1926800 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f192b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfe65aa740 10 client.0 _ll_register_callbacks cb 0x5576f1841950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfe0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f192aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.866 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfdbe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f192b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53298/0 (socket says 172.21.15.42:53298) 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfdbe00640 1 -- 172.21.15.42:0/777210724 learned_addr learned my addr 172.21.15.42:0/777210724 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfe1600640 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f1932470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfdbe00640 1 -- 172.21.15.42:0/777210724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 msgr2=0x5576f1932470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f1932470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfdbe00640 1 -- 172.21.15.42:0/777210724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 msgr2=0x5576f192aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f192aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.861+0000 7fcfdbe00640 1 -- 172.21.15.42:0/777210724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f1664620 con 0x5576f1921690 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.862+0000 7fcfe1600640 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f1932470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.862+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f192b430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009c30 tx=0x7fcfc4003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c973ed7f0e0e02d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.862+0000 7fcfca000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.862+0000 7fcfca000640 1 -- 172.21.15.42:0/777210724 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc401f040 con 0x5576f1921690 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.862+0000 7fcfca000640 1 -- 172.21.15.42:0/777210724 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fcfc4005050 con 0x5576f1921690 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.862+0000 7fcfe65aa740 1 -- 172.21.15.42:0/777210724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfc40043d0 con 0x5576f1921690 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfca000640 1 -- 172.21.15.42:0/777210724 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfc4009830 con 0x5576f1921690 2024-04-25T00:27:25.867 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfe65aa740 10 client.25871 fetch_fsmap learned FSMap version 30 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfe65aa740 1 -- 172.21.15.42:0/777210724 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5576f1935450 con 0x5576f1921690 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfca000640 1 -- 172.21.15.42:0/777210724 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfc40043d0 con 0x5576f1921690 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfca000640 1 -- 172.21.15.42:0/777210724 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fcfc4018070 con 0x5576f1921690 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfe65aa740 10 client.25871 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfe65aa740 10 client.25871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfe65aa740 20 client.25871 populate_metadata read hostname 'smithi042' 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5576f19368c0 0x5576f1938d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfe65aa740 4 client.25871 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.863+0000 7fcfe65aa740 1 -- 172.21.15.42:0/777210724 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576f1890ff0 con 0x5576f19368c0 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.864+0000 7fcfe0c00640 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5576f19368c0 0x5576f1938d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.864+0000 7fcfe0c00640 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5576f19368c0 0x5576f1938d60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5097d3627d0fdb3d server_cookie=6dbcce3d561ecf16 in_seq=0 out_seq=0 2024-04-25T00:27:25.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.864+0000 7fcfca000640 10 client.25871 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfca000640 1 -- 172.21.15.42:0/777210724 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576f1890ff0 con 0x5576f19368c0 2024-04-25T00:27:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfca000640 10 client.25871 handle_command_reply: tid=0 2024-04-25T00:27:25.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 client.25871 shutdown 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 -- 172.21.15.42:0/777210724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 msgr2=0x5576f192b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f192b430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009c30 tx=0x7fcfc4003820 comp rx=0 tx=0).stop 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 -- 172.21.15.42:0/777210724 shutdown_connections 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5576f19368c0 0x5576f1938d60 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f1932470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f192b430 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/777210724 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f192aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfca000640 0 client.25871 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 -- 172.21.15.42:0/777210724 >> 172.21.15.42:0/777210724 conn(0x5576f191c310 msgr2=0x5576f191b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 -- 172.21.15.42:0/777210724 shutdown_connections 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.865+0000 7fcfe65aa740 1 -- 172.21.15.42:0/777210724 wait complete. 2024-04-25T00:27:25.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.866+0000 7fcfe65aa740 20 client.25871 trim_cache size 0 max 16384 2024-04-25T00:27:25.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:25 smithi042 ceph-mon[28130]: pgmap v648: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-04-25T00:27:25.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.883+0000 7fcfe65aa740 1 Processor -- start 2024-04-25T00:27:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.883+0000 7fcfe65aa740 1 -- start start 2024-04-25T00:27:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f1925ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f1932430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f193e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1ad5d0 con 0x5576f1920ca0 2024-04-25T00:27:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1aae70 con 0x5576f1921690 2024-04-25T00:27:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1a8cc0 con 0x5576f1926800 2024-04-25T00:27:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f193e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f193e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57346/0 (socket says 172.21.15.42:57346) 2024-04-25T00:27:25.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe1600640 1 -- 172.21.15.42:0/389378065 learned_addr learned my addr 172.21.15.42:0/389378065 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f1925ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/389378065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f1932430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe1600640 1 -- 172.21.15.42:0/389378065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 msgr2=0x5576f1932430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe1600640 1 --2- 172.21.15.42:0/389378065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f1932430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe1600640 1 -- 172.21.15.42:0/389378065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 msgr2=0x5576f1925ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe1600640 1 --2- 172.21.15.42:0/389378065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f1925ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe1600640 1 -- 172.21.15.42:0/389378065 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576f1922360 con 0x5576f1926800 2024-04-25T00:27:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/389378065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f1932430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:27:25.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.884+0000 7fcfe0c00640 1 --2- 172.21.15.42:0/389378065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f1925ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:27:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.885+0000 7fcfe1600640 1 --2- 172.21.15.42:0/389378065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f193e630 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0317a0 tx=0x7fcfcc041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e4c61e19834ae4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:25.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.885+0000 7fcfd1200640 1 -- 172.21.15.42:0/389378065 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc0317d0 con 0x5576f1926800 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.885+0000 7fcfd1200640 1 -- 172.21.15.42:0/389378065 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc036050 con 0x5576f1926800 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.885+0000 7fcfd1200640 1 -- 172.21.15.42:0/389378065 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc02fa70 con 0x5576f1926800 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.886+0000 7fcfe65aa740 1 -- 172.21.15.42:0/389378065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 msgr2=0x5576f193e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.886+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/389378065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f193e630 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0317a0 tx=0x7fcfcc041000 comp rx=0 tx=0).stop 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.886+0000 7fcfe65aa740 1 -- 172.21.15.42:0/389378065 shutdown_connections 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.886+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/389378065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1926800 0x5576f193e630 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.886+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/389378065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1921690 0x5576f1932430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.886+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/389378065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1920ca0 0x5576f1925ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.886+0000 7fcfe65aa740 1 -- 172.21.15.42:0/389378065 >> 172.21.15.42:0/389378065 conn(0x5576f191c310 msgr2=0x5576f1916fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.886+0000 7fcfe65aa740 1 -- 172.21.15.42:0/389378065 shutdown_connections 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.886+0000 7fcfe65aa740 1 -- 172.21.15.42:0/389378065 wait complete. 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.887+0000 7fcfe65aa740 1 -- start start 2024-04-25T00:27:25.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.887+0000 7fcfe65aa740 1 Processor -- start 2024-04-25T00:27:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1920ca0 0x5576f192fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1921690 0x5576f192fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe65aa740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 0x5576f1931000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1a9330 con 0x5576f1921690 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1aa7a0 con 0x5576f1926800 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe65aa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1a9a00 con 0x5576f1920ca0 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe65aa740 10 client.0 _ll_register_callbacks cb 0x7fcfc4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1920ca0 0x5576f192fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1920ca0 0x5576f192fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57350/0 (socket says 172.21.15.42:57350) 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe0c00640 1 -- 172.21.15.42:0/3086222629 learned_addr learned my addr 172.21.15.42:0/3086222629 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1921690 0x5576f192fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe1600640 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 0x5576f1931000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe0c00640 1 -- 172.21.15.42:0/3086222629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 msgr2=0x5576f1931000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:25.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe0c00640 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 0x5576f1931000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe0c00640 1 -- 172.21.15.42:0/3086222629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1921690 msgr2=0x5576f192fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe0c00640 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1921690 0x5576f192fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.888+0000 7fcfe0c00640 1 -- 172.21.15.42:0/3086222629 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576f19315f0 con 0x5576f1920ca0 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.889+0000 7fcfe0c00640 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1920ca0 0x5576f192fab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcfd409b730 tx=0x7fcfd400cf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b066cdb19b8abd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.889+0000 7fcfca000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.889+0000 7fcfca000640 1 -- 172.21.15.42:0/3086222629 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd409c130 con 0x5576f1920ca0 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.889+0000 7fcfca000640 1 -- 172.21.15.42:0/3086222629 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fcfd40d2d30 con 0x5576f1920ca0 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.889+0000 7fcfe65aa740 1 -- 172.21.15.42:0/3086222629 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfcc036050 con 0x5576f1920ca0 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.889+0000 7fcfca000640 1 -- 172.21.15.42:0/3086222629 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfd409ba30 con 0x5576f1920ca0 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.889+0000 7fcfe65aa740 10 client.25885 fetch_fsmap learned FSMap version 30 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.889+0000 7fcfe65aa740 1 -- 172.21.15.42:0/3086222629 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5576f199cf00 con 0x5576f1920ca0 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.889+0000 7fcfca000640 1 -- 172.21.15.42:0/3086222629 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcfcc036050 con 0x5576f1920ca0 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.890+0000 7fcfca000640 1 -- 172.21.15.42:0/3086222629 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fcfd40a4210 con 0x5576f1920ca0 2024-04-25T00:27:25.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.890+0000 7fcfe65aa740 10 client.25885 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:27:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.890+0000 7fcfe65aa740 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.890+0000 7fcfe65aa740 20 client.25885 populate_metadata read hostname 'smithi042' 2024-04-25T00:27:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.890+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5576f199d880 0x5576f199fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.890+0000 7fcfe65aa740 4 client.25885 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:27:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.890+0000 7fcfe65aa740 1 -- 172.21.15.42:0/3086222629 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576f188b340 con 0x5576f199d880 2024-04-25T00:27:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.890+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5576f199d880 0x5576f199fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.890+0000 7fcfdbe00640 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5576f199d880 0x5576f199fca0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17170597f484bf46 server_cookie=6248e70c374f9a87 in_seq=0 out_seq=0 2024-04-25T00:27:25.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.890+0000 7fcfca000640 10 client.25885 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:25.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.947+0000 7fcfca000640 1 -- 172.21.15.42:0/3086222629 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576f188b340 con 0x5576f199d880 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.947+0000 7fcfca000640 10 client.25885 handle_command_reply: tid=0 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.947+0000 7fcfe65aa740 1 client.25885 shutdown 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 1 -- 172.21.15.42:0/3086222629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1920ca0 msgr2=0x5576f192fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1920ca0 0x5576f192fab0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fcfd409b730 tx=0x7fcfd400cf60 comp rx=0 tx=0).stop 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 1 -- 172.21.15.42:0/3086222629 shutdown_connections 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5576f199d880 0x5576f199fca0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5576f1926800 0x5576f1931000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5576f1921690 0x5576f192fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 1 --2- 172.21.15.42:0/3086222629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5576f1920ca0 0x5576f192fab0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfca000640 0 client.25885 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 1 -- 172.21.15.42:0/3086222629 >> 172.21.15.42:0/3086222629 conn(0x5576f191c310 msgr2=0x5576f1925ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 1 -- 172.21.15.42:0/3086222629 shutdown_connections 2024-04-25T00:27:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 1 -- 172.21.15.42:0/3086222629 wait complete. 2024-04-25T00:27:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.948+0000 7fcfe65aa740 20 client.25885 trim_cache size 0 max 16384 2024-04-25T00:27:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:27:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:27:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:27:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:27:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 -- 172.21.15.42:0/3963117010 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfc00a2da0 msgr2=0x7fcfc00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfc00a2da0 0x7fcfc00a5260 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x5576f1921ef0 tx=0x7fcfcc031bb0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 -- 172.21.15.42:0/3963117010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfdc1022e0 msgr2=0x7fcfdc101c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfdc1022e0 0x7fcfdc101c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcfd400e9f0 tx=0x7fcfd400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 -- 172.21.15.42:0/3963117010 shutdown_connections 2024-04-25T00:27:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcfc00a2da0 0x7fcfc00a5260 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcfdc107540 0x7fcfdc1007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcfdc104c10 0x7fcfdc100290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 --2- 172.21.15.42:0/3963117010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfdc1022e0 0x7fcfdc101c50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 -- 172.21.15.42:0/3963117010 >> 172.21.15.42:0/3963117010 conn(0x7fcfdc0fbc10 msgr2=0x7fcfdc102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 -- 172.21.15.42:0/3963117010 shutdown_connections 2024-04-25T00:27:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:25.949+0000 7fcfe3600640 1 -- 172.21.15.42:0/3963117010 wait complete. 2024-04-25T00:27:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:25 smithi136 ceph-mon[32923]: pgmap v648: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 1.2 MiB/s wr, 171 op/s 2024-04-25T00:27:26.255 INFO:tasks.workunit.client.0.smithi042.stdout: 1 519218 6.30 MB/sec execute 316 sec latency 30.036 ms 2024-04-25T00:27:27.255 INFO:tasks.workunit.client.0.smithi042.stdout: 1 520359 6.29 MB/sec execute 317 sec latency 24.331 ms 2024-04-25T00:27:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:27 smithi106 ceph-mon[32510]: pgmap v649: 97 pgs: 97 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-04-25T00:27:27.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:27 smithi042 ceph-mon[28130]: pgmap v649: 97 pgs: 97 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-04-25T00:27:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:27 smithi136 ceph-mon[32923]: pgmap v649: 97 pgs: 97 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 580 KiB/s rd, 1.9 MiB/s wr, 243 op/s 2024-04-25T00:27:28.255 INFO:tasks.workunit.client.0.smithi042.stdout: 1 522304 6.31 MB/sec execute 318 sec latency 25.258 ms 2024-04-25T00:27:29.255 INFO:tasks.workunit.client.0.smithi042.stdout: 1 522971 6.30 MB/sec execute 319 sec latency 20.057 ms 2024-04-25T00:27:29.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:29 smithi106 ceph-mon[32510]: pgmap v650: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-04-25T00:27:29.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:29 smithi042 ceph-mon[28130]: pgmap v650: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-04-25T00:27:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:29 smithi136 ceph-mon[32923]: pgmap v650: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.6 MiB/s wr, 166 op/s 2024-04-25T00:27:30.255 INFO:tasks.workunit.client.0.smithi042.stdout: 1 524398 6.30 MB/sec execute 320 sec latency 24.530 ms 2024-04-25T00:27:31.256 INFO:tasks.workunit.client.0.smithi042.stdout: 1 526033 6.32 MB/sec execute 321 sec latency 15.483 ms 2024-04-25T00:27:31.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:31 smithi106 ceph-mon[32510]: pgmap v651: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-25T00:27:31.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:31 smithi042 ceph-mon[28130]: pgmap v651: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-25T00:27:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:31 smithi136 ceph-mon[32923]: pgmap v651: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-04-25T00:27:32.256 INFO:tasks.workunit.client.0.smithi042.stdout: 1 526748 6.30 MB/sec execute 322 sec latency 31.900 ms 2024-04-25T00:27:33.256 INFO:tasks.workunit.client.0.smithi042.stdout: 1 528439 6.31 MB/sec execute 323 sec latency 24.739 ms 2024-04-25T00:27:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:33 smithi106 ceph-mon[32510]: pgmap v652: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.7 MiB/s wr, 242 op/s 2024-04-25T00:27:33.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:33 smithi042 ceph-mon[28130]: pgmap v652: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.7 MiB/s wr, 242 op/s 2024-04-25T00:27:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:33 smithi136 ceph-mon[32923]: pgmap v652: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 1.7 MiB/s wr, 242 op/s 2024-04-25T00:27:34.256 INFO:tasks.workunit.client.0.smithi042.stdout: 1 529723 6.32 MB/sec execute 324 sec latency 14.547 ms 2024-04-25T00:27:35.256 INFO:tasks.workunit.client.0.smithi042.stdout: 1 530759 6.32 MB/sec execute 325 sec latency 21.004 ms 2024-04-25T00:27:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:35 smithi106 ceph-mon[32510]: pgmap v653: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-25T00:27:35.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:35 smithi042 ceph-mon[28130]: pgmap v653: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-25T00:27:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:35 smithi136 ceph-mon[32923]: pgmap v653: 97 pgs: 97 active+clean; 104 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 1.3 MiB/s wr, 179 op/s 2024-04-25T00:27:36.256 INFO:tasks.workunit.client.0.smithi042.stdout: 1 532526 6.32 MB/sec execute 326 sec latency 18.281 ms 2024-04-25T00:27:37.256 INFO:tasks.workunit.client.0.smithi042.stdout: 1 533327 6.32 MB/sec execute 327 sec latency 78.390 ms 2024-04-25T00:27:37.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:37 smithi042 ceph-mon[28130]: pgmap v654: 97 pgs: 97 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 2.1 MiB/s wr, 249 op/s 2024-04-25T00:27:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:37 smithi136 ceph-mon[32923]: pgmap v654: 97 pgs: 97 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 2.1 MiB/s wr, 249 op/s 2024-04-25T00:27:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:37 smithi106 ceph-mon[32510]: pgmap v654: 97 pgs: 97 active+clean; 106 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 599 KiB/s rd, 2.1 MiB/s wr, 249 op/s 2024-04-25T00:27:38.256 INFO:tasks.workunit.client.0.smithi042.stdout: 1 534410 6.32 MB/sec execute 328 sec latency 28.958 ms 2024-04-25T00:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.100+0000 7fe596a00640 1 -- 172.21.15.42:0/2693446814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5901016b0 msgr2=0x7fe590101b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.100+0000 7fe596a00640 1 --2- 172.21.15.42:0/2693446814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5901016b0 0x7fe590101b30 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe57c009900 tx=0x7fe57c02a400 comp rx=0 tx=0).stop 2024-04-25T00:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.100+0000 7fe596a00640 1 -- 172.21.15.42:0/2693446814 shutdown_connections 2024-04-25T00:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.100+0000 7fe596a00640 1 --2- 172.21.15.42:0/2693446814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590102070 0x7fe590108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.100+0000 7fe596a00640 1 --2- 172.21.15.42:0/2693446814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5901016b0 0x7fe590101b30 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.100+0000 7fe596a00640 1 --2- 172.21.15.42:0/2693446814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5901004c0 0x7fe5901008e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.100+0000 7fe596a00640 1 -- 172.21.15.42:0/2693446814 >> 172.21.15.42:0/2693446814 conn(0x7fe5900fba30 msgr2=0x7fe5900fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.101+0000 7fe596a00640 1 -- 172.21.15.42:0/2693446814 shutdown_connections 2024-04-25T00:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.101+0000 7fe596a00640 1 -- 172.21.15.42:0/2693446814 wait complete. 2024-04-25T00:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.102+0000 7fe596a00640 1 Processor -- start 2024-04-25T00:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.102+0000 7fe596a00640 1 -- start start 2024-04-25T00:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.102+0000 7fe596a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5901004c0 0x7fe590104450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.102+0000 7fe596a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5901016b0 0x7fe590102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.102+0000 7fe596a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590102070 0x7fe590102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.102+0000 7fe596a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010c1a0 con 0x7fe5901016b0 2024-04-25T00:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.102+0000 7fe596a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010c020 con 0x7fe5901004c0 2024-04-25T00:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.102+0000 7fe596a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe59010c320 con 0x7fe590102070 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.102+0000 7fe595600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5901004c0 0x7fe590104450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe595600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5901004c0 0x7fe590104450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34710/0 (socket says 172.21.15.42:34710) 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe595600640 1 -- 172.21.15.42:0/946696189 learned_addr learned my addr 172.21.15.42:0/946696189 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe594c00640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5901016b0 0x7fe590102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe596000640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590102070 0x7fe590102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe595600640 1 -- 172.21.15.42:0/946696189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590102070 msgr2=0x7fe590102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe595600640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590102070 0x7fe590102fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe595600640 1 -- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5901016b0 msgr2=0x7fe590102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe595600640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5901016b0 0x7fe590102a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe595600640 1 -- 172.21.15.42:0/946696189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe590103860 con 0x7fe5901004c0 2024-04-25T00:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe594c00640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5901016b0 0x7fe590102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.103+0000 7fe595600640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5901004c0 0x7fe590104450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe58400eb70 tx=0x7fe58400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.104+0000 7fe58e000640 1 -- 172.21.15.42:0/946696189 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584003b70 con 0x7fe5901004c0 2024-04-25T00:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.104+0000 7fe58e000640 1 -- 172.21.15.42:0/946696189 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58400ee70 con 0x7fe5901004c0 2024-04-25T00:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.104+0000 7fe58e000640 1 -- 172.21.15.42:0/946696189 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe584010830 con 0x7fe5901004c0 2024-04-25T00:27:39.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.104+0000 7fe596a00640 1 -- 172.21.15.42:0/946696189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5901a1a30 con 0x7fe5901004c0 2024-04-25T00:27:39.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.104+0000 7fe596a00640 1 -- 172.21.15.42:0/946696189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5901a1f00 con 0x7fe5901004c0 2024-04-25T00:27:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.107+0000 7fe59ade6740 1 Processor -- start 2024-04-25T00:27:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.107+0000 7fe58e000640 1 -- 172.21.15.42:0/946696189 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe584004020 con 0x7fe5901004c0 2024-04-25T00:27:39.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe59ade6740 1 -- start start 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a797c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a7981340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a79819b0 0x5585a7983e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe59ade6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5585a78deab0 con 0x5585a79819b0 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe59ade6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5585a7887640 con 0x5585a797c840 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe59ade6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5585a78daa70 con 0x5585a797be50 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe596000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a797c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe595600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a7981340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe596000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a797c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60314/0 (socket says 172.21.15.42:60314) 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe596000640 1 -- 172.21.15.42:0/2207393934 learned_addr learned my addr 172.21.15.42:0/2207393934 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.108+0000 7fe594c00640 1 --2- 172.21.15.42:0/2207393934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a79819b0 0x5585a7983e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.109+0000 7fe596000640 1 -- 172.21.15.42:0/2207393934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 msgr2=0x5585a7981340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.109+0000 7fe596000640 1 --2- 172.21.15.42:0/2207393934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a7981340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.109+0000 7fe596000640 1 -- 172.21.15.42:0/2207393934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a79819b0 msgr2=0x5585a7983e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.109+0000 7fe596000640 1 --2- 172.21.15.42:0/2207393934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a79819b0 0x5585a7983e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.109+0000 7fe596000640 1 -- 172.21.15.42:0/2207393934 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a75f3210 con 0x5585a797be50 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.109+0000 7fe596000640 1 --2- 172.21.15.42:0/2207393934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a797c270 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b6b0 tx=0x7fe58800bb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b956d7bf83842728 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.109+0000 7fe582600640 1 -- 172.21.15.42:0/2207393934 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588003bd0 con 0x5585a797be50 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.109+0000 7fe582600640 1 -- 172.21.15.42:0/2207393934 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe588015070 con 0x5585a797be50 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.109+0000 7fe582600640 1 -- 172.21.15.42:0/2207393934 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5880116a0 con 0x5585a797be50 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.109+0000 7fe58e000640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5780a2c00 0x7fe5780a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe58e000640 1 -- 172.21.15.42:0/946696189 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fe5840cf770 con 0x7fe5901004c0 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe595600640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5780a2c00 0x7fe5780a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe59ade6740 1 -- 172.21.15.42:0/2207393934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 msgr2=0x5585a797c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe59ade6740 1 --2- 172.21.15.42:0/2207393934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a797c270 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b6b0 tx=0x7fe58800bb70 comp rx=0 tx=0).stop 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe59ade6740 1 -- 172.21.15.42:0/2207393934 shutdown_connections 2024-04-25T00:27:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe59ade6740 1 --2- 172.21.15.42:0/2207393934 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a79819b0 0x5585a7983e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe59ade6740 1 --2- 172.21.15.42:0/2207393934 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a7981340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe59ade6740 1 --2- 172.21.15.42:0/2207393934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a797c270 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe59ade6740 1 -- 172.21.15.42:0/2207393934 >> 172.21.15.42:0/2207393934 conn(0x5585a79774c0 msgr2=0x5585a79778c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe59ade6740 1 -- 172.21.15.42:0/2207393934 shutdown_connections 2024-04-25T00:27:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.110+0000 7fe59ade6740 1 -- 172.21.15.42:0/2207393934 wait complete. 2024-04-25T00:27:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.111+0000 7fe595600640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5780a2c00 0x7fe5780a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe5840a46e0 tx=0x7fe5840d5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.111+0000 7fe59ade6740 1 -- start start 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.111+0000 7fe59ade6740 1 Processor -- start 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a79860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a7986610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a79819b0 0x5585a798d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe59ade6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5585a7984960 con 0x5585a79819b0 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe59ade6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5585a789a990 con 0x5585a797c840 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe59ade6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5585a7984ae0 con 0x5585a797be50 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe59ade6740 10 client.0 _ll_register_callbacks cb 0x5585a789c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe594c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a7986610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe594c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a7986610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34724/0 (socket says 172.21.15.42:34724) 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe594c00640 1 -- 172.21.15.42:0/227744159 learned_addr learned my addr 172.21.15.42:0/227744159 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe596000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a79860d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe596000640 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a79819b0 0x5585a798d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe594c00640 1 -- 172.21.15.42:0/227744159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 msgr2=0x5585a79860d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe594c00640 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a79860d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe594c00640 1 -- 172.21.15.42:0/227744159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a79819b0 msgr2=0x5585a798d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe594c00640 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a79819b0 0x5585a798d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.112+0000 7fe594c00640 1 -- 172.21.15.42:0/227744159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a76bf620 con 0x5585a797c840 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.113+0000 7fe594c00640 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a7986610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe57c000c00 tx=0x7fe57c0052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3e65ed6c322e968 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.113+0000 7fe576a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.113+0000 7fe576a00640 1 -- 172.21.15.42:0/227744159 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c035030 con 0x5585a797c840 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.113+0000 7fe576a00640 1 -- 172.21.15.42:0/227744159 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fe57c0318b0 con 0x5585a797c840 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.113+0000 7fe59ade6740 1 -- 172.21.15.42:0/227744159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe588015070 con 0x5585a797c840 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.113+0000 7fe576a00640 1 -- 172.21.15.42:0/227744159 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe57c005d50 con 0x5585a797c840 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.113+0000 7fe59ade6740 10 client.25889 fetch_fsmap learned FSMap version 30 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.113+0000 7fe59ade6740 1 -- 172.21.15.42:0/227744159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5585a79906a0 con 0x5585a797c840 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.114+0000 7fe576a00640 1 -- 172.21.15.42:0/227744159 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe588015070 con 0x5585a797c840 2024-04-25T00:27:39.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.114+0000 7fe576a00640 1 -- 172.21.15.42:0/227744159 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fe57c0035f0 con 0x5585a797c840 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.114+0000 7fe59ade6740 10 client.25889 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.114+0000 7fe59ade6740 10 client.25889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.114+0000 7fe59ade6740 20 client.25889 populate_metadata read hostname 'smithi042' 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.114+0000 7fe59ade6740 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5585a7991d80 0x5585a7994190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.114+0000 7fe59ade6740 4 client.25889 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.114+0000 7fe59ade6740 1 -- 172.21.15.42:0/227744159 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585a78ebff0 con 0x5585a7991d80 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.115+0000 7fe596000640 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5585a7991d80 0x5585a7994190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.115+0000 7fe596000640 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5585a7991d80 0x5585a7994190 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd054ea0faf0818 server_cookie=b0aacf83799308eb in_seq=0 out_seq=0 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.115+0000 7fe576a00640 10 client.25889 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe576a00640 1 -- 172.21.15.42:0/227744159 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5585a78ebff0 con 0x5585a7991d80 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe576a00640 10 client.25889 handle_command_reply: tid=0 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 client.25889 shutdown 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 -- 172.21.15.42:0/227744159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 msgr2=0x5585a7986610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a7986610 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe57c000c00 tx=0x7fe57c0052f0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 -- 172.21.15.42:0/227744159 shutdown_connections 2024-04-25T00:27:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5585a7991d80 0x5585a7994190 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a79819b0 0x5585a798d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a7986610 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 --2- 172.21.15.42:0/227744159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a79860d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe576a00640 0 client.25889 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 -- 172.21.15.42:0/227744159 >> 172.21.15.42:0/227744159 conn(0x5585a79774c0 msgr2=0x5585a7976620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 -- 172.21.15.42:0/227744159 shutdown_connections 2024-04-25T00:27:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 1 -- 172.21.15.42:0/227744159 wait complete. 2024-04-25T00:27:39.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.116+0000 7fe59ade6740 20 client.25889 trim_cache size 0 max 16384 2024-04-25T00:27:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.133+0000 7fe59ade6740 1 Processor -- start 2024-04-25T00:27:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.133+0000 7fe59ade6740 1 -- start start 2024-04-25T00:27:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797be50 0x5585a797a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a798d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a79819b0 0x5585a7998a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe59ade6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901ad4a0 con 0x5585a797be50 2024-04-25T00:27:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe59ade6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901aad40 con 0x5585a797c840 2024-04-25T00:27:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe59ade6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901a8b90 con 0x5585a79819b0 2024-04-25T00:27:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe594c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a798d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe594c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a798d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34740/0 (socket says 172.21.15.42:34740) 2024-04-25T00:27:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe596000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797be50 0x5585a797a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe594c00640 1 -- 172.21.15.42:0/3505300368 learned_addr learned my addr 172.21.15.42:0/3505300368 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe594c00640 1 -- 172.21.15.42:0/3505300368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a79819b0 msgr2=0x5585a7998a90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe594c00640 1 --2- 172.21.15.42:0/3505300368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a79819b0 0x5585a7998a90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe594c00640 1 -- 172.21.15.42:0/3505300368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797be50 msgr2=0x5585a797a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe594c00640 1 --2- 172.21.15.42:0/3505300368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797be50 0x5585a797a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.134+0000 7fe594c00640 1 -- 172.21.15.42:0/3505300368 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585a79772f0 con 0x5585a797c840 2024-04-25T00:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.135+0000 7fe594c00640 1 --2- 172.21.15.42:0/3505300368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a798d610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe57c003a90 tx=0x7fe57c003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=956a6df0a7083dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.135+0000 7fe581c00640 1 -- 172.21.15.42:0/3505300368 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c035030 con 0x5585a797c840 2024-04-25T00:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.135+0000 7fe581c00640 1 -- 172.21.15.42:0/3505300368 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57c053020 con 0x5585a797c840 2024-04-25T00:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.135+0000 7fe581c00640 1 -- 172.21.15.42:0/3505300368 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c0035f0 con 0x5585a797c840 2024-04-25T00:27:39.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.136+0000 7fe59ade6740 1 -- 172.21.15.42:0/3505300368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 msgr2=0x5585a798d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.136+0000 7fe59ade6740 1 --2- 172.21.15.42:0/3505300368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a798d610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe57c003a90 tx=0x7fe57c003f10 comp rx=0 tx=0).stop 2024-04-25T00:27:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.136+0000 7fe59ade6740 1 -- 172.21.15.42:0/3505300368 shutdown_connections 2024-04-25T00:27:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.136+0000 7fe59ade6740 1 --2- 172.21.15.42:0/3505300368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a79819b0 0x5585a7998a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.136+0000 7fe59ade6740 1 --2- 172.21.15.42:0/3505300368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a797c840 0x5585a798d610 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.136+0000 7fe59ade6740 1 --2- 172.21.15.42:0/3505300368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797be50 0x5585a797a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.136+0000 7fe59ade6740 1 -- 172.21.15.42:0/3505300368 >> 172.21.15.42:0/3505300368 conn(0x5585a79774c0 msgr2=0x5585a797a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.136+0000 7fe59ade6740 1 -- 172.21.15.42:0/3505300368 shutdown_connections 2024-04-25T00:27:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.136+0000 7fe59ade6740 1 -- 172.21.15.42:0/3505300368 wait complete. 2024-04-25T00:27:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.136+0000 7fe59ade6740 1 -- start start 2024-04-25T00:27:39.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe59ade6740 1 Processor -- start 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a798b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797c840 0x5585a798b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe59ade6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a79819b0 0x5585a798c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe59ade6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901a9200 con 0x5585a797c840 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe59ade6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901aa670 con 0x5585a79819b0 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe59ade6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901a98d0 con 0x5585a797be50 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe59ade6740 10 client.0 _ll_register_callbacks cb 0x7fe5840093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe596000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a798b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe596000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a798b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60332/0 (socket says 172.21.15.42:60332) 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe596000640 1 -- 172.21.15.42:0/4135586990 learned_addr learned my addr 172.21.15.42:0/4135586990 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:39.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe596000640 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a79819b0 0x5585a798c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.137+0000 7fe594c00640 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797c840 0x5585a798b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe596000640 1 -- 172.21.15.42:0/4135586990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a79819b0 msgr2=0x5585a798c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe596000640 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a79819b0 0x5585a798c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe596000640 1 -- 172.21.15.42:0/4135586990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797c840 msgr2=0x5585a798b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe596000640 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797c840 0x5585a798b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe596000640 1 -- 172.21.15.42:0/4135586990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585a798cc30 con 0x5585a797be50 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe594c00640 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797c840 0x5585a798b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe596000640 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a798b140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b6b0 tx=0x7fe588013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce1f22118372317a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe576a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe576a00640 1 -- 172.21.15.42:0/4135586990 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe588023040 con 0x5585a797be50 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe576a00640 1 -- 172.21.15.42:0/4135586990 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fe58801a670 con 0x5585a797be50 2024-04-25T00:27:39.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.138+0000 7fe59ade6740 1 -- 172.21.15.42:0/4135586990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe57c053020 con 0x5585a797be50 2024-04-25T00:27:39.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.139+0000 7fe576a00640 1 -- 172.21.15.42:0/4135586990 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe58801a870 con 0x5585a797be50 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.139+0000 7fe59ade6740 10 client.25900 fetch_fsmap learned FSMap version 30 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.139+0000 7fe59ade6740 1 -- 172.21.15.42:0/4135586990 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5585a79f86c0 con 0x5585a797be50 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.139+0000 7fe576a00640 1 -- 172.21.15.42:0/4135586990 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe57c053020 con 0x5585a797be50 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.139+0000 7fe576a00640 1 -- 172.21.15.42:0/4135586990 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fe58803b470 con 0x5585a797be50 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.139+0000 7fe59ade6740 10 client.25900 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.139+0000 7fe59ade6740 10 client.25900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.139+0000 7fe59ade6740 20 client.25900 populate_metadata read hostname 'smithi042' 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.140+0000 7fe59ade6740 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5585a79f8ed0 0x5585a79fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.140+0000 7fe59ade6740 4 client.25900 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.140+0000 7fe59ade6740 1 -- 172.21.15.42:0/4135586990 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585a78e6340 con 0x5585a79f8ed0 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.140+0000 7fe594c00640 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5585a79f8ed0 0x5585a79fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.140+0000 7fe594c00640 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5585a79f8ed0 0x5585a79fb2f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8136f7097a4f53af server_cookie=73931244aade1418 in_seq=0 out_seq=0 2024-04-25T00:27:39.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.140+0000 7fe576a00640 10 client.25900 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe576a00640 1 -- 172.21.15.42:0/4135586990 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5585a78e6340 con 0x5585a79f8ed0 2024-04-25T00:27:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe576a00640 10 client.25900 handle_command_reply: tid=0 2024-04-25T00:27:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe59ade6740 1 client.25900 shutdown 2024-04-25T00:27:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe59ade6740 1 -- 172.21.15.42:0/4135586990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 msgr2=0x5585a798b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe59ade6740 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a798b140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe58800b6b0 tx=0x7fe588013040 comp rx=0 tx=0).stop 2024-04-25T00:27:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe59ade6740 1 -- 172.21.15.42:0/4135586990 shutdown_connections 2024-04-25T00:27:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe59ade6740 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5585a79f8ed0 0x5585a79fb2f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe59ade6740 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5585a79819b0 0x5585a798c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe59ade6740 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5585a797c840 0x5585a798b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe576a00640 0 client.25900 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe59ade6740 1 --2- 172.21.15.42:0/4135586990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5585a797be50 0x5585a798b140 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.198+0000 7fe59ade6740 1 -- 172.21.15.42:0/4135586990 >> 172.21.15.42:0/4135586990 conn(0x5585a79774c0 msgr2=0x5585a797aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.199+0000 7fe59ade6740 1 -- 172.21.15.42:0/4135586990 shutdown_connections 2024-04-25T00:27:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.199+0000 7fe59ade6740 1 -- 172.21.15.42:0/4135586990 wait complete. 2024-04-25T00:27:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.199+0000 7fe59ade6740 20 client.25900 trim_cache size 0 max 16384 2024-04-25T00:27:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:27:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:27:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:27:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:27:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.199+0000 7fe596a00640 1 -- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5780a2c00 msgr2=0x7fe5780a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.199+0000 7fe596a00640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5780a2c00 0x7fe5780a50c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe5840a46e0 tx=0x7fe5840d5530 comp rx=0 tx=0).stop 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.199+0000 7fe596a00640 1 -- 172.21.15.42:0/946696189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5901004c0 msgr2=0x7fe590104450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.199+0000 7fe596a00640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5901004c0 0x7fe590104450 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe58400eb70 tx=0x7fe58400c6a0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.200+0000 7fe596a00640 1 -- 172.21.15.42:0/946696189 shutdown_connections 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.200+0000 7fe596a00640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5780a2c00 0x7fe5780a50c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.200+0000 7fe596a00640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590102070 0x7fe590102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.200+0000 7fe596a00640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5901016b0 0x7fe590102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.200+0000 7fe596a00640 1 --2- 172.21.15.42:0/946696189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5901004c0 0x7fe590104450 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.200+0000 7fe596a00640 1 -- 172.21.15.42:0/946696189 >> 172.21.15.42:0/946696189 conn(0x7fe5900fba30 msgr2=0x7fe5900fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.200+0000 7fe596a00640 1 -- 172.21.15.42:0/946696189 shutdown_connections 2024-04-25T00:27:39.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:39.200+0000 7fe596a00640 1 -- 172.21.15.42:0/946696189 wait complete. 2024-04-25T00:27:39.256 INFO:tasks.workunit.client.0.smithi042.stdout: 1 536384 6.33 MB/sec execute 329 sec latency 17.978 ms 2024-04-25T00:27:39.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:39 smithi042 ceph-mon[28130]: pgmap v655: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-04-25T00:27:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:39 smithi136 ceph-mon[32923]: pgmap v655: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-04-25T00:27:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:39 smithi106 ceph-mon[32510]: pgmap v655: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-04-25T00:27:40.257 INFO:tasks.workunit.client.0.smithi042.stdout: 1 537030 6.32 MB/sec execute 330 sec latency 20.626 ms 2024-04-25T00:27:41.257 INFO:tasks.workunit.client.0.smithi042.stdout: 1 538135 6.32 MB/sec execute 331 sec latency 22.058 ms 2024-04-25T00:27:41.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:41 smithi042 ceph-mon[28130]: pgmap v656: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-04-25T00:27:42.015 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:41 smithi136 ceph-mon[32923]: pgmap v656: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-04-25T00:27:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:41 smithi106 ceph-mon[32510]: pgmap v656: 97 pgs: 97 active+clean; 107 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.5 MiB/s wr, 190 op/s 2024-04-25T00:27:42.257 INFO:tasks.workunit.client.0.smithi042.stdout: 1 539980 6.34 MB/sec execute 332 sec latency 49.919 ms 2024-04-25T00:27:43.257 INFO:tasks.workunit.client.0.smithi042.stdout: 1 540530 6.32 MB/sec execute 333 sec latency 19.690 ms 2024-04-25T00:27:43.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:43 smithi042 ceph-mon[28130]: pgmap v657: 97 pgs: 97 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-04-25T00:27:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:43 smithi136 ceph-mon[32923]: pgmap v657: 97 pgs: 97 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-04-25T00:27:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:43 smithi106 ceph-mon[32510]: pgmap v657: 97 pgs: 97 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 2.1 MiB/s wr, 259 op/s 2024-04-25T00:27:44.257 INFO:tasks.workunit.client.0.smithi042.stdout: 1 541538 6.32 MB/sec execute 334 sec latency 25.540 ms 2024-04-25T00:27:45.257 INFO:tasks.workunit.client.0.smithi042.stdout: 1 543146 6.32 MB/sec execute 335 sec latency 23.630 ms 2024-04-25T00:27:45.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:45 smithi042 ceph-mon[28130]: pgmap v658: 97 pgs: 97 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-04-25T00:27:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:45 smithi136 ceph-mon[32923]: pgmap v658: 97 pgs: 97 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-04-25T00:27:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:45 smithi106 ceph-mon[32510]: pgmap v658: 97 pgs: 97 active+clean; 109 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.7 MiB/s wr, 185 op/s 2024-04-25T00:27:46.257 INFO:tasks.workunit.client.0.smithi042.stdout: 1 543924 6.32 MB/sec execute 336 sec latency 19.188 ms 2024-04-25T00:27:47.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 544801 6.31 MB/sec execute 337 sec latency 67.251 ms 2024-04-25T00:27:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:47 smithi042 ceph-mon[28130]: pgmap v659: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 2.3 MiB/s wr, 256 op/s 2024-04-25T00:27:47.889 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:47 smithi106 ceph-mon[32510]: pgmap v659: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 2.3 MiB/s wr, 256 op/s 2024-04-25T00:27:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:47 smithi136 ceph-mon[32923]: pgmap v659: 97 pgs: 97 active+clean; 111 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 2.3 MiB/s wr, 256 op/s 2024-04-25T00:27:48.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 546176 6.31 MB/sec execute 338 sec latency 26.237 ms 2024-04-25T00:27:49.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 547370 6.32 MB/sec execute 339 sec latency 18.496 ms 2024-04-25T00:27:49.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:49 smithi042 ceph-mon[28130]: pgmap v660: 97 pgs: 97 active+clean; 110 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-04-25T00:27:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:49 smithi136 ceph-mon[32923]: pgmap v660: 97 pgs: 97 active+clean; 110 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-04-25T00:27:50.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:49 smithi106 ceph-mon[32510]: pgmap v660: 97 pgs: 97 active+clean; 110 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.7 MiB/s wr, 189 op/s 2024-04-25T00:27:50.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 548014 6.31 MB/sec execute 340 sec latency 19.399 ms 2024-04-25T00:27:51.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 549416 6.31 MB/sec execute 341 sec latency 31.218 ms 2024-04-25T00:27:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:51 smithi136 ceph-mon[32923]: pgmap v661: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-04-25T00:27:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:51 smithi106 ceph-mon[32510]: pgmap v661: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-04-25T00:27:52.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:51 smithi042 ceph-mon[28130]: pgmap v661: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 460 KiB/s rd, 1.7 MiB/s wr, 201 op/s 2024-04-25T00:27:52.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 550823 6.32 MB/sec execute 342 sec latency 72.841 ms 2024-04-25T00:27:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.353+0000 7faf06400640 1 -- 172.21.15.42:0/2648939961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00077ce0 msgr2=0x7faf00078580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.353+0000 7faf06400640 1 --2- 172.21.15.42:0/2648939961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00077ce0 0x7faf00078580 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faef8009ca0 tx=0x7faef802a8c0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.353+0000 7faf06400640 1 -- 172.21.15.42:0/2648939961 shutdown_connections 2024-04-25T00:27:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.353+0000 7faf06400640 1 --2- 172.21.15.42:0/2648939961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00077ce0 0x7faf00078580 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.353+0000 7faf06400640 1 --2- 172.21.15.42:0/2648939961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00077340 0x7faf000777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.353+0000 7faf06400640 1 --2- 172.21.15.42:0/2648939961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00076100 0x7faf00076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.353+0000 7faf06400640 1 -- 172.21.15.42:0/2648939961 >> 172.21.15.42:0/2648939961 conn(0x7faf000fed50 msgr2=0x7faf00101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.353+0000 7faf06400640 1 -- 172.21.15.42:0/2648939961 shutdown_connections 2024-04-25T00:27:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.353+0000 7faf06400640 1 -- 172.21.15.42:0/2648939961 wait complete. 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.354+0000 7faf06400640 1 Processor -- start 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.354+0000 7faf06400640 1 -- start start 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faf06400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00076100 0x7faf0019deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faf06400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00077340 0x7faf0019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faf06400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00077ce0 0x7faf001a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faf06400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faf001138b0 con 0x7faf00077340 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faf06400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faf00113730 con 0x7faf00076100 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faf06400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faf00113a30 con 0x7faf00077ce0 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faf05000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00076100 0x7faf0019deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faf05000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00076100 0x7faf0019deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:50708/0 (socket says 172.21.15.42:50708) 2024-04-25T00:27:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faf05000640 1 -- 172.21.15.42:0/2567301899 learned_addr learned my addr 172.21.15.42:0/2567301899 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faf05a00640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00077ce0 0x7faf001a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.355+0000 7faeffe00640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00077340 0x7faf0019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faf05000640 1 -- 172.21.15.42:0/2567301899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00077ce0 msgr2=0x7faf001a5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faf05000640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00077ce0 0x7faf001a5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faf05000640 1 -- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00077340 msgr2=0x7faf0019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faf05000640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00077340 0x7faf0019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faf05000640 1 -- 172.21.15.42:0/2567301899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf001a5a80 con 0x7faf00076100 2024-04-25T00:27:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faeffe00640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00077340 0x7faf0019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faf05000640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00076100 0x7faf0019deb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faef400e9f0 tx=0x7faef400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faefd600640 1 -- 172.21.15.42:0/2567301899 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef400cd80 con 0x7faf00076100 2024-04-25T00:27:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faefd600640 1 -- 172.21.15.42:0/2567301899 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef4005500 con 0x7faf00076100 2024-04-25T00:27:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faf06400640 1 -- 172.21.15.42:0/2567301899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf001a5d70 con 0x7faf00076100 2024-04-25T00:27:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.356+0000 7faf06400640 1 -- 172.21.15.42:0/2567301899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf001a6200 con 0x7faf00076100 2024-04-25T00:27:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.357+0000 7faefd600640 1 -- 172.21.15.42:0/2567301899 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4010690 con 0x7faf00076100 2024-04-25T00:27:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.359+0000 7faefd600640 1 -- 172.21.15.42:0/2567301899 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faef40046c0 con 0x7faf00076100 2024-04-25T00:27:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.360+0000 7faf0a97e740 1 Processor -- start 2024-04-25T00:27:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.360+0000 7faf0a97e740 1 -- start start 2024-04-25T00:27:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.360+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e93270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.360+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 0x5572e1e98340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1e9ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf0a97e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5572e1df5ab0 con 0x5572e1e93840 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf0a97e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5572e1d9e640 con 0x5572e1e989b0 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf0a97e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5572e1df1a70 con 0x5572e1e92e50 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf05a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e93270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf05a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e93270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:46616/0 (socket says 172.21.15.42:46616) 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf05a00640 1 -- 172.21.15.42:0/2955475947 learned_addr learned my addr 172.21.15.42:0/2955475947 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf05000640 1 --2- 172.21.15.42:0/2955475947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 0x5572e1e98340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faeffe00640 1 --2- 172.21.15.42:0/2955475947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1e9ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf05a00640 1 -- 172.21.15.42:0/2955475947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 msgr2=0x5572e1e9ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf05a00640 1 --2- 172.21.15.42:0/2955475947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1e9ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf05a00640 1 -- 172.21.15.42:0/2955475947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 msgr2=0x5572e1e98340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf05a00640 1 --2- 172.21.15.42:0/2955475947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 0x5572e1e98340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.361+0000 7faf05a00640 1 -- 172.21.15.42:0/2955475947 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e1b0a210 con 0x5572e1e92e50 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.362+0000 7faf05a00640 1 --2- 172.21.15.42:0/2955475947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e93270 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faef8009ca0 tx=0x7faef80093c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7d4a8ddf5e63a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.362+0000 7faefd600640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2c00 0x7faee80a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.362+0000 7faefd600640 1 -- 172.21.15.42:0/2567301899 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7faef40cf410 con 0x7faf00076100 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.362+0000 7faef2000640 1 -- 172.21.15.42:0/2955475947 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8031650 con 0x5572e1e92e50 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.362+0000 7faef2000640 1 -- 172.21.15.42:0/2955475947 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef8036030 con 0x5572e1e92e50 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.362+0000 7faf05000640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2c00 0x7faee80a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.362+0000 7faef2000640 1 -- 172.21.15.42:0/2955475947 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8004740 con 0x5572e1e92e50 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf0a97e740 1 -- 172.21.15.42:0/2955475947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 msgr2=0x5572e1e93270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf0a97e740 1 --2- 172.21.15.42:0/2955475947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e93270 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faef8009ca0 tx=0x7faef80093c0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf05000640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2c00 0x7faee80a50c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faef409b940 tx=0x7faef40d2b80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf0a97e740 1 -- 172.21.15.42:0/2955475947 shutdown_connections 2024-04-25T00:27:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf0a97e740 1 --2- 172.21.15.42:0/2955475947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1e9ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf0a97e740 1 --2- 172.21.15.42:0/2955475947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 0x5572e1e98340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf0a97e740 1 --2- 172.21.15.42:0/2955475947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e93270 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf0a97e740 1 -- 172.21.15.42:0/2955475947 >> 172.21.15.42:0/2955475947 conn(0x5572e1e8e4c0 msgr2=0x5572e1e8e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf0a97e740 1 -- 172.21.15.42:0/2955475947 shutdown_connections 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf0a97e740 1 -- 172.21.15.42:0/2955475947 wait complete. 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.363+0000 7faf0a97e740 1 -- start start 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faf0a97e740 1 Processor -- start 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e9cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e93840 0x5572e1e9d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e989b0 0x5572e1ea4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faf0a97e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5572e1e9b780 con 0x5572e1e989b0 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faf0a97e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5572e1db1990 con 0x5572e1e93840 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faf0a97e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5572e1e9b900 con 0x5572e1e92e50 2024-04-25T00:27:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faf0a97e740 10 client.0 _ll_register_callbacks cb 0x5572e1db3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faeffe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e93840 0x5572e1e9d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faf05a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e9cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faeffe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e93840 0x5572e1e9d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:50730/0 (socket says 172.21.15.42:50730) 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.364+0000 7faeffe00640 1 -- 172.21.15.42:0/3471520550 learned_addr learned my addr 172.21.15.42:0/3471520550 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faf05a00640 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e989b0 0x5572e1ea4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faeffe00640 1 -- 172.21.15.42:0/3471520550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 msgr2=0x5572e1e9cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faeffe00640 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e9cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faeffe00640 1 -- 172.21.15.42:0/3471520550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e989b0 msgr2=0x5572e1ea4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faeffe00640 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e989b0 0x5572e1ea4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faeffe00640 1 -- 172.21.15.42:0/3471520550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e1bd6620 con 0x5572e1e93840 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faf05a00640 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e9cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faeffe00640 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e93840 0x5572e1e9d420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faeec013790 tx=0x7faeec013c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed2de8bd0d659bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faee6a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:27:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faee6a00640 1 -- 172.21.15.42:0/3471520550 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec020060 con 0x5572e1e93840 2024-04-25T00:27:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.365+0000 7faee6a00640 1 -- 172.21.15.42:0/3471520550 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7faeec011dd0 con 0x5572e1e93840 2024-04-25T00:27:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.366+0000 7faf0a97e740 1 -- 172.21.15.42:0/3471520550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef8036030 con 0x5572e1e93840 2024-04-25T00:27:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.366+0000 7faee6a00640 1 -- 172.21.15.42:0/3471520550 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faeec00f450 con 0x5572e1e93840 2024-04-25T00:27:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.366+0000 7faf0a97e740 10 client.25907 fetch_fsmap learned FSMap version 30 2024-04-25T00:27:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.366+0000 7faf0a97e740 1 -- 172.21.15.42:0/3471520550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5572e1ea74e0 con 0x5572e1e93840 2024-04-25T00:27:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.366+0000 7faee6a00640 1 -- 172.21.15.42:0/3471520550 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef8036030 con 0x5572e1e93840 2024-04-25T00:27:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.366+0000 7faee6a00640 1 -- 172.21.15.42:0/3471520550 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7faeec01a030 con 0x5572e1e93840 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.367+0000 7faf0a97e740 10 client.25907 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.367+0000 7faf0a97e740 10 client.25907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.367+0000 7faf0a97e740 20 client.25907 populate_metadata read hostname 'smithi042' 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.367+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5572e1ea8c20 0x5572e1eab060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.367+0000 7faf0a97e740 4 client.25907 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.367+0000 7faf0a97e740 1 -- 172.21.15.42:0/3471520550 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572e1e02ff0 con 0x5572e1ea8c20 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.367+0000 7faf05a00640 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5572e1ea8c20 0x5572e1eab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.368+0000 7faf05a00640 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5572e1ea8c20 0x5572e1eab060 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a9fa23522270366 server_cookie=90075835e44580e4 in_seq=0 out_seq=0 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.368+0000 7faee6a00640 10 client.25907 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.368+0000 7faee6a00640 1 -- 172.21.15.42:0/3471520550 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572e1e02ff0 con 0x5572e1ea8c20 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faee6a00640 10 client.25907 handle_command_reply: tid=0 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 client.25907 shutdown 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 -- 172.21.15.42:0/3471520550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e93840 msgr2=0x5572e1e9d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:52.372 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e93840 0x5572e1e9d420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7faeec013790 tx=0x7faeec013c50 comp rx=0 tx=0).stop 2024-04-25T00:27:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 -- 172.21.15.42:0/3471520550 shutdown_connections 2024-04-25T00:27:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5572e1ea8c20 0x5572e1eab060 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e989b0 0x5572e1ea4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e93840 0x5572e1e9d420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3471520550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e9cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 -- 172.21.15.42:0/3471520550 >> 172.21.15.42:0/3471520550 conn(0x5572e1e8e4c0 msgr2=0x5572e1e8d620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faee6a00640 0 client.25907 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 -- 172.21.15.42:0/3471520550 shutdown_connections 2024-04-25T00:27:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.369+0000 7faf0a97e740 1 -- 172.21.15.42:0/3471520550 wait complete. 2024-04-25T00:27:52.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.370+0000 7faf0a97e740 20 client.25907 trim_cache size 0 max 16384 2024-04-25T00:27:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.387+0000 7faf0a97e740 1 Processor -- start 2024-04-25T00:27:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf0a97e740 1 -- start start 2024-04-25T00:27:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e91bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 0x5572e1ea4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1e89110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf0a97e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faee0010ca0 con 0x5572e1e93840 2024-04-25T00:27:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf0a97e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faee000e540 con 0x5572e1e989b0 2024-04-25T00:27:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf0a97e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faee000c390 con 0x5572e1e92e50 2024-04-25T00:27:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf05a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e91bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf05a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e91bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:46636/0 (socket says 172.21.15.42:46636) 2024-04-25T00:27:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf05a00640 1 -- 172.21.15.42:0/2958739444 learned_addr learned my addr 172.21.15.42:0/2958739444 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faf05a00640 1 --2- 172.21.15.42:0/2958739444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1e89110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.388+0000 7faeffe00640 1 --2- 172.21.15.42:0/2958739444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 0x5572e1ea4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.389+0000 7faf05a00640 1 -- 172.21.15.42:0/2958739444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 msgr2=0x5572e1e89110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.389+0000 7faf05a00640 1 --2- 172.21.15.42:0/2958739444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1e89110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.389+0000 7faf05a00640 1 -- 172.21.15.42:0/2958739444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 msgr2=0x5572e1ea4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.389+0000 7faf05a00640 1 --2- 172.21.15.42:0/2958739444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 0x5572e1ea4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.389+0000 7faf05a00640 1 -- 172.21.15.42:0/2958739444 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572e1e8e2f0 con 0x5572e1e92e50 2024-04-25T00:27:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.389+0000 7faeffe00640 1 --2- 172.21.15.42:0/2958739444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 0x5572e1ea4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.389+0000 7faf05a00640 1 --2- 172.21.15.42:0/2958739444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e91bc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faef8006010 tx=0x7faef8031bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=571d9770d30df99f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.389+0000 7faef1600640 1 -- 172.21.15.42:0/2958739444 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef80047a0 con 0x5572e1e92e50 2024-04-25T00:27:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.389+0000 7faef1600640 1 -- 172.21.15.42:0/2958739444 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef8036040 con 0x5572e1e92e50 2024-04-25T00:27:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.389+0000 7faef1600640 1 -- 172.21.15.42:0/2958739444 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8057910 con 0x5572e1e92e50 2024-04-25T00:27:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.390+0000 7faf0a97e740 1 -- 172.21.15.42:0/2958739444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 msgr2=0x5572e1e91bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.390+0000 7faf0a97e740 1 --2- 172.21.15.42:0/2958739444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e91bc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faef8006010 tx=0x7faef8031bb0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.390+0000 7faf0a97e740 1 -- 172.21.15.42:0/2958739444 shutdown_connections 2024-04-25T00:27:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.390+0000 7faf0a97e740 1 --2- 172.21.15.42:0/2958739444 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1e89110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.390+0000 7faf0a97e740 1 --2- 172.21.15.42:0/2958739444 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e93840 0x5572e1ea4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.390+0000 7faf0a97e740 1 --2- 172.21.15.42:0/2958739444 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e92e50 0x5572e1e91bc0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.390+0000 7faf0a97e740 1 -- 172.21.15.42:0/2958739444 >> 172.21.15.42:0/2958739444 conn(0x5572e1e8e4c0 msgr2=0x5572e1e914b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.390+0000 7faf0a97e740 1 -- 172.21.15.42:0/2958739444 shutdown_connections 2024-04-25T00:27:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.390+0000 7faf0a97e740 1 -- 172.21.15.42:0/2958739444 wait complete. 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.391+0000 7faf0a97e740 1 -- start start 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.391+0000 7faf0a97e740 1 Processor -- start 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.391+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e92e50 0x5572e1ea1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.391+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e93840 0x5572e1ea1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.391+0000 7faf0a97e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1ea2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.391+0000 7faf0a97e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7faee000ca00 con 0x5572e1e92e50 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.391+0000 7faf0a97e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7faee000de70 con 0x5572e1e989b0 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.391+0000 7faf0a97e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faee000d0d0 con 0x5572e1e93840 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.391+0000 7faf0a97e740 10 client.0 _ll_register_callbacks cb 0x7faeec0096d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faf05a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e92e50 0x5572e1ea1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faeffe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e93840 0x5572e1ea1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faf05a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e92e50 0x5572e1ea1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:38790/0 (socket says 172.21.15.42:38790) 2024-04-25T00:27:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faf05a00640 1 -- 172.21.15.42:0/3308614394 learned_addr learned my addr 172.21.15.42:0/3308614394 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faf05a00640 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1ea2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faeffe00640 1 -- 172.21.15.42:0/3308614394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 msgr2=0x5572e1ea2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faeffe00640 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1ea2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faeffe00640 1 -- 172.21.15.42:0/3308614394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e92e50 msgr2=0x5572e1ea1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faeffe00640 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e92e50 0x5572e1ea1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faeffe00640 1 -- 172.21.15.42:0/3308614394 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572e1e8f190 con 0x5572e1e93840 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faf05a00640 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1ea2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faeffe00640 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e93840 0x5572e1ea1fd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faeec026900 tx=0x7faeec009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=824c4a0033edb532 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.392+0000 7faee6a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.393+0000 7faee6a00640 1 -- 172.21.15.42:0/3308614394 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec02b070 con 0x5572e1e93840 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.393+0000 7faf0a97e740 1 -- 172.21.15.42:0/3308614394 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faef8036040 con 0x5572e1e93840 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.393+0000 7faee6a00640 1 -- 172.21.15.42:0/3308614394 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7faeec0263b0 con 0x5572e1e93840 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.393+0000 7faee6a00640 1 -- 172.21.15.42:0/3308614394 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7faeec026610 con 0x5572e1e93840 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.393+0000 7faf0a97e740 10 client.25918 fetch_fsmap learned FSMap version 30 2024-04-25T00:27:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.393+0000 7faf0a97e740 1 -- 172.21.15.42:0/3308614394 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5572e1ea4710 con 0x5572e1e93840 2024-04-25T00:27:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.393+0000 7faee6a00640 1 -- 172.21.15.42:0/3308614394 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faef8036040 con 0x5572e1e93840 2024-04-25T00:27:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.394+0000 7faee6a00640 1 -- 172.21.15.42:0/3308614394 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7faeec01a040 con 0x5572e1e93840 2024-04-25T00:27:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.394+0000 7faf0a97e740 10 client.25918 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:27:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.394+0000 7faf0a97e740 10 client.25918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:27:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.394+0000 7faf0a97e740 20 client.25918 populate_metadata read hostname 'smithi042' 2024-04-25T00:27:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.394+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5572e1f0f9b0 0x5572e1f11dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:27:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.394+0000 7faf0a97e740 4 client.25918 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:27:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.394+0000 7faf0a97e740 1 -- 172.21.15.42:0/3308614394 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5572e1dfd340 con 0x5572e1f0f9b0 2024-04-25T00:27:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.394+0000 7faf05a00640 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5572e1f0f9b0 0x5572e1f11dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:27:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.394+0000 7faf05a00640 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5572e1f0f9b0 0x5572e1f11dd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5db633d5535f002 server_cookie=d36f19e0f85f0768 in_seq=0 out_seq=0 2024-04-25T00:27:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.395+0000 7faee6a00640 10 client.25918 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.442+0000 7faee6a00640 1 -- 172.21.15.42:0/3308614394 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5572e1dfd340 con 0x5572e1f0f9b0 2024-04-25T00:27:52.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.442+0000 7faee6a00640 10 client.25918 handle_command_reply: tid=0 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.442+0000 7faf0a97e740 1 client.25918 shutdown 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 1 -- 172.21.15.42:0/3308614394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e93840 msgr2=0x5572e1ea1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e93840 0x5572e1ea1fd0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7faeec026900 tx=0x7faeec009fa0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 1 -- 172.21.15.42:0/3308614394 shutdown_connections 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5572e1f0f9b0 0x5572e1f11dd0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5572e1e989b0 0x5572e1ea2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5572e1e93840 0x5572e1ea1fd0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 1 --2- 172.21.15.42:0/3308614394 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5572e1e92e50 0x5572e1ea1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 1 -- 172.21.15.42:0/3308614394 >> 172.21.15.42:0/3308614394 conn(0x5572e1e8e4c0 msgr2=0x5572e1e9bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faee6a00640 0 client.25918 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 1 -- 172.21.15.42:0/3308614394 shutdown_connections 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 1 -- 172.21.15.42:0/3308614394 wait complete. 2024-04-25T00:27:52.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.443+0000 7faf0a97e740 20 client.25918 trim_cache size 0 max 16384 2024-04-25T00:27:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:27:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:27:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:27:52.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 -- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2c00 msgr2=0x7faee80a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2c00 0x7faee80a50c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7faef409b940 tx=0x7faef40d2b80 comp rx=0 tx=0).stop 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 -- 172.21.15.42:0/2567301899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00076100 msgr2=0x7faf0019deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00076100 0x7faf0019deb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7faef400e9f0 tx=0x7faef400edf0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 -- 172.21.15.42:0/2567301899 shutdown_connections 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faee80a2c00 0x7faee80a50c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faf00077ce0 0x7faf001a5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7faf00077340 0x7faf0019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 --2- 172.21.15.42:0/2567301899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf00076100 0x7faf0019deb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 -- 172.21.15.42:0/2567301899 >> 172.21.15.42:0/2567301899 conn(0x7faf000fed50 msgr2=0x7faf00100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:27:52.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 -- 172.21.15.42:0/2567301899 shutdown_connections 2024-04-25T00:27:52.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:27:52.444+0000 7faf06400640 1 -- 172.21.15.42:0/2567301899 wait complete. 2024-04-25T00:27:53.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 551335 6.30 MB/sec execute 343 sec latency 35.023 ms 2024-04-25T00:27:53.929 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:53 smithi042 ceph-mon[28130]: pgmap v662: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-04-25T00:27:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:53 smithi136 ceph-mon[32923]: pgmap v662: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-04-25T00:27:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:53 smithi106 ceph-mon[32510]: pgmap v662: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.0 MiB/s wr, 226 op/s 2024-04-25T00:27:54.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 552317 6.30 MB/sec execute 344 sec latency 28.883 ms 2024-04-25T00:27:55.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 554111 6.32 MB/sec execute 345 sec latency 16.335 ms 2024-04-25T00:27:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:55 smithi136 ceph-mon[32923]: pgmap v663: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-04-25T00:27:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:55 smithi106 ceph-mon[32510]: pgmap v663: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-04-25T00:27:56.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:55 smithi042 ceph-mon[28130]: pgmap v663: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 354 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-04-25T00:27:56.258 INFO:tasks.workunit.client.0.smithi042.stdout: 1 554633 6.30 MB/sec execute 346 sec latency 20.198 ms 2024-04-25T00:27:57.259 INFO:tasks.workunit.client.0.smithi042.stdout: 1 555595 6.30 MB/sec execute 347 sec latency 83.207 ms 2024-04-25T00:27:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:57 smithi136 ceph-mon[32923]: pgmap v664: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-04-25T00:27:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:57 smithi106 ceph-mon[32510]: pgmap v664: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-04-25T00:27:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:57 smithi042 ceph-mon[28130]: pgmap v664: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-04-25T00:27:58.259 INFO:tasks.workunit.client.0.smithi042.stdout: 1 556939 6.30 MB/sec execute 348 sec latency 23.349 ms 2024-04-25T00:27:59.259 INFO:tasks.workunit.client.0.smithi042.stdout: 1 558026 6.30 MB/sec execute 349 sec latency 19.171 ms 2024-04-25T00:28:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:27:59 smithi136 ceph-mon[32923]: pgmap v665: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-25T00:28:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:27:59 smithi106 ceph-mon[32510]: pgmap v665: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-25T00:28:00.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:27:59 smithi042 ceph-mon[28130]: pgmap v665: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-25T00:28:00.259 INFO:tasks.workunit.client.0.smithi042.stdout: 1 558649 6.29 MB/sec execute 350 sec latency 44.259 ms 2024-04-25T00:28:01.259 INFO:tasks.workunit.client.0.smithi042.stdout: 1 560172 6.29 MB/sec execute 351 sec latency 21.268 ms 2024-04-25T00:28:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:01 smithi136 ceph-mon[32923]: pgmap v666: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-04-25T00:28:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:01 smithi106 ceph-mon[32510]: pgmap v666: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-04-25T00:28:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:01 smithi042 ceph-mon[28130]: pgmap v666: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 406 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-04-25T00:28:02.259 INFO:tasks.workunit.client.0.smithi042.stdout: 1 561631 6.31 MB/sec execute 352 sec latency 35.821 ms 2024-04-25T00:28:03.260 INFO:tasks.workunit.client.0.smithi042.stdout: 1 562664 6.30 MB/sec execute 353 sec latency 23.174 ms 2024-04-25T00:28:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:03 smithi136 ceph-mon[32923]: pgmap v667: 97 pgs: 97 active+clean; 110 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-04-25T00:28:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:03 smithi106 ceph-mon[32510]: pgmap v667: 97 pgs: 97 active+clean; 110 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-04-25T00:28:04.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:03 smithi042 ceph-mon[28130]: pgmap v667: 97 pgs: 97 active+clean; 110 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 480 KiB/s rd, 1.9 MiB/s wr, 204 op/s 2024-04-25T00:28:04.260 INFO:tasks.workunit.client.0.smithi042.stdout: 1 564547 6.31 MB/sec execute 354 sec latency 17.987 ms 2024-04-25T00:28:05.260 INFO:tasks.workunit.client.0.smithi042.stdout: 1 565342 6.31 MB/sec execute 355 sec latency 19.760 ms 2024-04-25T00:28:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.608+0000 7f0c57200640 1 -- 172.21.15.42:0/3743225569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c581049f0 msgr2=0x7f0c58104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.608+0000 7f0c57200640 1 --2- 172.21.15.42:0/3743225569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c581049f0 0x7f0c58104e70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0c44009900 tx=0x7f0c4402a400 comp rx=0 tx=0).stop 2024-04-25T00:28:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.608+0000 7f0c57200640 1 -- 172.21.15.42:0/3743225569 shutdown_connections 2024-04-25T00:28:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.608+0000 7f0c57200640 1 --2- 172.21.15.42:0/3743225569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581053b0 0x7f0c5810bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.608+0000 7f0c57200640 1 --2- 172.21.15.42:0/3743225569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c581049f0 0x7f0c58104e70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.608+0000 7f0c57200640 1 --2- 172.21.15.42:0/3743225569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c58103800 0x7f0c58103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.608+0000 7f0c57200640 1 -- 172.21.15.42:0/3743225569 >> 172.21.15.42:0/3743225569 conn(0x7f0c580fed70 msgr2=0x7f0c581011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.608+0000 7f0c57200640 1 -- 172.21.15.42:0/3743225569 shutdown_connections 2024-04-25T00:28:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.608+0000 7f0c57200640 1 -- 172.21.15.42:0/3743225569 wait complete. 2024-04-25T00:28:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.609+0000 7f0c57200640 1 Processor -- start 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c57200640 1 -- start start 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c57200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c58103800 0x7f0c58199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c57200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581049f0 0x7f0c5819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c57200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c581053b0 0x7f0c581a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c57200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5810f4b0 con 0x7f0c58103800 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c57200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5810f330 con 0x7f0c581053b0 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c57200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0c5810f630 con 0x7f0c581049f0 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c55e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c58103800 0x7f0c58199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c55400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581049f0 0x7f0c5819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c55e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c58103800 0x7f0c58199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58322/0 (socket says 172.21.15.42:58322) 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c55e00640 1 -- 172.21.15.42:0/1672571398 learned_addr learned my addr 172.21.15.42:0/1672571398 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.610+0000 7f0c56800640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c581053b0 0x7f0c581a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.611+0000 7f0c55400640 1 -- 172.21.15.42:0/1672571398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c581053b0 msgr2=0x7f0c581a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.611+0000 7f0c55400640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c581053b0 0x7f0c581a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.611+0000 7f0c55400640 1 -- 172.21.15.42:0/1672571398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c58103800 msgr2=0x7f0c58199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.611+0000 7f0c55400640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c58103800 0x7f0c58199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.611+0000 7f0c55400640 1 -- 172.21.15.42:0/1672571398 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c581a1710 con 0x7f0c581049f0 2024-04-25T00:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.611+0000 7f0c56800640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c581053b0 0x7f0c581a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.611+0000 7f0c55400640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581049f0 0x7f0c5819a030 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c4402a920 tx=0x7f0c44032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.611+0000 7f0c4ea00640 1 -- 172.21.15.42:0/1672571398 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4403f030 con 0x7f0c581049f0 2024-04-25T00:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.612+0000 7f0c57200640 1 -- 172.21.15.42:0/1672571398 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c581a19a0 con 0x7f0c581049f0 2024-04-25T00:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.612+0000 7f0c4ea00640 1 -- 172.21.15.42:0/1672571398 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c44003ce0 con 0x7f0c581049f0 2024-04-25T00:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.612+0000 7f0c4ea00640 1 -- 172.21.15.42:0/1672571398 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44035070 con 0x7f0c581049f0 2024-04-25T00:28:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.612+0000 7f0c57200640 1 -- 172.21.15.42:0/1672571398 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c581a1eb0 con 0x7f0c581049f0 2024-04-25T00:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.615+0000 7f0c5fc19740 1 Processor -- start 2024-04-25T00:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.615+0000 7f0c4ea00640 1 -- 172.21.15.42:0/1672571398 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0c44042030 con 0x7f0c581049f0 2024-04-25T00:28:05.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.615+0000 7f0c5fc19740 1 -- start start 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.615+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 0x55c28adf5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf5840 0x55c28adfa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adfa9b0 0x55c28adfce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c28ad57ab0 con 0x55c28adfa9b0 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c28ad00640 con 0x55c28adf4e50 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c28ad53a70 con 0x55c28adf5840 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c56800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 0x55c28adf5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c55e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf5840 0x55c28adfa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c56800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 0x55c28adf5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35742/0 (socket says 172.21.15.42:35742) 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c56800640 1 -- 172.21.15.42:0/2522000671 learned_addr learned my addr 172.21.15.42:0/2522000671 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c55400640 1 --2- 172.21.15.42:0/2522000671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adfa9b0 0x55c28adfce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c56800640 1 -- 172.21.15.42:0/2522000671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf5840 msgr2=0x55c28adfa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c56800640 1 --2- 172.21.15.42:0/2522000671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf5840 0x55c28adfa340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c56800640 1 -- 172.21.15.42:0/2522000671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adfa9b0 msgr2=0x55c28adfce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c56800640 1 --2- 172.21.15.42:0/2522000671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adfa9b0 0x55c28adfce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c56800640 1 -- 172.21.15.42:0/2522000671 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28aa6c210 con 0x55c28adf4e50 2024-04-25T00:28:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.616+0000 7f0c56800640 1 --2- 172.21.15.42:0/2522000671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 0x55c28adf5270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0c50010750 tx=0x7f0c50010c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a6f29504646e202 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.617+0000 7f0c43400640 1 -- 172.21.15.42:0/2522000671 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5000ead0 con 0x55c28adf4e50 2024-04-25T00:28:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.617+0000 7f0c43400640 1 -- 172.21.15.42:0/2522000671 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c50010e60 con 0x55c28adf4e50 2024-04-25T00:28:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.617+0000 7f0c43400640 1 -- 172.21.15.42:0/2522000671 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50011640 con 0x55c28adf4e50 2024-04-25T00:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.617+0000 7f0c4ea00640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0c3c0a2c00 0x7f0c3c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.617+0000 7f0c4ea00640 1 -- 172.21.15.42:0/1672571398 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f0c440ef410 con 0x7f0c581049f0 2024-04-25T00:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.617+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2522000671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 msgr2=0x55c28adf5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.617+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/2522000671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 0x55c28adf5270 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0c50010750 tx=0x7f0c50010c10 comp rx=0 tx=0).stop 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.618+0000 7f0c55e00640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0c3c0a2c00 0x7f0c3c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.618+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2522000671 shutdown_connections 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.618+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/2522000671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adfa9b0 0x55c28adfce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.618+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/2522000671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf5840 0x55c28adfa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.618+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/2522000671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 0x55c28adf5270 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.618+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2522000671 >> 172.21.15.42:0/2522000671 conn(0x55c28adf04c0 msgr2=0x55c28adf08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.618+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2522000671 shutdown_connections 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.618+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2522000671 wait complete. 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.619+0000 7f0c55e00640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0c3c0a2c00 0x7f0c3c0a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55c28adf1390 tx=0x7f0c48005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.619+0000 7f0c5fc19740 1 -- start start 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.619+0000 7f0c5fc19740 1 Processor -- start 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf4e50 0x55c28ae078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf5840 0x55c28ae07e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 0x55c28ae0ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c28adfd960 con 0x55c28adf4e50 2024-04-25T00:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c28ad13990 con 0x55c28adf5840 2024-04-25T00:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c28adfdae0 con 0x55c28adfa9b0 2024-04-25T00:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c5fc19740 10 client.0 _ll_register_callbacks cb 0x55c28ad15950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c55400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf5840 0x55c28ae07e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c55e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf4e50 0x55c28ae078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c55400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf5840 0x55c28ae07e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35750/0 (socket says 172.21.15.42:35750) 2024-04-25T00:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c55400640 1 -- 172.21.15.42:0/2787020898 learned_addr learned my addr 172.21.15.42:0/2787020898 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.620+0000 7f0c56800640 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 0x55c28ae0ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.621+0000 7f0c55e00640 1 -- 172.21.15.42:0/2787020898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 msgr2=0x55c28ae0ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.621+0000 7f0c55e00640 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 0x55c28ae0ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.621+0000 7f0c55e00640 1 -- 172.21.15.42:0/2787020898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf5840 msgr2=0x55c28ae07e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.621+0000 7f0c55e00640 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf5840 0x55c28ae07e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.621+0000 7f0c55e00640 1 -- 172.21.15.42:0/2787020898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28ab38620 con 0x55c28adf4e50 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.621+0000 7f0c55e00640 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf4e50 0x55c28ae078d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55c28adf5c40 tx=0x7f0c48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798321bbba2cb93b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.621+0000 7f0c37e00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.621+0000 7f0c37e00640 1 -- 172.21.15.42:0/2787020898 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4801f640 con 0x55c28adf4e50 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.622+0000 7f0c37e00640 1 -- 172.21.15.42:0/2787020898 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f0c4801f7e0 con 0x55c28adf4e50 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.622+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2787020898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c50010e60 con 0x55c28adf4e50 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.622+0000 7f0c37e00640 1 -- 172.21.15.42:0/2787020898 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c4801f9e0 con 0x55c28adf4e50 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.622+0000 7f0c5fc19740 10 client.15801 fetch_fsmap learned FSMap version 30 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.622+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2787020898 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c28ae11c90 con 0x55c28adf4e50 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.622+0000 7f0c37e00640 1 -- 172.21.15.42:0/2787020898 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c50010e60 con 0x55c28adf4e50 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.623+0000 7f0c37e00640 1 -- 172.21.15.42:0/2787020898 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f0c48027070 con 0x55c28adf4e50 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.623+0000 7f0c5fc19740 10 client.15801 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.623+0000 7f0c5fc19740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.623+0000 7f0c5fc19740 20 client.15801 populate_metadata read hostname 'smithi042' 2024-04-25T00:28:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.623+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c28ae133d0 0x55c28ae157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.623+0000 7f0c5fc19740 4 client.15801 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.623+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2787020898 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c28ad64ff0 con 0x55c28ae133d0 2024-04-25T00:28:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.623+0000 7f0c55400640 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c28ae133d0 0x55c28ae157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.624+0000 7f0c55400640 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c28ae133d0 0x55c28ae157e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73ecc0fd2932ff49 server_cookie=41c035bc5d100662 in_seq=0 out_seq=0 2024-04-25T00:28:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.624+0000 7f0c37e00640 10 client.15801 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c37e00640 1 -- 172.21.15.42:0/2787020898 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c28ad64ff0 con 0x55c28ae133d0 2024-04-25T00:28:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c37e00640 10 client.15801 handle_command_reply: tid=0 2024-04-25T00:28:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 client.15801 shutdown 2024-04-25T00:28:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2787020898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf4e50 msgr2=0x55c28ae078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf4e50 0x55c28ae078d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x55c28adf5c40 tx=0x7f0c48004730 comp rx=0 tx=0).stop 2024-04-25T00:28:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2787020898 shutdown_connections 2024-04-25T00:28:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c28ae133d0 0x55c28ae157e0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 0x55c28ae0ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf5840 0x55c28ae07e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/2787020898 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf4e50 0x55c28ae078d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c37e00640 0 client.15801 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2787020898 >> 172.21.15.42:0/2787020898 conn(0x55c28adf04c0 msgr2=0x55c28adef620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2787020898 shutdown_connections 2024-04-25T00:28:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.625+0000 7f0c5fc19740 1 -- 172.21.15.42:0/2787020898 wait complete. 2024-04-25T00:28:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.626+0000 7f0c5fc19740 20 client.15801 trim_cache size 0 max 16384 2024-04-25T00:28:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c5fc19740 1 Processor -- start 2024-04-25T00:28:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c5fc19740 1 -- start start 2024-04-25T00:28:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 0x55c28ae07e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf5840 0x55c28ae0ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 0x55c28adf3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581a90f0 con 0x55c28adf5840 2024-04-25T00:28:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581a6990 con 0x55c28adf4e50 2024-04-25T00:28:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581a4800 con 0x55c28adfa9b0 2024-04-25T00:28:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c56800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 0x55c28adf3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c55400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf5840 0x55c28ae0ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c56800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 0x55c28adf3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:58614/0 (socket says 172.21.15.42:58614) 2024-04-25T00:28:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c56800640 1 -- 172.21.15.42:0/1520573897 learned_addr learned my addr 172.21.15.42:0/1520573897 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.647+0000 7f0c55e00640 1 --2- 172.21.15.42:0/1520573897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 0x55c28ae07e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.648+0000 7f0c56800640 1 -- 172.21.15.42:0/1520573897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 msgr2=0x55c28ae07e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.648+0000 7f0c56800640 1 --2- 172.21.15.42:0/1520573897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 0x55c28ae07e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.648+0000 7f0c56800640 1 -- 172.21.15.42:0/1520573897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf5840 msgr2=0x55c28ae0ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.648+0000 7f0c56800640 1 --2- 172.21.15.42:0/1520573897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf5840 0x55c28ae0ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.648+0000 7f0c56800640 1 -- 172.21.15.42:0/1520573897 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c28adeea10 con 0x55c28adfa9b0 2024-04-25T00:28:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.648+0000 7f0c56800640 1 --2- 172.21.15.42:0/1520573897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 0x55c28adf3ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009640 tx=0x7f0c500106a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4bbfb2defe1ebe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.648+0000 7f0c42a00640 1 -- 172.21.15.42:0/1520573897 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c5002c070 con 0x55c28adfa9b0 2024-04-25T00:28:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.648+0000 7f0c42a00640 1 -- 172.21.15.42:0/1520573897 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c50010e60 con 0x55c28adfa9b0 2024-04-25T00:28:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.648+0000 7f0c42a00640 1 -- 172.21.15.42:0/1520573897 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c50012950 con 0x55c28adfa9b0 2024-04-25T00:28:05.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.649+0000 7f0c5fc19740 1 -- 172.21.15.42:0/1520573897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 msgr2=0x55c28adf3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.649+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/1520573897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 0x55c28adf3ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0c50009640 tx=0x7f0c500106a0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.649+0000 7f0c5fc19740 1 -- 172.21.15.42:0/1520573897 shutdown_connections 2024-04-25T00:28:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.649+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/1520573897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adfa9b0 0x55c28adf3ae0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.649+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/1520573897 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf5840 0x55c28ae0ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.649+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/1520573897 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adf4e50 0x55c28ae07e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.649+0000 7f0c5fc19740 1 -- 172.21.15.42:0/1520573897 >> 172.21.15.42:0/1520573897 conn(0x55c28adf04c0 msgr2=0x55c28adf33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.650+0000 7f0c5fc19740 1 -- 172.21.15.42:0/1520573897 shutdown_connections 2024-04-25T00:28:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.650+0000 7f0c5fc19740 1 -- 172.21.15.42:0/1520573897 wait complete. 2024-04-25T00:28:05.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.650+0000 7f0c5fc19740 1 -- start start 2024-04-25T00:28:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c5fc19740 1 Processor -- start 2024-04-25T00:28:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf4e50 0x55c28ae0c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf5840 0x55c28ae0ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c5fc19740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adfa9b0 0x55c28ae0daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581a4e50 con 0x55c28adf5840 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581a62c0 con 0x55c28adfa9b0 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c5fc19740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0c581a5520 con 0x55c28adf4e50 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c5fc19740 10 client.0 _ll_register_callbacks cb 0x7f0c480098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c56800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adfa9b0 0x55c28ae0daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c55e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf4e50 0x55c28ae0c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c56800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adfa9b0 0x55c28ae0daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35768/0 (socket says 172.21.15.42:35768) 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c56800640 1 -- 172.21.15.42:0/160642548 learned_addr learned my addr 172.21.15.42:0/160642548 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c55e00640 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf4e50 0x55c28ae0c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:58616/0 (socket says 172.21.15.42:58616) 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.651+0000 7f0c55400640 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf5840 0x55c28ae0ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.652+0000 7f0c55e00640 1 -- 172.21.15.42:0/160642548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adfa9b0 msgr2=0x55c28ae0daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.652+0000 7f0c55e00640 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adfa9b0 0x55c28ae0daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.652+0000 7f0c55e00640 1 -- 172.21.15.42:0/160642548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf5840 msgr2=0x55c28ae0ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.652+0000 7f0c55e00640 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf5840 0x55c28ae0ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.652+0000 7f0c55e00640 1 -- 172.21.15.42:0/160642548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c28ae0e090 con 0x55c28adf4e50 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.652+0000 7f0c55e00640 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf4e50 0x55c28ae0c550 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0c48019040 tx=0x7f0c48009970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9e244d084a26839 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.652+0000 7f0c37e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.652+0000 7f0c37e00640 1 -- 172.21.15.42:0/160642548 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c480043e0 con 0x55c28adf4e50 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.652+0000 7f0c5fc19740 1 -- 172.21.15.42:0/160642548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c50010e60 con 0x55c28adf4e50 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.652+0000 7f0c37e00640 1 -- 172.21.15.42:0/160642548 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f0c4801fd90 con 0x55c28adf4e50 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.653+0000 7f0c37e00640 1 -- 172.21.15.42:0/160642548 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0c48045070 con 0x55c28adf4e50 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.653+0000 7f0c5fc19740 10 client.25936 fetch_fsmap learned FSMap version 30 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.653+0000 7f0c5fc19740 1 -- 172.21.15.42:0/160642548 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c28ae79960 con 0x55c28adf4e50 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.653+0000 7f0c37e00640 1 -- 172.21.15.42:0/160642548 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c50010e60 con 0x55c28adf4e50 2024-04-25T00:28:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.653+0000 7f0c37e00640 1 -- 172.21.15.42:0/160642548 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f0c48027040 con 0x55c28adf4e50 2024-04-25T00:28:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.654+0000 7f0c5fc19740 10 client.25936 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:28:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.654+0000 7f0c5fc19740 10 client.25936 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.654+0000 7f0c5fc19740 20 client.25936 populate_metadata read hostname 'smithi042' 2024-04-25T00:28:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.654+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c28ae7a2a0 0x55c28ae7c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.654+0000 7f0c5fc19740 4 client.25936 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:28:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.654+0000 7f0c5fc19740 1 -- 172.21.15.42:0/160642548 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c28ad5f340 con 0x55c28ae7a2a0 2024-04-25T00:28:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.654+0000 7f0c55400640 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c28ae7a2a0 0x55c28ae7c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.654+0000 7f0c55400640 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c28ae7a2a0 0x55c28ae7c690 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ab3b49b8d875d8 server_cookie=3c89b7f1f8f0c359 in_seq=0 out_seq=0 2024-04-25T00:28:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.654+0000 7f0c37e00640 10 client.25936 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c37e00640 1 -- 172.21.15.42:0/160642548 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c28ad5f340 con 0x55c28ae7a2a0 2024-04-25T00:28:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c37e00640 10 client.25936 handle_command_reply: tid=0 2024-04-25T00:28:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c5fc19740 1 client.25936 shutdown 2024-04-25T00:28:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c5fc19740 1 -- 172.21.15.42:0/160642548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf4e50 msgr2=0x55c28ae0c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf4e50 0x55c28ae0c550 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0c48019040 tx=0x7f0c48009970 comp rx=0 tx=0).stop 2024-04-25T00:28:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c5fc19740 1 -- 172.21.15.42:0/160642548 shutdown_connections 2024-04-25T00:28:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c28ae7a2a0 0x55c28ae7c690 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c28adfa9b0 0x55c28ae0daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c28adf5840 0x55c28ae0ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c5fc19740 1 --2- 172.21.15.42:0/160642548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c28adf4e50 0x55c28ae0c550 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c5fc19740 1 -- 172.21.15.42:0/160642548 >> 172.21.15.42:0/160642548 conn(0x55c28adf04c0 msgr2=0x55c28ae07cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.702+0000 7f0c37e00640 0 client.25936 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.703+0000 7f0c5fc19740 1 -- 172.21.15.42:0/160642548 shutdown_connections 2024-04-25T00:28:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.703+0000 7f0c5fc19740 1 -- 172.21.15.42:0/160642548 wait complete. 2024-04-25T00:28:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.703+0000 7f0c5fc19740 20 client.25936 trim_cache size 0 max 16384 2024-04-25T00:28:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:28:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:28:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:28:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:28:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 -- 172.21.15.42:0/1672571398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0c3c0a2c00 msgr2=0x7f0c3c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0c3c0a2c00 0x7f0c3c0a50c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x55c28adf1390 tx=0x7f0c48005f90 comp rx=0 tx=0).stop 2024-04-25T00:28:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 -- 172.21.15.42:0/1672571398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581049f0 msgr2=0x7f0c5819a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581049f0 0x7f0c5819a030 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c4402a920 tx=0x7f0c44032ce0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 -- 172.21.15.42:0/1672571398 shutdown_connections 2024-04-25T00:28:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0c3c0a2c00 0x7f0c3c0a50c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0c581053b0 0x7f0c581a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0c581049f0 0x7f0c5819a030 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 --2- 172.21.15.42:0/1672571398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0c58103800 0x7f0c58199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 -- 172.21.15.42:0/1672571398 >> 172.21.15.42:0/1672571398 conn(0x7f0c580fed70 msgr2=0x7f0c58107d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.704+0000 7f0c57200640 1 -- 172.21.15.42:0/1672571398 shutdown_connections 2024-04-25T00:28:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:05.705+0000 7f0c57200640 1 -- 172.21.15.42:0/1672571398 wait complete. 2024-04-25T00:28:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:05 smithi136 ceph-mon[32923]: pgmap v668: 97 pgs: 97 active+clean; 110 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-04-25T00:28:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:05 smithi106 ceph-mon[32510]: pgmap v668: 97 pgs: 97 active+clean; 110 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-04-25T00:28:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:05 smithi042 ceph-mon[28130]: pgmap v668: 97 pgs: 97 active+clean; 110 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-04-25T00:28:06.260 INFO:tasks.workunit.client.0.smithi042.stdout: 1 566501 6.30 MB/sec execute 356 sec latency 35.990 ms 2024-04-25T00:28:07.260 INFO:tasks.workunit.client.0.smithi042.stdout: 1 568481 6.32 MB/sec execute 357 sec latency 25.730 ms 2024-04-25T00:28:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:07 smithi136 ceph-mon[32923]: pgmap v669: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-04-25T00:28:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:07 smithi106 ceph-mon[32510]: pgmap v669: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-04-25T00:28:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:07 smithi042 ceph-mon[28130]: pgmap v669: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 516 KiB/s rd, 2.1 MiB/s wr, 238 op/s 2024-04-25T00:28:08.260 INFO:tasks.workunit.client.0.smithi042.stdout: 1 569089 6.31 MB/sec execute 358 sec latency 21.118 ms 2024-04-25T00:28:09.260 INFO:tasks.workunit.client.0.smithi042.stdout: 1 570544 6.31 MB/sec execute 359 sec latency 28.524 ms 2024-04-25T00:28:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:09 smithi136 ceph-mon[32923]: pgmap v670: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-04-25T00:28:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:09 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:09 smithi106 ceph-mon[32510]: pgmap v670: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-04-25T00:28:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:09 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:10.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:09 smithi042 ceph-mon[28130]: pgmap v670: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.6 MiB/s wr, 170 op/s 2024-04-25T00:28:10.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:09 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:28:10.260 INFO:tasks.workunit.client.0.smithi042.stdout: 1 572188 6.32 MB/sec execute 360 sec latency 15.967 ms 2024-04-25T00:28:11.261 INFO:tasks.workunit.client.0.smithi042.stdout: 1 573290 6.32 MB/sec execute 361 sec latency 24.122 ms 2024-04-25T00:28:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:11 smithi136 ceph-mon[32923]: pgmap v671: 97 pgs: 97 active+clean; 110 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-04-25T00:28:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:28:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:11 smithi106 ceph-mon[32510]: pgmap v671: 97 pgs: 97 active+clean; 110 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-04-25T00:28:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:12.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:28:12.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:11 smithi042 ceph-mon[28130]: pgmap v671: 97 pgs: 97 active+clean; 110 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.9 MiB/s wr, 193 op/s 2024-04-25T00:28:12.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:28:12.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:28:12.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:28:12.260 INFO:tasks.workunit.client.0.smithi042.stdout: 1 574868 6.32 MB/sec execute 362 sec latency 57.778 ms 2024-04-25T00:28:13.262 INFO:tasks.workunit.client.0.smithi042.stdout: 1 575874 6.33 MB/sec execute 363 sec latency 20.457 ms 2024-04-25T00:28:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:13 smithi136 ceph-mon[32923]: pgmap v672: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-04-25T00:28:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:13 smithi106 ceph-mon[32510]: pgmap v672: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-04-25T00:28:14.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:13 smithi042 ceph-mon[28130]: pgmap v672: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 509 KiB/s rd, 2.2 MiB/s wr, 225 op/s 2024-04-25T00:28:14.261 INFO:tasks.workunit.client.0.smithi042.stdout: 1 576957 6.32 MB/sec execute 364 sec latency 37.609 ms 2024-04-25T00:28:15.261 INFO:tasks.workunit.client.0.smithi042.stdout: 1 578811 6.33 MB/sec execute 365 sec latency 17.941 ms 2024-04-25T00:28:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:15 smithi136 ceph-mon[32923]: pgmap v673: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-04-25T00:28:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:15 smithi106 ceph-mon[32510]: pgmap v673: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-04-25T00:28:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:15 smithi042 ceph-mon[28130]: pgmap v673: 97 pgs: 97 active+clean; 111 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-04-25T00:28:16.261 INFO:tasks.workunit.client.0.smithi042.stdout: 1 579513 6.33 MB/sec execute 366 sec latency 77.409 ms 2024-04-25T00:28:17.261 INFO:tasks.workunit.client.0.smithi042.stdout: 1 580535 6.32 MB/sec execute 367 sec latency 131.132 ms 2024-04-25T00:28:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:17 smithi136 ceph-mon[32923]: pgmap v674: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-04-25T00:28:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:17 smithi106 ceph-mon[32510]: pgmap v674: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-04-25T00:28:18.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:17 smithi042 ceph-mon[28130]: pgmap v674: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 621 KiB/s rd, 2.3 MiB/s wr, 254 op/s 2024-04-25T00:28:18.261 INFO:tasks.workunit.client.0.smithi042.stdout: 1 581989 6.33 MB/sec execute 368 sec latency 47.037 ms 2024-04-25T00:28:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.867+0000 7f695dc00640 1 -- 172.21.15.42:0/257558407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69581027f0 msgr2=0x7f6958102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.867+0000 7f695dc00640 1 --2- 172.21.15.42:0/257558407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69581027f0 0x7f6958102c70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6944009900 tx=0x7f694402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.867+0000 7f695dc00640 1 -- 172.21.15.42:0/257558407 shutdown_connections 2024-04-25T00:28:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.867+0000 7f695dc00640 1 --2- 172.21.15.42:0/257558407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69581031b0 0x7f6958109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.867+0000 7f695dc00640 1 --2- 172.21.15.42:0/257558407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f69581027f0 0x7f6958102c70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.867+0000 7f695dc00640 1 --2- 172.21.15.42:0/257558407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6958101600 0x7f6958101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.867+0000 7f695dc00640 1 -- 172.21.15.42:0/257558407 >> 172.21.15.42:0/257558407 conn(0x7f69580fcb70 msgr2=0x7f69580fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.867+0000 7f695dc00640 1 -- 172.21.15.42:0/257558407 shutdown_connections 2024-04-25T00:28:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.867+0000 7f695dc00640 1 -- 172.21.15.42:0/257558407 wait complete. 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.868+0000 7f695dc00640 1 Processor -- start 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f695dc00640 1 -- start start 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f695dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6958101600 0x7f69581978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f695dc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69581027f0 0x7f6958197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f695dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581031b0 0x7f695819ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f695dc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f695810d2c0 con 0x7f69581031b0 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f695dc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f695810d140 con 0x7f69581027f0 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f695dc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f695810d440 con 0x7f6958101600 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f6957400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69581027f0 0x7f6958197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f6957e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6958101600 0x7f69581978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f6957e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6958101600 0x7f69581978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:50428/0 (socket says 172.21.15.42:50428) 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f6957400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69581027f0 0x7f6958197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40594/0 (socket says 172.21.15.42:40594) 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f6957e00640 1 -- 172.21.15.42:0/1227593070 learned_addr learned my addr 172.21.15.42:0/1227593070 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.869+0000 7f695d200640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581031b0 0x7f695819ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f6957400640 1 -- 172.21.15.42:0/1227593070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6958101600 msgr2=0x7f69581978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f6957400640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6958101600 0x7f69581978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f6957400640 1 -- 172.21.15.42:0/1227593070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581031b0 msgr2=0x7f695819ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f6957400640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581031b0 0x7f695819ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f6957400640 1 -- 172.21.15.42:0/1227593070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695819f4f0 con 0x7f69581027f0 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f6957e00640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6958101600 0x7f69581978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f6957400640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69581027f0 0x7f6958197e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f694402a910 tx=0x7f6944032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f6954c00640 1 -- 172.21.15.42:0/1227593070 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694403f030 con 0x7f69581027f0 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f6954c00640 1 -- 172.21.15.42:0/1227593070 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6944003ce0 con 0x7f69581027f0 2024-04-25T00:28:18.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f6954c00640 1 -- 172.21.15.42:0/1227593070 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944035070 con 0x7f69581027f0 2024-04-25T00:28:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.870+0000 7f695dc00640 1 -- 172.21.15.42:0/1227593070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695819f780 con 0x7f69581027f0 2024-04-25T00:28:18.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.871+0000 7f695dc00640 1 -- 172.21.15.42:0/1227593070 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695819fbf0 con 0x7f69581027f0 2024-04-25T00:28:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.875+0000 7f6962029740 1 Processor -- start 2024-04-25T00:28:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.875+0000 7f6962029740 1 -- start start 2024-04-25T00:28:18.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.875+0000 7f6954c00640 1 -- 172.21.15.42:0/1227593070 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6944042030 con 0x7f69581027f0 2024-04-25T00:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.875+0000 7f6962029740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 0x55a1ea45d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6962029740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45dc50 0x55a1ea462750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6962029740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea462dc0 0x55a1ea465250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6962029740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a1ea3bbab0 con 0x55a1ea45d260 2024-04-25T00:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6962029740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a1ea364640 con 0x55a1ea45dc50 2024-04-25T00:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6962029740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a1ea3b7a70 con 0x55a1ea462dc0 2024-04-25T00:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f695d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 0x55a1ea45d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6957e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45dc50 0x55a1ea462750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6957e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45dc50 0x55a1ea462750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40598/0 (socket says 172.21.15.42:40598) 2024-04-25T00:28:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6957e00640 1 -- 172.21.15.42:0/3147942651 learned_addr learned my addr 172.21.15.42:0/3147942651 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f695d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 0x55a1ea45d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:39930/0 (socket says 172.21.15.42:39930) 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6957400640 1 --2- 172.21.15.42:0/3147942651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea462dc0 0x55a1ea465250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6957e00640 1 -- 172.21.15.42:0/3147942651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea462dc0 msgr2=0x55a1ea465250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6957e00640 1 --2- 172.21.15.42:0/3147942651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea462dc0 0x55a1ea465250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6957e00640 1 -- 172.21.15.42:0/3147942651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 msgr2=0x55a1ea45d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6957e00640 1 --2- 172.21.15.42:0/3147942651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 0x55a1ea45d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f6957e00640 1 -- 172.21.15.42:0/3147942651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ea0d0210 con 0x55a1ea45dc50 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.876+0000 7f695d200640 1 --2- 172.21.15.42:0/3147942651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 0x55a1ea45d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.877+0000 7f6957e00640 1 --2- 172.21.15.42:0/3147942651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45dc50 0x55a1ea462750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f69480115c0 tx=0x7f6948011a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b835faf8df1cb49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.877+0000 7f694d600640 1 -- 172.21.15.42:0/3147942651 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948005020 con 0x55a1ea45dc50 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.877+0000 7f694d600640 1 -- 172.21.15.42:0/3147942651 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69480047a0 con 0x55a1ea45dc50 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.877+0000 7f694d600640 1 -- 172.21.15.42:0/3147942651 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694800fa00 con 0x55a1ea45dc50 2024-04-25T00:28:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.877+0000 7f6954c00640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f693c0a2b80 0x7f693c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.877+0000 7f6954c00640 1 -- 172.21.15.42:0/1227593070 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f69440ef4c0 con 0x7f69581027f0 2024-04-25T00:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.878+0000 7f695d200640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f693c0a2b80 0x7f693c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.878+0000 7f6962029740 1 -- 172.21.15.42:0/3147942651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45dc50 msgr2=0x55a1ea462750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.878+0000 7f6962029740 1 --2- 172.21.15.42:0/3147942651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45dc50 0x55a1ea462750 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f69480115c0 tx=0x7f6948011a80 comp rx=0 tx=0).stop 2024-04-25T00:28:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.878+0000 7f6962029740 1 -- 172.21.15.42:0/3147942651 shutdown_connections 2024-04-25T00:28:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.878+0000 7f6962029740 1 --2- 172.21.15.42:0/3147942651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea462dc0 0x55a1ea465250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.878+0000 7f6962029740 1 --2- 172.21.15.42:0/3147942651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45dc50 0x55a1ea462750 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.878+0000 7f6962029740 1 --2- 172.21.15.42:0/3147942651 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 0x55a1ea45d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.878+0000 7f6962029740 1 -- 172.21.15.42:0/3147942651 >> 172.21.15.42:0/3147942651 conn(0x55a1ea4544c0 msgr2=0x55a1ea4548c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.878+0000 7f6962029740 1 -- 172.21.15.42:0/3147942651 shutdown_connections 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.878+0000 7f695d200640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f693c0a2b80 0x7f693c0a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55a1ea45e4b0 tx=0x7f6950009600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.879+0000 7f6962029740 1 -- 172.21.15.42:0/3147942651 wait complete. 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.879+0000 7f6962029740 1 -- start start 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.879+0000 7f6962029740 1 Processor -- start 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f6962029740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45d260 0x55a1ea4674d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f6962029740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45dc50 0x55a1ea467a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f6962029740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea46ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f6962029740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a1ea465d70 con 0x55a1ea45dc50 2024-04-25T00:28:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f6962029740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a1ea377990 con 0x55a1ea462dc0 2024-04-25T00:28:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f6962029740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a1ea465ef0 con 0x55a1ea45d260 2024-04-25T00:28:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f6962029740 10 client.0 _ll_register_callbacks cb 0x55a1ea379950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f695d200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea46ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f6957e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45d260 0x55a1ea4674d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f695d200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea46ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40614/0 (socket says 172.21.15.42:40614) 2024-04-25T00:28:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.880+0000 7f695d200640 1 -- 172.21.15.42:0/3278584992 learned_addr learned my addr 172.21.15.42:0/3278584992 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f6957400640 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45dc50 0x55a1ea467a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f695d200640 1 -- 172.21.15.42:0/3278584992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45d260 msgr2=0x55a1ea4674d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f695d200640 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45d260 0x55a1ea4674d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f695d200640 1 -- 172.21.15.42:0/3278584992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45dc50 msgr2=0x55a1ea467a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f695d200640 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45dc50 0x55a1ea467a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f695d200640 1 -- 172.21.15.42:0/3278584992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ea19c620 con 0x55a1ea462dc0 2024-04-25T00:28:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f6957400640 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45dc50 0x55a1ea467a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f695d200640 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea46ea50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6950011ef0 tx=0x7f6950004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d246debf0d00e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f6942000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:28:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f6942000640 1 -- 172.21.15.42:0/3278584992 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695001ecb0 con 0x55a1ea462dc0 2024-04-25T00:28:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f6962029740 1 -- 172.21.15.42:0/3278584992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69480047a0 con 0x55a1ea462dc0 2024-04-25T00:28:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.881+0000 7f6942000640 1 -- 172.21.15.42:0/3278584992 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f695002a480 con 0x55a1ea462dc0 2024-04-25T00:28:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6942000640 1 -- 172.21.15.42:0/3278584992 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f695001ee50 con 0x55a1ea462dc0 2024-04-25T00:28:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6962029740 10 client.25943 fetch_fsmap learned FSMap version 30 2024-04-25T00:28:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6962029740 1 -- 172.21.15.42:0/3278584992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1ea471aa0 con 0x55a1ea462dc0 2024-04-25T00:28:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6942000640 1 -- 172.21.15.42:0/3278584992 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695001ee50 con 0x55a1ea462dc0 2024-04-25T00:28:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6942000640 1 -- 172.21.15.42:0/3278584992 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f695000f040 con 0x55a1ea462dc0 2024-04-25T00:28:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6962029740 10 client.25943 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:28:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6962029740 10 client.25943 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6962029740 20 client.25943 populate_metadata read hostname 'smithi042' 2024-04-25T00:28:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6962029740 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1ea473190 0x55a1ea4755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6962029740 4 client.25943 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.882+0000 7f6962029740 1 -- 172.21.15.42:0/3278584992 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1ea3c8ff0 con 0x55a1ea473190 2024-04-25T00:28:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.883+0000 7f6957e00640 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1ea473190 0x55a1ea4755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.883+0000 7f6957e00640 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1ea473190 0x55a1ea4755c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8c112d134736fc3 server_cookie=e4c7598f2477cd24 in_seq=0 out_seq=0 2024-04-25T00:28:18.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.883+0000 7f6942000640 10 client.25943 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.884+0000 7f6942000640 1 -- 172.21.15.42:0/3278584992 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1ea3c8ff0 con 0x55a1ea473190 2024-04-25T00:28:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.884+0000 7f6942000640 10 client.25943 handle_command_reply: tid=0 2024-04-25T00:28:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.884+0000 7f6962029740 1 client.25943 shutdown 2024-04-25T00:28:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.884+0000 7f6962029740 1 -- 172.21.15.42:0/3278584992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 msgr2=0x55a1ea46ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.884+0000 7f6962029740 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea46ea50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6950011ef0 tx=0x7f6950004a40 comp rx=0 tx=0).stop 2024-04-25T00:28:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.885+0000 7f6962029740 1 -- 172.21.15.42:0/3278584992 shutdown_connections 2024-04-25T00:28:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.885+0000 7f6962029740 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1ea473190 0x55a1ea4755c0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.885+0000 7f6962029740 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea46ea50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.885+0000 7f6962029740 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45dc50 0x55a1ea467a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.885+0000 7f6962029740 1 --2- 172.21.15.42:0/3278584992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45d260 0x55a1ea4674d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.885+0000 7f6942000640 0 client.25943 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.885+0000 7f6962029740 1 -- 172.21.15.42:0/3278584992 >> 172.21.15.42:0/3278584992 conn(0x55a1ea4544c0 msgr2=0x55a1ea453620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.885+0000 7f6962029740 1 -- 172.21.15.42:0/3278584992 shutdown_connections 2024-04-25T00:28:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.885+0000 7f6962029740 1 -- 172.21.15.42:0/3278584992 wait complete. 2024-04-25T00:28:18.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.885+0000 7f6962029740 20 client.25943 trim_cache size 0 max 16384 2024-04-25T00:28:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.903+0000 7f6962029740 1 Processor -- start 2024-04-25T00:28:18.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.903+0000 7f6962029740 1 -- start start 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6962029740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 0x55a1ea479ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6962029740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6962029740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea47a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6962029740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69581a91e0 con 0x55a1ea45d260 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6962029740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f69581a6a80 con 0x55a1ea462dc0 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6962029740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f69581a4870 con 0x55a1ea45dc50 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6957400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6957400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:50460/0 (socket says 172.21.15.42:50460) 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6957400640 1 -- 172.21.15.42:0/2483986981 learned_addr learned my addr 172.21.15.42:0/2483986981 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6957e00640 1 --2- 172.21.15.42:0/2483986981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 0x55a1ea479ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f695d200640 1 --2- 172.21.15.42:0/2483986981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea47a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6957400640 1 -- 172.21.15.42:0/2483986981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 msgr2=0x55a1ea47a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6957400640 1 --2- 172.21.15.42:0/2483986981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea47a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6957400640 1 -- 172.21.15.42:0/2483986981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 msgr2=0x55a1ea479ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6957400640 1 --2- 172.21.15.42:0/2483986981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 0x55a1ea479ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.904+0000 7f6957400640 1 -- 172.21.15.42:0/2483986981 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ea452a10 con 0x55a1ea45dc50 2024-04-25T00:28:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.905+0000 7f695d200640 1 --2- 172.21.15.42:0/2483986981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea47a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.905+0000 7f6957400640 1 --2- 172.21.15.42:0/2483986981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46ea10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f69440b8770 tx=0x7f6944032f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d50b02beb799f33b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.905+0000 7f694cc00640 1 -- 172.21.15.42:0/2483986981 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944049070 con 0x55a1ea45dc50 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.905+0000 7f694cc00640 1 -- 172.21.15.42:0/2483986981 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69440bbda0 con 0x55a1ea45dc50 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.905+0000 7f694cc00640 1 -- 172.21.15.42:0/2483986981 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694403bd30 con 0x55a1ea45dc50 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.906+0000 7f6962029740 1 -- 172.21.15.42:0/2483986981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 msgr2=0x55a1ea46ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.906+0000 7f6962029740 1 --2- 172.21.15.42:0/2483986981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46ea10 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f69440b8770 tx=0x7f6944032f90 comp rx=0 tx=0).stop 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.906+0000 7f6962029740 1 -- 172.21.15.42:0/2483986981 shutdown_connections 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.906+0000 7f6962029740 1 --2- 172.21.15.42:0/2483986981 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea462dc0 0x55a1ea47a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.906+0000 7f6962029740 1 --2- 172.21.15.42:0/2483986981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46ea10 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.906+0000 7f6962029740 1 --2- 172.21.15.42:0/2483986981 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea45d260 0x55a1ea479ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.906+0000 7f6962029740 1 -- 172.21.15.42:0/2483986981 >> 172.21.15.42:0/2483986981 conn(0x55a1ea4544c0 msgr2=0x55a1ea453620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.906+0000 7f6962029740 1 -- 172.21.15.42:0/2483986981 shutdown_connections 2024-04-25T00:28:18.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.906+0000 7f6962029740 1 -- 172.21.15.42:0/2483986981 wait complete. 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.907+0000 7f6962029740 1 -- start start 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.907+0000 7f6962029740 1 Processor -- start 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.907+0000 7f6962029740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45d260 0x55a1ea46e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.907+0000 7f6962029740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.907+0000 7f6962029740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea462dc0 0x55a1ea46cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.907+0000 7f6962029740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f69581a4f40 con 0x55a1ea462dc0 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.907+0000 7f6962029740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f69581a63b0 con 0x55a1ea45d260 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.907+0000 7f6962029740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f69581a5610 con 0x55a1ea45dc50 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.907+0000 7f6962029740 10 client.0 _ll_register_callbacks cb 0x7f6950009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45d260 0x55a1ea46e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45d260 0x55a1ea46e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40634/0 (socket says 172.21.15.42:40634) 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957e00640 1 -- 172.21.15.42:0/2526349481 learned_addr learned my addr 172.21.15.42:0/2526349481 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957400640 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f695d200640 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea462dc0 0x55a1ea46cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957400640 1 -- 172.21.15.42:0/2526349481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45d260 msgr2=0x55a1ea46e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957400640 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45d260 0x55a1ea46e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957400640 1 -- 172.21.15.42:0/2526349481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea462dc0 msgr2=0x55a1ea46cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957400640 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea462dc0 0x55a1ea46cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957400640 1 -- 172.21.15.42:0/2526349481 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ea46d2e0 con 0x55a1ea45dc50 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f695d200640 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea462dc0 0x55a1ea46cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957e00640 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45d260 0x55a1ea46e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6957400640 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46e980 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69440bbaa0 tx=0x7f69440bbca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=690c1ec891433374 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.908+0000 7f6942000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6942000640 1 -- 172.21.15.42:0/2526349481 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6944049040 con 0x55a1ea45dc50 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6942000640 1 -- 172.21.15.42:0/2526349481 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f69440b8410 con 0x55a1ea45dc50 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6962029740 1 -- 172.21.15.42:0/2526349481 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69440bbda0 con 0x55a1ea45dc50 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6942000640 1 -- 172.21.15.42:0/2526349481 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f69440da020 con 0x55a1ea45dc50 2024-04-25T00:28:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6962029740 10 client.25954 fetch_fsmap learned FSMap version 30 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6962029740 1 -- 172.21.15.42:0/2526349481 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a1ea4da390 con 0x55a1ea45dc50 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6942000640 1 -- 172.21.15.42:0/2526349481 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69440bbda0 con 0x55a1ea45dc50 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6942000640 1 -- 172.21.15.42:0/2526349481 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f69440ef070 con 0x55a1ea45dc50 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6962029740 10 client.25954 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6962029740 10 client.25954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6962029740 20 client.25954 populate_metadata read hostname 'smithi042' 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6962029740 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1ea4dacd0 0x55a1ea4dd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6962029740 4 client.25954 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.909+0000 7f6962029740 1 -- 172.21.15.42:0/2526349481 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1ea3c3340 con 0x55a1ea4dacd0 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.910+0000 7f6957e00640 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1ea4dacd0 0x55a1ea4dd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.910+0000 7f6957e00640 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1ea4dacd0 0x55a1ea4dd0f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4265bc3140f0b87 server_cookie=433002007ce82129 in_seq=0 out_seq=0 2024-04-25T00:28:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.910+0000 7f6942000640 10 client.25954 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6942000640 1 -- 172.21.15.42:0/2526349481 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1ea3c3340 con 0x55a1ea4dacd0 2024-04-25T00:28:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6942000640 10 client.25954 handle_command_reply: tid=0 2024-04-25T00:28:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6962029740 1 client.25954 shutdown 2024-04-25T00:28:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6962029740 1 -- 172.21.15.42:0/2526349481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 msgr2=0x55a1ea46e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6962029740 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46e980 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f69440bbaa0 tx=0x7f69440bbca0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6962029740 1 -- 172.21.15.42:0/2526349481 shutdown_connections 2024-04-25T00:28:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6962029740 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1ea4dacd0 0x55a1ea4dd0f0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6962029740 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1ea462dc0 0x55a1ea46cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6962029740 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1ea45dc50 0x55a1ea46e980 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6942000640 0 client.25954 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6962029740 1 --2- 172.21.15.42:0/2526349481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1ea45d260 0x55a1ea46e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.966+0000 7f6962029740 1 -- 172.21.15.42:0/2526349481 >> 172.21.15.42:0/2526349481 conn(0x55a1ea4544c0 msgr2=0x55a1ea4573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.967+0000 7f6962029740 1 -- 172.21.15.42:0/2526349481 shutdown_connections 2024-04-25T00:28:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.967+0000 7f6962029740 1 -- 172.21.15.42:0/2526349481 wait complete. 2024-04-25T00:28:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.967+0000 7f6962029740 20 client.25954 trim_cache size 0 max 16384 2024-04-25T00:28:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:28:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:28:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:28:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:28:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.967+0000 7f695dc00640 1 -- 172.21.15.42:0/1227593070 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f693c0a2b80 msgr2=0x7f693c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.967+0000 7f695dc00640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f693c0a2b80 0x7f693c0a5040 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x55a1ea45e4b0 tx=0x7f6950009600 comp rx=0 tx=0).stop 2024-04-25T00:28:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.967+0000 7f695dc00640 1 -- 172.21.15.42:0/1227593070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69581027f0 msgr2=0x7f6958197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.967+0000 7f695dc00640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69581027f0 0x7f6958197e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f694402a910 tx=0x7f6944032ce0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.968+0000 7f695dc00640 1 -- 172.21.15.42:0/1227593070 shutdown_connections 2024-04-25T00:28:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.968+0000 7f695dc00640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f693c0a2b80 0x7f693c0a5040 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.968+0000 7f695dc00640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f69581031b0 0x7f695819ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.968+0000 7f695dc00640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69581027f0 0x7f6958197e10 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.968+0000 7f695dc00640 1 --2- 172.21.15.42:0/1227593070 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6958101600 0x7f69581978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.968+0000 7f695dc00640 1 -- 172.21.15.42:0/1227593070 >> 172.21.15.42:0/1227593070 conn(0x7f69580fcb70 msgr2=0x7f6958105af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.968+0000 7f695dc00640 1 -- 172.21.15.42:0/1227593070 shutdown_connections 2024-04-25T00:28:18.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:18.968+0000 7f695dc00640 1 -- 172.21.15.42:0/1227593070 wait complete. 2024-04-25T00:28:19.261 INFO:tasks.workunit.client.0.smithi042.stdout: 1 582916 6.33 MB/sec execute 369 sec latency 48.383 ms 2024-04-25T00:28:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:19 smithi136 ceph-mon[32923]: pgmap v675: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.7 MiB/s wr, 182 op/s 2024-04-25T00:28:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:19 smithi106 ceph-mon[32510]: pgmap v675: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.7 MiB/s wr, 182 op/s 2024-04-25T00:28:20.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:19 smithi042 ceph-mon[28130]: pgmap v675: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 1.7 MiB/s wr, 182 op/s 2024-04-25T00:28:20.261 INFO:tasks.workunit.client.0.smithi042.stdout: 1 583710 6.31 MB/sec execute 370 sec latency 25.038 ms 2024-04-25T00:28:21.262 INFO:tasks.workunit.client.0.smithi042.stdout: 1 585082 6.31 MB/sec execute 371 sec latency 26.735 ms 2024-04-25T00:28:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:21 smithi136 ceph-mon[32923]: pgmap v676: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-04-25T00:28:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:21 smithi106 ceph-mon[32510]: pgmap v676: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-04-25T00:28:22.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:21 smithi042 ceph-mon[28130]: pgmap v676: 97 pgs: 97 active+clean; 113 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 1.8 MiB/s wr, 194 op/s 2024-04-25T00:28:22.262 INFO:tasks.workunit.client.0.smithi042.stdout: 1 586343 6.33 MB/sec execute 372 sec latency 21.750 ms 2024-04-25T00:28:23.262 INFO:tasks.workunit.client.0.smithi042.stdout: 1 586974 6.31 MB/sec execute 373 sec latency 47.079 ms 2024-04-25T00:28:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:23 smithi136 ceph-mon[32923]: pgmap v677: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-04-25T00:28:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:23 smithi106 ceph-mon[32510]: pgmap v677: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-04-25T00:28:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:23 smithi042 ceph-mon[28130]: pgmap v677: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-04-25T00:28:24.262 INFO:tasks.workunit.client.0.smithi042.stdout: 1 588255 6.31 MB/sec execute 374 sec latency 39.464 ms 2024-04-25T00:28:25.262 INFO:tasks.workunit.client.0.smithi042.stdout: 1 589758 6.32 MB/sec execute 375 sec latency 25.014 ms 2024-04-25T00:28:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:25 smithi136 ceph-mon[32923]: pgmap v678: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-04-25T00:28:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:25 smithi106 ceph-mon[32510]: pgmap v678: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-04-25T00:28:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:25 smithi042 ceph-mon[28130]: pgmap v678: 97 pgs: 97 active+clean; 111 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.3 MiB/s wr, 177 op/s 2024-04-25T00:28:26.262 INFO:tasks.workunit.client.0.smithi042.stdout: 1 590302 6.31 MB/sec execute 376 sec latency 23.964 ms 2024-04-25T00:28:27.262 INFO:tasks.workunit.client.0.smithi042.stdout: 1 591313 6.31 MB/sec execute 377 sec latency 33.758 ms 2024-04-25T00:28:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:27 smithi136 ceph-mon[32923]: pgmap v679: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-04-25T00:28:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:27 smithi106 ceph-mon[32510]: pgmap v679: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-04-25T00:28:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:27 smithi042 ceph-mon[28130]: pgmap v679: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 593 KiB/s rd, 1.8 MiB/s wr, 234 op/s 2024-04-25T00:28:28.262 INFO:tasks.workunit.client.0.smithi042.stdout: 1 592982 6.32 MB/sec execute 378 sec latency 21.082 ms 2024-04-25T00:28:29.263 INFO:tasks.workunit.client.0.smithi042.stdout: 1 593611 6.31 MB/sec execute 379 sec latency 20.139 ms 2024-04-25T00:28:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:29 smithi136 ceph-mon[32923]: pgmap v680: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-04-25T00:28:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:29 smithi106 ceph-mon[32510]: pgmap v680: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-04-25T00:28:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:29 smithi042 ceph-mon[28130]: pgmap v680: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-04-25T00:28:30.263 INFO:tasks.workunit.client.0.smithi042.stdout: 1 594574 6.31 MB/sec execute 380 sec latency 29.350 ms 2024-04-25T00:28:31.263 INFO:tasks.workunit.client.0.smithi042.stdout: 1 595768 6.30 MB/sec execute 381 sec latency 25.719 ms 2024-04-25T00:28:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:31 smithi136 ceph-mon[32923]: pgmap v681: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:28:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:31 smithi106 ceph-mon[32510]: pgmap v681: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:28:32.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.123+0000 7f1f41a00640 1 -- 172.21.15.42:0/427723202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f3c103190 msgr2=0x7f1f3c109a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.123+0000 7f1f41a00640 1 --2- 172.21.15.42:0/427723202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f3c103190 0x7f1f3c109a20 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009ca0 tx=0x7f1f3402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.123+0000 7f1f41a00640 1 -- 172.21.15.42:0/427723202 shutdown_connections 2024-04-25T00:28:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.123+0000 7f1f41a00640 1 --2- 172.21.15.42:0/427723202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f3c103190 0x7f1f3c109a20 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.123+0000 7f1f41a00640 1 --2- 172.21.15.42:0/427723202 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f3c1027d0 0x7f1f3c102c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.123+0000 7f1f41a00640 1 --2- 172.21.15.42:0/427723202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c1015e0 0x7f1f3c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.123+0000 7f1f41a00640 1 -- 172.21.15.42:0/427723202 >> 172.21.15.42:0/427723202 conn(0x7f1f3c0fcb50 msgr2=0x7f1f3c0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.123+0000 7f1f41a00640 1 -- 172.21.15.42:0/427723202 shutdown_connections 2024-04-25T00:28:32.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.123+0000 7f1f41a00640 1 -- 172.21.15.42:0/427723202 wait complete. 2024-04-25T00:28:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.124+0000 7f1f41a00640 1 Processor -- start 2024-04-25T00:28:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.124+0000 7f1f41a00640 1 -- start start 2024-04-25T00:28:32.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f41a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f3c1015e0 0x7f1f3c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f41a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c1027d0 0x7f1f3c19a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f41a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f3c103190 0x7f1f3c1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f41a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c10d290 con 0x7f1f3c103190 2024-04-25T00:28:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f41a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c10d110 con 0x7f1f3c1027d0 2024-04-25T00:28:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f41a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c10d410 con 0x7f1f3c1015e0 2024-04-25T00:28:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f3be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f3c1015e0 0x7f1f3c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f3b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c1027d0 0x7f1f3c19a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f3be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f3c1015e0 0x7f1f3c199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:52456/0 (socket says 172.21.15.42:52456) 2024-04-25T00:28:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f3be00640 1 -- 172.21.15.42:0/743653488 learned_addr learned my addr 172.21.15.42:0/743653488 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:32.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.125+0000 7f1f41000640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f3c103190 0x7f1f3c1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f3b400640 1 -- 172.21.15.42:0/743653488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f3c1015e0 msgr2=0x7f1f3c199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f3b400640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f3c1015e0 0x7f1f3c199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f3b400640 1 -- 172.21.15.42:0/743653488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f3c103190 msgr2=0x7f1f3c1a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f3b400640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f3c103190 0x7f1f3c1a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f3b400640 1 -- 172.21.15.42:0/743653488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c1a1690 con 0x7f1f3c1027d0 2024-04-25T00:28:32.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f3be00640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f3c1015e0 0x7f1f3c199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:32.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:31 smithi042 ceph-mon[28130]: pgmap v681: 97 pgs: 97 active+clean; 113 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 408 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:28:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f3b400640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c1027d0 0x7f1f3c19a000 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400e9f0 tx=0x7f1f2400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f38c00640 1 -- 172.21.15.42:0/743653488 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2400ccc0 con 0x7f1f3c1027d0 2024-04-25T00:28:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f41a00640 1 -- 172.21.15.42:0/743653488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c1a1980 con 0x7f1f3c1027d0 2024-04-25T00:28:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f38c00640 1 -- 172.21.15.42:0/743653488 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f24010910 con 0x7f1f3c1027d0 2024-04-25T00:28:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.126+0000 7f1f38c00640 1 -- 172.21.15.42:0/743653488 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f24018960 con 0x7f1f3c1027d0 2024-04-25T00:28:32.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.127+0000 7f1f41a00640 1 -- 172.21.15.42:0/743653488 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c1a1f20 con 0x7f1f3c1027d0 2024-04-25T00:28:32.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.130+0000 7f1f38c00640 1 -- 172.21.15.42:0/743653488 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f24010ac0 con 0x7f1f3c1027d0 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.130+0000 7f1f45fdc740 1 Processor -- start 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.130+0000 7f1f45fdc740 1 -- start start 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.130+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501b9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.130+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 0x559d501be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.130+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501c0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.130+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559d5011bab0 con 0x559d501be9b0 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.130+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559d500c4640 con 0x559d501b8e50 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.130+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d50117a70 con 0x559d501b9840 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f41000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f41000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501b9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35292/0 (socket says 172.21.15.42:35292) 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f41000640 1 -- 172.21.15.42:0/1519483249 learned_addr learned my addr 172.21.15.42:0/1519483249 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f3be00640 1 --2- 172.21.15.42:0/1519483249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 0x559d501be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f3b400640 1 --2- 172.21.15.42:0/1519483249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501c0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f41000640 1 -- 172.21.15.42:0/1519483249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 msgr2=0x559d501be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f41000640 1 --2- 172.21.15.42:0/1519483249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 0x559d501be340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f41000640 1 -- 172.21.15.42:0/1519483249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 msgr2=0x559d501c0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f41000640 1 --2- 172.21.15.42:0/1519483249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501c0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f41000640 1 -- 172.21.15.42:0/1519483249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d4fe30210 con 0x559d501b8e50 2024-04-25T00:28:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.131+0000 7f1f3b400640 1 --2- 172.21.15.42:0/1519483249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501c0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.132+0000 7f1f41000640 1 --2- 172.21.15.42:0/1519483249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501b9270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009ca0 tx=0x7f1f340093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f99ba0b34a3c208f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.132+0000 7f1f31600640 1 -- 172.21.15.42:0/1519483249 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34031650 con 0x559d501b8e50 2024-04-25T00:28:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.132+0000 7f1f31600640 1 -- 172.21.15.42:0/1519483249 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f34036030 con 0x559d501b8e50 2024-04-25T00:28:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.132+0000 7f1f31600640 1 -- 172.21.15.42:0/1519483249 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f34004740 con 0x559d501b8e50 2024-04-25T00:28:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.132+0000 7f1f38c00640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f200a2b80 0x7f1f200a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.132+0000 7f1f38c00640 1 -- 172.21.15.42:0/743653488 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1f240cf3a0 con 0x7f1f3c1027d0 2024-04-25T00:28:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.133+0000 7f1f3be00640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f200a2b80 0x7f1f200a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.133+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1519483249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 msgr2=0x559d501b9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.133+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/1519483249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501b9270 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009ca0 tx=0x7f1f340093c0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.133+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1519483249 shutdown_connections 2024-04-25T00:28:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.133+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/1519483249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501c0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.133+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/1519483249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 0x559d501be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.133+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/1519483249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501b9270 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.133+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1519483249 >> 172.21.15.42:0/1519483249 conn(0x559d501b44c0 msgr2=0x559d501b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.133+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1519483249 shutdown_connections 2024-04-25T00:28:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.133+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1519483249 wait complete. 2024-04-25T00:28:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.134+0000 7f1f3be00640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f200a2b80 0x7f1f200a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x559d501b5390 tx=0x7f1f2c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.134+0000 7f1f45fdc740 1 -- start start 2024-04-25T00:28:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.134+0000 7f1f45fdc740 1 Processor -- start 2024-04-25T00:28:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501c2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b9840 0x559d501c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501ca470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x559d501c1780 con 0x559d501be9b0 2024-04-25T00:28:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x559d500d7990 con 0x559d501b9840 2024-04-25T00:28:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559d501c1900 con 0x559d501b8e50 2024-04-25T00:28:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f45fdc740 10 client.0 _ll_register_callbacks cb 0x559d500d9950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f3be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501c2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f3b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b9840 0x559d501c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f3be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501c2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:52470/0 (socket says 172.21.15.42:52470) 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f41000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501ca470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f3b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b9840 0x559d501c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35306/0 (socket says 172.21.15.42:35306) 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.135+0000 7f1f3be00640 1 -- 172.21.15.42:0/1425467303 learned_addr learned my addr 172.21.15.42:0/1425467303 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.136+0000 7f1f3be00640 1 -- 172.21.15.42:0/1425467303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b9840 msgr2=0x559d501c3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.136+0000 7f1f3be00640 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b9840 0x559d501c3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.136+0000 7f1f3be00640 1 -- 172.21.15.42:0/1425467303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 msgr2=0x559d501ca470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.136+0000 7f1f3be00640 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501ca470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.136+0000 7f1f3be00640 1 -- 172.21.15.42:0/1425467303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d4fefc620 con 0x559d501b8e50 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.136+0000 7f1f3be00640 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501c2ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00f750 tx=0x7f1f2c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bada181f9d7ec46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.136+0000 7f1f2a000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.137+0000 7f1f2a000640 1 -- 172.21.15.42:0/1425467303 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0118c0 con 0x559d501b8e50 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.137+0000 7f1f2a000640 1 -- 172.21.15.42:0/1425467303 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1f2c011a60 con 0x559d501b8e50 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.137+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1425467303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f34036030 con 0x559d501b8e50 2024-04-25T00:28:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.137+0000 7f1f2a000640 1 -- 172.21.15.42:0/1425467303 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f2c011ca0 con 0x559d501b8e50 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.137+0000 7f1f45fdc740 10 client.25966 fetch_fsmap learned FSMap version 30 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.137+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1425467303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d501cd520 con 0x559d501b8e50 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.137+0000 7f1f2a000640 1 -- 172.21.15.42:0/1425467303 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f34036030 con 0x559d501b8e50 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.138+0000 7f1f2a000640 1 -- 172.21.15.42:0/1425467303 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f1f2c027070 con 0x559d501b8e50 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.138+0000 7f1f45fdc740 10 client.25966 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.138+0000 7f1f45fdc740 10 client.25966 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.138+0000 7f1f45fdc740 20 client.25966 populate_metadata read hostname 'smithi042' 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.138+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x559d501cec60 0x559d501d10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.138+0000 7f1f45fdc740 4 client.25966 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.138+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1425467303 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d50128ff0 con 0x559d501cec60 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.138+0000 7f1f3b400640 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x559d501cec60 0x559d501d10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.138+0000 7f1f3b400640 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x559d501cec60 0x559d501d10d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb6cb250acb5ae9 server_cookie=a016e8857f9df6d8 in_seq=0 out_seq=0 2024-04-25T00:28:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.138+0000 7f1f2a000640 10 client.25966 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.139+0000 7f1f2a000640 1 -- 172.21.15.42:0/1425467303 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d50128ff0 con 0x559d501cec60 2024-04-25T00:28:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.139+0000 7f1f2a000640 10 client.25966 handle_command_reply: tid=0 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.139+0000 7f1f45fdc740 1 client.25966 shutdown 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1425467303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 msgr2=0x559d501c2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501c2ef0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00f750 tx=0x7f1f2c004730 comp rx=0 tx=0).stop 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1425467303 shutdown_connections 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x559d501cec60 0x559d501d10d0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501ca470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b9840 0x559d501c3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f2a000640 0 client.25966 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/1425467303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501c2ef0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1425467303 >> 172.21.15.42:0/1425467303 conn(0x559d501b44c0 msgr2=0x559d501b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1425467303 shutdown_connections 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 1 -- 172.21.15.42:0/1425467303 wait complete. 2024-04-25T00:28:32.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.140+0000 7f1f45fdc740 20 client.25966 trim_cache size 0 max 16384 2024-04-25T00:28:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.159+0000 7f1f45fdc740 1 Processor -- start 2024-04-25T00:28:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.159+0000 7f1f45fdc740 1 -- start start 2024-04-25T00:28:32.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.159+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.159+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501b9840 0x559d501c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.159+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501be9b0 0x559d501c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.159+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c1ad450 con 0x559d501b9840 2024-04-25T00:28:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.159+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c1aacf0 con 0x559d501be9b0 2024-04-25T00:28:32.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.159+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c1a8b40 con 0x559d501b8e50 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.159+0000 7f1f3be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.160+0000 7f1f41000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501be9b0 0x559d501c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.160+0000 7f1f3be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501be0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:52486/0 (socket says 172.21.15.42:52486) 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.160+0000 7f1f3be00640 1 -- 172.21.15.42:0/393688608 learned_addr learned my addr 172.21.15.42:0/393688608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.160+0000 7f1f3b400640 1 --2- 172.21.15.42:0/393688608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501b9840 0x559d501c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.160+0000 7f1f3be00640 1 -- 172.21.15.42:0/393688608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501be9b0 msgr2=0x559d501c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.160+0000 7f1f3be00640 1 --2- 172.21.15.42:0/393688608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501be9b0 0x559d501c3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.160+0000 7f1f3be00640 1 -- 172.21.15.42:0/393688608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501b9840 msgr2=0x559d501c2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.160+0000 7f1f3be00640 1 --2- 172.21.15.42:0/393688608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501b9840 0x559d501c2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.160+0000 7f1f3be00640 1 -- 172.21.15.42:0/393688608 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d501b2a10 con 0x559d501b8e50 2024-04-25T00:28:32.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.160+0000 7f1f3be00640 1 --2- 172.21.15.42:0/393688608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501be0d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c034670 tx=0x7f1f2c012570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99dfc403fbff1e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.161+0000 7f1f30c00640 1 -- 172.21.15.42:0/393688608 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0118c0 con 0x559d501b8e50 2024-04-25T00:28:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.161+0000 7f1f30c00640 1 -- 172.21.15.42:0/393688608 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f2c038030 con 0x559d501b8e50 2024-04-25T00:28:32.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.161+0000 7f1f30c00640 1 -- 172.21.15.42:0/393688608 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c005580 con 0x559d501b8e50 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.162+0000 7f1f45fdc740 1 -- 172.21.15.42:0/393688608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 msgr2=0x559d501be0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.162+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/393688608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501be0d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c034670 tx=0x7f1f2c012570 comp rx=0 tx=0).stop 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.162+0000 7f1f45fdc740 1 -- 172.21.15.42:0/393688608 shutdown_connections 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.162+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/393688608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501be9b0 0x559d501c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.162+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/393688608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501b9840 0x559d501c2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.162+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/393688608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b8e50 0x559d501be0d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.162+0000 7f1f45fdc740 1 -- 172.21.15.42:0/393688608 >> 172.21.15.42:0/393688608 conn(0x559d501b44c0 msgr2=0x559d501b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.162+0000 7f1f45fdc740 1 -- 172.21.15.42:0/393688608 shutdown_connections 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.162+0000 7f1f45fdc740 1 -- 172.21.15.42:0/393688608 wait complete. 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.162+0000 7f1f45fdc740 1 -- start start 2024-04-25T00:28:32.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f45fdc740 1 Processor -- start 2024-04-25T00:28:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 0x559d501c7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f45fdc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501c8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c1a91b0 con 0x559d501be9b0 2024-04-25T00:28:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c1aa620 con 0x559d501b8e50 2024-04-25T00:28:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f45fdc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c1a9880 con 0x559d501b9840 2024-04-25T00:28:32.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f45fdc740 10 client.0 _ll_register_callbacks cb 0x7f1f2c00f5e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f3be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f3be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501c6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35316/0 (socket says 172.21.15.42:35316) 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f3be00640 1 -- 172.21.15.42:0/3729639688 learned_addr learned my addr 172.21.15.42:0/3729639688 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f3b400640 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 0x559d501c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.163+0000 7f1f41000640 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f3be00640 1 -- 172.21.15.42:0/3729639688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 msgr2=0x559d501c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f3be00640 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 0x559d501c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f3be00640 1 -- 172.21.15.42:0/3729639688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 msgr2=0x559d501c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f3be00640 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f3be00640 1 -- 172.21.15.42:0/3729639688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d501c8940 con 0x559d501b8e50 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f3b400640 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 0x559d501c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f3be00640 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501c6e00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00f750 tx=0x7f1f2c0346f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87f485b56d43554 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f2a000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f2a000640 1 -- 172.21.15.42:0/3729639688 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c042070 con 0x559d501b8e50 2024-04-25T00:28:32.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f2a000640 1 -- 172.21.15.42:0/3729639688 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1f2c005500 con 0x559d501b8e50 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.164+0000 7f1f45fdc740 1 -- 172.21.15.42:0/3729639688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f2c038030 con 0x559d501b8e50 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f2a000640 1 -- 172.21.15.42:0/3729639688 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f2c04e050 con 0x559d501b8e50 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f45fdc740 10 client.25967 fetch_fsmap learned FSMap version 30 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f45fdc740 1 -- 172.21.15.42:0/3729639688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x559d501c9c50 con 0x559d501b8e50 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f2a000640 1 -- 172.21.15.42:0/3729639688 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f2c038030 con 0x559d501b8e50 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f2a000640 1 -- 172.21.15.42:0/3729639688 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f1f2c027070 con 0x559d501b8e50 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f45fdc740 10 client.25967 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f45fdc740 10 client.25967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f45fdc740 20 client.25967 populate_metadata read hostname 'smithi042' 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x559d502358c0 0x559d50237ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f45fdc740 4 client.25967 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f45fdc740 1 -- 172.21.15.42:0/3729639688 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d50123340 con 0x559d502358c0 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.165+0000 7f1f3b400640 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x559d502358c0 0x559d50237ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.166+0000 7f1f3b400640 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x559d502358c0 0x559d50237ce0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9255676b809e4938 server_cookie=ea8d0f6ffbb0891a in_seq=0 out_seq=0 2024-04-25T00:28:32.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.166+0000 7f1f2a000640 10 client.25967 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:32.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.223+0000 7f1f2a000640 1 -- 172.21.15.42:0/3729639688 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d50123340 con 0x559d502358c0 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.223+0000 7f1f2a000640 10 client.25967 handle_command_reply: tid=0 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.223+0000 7f1f45fdc740 1 client.25967 shutdown 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 1 -- 172.21.15.42:0/3729639688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 msgr2=0x559d501c6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501c6e00 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00f750 tx=0x7f1f2c0346f0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 1 -- 172.21.15.42:0/3729639688 shutdown_connections 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x559d502358c0 0x559d50237ce0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x559d501be9b0 0x559d501c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559d501b9840 0x559d501c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f2a000640 0 client.25967 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 1 --2- 172.21.15.42:0/3729639688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559d501b8e50 0x559d501c6e00 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 1 -- 172.21.15.42:0/3729639688 >> 172.21.15.42:0/3729639688 conn(0x559d501b44c0 msgr2=0x559d501bdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 1 -- 172.21.15.42:0/3729639688 shutdown_connections 2024-04-25T00:28:32.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 1 -- 172.21.15.42:0/3729639688 wait complete. 2024-04-25T00:28:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.224+0000 7f1f45fdc740 20 client.25967 trim_cache size 0 max 16384 2024-04-25T00:28:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:28:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:28:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:28:32.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 -- 172.21.15.42:0/743653488 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f200a2b80 msgr2=0x7f1f200a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f200a2b80 0x7f1f200a5040 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x559d501b5390 tx=0x7f1f2c005f90 comp rx=0 tx=0).stop 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 -- 172.21.15.42:0/743653488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c1027d0 msgr2=0x7f1f3c19a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c1027d0 0x7f1f3c19a000 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400e9f0 tx=0x7f1f2400eeb0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 -- 172.21.15.42:0/743653488 shutdown_connections 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1f200a2b80 0x7f1f200a5040 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1f3c103190 0x7f1f3c1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3c1027d0 0x7f1f3c19a000 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 --2- 172.21.15.42:0/743653488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1f3c1015e0 0x7f1f3c199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 -- 172.21.15.42:0/743653488 >> 172.21.15.42:0/743653488 conn(0x7f1f3c0fcb50 msgr2=0x7f1f3c073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 -- 172.21.15.42:0/743653488 shutdown_connections 2024-04-25T00:28:32.227 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:32.225+0000 7f1f41a00640 1 -- 172.21.15.42:0/743653488 wait complete. 2024-04-25T00:28:32.263 INFO:tasks.workunit.client.0.smithi042.stdout: 1 597005 6.31 MB/sec execute 382 sec latency 47.136 ms 2024-04-25T00:28:33.263 INFO:tasks.workunit.client.0.smithi042.stdout: 1 597655 6.30 MB/sec execute 383 sec latency 50.934 ms 2024-04-25T00:28:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:33 smithi136 ceph-mon[32923]: pgmap v682: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-25T00:28:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:33 smithi106 ceph-mon[32510]: pgmap v682: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-25T00:28:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:33 smithi042 ceph-mon[28130]: pgmap v682: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-25T00:28:34.263 INFO:tasks.workunit.client.0.smithi042.stdout: 1 598911 6.29 MB/sec execute 384 sec latency 38.075 ms 2024-04-25T00:28:35.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:34 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:28:35.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:34 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:28:35.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:34 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:28:35.263 INFO:tasks.workunit.client.0.smithi042.stdout: 1 600377 6.31 MB/sec execute 385 sec latency 15.790 ms 2024-04-25T00:28:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:35 smithi136 ceph-mon[32923]: pgmap v683: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-04-25T00:28:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:35 smithi106 ceph-mon[32510]: pgmap v683: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-04-25T00:28:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:35 smithi042 ceph-mon[28130]: pgmap v683: 97 pgs: 97 active+clean; 114 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 415 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-04-25T00:28:36.263 INFO:tasks.workunit.client.0.smithi042.stdout: 1 600857 6.30 MB/sec execute 386 sec latency 20.824 ms 2024-04-25T00:28:37.263 INFO:tasks.workunit.client.0.smithi042.stdout: 1 601895 6.29 MB/sec execute 387 sec latency 22.070 ms 2024-04-25T00:28:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:37 smithi136 ceph-mon[32923]: pgmap v684: 97 pgs: 97 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-04-25T00:28:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:37 smithi106 ceph-mon[32510]: pgmap v684: 97 pgs: 97 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-04-25T00:28:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:37 smithi042 ceph-mon[28130]: pgmap v684: 97 pgs: 97 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 630 KiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-04-25T00:28:38.264 INFO:tasks.workunit.client.0.smithi042.stdout: 1 603531 6.30 MB/sec execute 388 sec latency 64.533 ms 2024-04-25T00:28:39.264 INFO:tasks.workunit.client.0.smithi042.stdout: 1 604241 6.30 MB/sec execute 389 sec latency 22.503 ms 2024-04-25T00:28:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:39 smithi106 ceph-mon[32510]: pgmap v685: 97 pgs: 97 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-25T00:28:40.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:39 smithi042 ceph-mon[28130]: pgmap v685: 97 pgs: 97 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-25T00:28:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:39 smithi136 ceph-mon[32923]: pgmap v685: 97 pgs: 97 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.5 MiB/s wr, 173 op/s 2024-04-25T00:28:40.264 INFO:tasks.workunit.client.0.smithi042.stdout: 1 605213 6.29 MB/sec execute 390 sec latency 29.616 ms 2024-04-25T00:28:41.264 INFO:tasks.workunit.client.0.smithi042.stdout: 1 606634 6.29 MB/sec execute 391 sec latency 21.272 ms 2024-04-25T00:28:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:41 smithi106 ceph-mon[32510]: pgmap v686: 97 pgs: 97 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-04-25T00:28:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:41 smithi042 ceph-mon[28130]: pgmap v686: 97 pgs: 97 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-04-25T00:28:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:41 smithi136 ceph-mon[32923]: pgmap v686: 97 pgs: 97 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-04-25T00:28:42.264 INFO:tasks.workunit.client.0.smithi042.stdout: 1 607715 6.30 MB/sec execute 392 sec latency 17.139 ms 2024-04-25T00:28:43.264 INFO:tasks.workunit.client.0.smithi042.stdout: 1 608452 6.29 MB/sec execute 393 sec latency 69.721 ms 2024-04-25T00:28:44.064 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:43 smithi136 ceph-mon[32923]: pgmap v687: 97 pgs: 97 active+clean; 118 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-04-25T00:28:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:43 smithi106 ceph-mon[32510]: pgmap v687: 97 pgs: 97 active+clean; 118 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-04-25T00:28:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:43 smithi042 ceph-mon[28130]: pgmap v687: 97 pgs: 97 active+clean; 118 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 590 KiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-04-25T00:28:44.264 INFO:tasks.workunit.client.0.smithi042.stdout: 1 609786 6.28 MB/sec execute 394 sec latency 28.840 ms 2024-04-25T00:28:45.264 INFO:tasks.workunit.client.0.smithi042.stdout: 1 611140 6.30 MB/sec execute 395 sec latency 20.177 ms 2024-04-25T00:28:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.379+0000 7f60a0a00640 1 -- 172.21.15.42:0/2208920431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609c075780 msgr2=0x7f609c073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.379+0000 7f60a0a00640 1 --2- 172.21.15.42:0/2208920431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609c075780 0x7f609c073bd0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f608c009900 tx=0x7f608c02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.380+0000 7f60a0a00640 1 -- 172.21.15.42:0/2208920431 shutdown_connections 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.380+0000 7f60a0a00640 1 --2- 172.21.15.42:0/2208920431 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f609c10dc30 0x7f609c110020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.380+0000 7f60a0a00640 1 --2- 172.21.15.42:0/2208920431 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f609c0741a0 0x7f609c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.380+0000 7f60a0a00640 1 --2- 172.21.15.42:0/2208920431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609c075780 0x7f609c073bd0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.380+0000 7f60a0a00640 1 -- 172.21.15.42:0/2208920431 >> 172.21.15.42:0/2208920431 conn(0x7f609c0fcbd0 msgr2=0x7f609c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.380+0000 7f60a0a00640 1 -- 172.21.15.42:0/2208920431 shutdown_connections 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.380+0000 7f60a0a00640 1 -- 172.21.15.42:0/2208920431 wait complete. 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.381+0000 7f60a0a00640 1 Processor -- start 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.381+0000 7f60a0a00640 1 -- start start 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.381+0000 7f60a0a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f609c0741a0 0x7f609c19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.381+0000 7f60a0a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609c075780 0x7f609c19e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.381+0000 7f60a0a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f609c10dc30 0x7f609c1a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.381+0000 7f60a0a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1138b0 con 0x7f609c10dc30 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.381+0000 7f60a0a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f609c113730 con 0x7f609c0741a0 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.381+0000 7f60a0a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f609c113a30 con 0x7f609c075780 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f6099e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609c075780 0x7f609c19e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f609a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f609c0741a0 0x7f609c19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f6099e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609c075780 0x7f609c19e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:56776/0 (socket says 172.21.15.42:56776) 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f6099e00640 1 -- 172.21.15.42:0/3558999140 learned_addr learned my addr 172.21.15.42:0/3558999140 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f609b200640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f609c10dc30 0x7f609c1a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f6099e00640 1 -- 172.21.15.42:0/3558999140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f609c0741a0 msgr2=0x7f609c19de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f6099e00640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f609c0741a0 0x7f609c19de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f6099e00640 1 -- 172.21.15.42:0/3558999140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f609c10dc30 msgr2=0x7f609c1a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f6099e00640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f609c10dc30 0x7f609c1a5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f6099e00640 1 -- 172.21.15.42:0/3558999140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f609c1a5a70 con 0x7f609c075780 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f609a800640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f609c0741a0 0x7f609c19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.382+0000 7f6099e00640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609c075780 0x7f609c19e390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f608800c920 tx=0x7f608800cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.383+0000 7f6093400640 1 -- 172.21.15.42:0/3558999140 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088007c70 con 0x7f609c075780 2024-04-25T00:28:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.383+0000 7f6093400640 1 -- 172.21.15.42:0/3558999140 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6088007e10 con 0x7f609c075780 2024-04-25T00:28:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.383+0000 7f60a0a00640 1 -- 172.21.15.42:0/3558999140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f609c1a5d60 con 0x7f609c075780 2024-04-25T00:28:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.383+0000 7f6093400640 1 -- 172.21.15.42:0/3558999140 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6088018890 con 0x7f609c075780 2024-04-25T00:28:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.383+0000 7f60a0a00640 1 -- 172.21.15.42:0/3558999140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f609c1a6270 con 0x7f609c075780 2024-04-25T00:28:45.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.386+0000 7f60a4253740 1 Processor -- start 2024-04-25T00:28:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f60a4253740 1 -- start start 2024-04-25T00:28:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f6093400640 1 -- 172.21.15.42:0/3558999140 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6088018b30 con 0x7f609c075780 2024-04-25T00:28:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f60a4253740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 0x56035af6f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f60a4253740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 0x56035af74340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f60a4253740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 0x56035af76e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f60a4253740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56035aed1ab0 con 0x56035af749b0 2024-04-25T00:28:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f60a4253740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56035ae7a640 con 0x56035af6ee50 2024-04-25T00:28:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f60a4253740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56035aecda70 con 0x56035af6f840 2024-04-25T00:28:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f609b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 0x56035af6f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f609b200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 0x56035af6f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40048/0 (socket says 172.21.15.42:40048) 2024-04-25T00:28:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f609b200640 1 -- 172.21.15.42:0/322963407 learned_addr learned my addr 172.21.15.42:0/322963407 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.387+0000 7f609a800640 1 --2- 172.21.15.42:0/322963407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 0x56035af74340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.388+0000 7f6099e00640 1 --2- 172.21.15.42:0/322963407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 0x56035af76e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.388+0000 7f609b200640 1 -- 172.21.15.42:0/322963407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 msgr2=0x56035af74340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.388+0000 7f609b200640 1 --2- 172.21.15.42:0/322963407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 0x56035af74340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.388+0000 7f609b200640 1 -- 172.21.15.42:0/322963407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 msgr2=0x56035af76e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.388+0000 7f609b200640 1 --2- 172.21.15.42:0/322963407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 0x56035af76e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.388+0000 7f609b200640 1 -- 172.21.15.42:0/322963407 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035abe6210 con 0x56035af6ee50 2024-04-25T00:28:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.388+0000 7f6099e00640 1 --2- 172.21.15.42:0/322963407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 0x56035af76e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:28:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.388+0000 7f609b200640 1 --2- 172.21.15.42:0/322963407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 0x56035af6f270 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6094009fd0 tx=0x7f60940118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a6366fc1e6f679f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.388+0000 7f6087400640 1 -- 172.21.15.42:0/322963407 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094004d20 con 0x56035af6ee50 2024-04-25T00:28:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.388+0000 7f6093400640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f60800a2c00 0x7f60800a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.389+0000 7f6087400640 1 -- 172.21.15.42:0/322963407 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6094004050 con 0x56035af6ee50 2024-04-25T00:28:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.389+0000 7f6087400640 1 -- 172.21.15.42:0/322963407 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6094005260 con 0x56035af6ee50 2024-04-25T00:28:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.389+0000 7f6093400640 1 -- 172.21.15.42:0/3558999140 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f60880cefd0 con 0x7f609c075780 2024-04-25T00:28:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.389+0000 7f609a800640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f60800a2c00 0x7f60800a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.389+0000 7f60a4253740 1 -- 172.21.15.42:0/322963407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 msgr2=0x56035af6f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.389+0000 7f60a4253740 1 --2- 172.21.15.42:0/322963407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 0x56035af6f270 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6094009fd0 tx=0x7f60940118d0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.390+0000 7f60a4253740 1 -- 172.21.15.42:0/322963407 shutdown_connections 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.390+0000 7f60a4253740 1 --2- 172.21.15.42:0/322963407 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 0x56035af76e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.390+0000 7f60a4253740 1 --2- 172.21.15.42:0/322963407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 0x56035af74340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.390+0000 7f60a4253740 1 --2- 172.21.15.42:0/322963407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 0x56035af6f270 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.390+0000 7f60a4253740 1 -- 172.21.15.42:0/322963407 >> 172.21.15.42:0/322963407 conn(0x56035af6a4c0 msgr2=0x56035af6a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.390+0000 7f60a4253740 1 -- 172.21.15.42:0/322963407 shutdown_connections 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.390+0000 7f609a800640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f60800a2c00 0x7f60800a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f608c005ab0 tx=0x7f608c005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.390+0000 7f60a4253740 1 -- 172.21.15.42:0/322963407 wait complete. 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.390+0000 7f60a4253740 1 -- start start 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.390+0000 7f60a4253740 1 Processor -- start 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f60a4253740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 0x56035af79090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f60a4253740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 0x56035af795d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f60a4253740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 0x56035af80610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f60a4253740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56035af77960 con 0x56035af749b0 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f60a4253740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56035ae8d990 con 0x56035af6ee50 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f60a4253740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56035af77ae0 con 0x56035af6f840 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f60a4253740 10 client.0 _ll_register_callbacks cb 0x56035ae8f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f6099e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 0x56035af795d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f6099e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 0x56035af795d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:56790/0 (socket says 172.21.15.42:56790) 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f6099e00640 1 -- 172.21.15.42:0/3537737573 learned_addr learned my addr 172.21.15.42:0/3537737573 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f609b200640 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 0x56035af80610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.391+0000 7f609a800640 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 0x56035af79090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f6099e00640 1 -- 172.21.15.42:0/3537737573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 msgr2=0x56035af79090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f6099e00640 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 0x56035af79090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f6099e00640 1 -- 172.21.15.42:0/3537737573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 msgr2=0x56035af80610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f6099e00640 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 0x56035af80610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f6099e00640 1 -- 172.21.15.42:0/3537737573 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035acb2620 con 0x56035af6f840 2024-04-25T00:28:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f6099e00640 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 0x56035af795d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f608809b5a0 tx=0x7f608809b200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f398eab42a51c009 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:45.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f607be00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:28:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f607be00640 1 -- 172.21.15.42:0/3537737573 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608809b8a0 con 0x56035af6f840 2024-04-25T00:28:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f607be00640 1 -- 172.21.15.42:0/3537737573 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f60880860e0 con 0x56035af6f840 2024-04-25T00:28:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f60a4253740 1 -- 172.21.15.42:0/3537737573 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6094004050 con 0x56035af6f840 2024-04-25T00:28:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f607be00640 1 -- 172.21.15.42:0/3537737573 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f60880862e0 con 0x56035af6f840 2024-04-25T00:28:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f60a4253740 10 client.25984 fetch_fsmap learned FSMap version 31 2024-04-25T00:28:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.392+0000 7f60a4253740 1 -- 172.21.15.42:0/3537737573 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56035af835d0 con 0x56035af6f840 2024-04-25T00:28:45.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.393+0000 7f607be00640 1 -- 172.21.15.42:0/3537737573 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6094004050 con 0x56035af6f840 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.393+0000 7f607be00640 1 -- 172.21.15.42:0/3537737573 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f60880a4080 con 0x56035af6f840 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.393+0000 7f60a4253740 10 client.25984 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.393+0000 7f60a4253740 10 client.25984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.393+0000 7f60a4253740 20 client.25984 populate_metadata read hostname 'smithi042' 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.393+0000 7f60a4253740 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56035af84d40 0x56035af87130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.393+0000 7f60a4253740 4 client.25984 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.393+0000 7f60a4253740 1 -- 172.21.15.42:0/3537737573 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56035aedeff0 con 0x56035af84d40 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.393+0000 7f609a800640 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56035af84d40 0x56035af87130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.394+0000 7f609a800640 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56035af84d40 0x56035af87130 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc93f6f63a2b279 server_cookie=e62dea930852c5bc in_seq=0 out_seq=0 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.394+0000 7f607be00640 10 client.25984 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.394+0000 7f607be00640 1 -- 172.21.15.42:0/3537737573 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56035aedeff0 con 0x56035af84d40 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.394+0000 7f607be00640 10 client.25984 handle_command_reply: tid=0 2024-04-25T00:28:45.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 client.25984 shutdown 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 -- 172.21.15.42:0/3537737573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 msgr2=0x56035af795d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 0x56035af795d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f608809b5a0 tx=0x7f608809b200 comp rx=0 tx=0).stop 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 -- 172.21.15.42:0/3537737573 shutdown_connections 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56035af84d40 0x56035af87130 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af749b0 0x56035af80610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6f840 0x56035af795d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 --2- 172.21.15.42:0/3537737573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6ee50 0x56035af79090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f607be00640 0 client.25984 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 -- 172.21.15.42:0/3537737573 >> 172.21.15.42:0/3537737573 conn(0x56035af6a4c0 msgr2=0x56035af69620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 -- 172.21.15.42:0/3537737573 shutdown_connections 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 1 -- 172.21.15.42:0/3537737573 wait complete. 2024-04-25T00:28:45.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.395+0000 7f60a4253740 20 client.25984 trim_cache size 0 max 16384 2024-04-25T00:28:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.412+0000 7f60a4253740 1 Processor -- start 2024-04-25T00:28:45.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.412+0000 7f60a4253740 1 -- start start 2024-04-25T00:28:45.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f60a4253740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6ee50 0x56035af70a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f60a4253740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6f840 0x56035af805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f60a4253740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af749b0 0x56035af70fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f60a4253740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1b17a0 con 0x56035af6f840 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f60a4253740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1af070 con 0x56035af749b0 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f60a4253740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1acec0 con 0x56035af6ee50 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f6099e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6f840 0x56035af805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f609a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6ee50 0x56035af70a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f6099e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6f840 0x56035af805d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50934/0 (socket says 172.21.15.42:50934) 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f6099e00640 1 -- 172.21.15.42:0/1333321104 learned_addr learned my addr 172.21.15.42:0/1333321104 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.413+0000 7f609b200640 1 --2- 172.21.15.42:0/1333321104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af749b0 0x56035af70fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.414+0000 7f6099e00640 1 -- 172.21.15.42:0/1333321104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6ee50 msgr2=0x56035af70a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.414+0000 7f6099e00640 1 --2- 172.21.15.42:0/1333321104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6ee50 0x56035af70a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.414+0000 7f6099e00640 1 -- 172.21.15.42:0/1333321104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af749b0 msgr2=0x56035af70fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.414+0000 7f6099e00640 1 --2- 172.21.15.42:0/1333321104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af749b0 0x56035af70fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.414+0000 7f6099e00640 1 -- 172.21.15.42:0/1333321104 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035af6ea90 con 0x56035af6f840 2024-04-25T00:28:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.414+0000 7f6099e00640 1 --2- 172.21.15.42:0/1333321104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6f840 0x56035af805d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f608800fee0 tx=0x7f608800ff30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404789d8b038ea9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.414+0000 7f6086a00640 1 -- 172.21.15.42:0/1333321104 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60880a4510 con 0x56035af6f840 2024-04-25T00:28:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.414+0000 7f6086a00640 1 -- 172.21.15.42:0/1333321104 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f608809aeb0 con 0x56035af6f840 2024-04-25T00:28:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.414+0000 7f6086a00640 1 -- 172.21.15.42:0/1333321104 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60880d23d0 con 0x56035af6f840 2024-04-25T00:28:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.415+0000 7f60a4253740 1 -- 172.21.15.42:0/1333321104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6f840 msgr2=0x56035af805d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.415+0000 7f60a4253740 1 --2- 172.21.15.42:0/1333321104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6f840 0x56035af805d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f608800fee0 tx=0x7f608800ff30 comp rx=0 tx=0).stop 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.415+0000 7f60a4253740 1 -- 172.21.15.42:0/1333321104 shutdown_connections 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.415+0000 7f60a4253740 1 --2- 172.21.15.42:0/1333321104 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af749b0 0x56035af70fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.415+0000 7f60a4253740 1 --2- 172.21.15.42:0/1333321104 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6f840 0x56035af805d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.415+0000 7f60a4253740 1 --2- 172.21.15.42:0/1333321104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af6ee50 0x56035af70a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.415+0000 7f60a4253740 1 -- 172.21.15.42:0/1333321104 >> 172.21.15.42:0/1333321104 conn(0x56035af6a4c0 msgr2=0x56035af65110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.415+0000 7f60a4253740 1 -- 172.21.15.42:0/1333321104 shutdown_connections 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.415+0000 7f60a4253740 1 -- 172.21.15.42:0/1333321104 wait complete. 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.416+0000 7f60a4253740 1 -- start start 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.416+0000 7f60a4253740 1 Processor -- start 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.416+0000 7f60a4253740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6ee50 0x56035af7dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.416+0000 7f60a4253740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6f840 0x56035af7e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.416+0000 7f60a4253740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af749b0 0x56035af7f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.416+0000 7f60a4253740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1ad530 con 0x56035af6ee50 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.416+0000 7f60a4253740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1ae9a0 con 0x56035af6f840 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.416+0000 7f60a4253740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f609c1adc00 con 0x56035af749b0 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.416+0000 7f60a4253740 10 client.0 _ll_register_callbacks cb 0x7f608800fe30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f609b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af749b0 0x56035af7f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f609a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6ee50 0x56035af7dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f6099e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6f840 0x56035af7e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f609b200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af749b0 0x56035af7f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:56810/0 (socket says 172.21.15.42:56810) 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f609b200640 1 -- 172.21.15.42:0/1715587967 learned_addr learned my addr 172.21.15.42:0/1715587967 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f6099e00640 1 -- 172.21.15.42:0/1715587967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af749b0 msgr2=0x56035af7f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f6099e00640 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af749b0 0x56035af7f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f6099e00640 1 -- 172.21.15.42:0/1715587967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6ee50 msgr2=0x56035af7dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f6099e00640 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6ee50 0x56035af7dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f6099e00640 1 -- 172.21.15.42:0/1715587967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035af7f690 con 0x56035af6f840 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f6099e00640 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6f840 0x56035af7e100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f608809b5a0 tx=0x7f608809b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5279db123254e466 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f607be00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f607be00640 1 -- 172.21.15.42:0/1715587967 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608809cdd0 con 0x56035af6f840 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f60a4253740 1 -- 172.21.15.42:0/1715587967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f608809aeb0 con 0x56035af6f840 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.417+0000 7f607be00640 1 -- 172.21.15.42:0/1715587967 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f60880975f0 con 0x56035af6f840 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f607be00640 1 -- 172.21.15.42:0/1715587967 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f60880a4510 con 0x56035af6f840 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f60a4253740 10 client.25985 fetch_fsmap learned FSMap version 31 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f60a4253740 1 -- 172.21.15.42:0/1715587967 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56035af7f690 con 0x56035af6f840 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f607be00640 1 -- 172.21.15.42:0/1715587967 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f608809aeb0 con 0x56035af6f840 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f607be00640 1 -- 172.21.15.42:0/1715587967 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f60880b4aa0 con 0x56035af6f840 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f60a4253740 10 client.25985 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f60a4253740 10 client.25985 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f60a4253740 20 client.25985 populate_metadata read hostname 'smithi042' 2024-04-25T00:28:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f60a4253740 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56035afeb620 0x56035afeda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f60a4253740 4 client.25985 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:28:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f60a4253740 1 -- 172.21.15.42:0/1715587967 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56035aed9340 con 0x56035afeb620 2024-04-25T00:28:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.418+0000 7f609a800640 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56035afeb620 0x56035afeda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.419+0000 7f607be00640 10 client.25985 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.419+0000 7f609a800640 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56035afeb620 0x56035afeda10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54d4965d4e9e52e server_cookie=877a8dad5feed184 in_seq=0 out_seq=0 2024-04-25T00:28:45.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.471+0000 7f607be00640 1 -- 172.21.15.42:0/1715587967 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56035aed9340 con 0x56035afeb620 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.471+0000 7f607be00640 10 client.25985 handle_command_reply: tid=0 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 client.25985 shutdown 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 -- 172.21.15.42:0/1715587967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6f840 msgr2=0x56035af7e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6f840 0x56035af7e100 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f608809b5a0 tx=0x7f608809b330 comp rx=0 tx=0).stop 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 -- 172.21.15.42:0/1715587967 shutdown_connections 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56035afeb620 0x56035afeda10 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56035af749b0 0x56035af7f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56035af6f840 0x56035af7e100 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 --2- 172.21.15.42:0/1715587967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56035af6ee50 0x56035af7dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f607be00640 0 client.25985 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 -- 172.21.15.42:0/1715587967 >> 172.21.15.42:0/1715587967 conn(0x56035af6a4c0 msgr2=0x56035af78fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:45.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 -- 172.21.15.42:0/1715587967 shutdown_connections 2024-04-25T00:28:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 1 -- 172.21.15.42:0/1715587967 wait complete. 2024-04-25T00:28:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.472+0000 7f60a4253740 20 client.25985 trim_cache size 0 max 16384 2024-04-25T00:28:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:28:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:28:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:28:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:28:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 -- 172.21.15.42:0/3558999140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f60800a2c00 msgr2=0x7f60800a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f60800a2c00 0x7f60800a50c0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f608c005ab0 tx=0x7f608c005a20 comp rx=0 tx=0).stop 2024-04-25T00:28:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 -- 172.21.15.42:0/3558999140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609c075780 msgr2=0x7f609c19e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609c075780 0x7f609c19e390 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f608800c920 tx=0x7f608800cde0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 -- 172.21.15.42:0/3558999140 shutdown_connections 2024-04-25T00:28:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f60800a2c00 0x7f60800a50c0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f609c10dc30 0x7f609c1a5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f609c075780 0x7f609c19e390 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 --2- 172.21.15.42:0/3558999140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f609c0741a0 0x7f609c19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 -- 172.21.15.42:0/3558999140 >> 172.21.15.42:0/3558999140 conn(0x7f609c0fcbd0 msgr2=0x7f609c100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 -- 172.21.15.42:0/3558999140 shutdown_connections 2024-04-25T00:28:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:45.473+0000 7f60a0a00640 1 -- 172.21.15.42:0/3558999140 wait complete. 2024-04-25T00:28:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:45 smithi106 ceph-mon[32510]: pgmap v688: 97 pgs: 97 active+clean; 118 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-25T00:28:46.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:45 smithi042 ceph-mon[28130]: pgmap v688: 97 pgs: 97 active+clean; 118 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-25T00:28:46.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:45 smithi136 ceph-mon[32923]: pgmap v688: 97 pgs: 97 active+clean; 118 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.6 MiB/s wr, 169 op/s 2024-04-25T00:28:46.264 INFO:tasks.workunit.client.0.smithi042.stdout: 1 611707 6.28 MB/sec execute 396 sec latency 48.133 ms 2024-04-25T00:28:47.265 INFO:tasks.workunit.client.0.smithi042.stdout: 1 612892 6.28 MB/sec execute 397 sec latency 24.834 ms 2024-04-25T00:28:48.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:47 smithi106 ceph-mon[32510]: pgmap v689: 97 pgs: 97 active+clean; 115 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.1 MiB/s wr, 239 op/s 2024-04-25T00:28:48.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:47 smithi042 ceph-mon[28130]: pgmap v689: 97 pgs: 97 active+clean; 115 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.1 MiB/s wr, 239 op/s 2024-04-25T00:28:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:47 smithi136 ceph-mon[32923]: pgmap v689: 97 pgs: 97 active+clean; 115 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 2.1 MiB/s wr, 239 op/s 2024-04-25T00:28:48.264 INFO:tasks.workunit.client.0.smithi042.stdout: 1 614543 6.29 MB/sec execute 398 sec latency 19.284 ms 2024-04-25T00:28:49.265 INFO:tasks.workunit.client.0.smithi042.stdout: 1 615045 6.28 MB/sec execute 399 sec latency 32.591 ms 2024-04-25T00:28:50.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:49 smithi106 ceph-mon[32510]: pgmap v690: 97 pgs: 97 active+clean; 115 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-04-25T00:28:50.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:49 smithi042 ceph-mon[28130]: pgmap v690: 97 pgs: 97 active+clean; 115 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-04-25T00:28:50.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:49 smithi136 ceph-mon[32923]: pgmap v690: 97 pgs: 97 active+clean; 115 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.6 MiB/s wr, 167 op/s 2024-04-25T00:28:50.265 INFO:tasks.workunit.client.0.smithi042.stdout: 1 616132 6.28 MB/sec execute 400 sec latency 21.273 ms 2024-04-25T00:28:51.265 INFO:tasks.workunit.client.0.smithi042.stdout: 1 617946 6.29 MB/sec execute 401 sec latency 18.816 ms 2024-04-25T00:28:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:51 smithi106 ceph-mon[32510]: pgmap v691: 97 pgs: 97 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-04-25T00:28:52.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:51 smithi042 ceph-mon[28130]: pgmap v691: 97 pgs: 97 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-04-25T00:28:52.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:51 smithi136 ceph-mon[32923]: pgmap v691: 97 pgs: 97 active+clean; 116 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 448 KiB/s rd, 1.8 MiB/s wr, 197 op/s 2024-04-25T00:28:52.265 INFO:tasks.workunit.client.0.smithi042.stdout: 1 618462 6.28 MB/sec execute 402 sec latency 19.315 ms 2024-04-25T00:28:53.265 INFO:tasks.workunit.client.0.smithi042.stdout: 1 619492 6.28 MB/sec execute 403 sec latency 47.855 ms 2024-04-25T00:28:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:53 smithi106 ceph-mon[32510]: pgmap v692: 97 pgs: 97 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-04-25T00:28:54.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:53 smithi042 ceph-mon[28130]: pgmap v692: 97 pgs: 97 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-04-25T00:28:54.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:53 smithi136 ceph-mon[32923]: pgmap v692: 97 pgs: 97 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.9 MiB/s wr, 200 op/s 2024-04-25T00:28:54.265 INFO:tasks.workunit.client.0.smithi042.stdout: 1 620993 6.28 MB/sec execute 404 sec latency 22.877 ms 2024-04-25T00:28:55.265 INFO:tasks.workunit.client.0.smithi042.stdout: 1 621915 6.28 MB/sec execute 405 sec latency 30.939 ms 2024-04-25T00:28:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:55 smithi106 ceph-mon[32510]: pgmap v693: 97 pgs: 97 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-04-25T00:28:56.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:55 smithi042 ceph-mon[28130]: pgmap v693: 97 pgs: 97 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-04-25T00:28:56.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:55 smithi136 ceph-mon[32923]: pgmap v693: 97 pgs: 97 active+clean; 117 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 369 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-04-25T00:28:56.265 INFO:tasks.workunit.client.0.smithi042.stdout: 1 622772 6.27 MB/sec execute 406 sec latency 22.884 ms 2024-04-25T00:28:57.266 INFO:tasks.workunit.client.0.smithi042.stdout: 1 624466 6.28 MB/sec execute 407 sec latency 21.039 ms 2024-04-25T00:28:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:57 smithi106 ceph-mon[32510]: pgmap v694: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-04-25T00:28:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:57 smithi042 ceph-mon[28130]: pgmap v694: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-04-25T00:28:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:57 smithi136 ceph-mon[32923]: pgmap v694: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 1.9 MiB/s wr, 225 op/s 2024-04-25T00:28:58.266 INFO:tasks.workunit.client.0.smithi042.stdout: 1 625506 6.28 MB/sec execute 408 sec latency 16.637 ms 2024-04-25T00:28:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.636+0000 7f6b73600640 1 -- 172.21.15.42:0/2909564672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b6c075780 msgr2=0x7f6b6c073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.636+0000 7f6b73600640 1 --2- 172.21.15.42:0/2909564672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b6c075780 0x7f6b6c073bd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009ca0 tx=0x7f6b6402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.636+0000 7f6b73600640 1 -- 172.21.15.42:0/2909564672 shutdown_connections 2024-04-25T00:28:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.636+0000 7f6b73600640 1 --2- 172.21.15.42:0/2909564672 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b6c109660 0x7f6b6c10baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.636+0000 7f6b73600640 1 --2- 172.21.15.42:0/2909564672 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b6c0741a0 0x7f6b6c074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.636+0000 7f6b73600640 1 --2- 172.21.15.42:0/2909564672 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b6c075780 0x7f6b6c073bd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.636+0000 7f6b73600640 1 -- 172.21.15.42:0/2909564672 >> 172.21.15.42:0/2909564672 conn(0x7f6b6c0fcd30 msgr2=0x7f6b6c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.636+0000 7f6b73600640 1 -- 172.21.15.42:0/2909564672 shutdown_connections 2024-04-25T00:28:58.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.637+0000 7f6b73600640 1 -- 172.21.15.42:0/2909564672 wait complete. 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.638+0000 7f6b73600640 1 Processor -- start 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.638+0000 7f6b73600640 1 -- start start 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.638+0000 7f6b73600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b6c0741a0 0x7f6b6c199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.638+0000 7f6b73600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b6c075780 0x7f6b6c19a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.638+0000 7f6b73600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b6c109660 0x7f6b6c1a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.638+0000 7f6b73600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10f630 con 0x7f6b6c075780 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.638+0000 7f6b73600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10f4b0 con 0x7f6b6c109660 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.638+0000 7f6b73600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c10f7b0 con 0x7f6b6c0741a0 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b6c0741a0 0x7f6b6c199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b71600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b6c109660 0x7f6b6c1a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b6c0741a0 0x7f6b6c199bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39390/0 (socket says 172.21.15.42:39390) 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b70c00640 1 -- 172.21.15.42:0/1981344714 learned_addr learned my addr 172.21.15.42:0/1981344714 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:58.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b6c075780 0x7f6b6c19a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b70c00640 1 -- 172.21.15.42:0/1981344714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b6c109660 msgr2=0x7f6b6c1a1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b70c00640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b6c109660 0x7f6b6c1a1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b70c00640 1 -- 172.21.15.42:0/1981344714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b6c075780 msgr2=0x7f6b6c19a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b70c00640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b6c075780 0x7f6b6c19a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b70c00640 1 -- 172.21.15.42:0/1981344714 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6c1a17d0 con 0x7f6b6c0741a0 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.639+0000 7f6b71600640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b6c109660 0x7f6b6c1a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.640+0000 7f6b70c00640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b6c0741a0 0x7f6b6c199bb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6b64000f80 tx=0x7f6b640319f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.640+0000 7f6b69600640 1 -- 172.21.15.42:0/1981344714 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64003bf0 con 0x7f6b6c0741a0 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.640+0000 7f6b69600640 1 -- 172.21.15.42:0/1981344714 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b64003d90 con 0x7f6b6c0741a0 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.640+0000 7f6b69600640 1 -- 172.21.15.42:0/1981344714 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6402fce0 con 0x7f6b6c0741a0 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.640+0000 7f6b73600640 1 -- 172.21.15.42:0/1981344714 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b6c1a1a60 con 0x7f6b6c0741a0 2024-04-25T00:28:58.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.640+0000 7f6b73600640 1 -- 172.21.15.42:0/1981344714 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b6c1a1f60 con 0x7f6b6c0741a0 2024-04-25T00:28:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.644+0000 7f6b69600640 1 -- 172.21.15.42:0/1981344714 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b64004040 con 0x7f6b6c0741a0 2024-04-25T00:28:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.644+0000 7f6b76379740 1 Processor -- start 2024-04-25T00:28:58.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b76379740 1 -- start start 2024-04-25T00:28:58.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b76379740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d5782910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b76379740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d578296190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b76379740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 0x55d578298c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b76379740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d5781f3ab0 con 0x55d578296800 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b76379740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d57819c640 con 0x55d578290ca0 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b76379740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d5781efa70 con 0x55d578291690 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b69600640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b500a2cb0 0x7f6b500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b71600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d5782910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d578296190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b71600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d5782910c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38218/0 (socket says 172.21.15.42:38218) 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b6be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 0x55d578298c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.645+0000 7f6b71600640 1 -- 172.21.15.42:0/1165686710 learned_addr learned my addr 172.21.15.42:0/1165686710 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:58.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b69600640 1 -- 172.21.15.42:0/1981344714 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6b640f0660 con 0x7f6b6c0741a0 2024-04-25T00:28:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b71600640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b500a2cb0 0x7f6b500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b70c00640 1 -- 172.21.15.42:0/1165686710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 msgr2=0x55d5782910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b70c00640 1 --2- 172.21.15.42:0/1165686710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d5782910c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b70c00640 1 -- 172.21.15.42:0/1165686710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 msgr2=0x55d578298c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b70c00640 1 --2- 172.21.15.42:0/1165686710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 0x55d578298c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b70c00640 1 -- 172.21.15.42:0/1165686710 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d577f08210 con 0x55d578291690 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b70c00640 1 --2- 172.21.15.42:0/1165686710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d578296190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6b640bcc20 tx=0x7f6b6404e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5256326ae8b8aefc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b61c00640 1 -- 172.21.15.42:0/1165686710 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b640bca60 con 0x55d578291690 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b61c00640 1 -- 172.21.15.42:0/1165686710 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b64036020 con 0x55d578291690 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.646+0000 7f6b61c00640 1 -- 172.21.15.42:0/1165686710 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b640bc880 con 0x55d578291690 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.647+0000 7f6b76379740 1 -- 172.21.15.42:0/1165686710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 msgr2=0x55d578296190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.647+0000 7f6b76379740 1 --2- 172.21.15.42:0/1165686710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d578296190 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6b640bcc20 tx=0x7f6b6404e000 comp rx=0 tx=0).stop 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.647+0000 7f6b71600640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b500a2cb0 0x7f6b500a5170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55d578291ef0 tx=0x7f6b5c004730 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.647+0000 7f6b76379740 1 -- 172.21.15.42:0/1165686710 shutdown_connections 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.647+0000 7f6b76379740 1 --2- 172.21.15.42:0/1165686710 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 0x55d578298c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.647+0000 7f6b76379740 1 --2- 172.21.15.42:0/1165686710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d578296190 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.647+0000 7f6b76379740 1 --2- 172.21.15.42:0/1165686710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d5782910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.647+0000 7f6b76379740 1 -- 172.21.15.42:0/1165686710 >> 172.21.15.42:0/1165686710 conn(0x55d57828c310 msgr2=0x55d57828c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.647+0000 7f6b76379740 1 -- 172.21.15.42:0/1165686710 shutdown_connections 2024-04-25T00:28:58.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.647+0000 7f6b76379740 1 -- 172.21.15.42:0/1165686710 wait complete. 2024-04-25T00:28:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.648+0000 7f6b76379740 1 -- start start 2024-04-25T00:28:58.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.648+0000 7f6b76379740 1 Processor -- start 2024-04-25T00:28:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b76379740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b76379740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d57829afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b76379740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 0x55d5782a1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b76379740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d5781f3ab0 con 0x55d578296800 2024-04-25T00:28:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b76379740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d5781af990 con 0x55d578290ca0 2024-04-25T00:28:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b76379740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d5781efa70 con 0x55d578291690 2024-04-25T00:28:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b76379740 10 client.0 _ll_register_callbacks cb 0x55d5781b1950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b70c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38220/0 (socket says 172.21.15.42:38220) 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b70c00640 1 -- 172.21.15.42:0/2765036311 learned_addr learned my addr 172.21.15.42:0/2765036311 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b71600640 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 0x55d5782a1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b6be00640 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d57829afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b70c00640 1 -- 172.21.15.42:0/2765036311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 msgr2=0x55d57829afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b70c00640 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d57829afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b70c00640 1 -- 172.21.15.42:0/2765036311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 msgr2=0x55d5782a1fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b70c00640 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 0x55d5782a1fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b70c00640 1 -- 172.21.15.42:0/2765036311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d577fd4620 con 0x55d578290ca0 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.649+0000 7f6b71600640 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 0x55d5782a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.650+0000 7f6b70c00640 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829aa60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b640b98f0 tx=0x7f6b64031bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ab21ba7fba4ac40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.650+0000 7f6b5a000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.650+0000 7f6b5a000640 1 -- 172.21.15.42:0/2765036311 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b640bca60 con 0x55d578290ca0 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.650+0000 7f6b5a000640 1 -- 172.21.15.42:0/2765036311 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6b640bd180 con 0x55d578290ca0 2024-04-25T00:28:58.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.650+0000 7f6b76379740 1 -- 172.21.15.42:0/2765036311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b64036020 con 0x55d578290ca0 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.650+0000 7f6b5a000640 1 -- 172.21.15.42:0/2765036311 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6b640bcf20 con 0x55d578290ca0 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.650+0000 7f6b76379740 10 client.25997 fetch_fsmap learned FSMap version 31 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.650+0000 7f6b76379740 1 -- 172.21.15.42:0/2765036311 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d5782a4fc0 con 0x55d578290ca0 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.650+0000 7f6b5a000640 1 -- 172.21.15.42:0/2765036311 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b64036020 con 0x55d578290ca0 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.651+0000 7f6b5a000640 1 -- 172.21.15.42:0/2765036311 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f6b640df020 con 0x55d578290ca0 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.651+0000 7f6b76379740 10 client.25997 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.651+0000 7f6b76379740 10 client.25997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.651+0000 7f6b76379740 20 client.25997 populate_metadata read hostname 'smithi042' 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.651+0000 7f6b76379740 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d5782a6590 0x55d5782a8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.651+0000 7f6b76379740 4 client.25997 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.651+0000 7f6b76379740 1 -- 172.21.15.42:0/2765036311 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d578200ff0 con 0x55d5782a6590 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.651+0000 7f6b6be00640 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d5782a6590 0x55d5782a8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.652+0000 7f6b6be00640 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d5782a6590 0x55d5782a8980 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a91148d4bac0a0 server_cookie=8d9e83060ce04e8b in_seq=0 out_seq=0 2024-04-25T00:28:58.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.652+0000 7f6b5a000640 10 client.25997 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.652+0000 7f6b5a000640 1 -- 172.21.15.42:0/2765036311 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d578200ff0 con 0x55d5782a6590 2024-04-25T00:28:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.652+0000 7f6b5a000640 10 client.25997 handle_command_reply: tid=0 2024-04-25T00:28:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.653+0000 7f6b76379740 1 client.25997 shutdown 2024-04-25T00:28:58.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.653+0000 7f6b76379740 1 -- 172.21.15.42:0/2765036311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 msgr2=0x55d57829aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.653+0000 7f6b76379740 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829aa60 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6b640b98f0 tx=0x7f6b64031bd0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.653+0000 7f6b76379740 1 -- 172.21.15.42:0/2765036311 shutdown_connections 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.653+0000 7f6b76379740 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d5782a6590 0x55d5782a8980 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.653+0000 7f6b76379740 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d578296800 0x55d5782a1fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.653+0000 7f6b76379740 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d57829afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.653+0000 7f6b76379740 1 --2- 172.21.15.42:0/2765036311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829aa60 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.653+0000 7f6b5a000640 0 client.25997 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.653+0000 7f6b76379740 1 -- 172.21.15.42:0/2765036311 >> 172.21.15.42:0/2765036311 conn(0x55d57828c310 msgr2=0x55d57828b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.654+0000 7f6b76379740 1 -- 172.21.15.42:0/2765036311 shutdown_connections 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.654+0000 7f6b76379740 1 -- 172.21.15.42:0/2765036311 wait complete. 2024-04-25T00:28:58.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.654+0000 7f6b76379740 20 client.25997 trim_cache size 0 max 16384 2024-04-25T00:28:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.676+0000 7f6b76379740 1 Processor -- start 2024-04-25T00:28:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b76379740 1 -- start start 2024-04-25T00:28:58.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b76379740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 0x55d57828f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b76379740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578290ca0 0x55d57828f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b76379740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578291690 0x55d57829a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b76379740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c1ad5e0 con 0x55d57828c310 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b76379740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c1aaeb0 con 0x55d578291690 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b76379740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c1a8d00 con 0x55d578290ca0 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b6be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578290ca0 0x55d57828f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b6be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578290ca0 0x55d57828f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39422/0 (socket says 172.21.15.42:39422) 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b6be00640 1 -- 172.21.15.42:0/3359600608 learned_addr learned my addr 172.21.15.42:0/3359600608 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b71600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578291690 0x55d57829a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.677+0000 7f6b70c00640 1 --2- 172.21.15.42:0/3359600608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 0x55d57828f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.678+0000 7f6b6be00640 1 -- 172.21.15.42:0/3359600608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578291690 msgr2=0x55d57829a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.678+0000 7f6b6be00640 1 --2- 172.21.15.42:0/3359600608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578291690 0x55d57829a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.678+0000 7f6b6be00640 1 -- 172.21.15.42:0/3359600608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 msgr2=0x55d57828f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:58.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.678+0000 7f6b6be00640 1 --2- 172.21.15.42:0/3359600608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 0x55d57828f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.678+0000 7f6b6be00640 1 -- 172.21.15.42:0/3359600608 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5782a8cf0 con 0x55d578290ca0 2024-04-25T00:28:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.678+0000 7f6b70c00640 1 --2- 172.21.15.42:0/3359600608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 0x55d57828f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:28:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.678+0000 7f6b6be00640 1 --2- 172.21.15.42:0/3359600608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578290ca0 0x55d57828f760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6b54005d00 tx=0x7f6b5400b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a56194b2bd54efe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.678+0000 7f6b61200640 1 -- 172.21.15.42:0/3359600608 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54004220 con 0x55d578290ca0 2024-04-25T00:28:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.678+0000 7f6b61200640 1 -- 172.21.15.42:0/3359600608 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b54024030 con 0x55d578290ca0 2024-04-25T00:28:58.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.678+0000 7f6b61200640 1 -- 172.21.15.42:0/3359600608 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54003b50 con 0x55d578290ca0 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.679+0000 7f6b76379740 1 -- 172.21.15.42:0/3359600608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578290ca0 msgr2=0x55d57828f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.679+0000 7f6b76379740 1 --2- 172.21.15.42:0/3359600608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578290ca0 0x55d57828f760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6b54005d00 tx=0x7f6b5400b410 comp rx=0 tx=0).stop 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.679+0000 7f6b76379740 1 -- 172.21.15.42:0/3359600608 shutdown_connections 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.679+0000 7f6b76379740 1 --2- 172.21.15.42:0/3359600608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578291690 0x55d57829a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.679+0000 7f6b76379740 1 --2- 172.21.15.42:0/3359600608 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578290ca0 0x55d57828f760 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.679+0000 7f6b76379740 1 --2- 172.21.15.42:0/3359600608 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 0x55d57828f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.679+0000 7f6b76379740 1 -- 172.21.15.42:0/3359600608 >> 172.21.15.42:0/3359600608 conn(0x55d578296340 msgr2=0x55d578286fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.679+0000 7f6b76379740 1 -- 172.21.15.42:0/3359600608 shutdown_connections 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.679+0000 7f6b76379740 1 -- 172.21.15.42:0/3359600608 wait complete. 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.680+0000 7f6b76379740 1 -- start start 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.680+0000 7f6b76379740 1 Processor -- start 2024-04-25T00:28:58.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.680+0000 7f6b76379740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 0x55d57829f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.680+0000 7f6b76379740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.680+0000 7f6b76379740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d5782a0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.680+0000 7f6b76379740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c1a9370 con 0x55d57828c310 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.680+0000 7f6b76379740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c1aa7e0 con 0x55d578290ca0 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.680+0000 7f6b76379740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6c1a9a40 con 0x55d578291690 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.680+0000 7f6b76379740 10 client.0 _ll_register_callbacks cb 0x7f6b5400b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b6be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b6be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:38246/0 (socket says 172.21.15.42:38246) 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b6be00640 1 -- 172.21.15.42:0/2836477130 learned_addr learned my addr 172.21.15.42:0/2836477130 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b70c00640 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 0x55d57829f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b71600640 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d5782a0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b6be00640 1 -- 172.21.15.42:0/2836477130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 msgr2=0x55d5782a0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b6be00640 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d5782a0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b6be00640 1 -- 172.21.15.42:0/2836477130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 msgr2=0x55d57829f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:28:58.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b6be00640 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 0x55d57829f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b6be00640 1 -- 172.21.15.42:0/2836477130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5782a0d80 con 0x55d578290ca0 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b6be00640 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829f7d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b54023040 tx=0x7f6b5400bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=232dbe7d93b3f029 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.681+0000 7f6b5a000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b5a000640 1 -- 172.21.15.42:0/2836477130 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b54004320 con 0x55d578290ca0 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b5a000640 1 -- 172.21.15.42:0/2836477130 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f6b540038d0 con 0x55d578290ca0 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b76379740 1 -- 172.21.15.42:0/2836477130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b54024030 con 0x55d578290ca0 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b5a000640 1 -- 172.21.15.42:0/2836477130 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6b54003ad0 con 0x55d578290ca0 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b76379740 10 client.26003 fetch_fsmap learned FSMap version 31 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b76379740 1 -- 172.21.15.42:0/2836477130 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d57830d130 con 0x55d578290ca0 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b5a000640 1 -- 172.21.15.42:0/2836477130 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b54024030 con 0x55d578290ca0 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b5a000640 1 -- 172.21.15.42:0/2836477130 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f6b54020490 con 0x55d578290ca0 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b76379740 10 client.26003 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b76379740 10 client.26003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b76379740 20 client.26003 populate_metadata read hostname 'smithi042' 2024-04-25T00:28:58.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b76379740 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d578296800 0x55d57830fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:28:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b76379740 4 client.26003 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:28:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.682+0000 7f6b76379740 1 -- 172.21.15.42:0/2836477130 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5781fb340 con 0x55d578296800 2024-04-25T00:28:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.683+0000 7f6b70c00640 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d578296800 0x55d57830fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:28:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.683+0000 7f6b70c00640 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d578296800 0x55d57830fae0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a107af597b0e081 server_cookie=f2340f2aa6d2ec5c in_seq=0 out_seq=0 2024-04-25T00:28:58.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.683+0000 7f6b5a000640 10 client.26003 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.735+0000 7f6b5a000640 1 -- 172.21.15.42:0/2836477130 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d5781fb340 con 0x55d578296800 2024-04-25T00:28:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.735+0000 7f6b5a000640 10 client.26003 handle_command_reply: tid=0 2024-04-25T00:28:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.735+0000 7f6b76379740 1 client.26003 shutdown 2024-04-25T00:28:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b76379740 1 -- 172.21.15.42:0/2836477130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 msgr2=0x55d57829f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:58.736 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b76379740 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829f7d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6b54023040 tx=0x7f6b5400bc70 comp rx=0 tx=0).stop 2024-04-25T00:28:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b76379740 1 -- 172.21.15.42:0/2836477130 shutdown_connections 2024-04-25T00:28:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b76379740 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d578296800 0x55d57830fae0 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b76379740 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d578291690 0x55d5782a0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b76379740 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d578290ca0 0x55d57829f7d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b76379740 1 --2- 172.21.15.42:0/2836477130 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d57828c310 0x55d57829f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b76379740 1 -- 172.21.15.42:0/2836477130 >> 172.21.15.42:0/2836477130 conn(0x55d578296340 msgr2=0x55d57828f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b5a000640 0 client.26003 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:28:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b76379740 1 -- 172.21.15.42:0/2836477130 shutdown_connections 2024-04-25T00:28:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.736+0000 7f6b76379740 1 -- 172.21.15.42:0/2836477130 wait complete. 2024-04-25T00:28:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b76379740 20 client.26003 trim_cache size 0 max 16384 2024-04-25T00:28:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:28:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:28:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:28:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:28:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b73600640 1 -- 172.21.15.42:0/1981344714 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b500a2cb0 msgr2=0x7f6b500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b73600640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b500a2cb0 0x7f6b500a5170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x55d578291ef0 tx=0x7f6b5c004730 comp rx=0 tx=0).stop 2024-04-25T00:28:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b73600640 1 -- 172.21.15.42:0/1981344714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b6c0741a0 msgr2=0x7f6b6c199bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:28:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b73600640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b6c0741a0 0x7f6b6c199bb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6b64000f80 tx=0x7f6b640319f0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b73600640 1 -- 172.21.15.42:0/1981344714 shutdown_connections 2024-04-25T00:28:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b73600640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f6b500a2cb0 0x7f6b500a5170 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b73600640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b6c109660 0x7f6b6c1a1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b73600640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6b6c075780 0x7f6b6c19a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b73600640 1 --2- 172.21.15.42:0/1981344714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b6c0741a0 0x7f6b6c199bb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:28:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.737+0000 7f6b73600640 1 -- 172.21.15.42:0/1981344714 >> 172.21.15.42:0/1981344714 conn(0x7f6b6c0fcd30 msgr2=0x7f6b6c105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:28:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.738+0000 7f6b73600640 1 -- 172.21.15.42:0/1981344714 shutdown_connections 2024-04-25T00:28:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:28:58.738+0000 7f6b73600640 1 -- 172.21.15.42:0/1981344714 wait complete. 2024-04-25T00:28:59.266 INFO:tasks.workunit.client.0.smithi042.stdout: 1 626599 6.28 MB/sec execute 409 sec latency 54.123 ms 2024-04-25T00:29:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:28:59 smithi106 ceph-mon[32510]: pgmap v695: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.5 MiB/s wr, 160 op/s 2024-04-25T00:29:00.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:28:59 smithi042 ceph-mon[28130]: pgmap v695: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.5 MiB/s wr, 160 op/s 2024-04-25T00:29:00.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:28:59 smithi136 ceph-mon[32923]: pgmap v695: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.5 MiB/s wr, 160 op/s 2024-04-25T00:29:00.266 INFO:tasks.workunit.client.0.smithi042.stdout: 1 628433 6.29 MB/sec execute 410 sec latency 49.450 ms 2024-04-25T00:29:01.266 INFO:tasks.workunit.client.0.smithi042.stdout: 1 629100 6.28 MB/sec execute 411 sec latency 19.638 ms 2024-04-25T00:29:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:01 smithi106 ceph-mon[32510]: pgmap v696: 97 pgs: 97 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-25T00:29:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:01 smithi042 ceph-mon[28130]: pgmap v696: 97 pgs: 97 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-25T00:29:02.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:01 smithi136 ceph-mon[32923]: pgmap v696: 97 pgs: 97 active+clean; 116 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 428 KiB/s rd, 1.6 MiB/s wr, 196 op/s 2024-04-25T00:29:02.266 INFO:tasks.workunit.client.0.smithi042.stdout: 1 630176 6.28 MB/sec execute 412 sec latency 28.873 ms 2024-04-25T00:29:03.267 INFO:tasks.workunit.client.0.smithi042.stdout: 1 632146 6.30 MB/sec execute 413 sec latency 18.788 ms 2024-04-25T00:29:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:03 smithi106 ceph-mon[32510]: pgmap v697: 97 pgs: 97 active+clean; 117 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-04-25T00:29:04.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:03 smithi042 ceph-mon[28130]: pgmap v697: 97 pgs: 97 active+clean; 117 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-04-25T00:29:04.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:03 smithi136 ceph-mon[32923]: pgmap v697: 97 pgs: 97 active+clean; 117 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-04-25T00:29:04.267 INFO:tasks.workunit.client.0.smithi042.stdout: 1 632706 6.29 MB/sec execute 414 sec latency 34.657 ms 2024-04-25T00:29:05.267 INFO:tasks.workunit.client.0.smithi042.stdout: 1 633592 6.28 MB/sec execute 415 sec latency 149.491 ms 2024-04-25T00:29:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:05 smithi106 ceph-mon[32510]: pgmap v698: 97 pgs: 97 active+clean; 117 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:29:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:05 smithi042 ceph-mon[28130]: pgmap v698: 97 pgs: 97 active+clean; 117 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:29:06.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:05 smithi136 ceph-mon[32923]: pgmap v698: 97 pgs: 97 active+clean; 117 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:29:06.267 INFO:tasks.workunit.client.0.smithi042.stdout: 1 635182 6.29 MB/sec execute 416 sec latency 21.706 ms 2024-04-25T00:29:07.267 INFO:tasks.workunit.client.0.smithi042.stdout: 1 636149 6.29 MB/sec execute 417 sec latency 18.403 ms 2024-04-25T00:29:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:07 smithi106 ceph-mon[32510]: pgmap v699: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.0 MiB/s wr, 230 op/s 2024-04-25T00:29:08.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:07 smithi042 ceph-mon[28130]: pgmap v699: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.0 MiB/s wr, 230 op/s 2024-04-25T00:29:08.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:07 smithi136 ceph-mon[32923]: pgmap v699: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 525 KiB/s rd, 2.0 MiB/s wr, 230 op/s 2024-04-25T00:29:08.267 INFO:tasks.workunit.client.0.smithi042.stdout: 1 637181 6.28 MB/sec execute 418 sec latency 20.203 ms 2024-04-25T00:29:09.267 INFO:tasks.workunit.client.0.smithi042.stdout: 1 639047 6.29 MB/sec execute 419 sec latency 47.179 ms 2024-04-25T00:29:10.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:09 smithi042 ceph-mon[28130]: pgmap v700: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-25T00:29:10.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:09 smithi136 ceph-mon[32923]: pgmap v700: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-25T00:29:10.267 INFO:tasks.workunit.client.0.smithi042.stdout: 1 639704 6.29 MB/sec execute 420 sec latency 22.788 ms 2024-04-25T00:29:10.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:09 smithi106 ceph-mon[32510]: pgmap v700: 97 pgs: 97 active+clean; 119 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-25T00:29:11.267 INFO:tasks.workunit.client.0.smithi042.stdout: 1 640744 6.28 MB/sec execute 421 sec latency 20.892 ms 2024-04-25T00:29:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.883+0000 7f8c36400640 1 -- 172.21.15.42:0/3475737726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c30075720 msgr2=0x7f8c30073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.883+0000 7f8c36400640 1 --2- 172.21.15.42:0/3475737726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c30075720 0x7f8c30073b70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f8c20009900 tx=0x7f8c2002a490 comp rx=0 tx=0).stop 2024-04-25T00:29:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.883+0000 7f8c36400640 1 -- 172.21.15.42:0/3475737726 shutdown_connections 2024-04-25T00:29:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.883+0000 7f8c36400640 1 --2- 172.21.15.42:0/3475737726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c30109630 0x7f8c3010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.883+0000 7f8c36400640 1 --2- 172.21.15.42:0/3475737726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c30074140 0x7f8c300745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.883+0000 7f8c36400640 1 --2- 172.21.15.42:0/3475737726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c30075720 0x7f8c30073b70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.883+0000 7f8c36400640 1 -- 172.21.15.42:0/3475737726 >> 172.21.15.42:0/3475737726 conn(0x7f8c300fcd10 msgr2=0x7f8c300ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.883+0000 7f8c36400640 1 -- 172.21.15.42:0/3475737726 shutdown_connections 2024-04-25T00:29:11.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.884+0000 7f8c36400640 1 -- 172.21.15.42:0/3475737726 wait complete. 2024-04-25T00:29:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c36400640 1 Processor -- start 2024-04-25T00:29:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c36400640 1 -- start start 2024-04-25T00:29:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c36400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c30074140 0x7f8c30199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c36400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c30075720 0x7f8c3019a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c36400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c30109630 0x7f8c301a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c36400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3010f640 con 0x7f8c30109630 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c36400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3010f4c0 con 0x7f8c30074140 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c36400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3010f7c0 con 0x7f8c30075720 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c2f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c30074140 0x7f8c30199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c2f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c30074140 0x7f8c30199be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39194/0 (socket says 172.21.15.42:39194) 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c30109630 0x7f8c301a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c2f400640 1 -- 172.21.15.42:0/17187549 learned_addr learned my addr 172.21.15.42:0/17187549 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.885+0000 7f8c26a00640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c30075720 0x7f8c3019a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.886+0000 7f8c2f400640 1 -- 172.21.15.42:0/17187549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c30075720 msgr2=0x7f8c3019a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.886+0000 7f8c2f400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c30075720 0x7f8c3019a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.886+0000 7f8c2f400640 1 -- 172.21.15.42:0/17187549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c30109630 msgr2=0x7f8c301a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.886+0000 7f8c2f400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c30109630 0x7f8c301a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.886+0000 7f8c2f400640 1 -- 172.21.15.42:0/17187549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c301a1800 con 0x7f8c30074140 2024-04-25T00:29:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.886+0000 7f8c2f400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c30074140 0x7f8c30199be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c20031440 tx=0x7f8c20032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.886+0000 7f8c2cc00640 1 -- 172.21.15.42:0/17187549 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2003f030 con 0x7f8c30074140 2024-04-25T00:29:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.887+0000 7f8c2cc00640 1 -- 172.21.15.42:0/17187549 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c20003ce0 con 0x7f8c30074140 2024-04-25T00:29:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.887+0000 7f8c36400640 1 -- 172.21.15.42:0/17187549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c301a1a90 con 0x7f8c30074140 2024-04-25T00:29:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.887+0000 7f8c2cc00640 1 -- 172.21.15.42:0/17187549 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c20035070 con 0x7f8c30074140 2024-04-25T00:29:11.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.887+0000 7f8c36400640 1 -- 172.21.15.42:0/17187549 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c301a1f80 con 0x7f8c30074140 2024-04-25T00:29:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.890+0000 7f8c391e0740 1 Processor -- start 2024-04-25T00:29:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.890+0000 7f8c2cc00640 1 -- 172.21.15.42:0/17187549 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c20042030 con 0x7f8c30074140 2024-04-25T00:29:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.890+0000 7f8c391e0740 1 -- start start 2024-04-25T00:29:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34acca0 0x55f3d34ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 0x55f3d34b2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34b2800 0x55f3d34b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c391e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f3d340fab0 con 0x55f3d34b2800 2024-04-25T00:29:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c391e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f3d33b8640 con 0x55f3d34ad690 2024-04-25T00:29:11.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c391e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f3d340ba70 con 0x55f3d34acca0 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c2fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34acca0 0x55f3d34ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c2fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34acca0 0x55f3d34ad0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:36678/0 (socket says 172.21.15.42:36678) 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c2f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 0x55f3d34b2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c2fe00640 1 -- 172.21.15.42:0/46757001 learned_addr learned my addr 172.21.15.42:0/46757001 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c26a00640 1 --2- 172.21.15.42:0/46757001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34b2800 0x55f3d34b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c2fe00640 1 -- 172.21.15.42:0/46757001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 msgr2=0x55f3d34b2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/46757001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 0x55f3d34b2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c2fe00640 1 -- 172.21.15.42:0/46757001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34b2800 msgr2=0x55f3d34b4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/46757001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34b2800 0x55f3d34b4c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c2fe00640 1 -- 172.21.15.42:0/46757001 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d3124210 con 0x55f3d34acca0 2024-04-25T00:29:11.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.891+0000 7f8c26a00640 1 --2- 172.21.15.42:0/46757001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34b2800 0x55f3d34b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.892+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/46757001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34acca0 0x55f3d34ad0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009fd0 tx=0x7f8c280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73f39841ab55469d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.892+0000 7f8c24c00640 1 -- 172.21.15.42:0/46757001 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28004d20 con 0x55f3d34acca0 2024-04-25T00:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.892+0000 7f8c2cc00640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8c100a2c30 0x7f8c100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.892+0000 7f8c24c00640 1 -- 172.21.15.42:0/46757001 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c28004050 con 0x55f3d34acca0 2024-04-25T00:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.892+0000 7f8c24c00640 1 -- 172.21.15.42:0/46757001 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c28005260 con 0x55f3d34acca0 2024-04-25T00:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.892+0000 7f8c2cc00640 1 -- 172.21.15.42:0/17187549 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f8c200ef5d0 con 0x7f8c30074140 2024-04-25T00:29:11.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.892+0000 7f8c2f400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8c100a2c30 0x7f8c100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.893+0000 7f8c391e0740 1 -- 172.21.15.42:0/46757001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34acca0 msgr2=0x55f3d34ad0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.893+0000 7f8c391e0740 1 --2- 172.21.15.42:0/46757001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34acca0 0x55f3d34ad0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009fd0 tx=0x7f8c280118d0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.893+0000 7f8c391e0740 1 -- 172.21.15.42:0/46757001 shutdown_connections 2024-04-25T00:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.893+0000 7f8c391e0740 1 --2- 172.21.15.42:0/46757001 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34b2800 0x55f3d34b4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.893+0000 7f8c391e0740 1 --2- 172.21.15.42:0/46757001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 0x55f3d34b2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.893+0000 7f8c391e0740 1 --2- 172.21.15.42:0/46757001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34acca0 0x55f3d34ad0c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.893+0000 7f8c391e0740 1 -- 172.21.15.42:0/46757001 >> 172.21.15.42:0/46757001 conn(0x55f3d34a8310 msgr2=0x55f3d34a8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.893+0000 7f8c2f400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8c100a2c30 0x7f8c100a50f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c200b8310 tx=0x7f8c200f2f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.893+0000 7f8c391e0740 1 -- 172.21.15.42:0/46757001 shutdown_connections 2024-04-25T00:29:11.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.893+0000 7f8c391e0740 1 -- 172.21.15.42:0/46757001 wait complete. 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.894+0000 7f8c391e0740 1 -- start start 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.894+0000 7f8c391e0740 1 Processor -- start 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.894+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34acca0 0x55f3d34b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.894+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34ad690 0x55f3d34b7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34b2800 0x55f3d34be490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c391e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f3d34b57b0 con 0x55f3d34ad690 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c391e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f3d33cb990 con 0x55f3d34acca0 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c391e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f3d34b5930 con 0x55f3d34b2800 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c391e0740 10 client.0 _ll_register_callbacks cb 0x55f3d33cd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c2fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34acca0 0x55f3d34b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c2fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34acca0 0x55f3d34b6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39198/0 (socket says 172.21.15.42:39198) 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c2fe00640 1 -- 172.21.15.42:0/2731269695 learned_addr learned my addr 172.21.15.42:0/2731269695 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:11.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34b2800 0x55f3d34be490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c26a00640 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34ad690 0x55f3d34b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c2fe00640 1 -- 172.21.15.42:0/2731269695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34b2800 msgr2=0x55f3d34be490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34b2800 0x55f3d34be490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c2fe00640 1 -- 172.21.15.42:0/2731269695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34ad690 msgr2=0x55f3d34b7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34ad690 0x55f3d34b7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.895+0000 7f8c2fe00640 1 -- 172.21.15.42:0/2731269695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d31f0620 con 0x55f3d34acca0 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.896+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34acca0 0x55f3d34b6f10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009de0 tx=0x7f8c28000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15a80c3340d4468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.896+0000 7f8c1d600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.896+0000 7f8c1d600640 1 -- 172.21.15.42:0/2731269695 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c2800f4e0 con 0x55f3d34acca0 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.896+0000 7f8c391e0740 1 -- 172.21.15.42:0/2731269695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c28004050 con 0x55f3d34acca0 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.896+0000 7f8c1d600640 1 -- 172.21.15.42:0/2731269695 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f8c2800f680 con 0x55f3d34acca0 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.897+0000 7f8c1d600640 1 -- 172.21.15.42:0/2731269695 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8c2800f880 con 0x55f3d34acca0 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.897+0000 7f8c391e0740 10 client.26015 fetch_fsmap learned FSMap version 31 2024-04-25T00:29:11.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.897+0000 7f8c391e0740 1 -- 172.21.15.42:0/2731269695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f3d34c14c0 con 0x55f3d34acca0 2024-04-25T00:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.897+0000 7f8c1d600640 1 -- 172.21.15.42:0/2731269695 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c28004050 con 0x55f3d34acca0 2024-04-25T00:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.897+0000 7f8c1d600640 1 -- 172.21.15.42:0/2731269695 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f8c28025040 con 0x55f3d34acca0 2024-04-25T00:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.897+0000 7f8c391e0740 10 client.26015 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.897+0000 7f8c391e0740 10 client.26015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.897+0000 7f8c391e0740 20 client.26015 populate_metadata read hostname 'smithi042' 2024-04-25T00:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.897+0000 7f8c391e0740 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3d34c2930 0x55f3d34c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.898+0000 7f8c391e0740 4 client.26015 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.898+0000 7f8c391e0740 1 -- 172.21.15.42:0/2731269695 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3d341cff0 con 0x55f3d34c2930 2024-04-25T00:29:11.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.898+0000 7f8c26a00640 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3d34c2930 0x55f3d34c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.898+0000 7f8c26a00640 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3d34c2930 0x55f3d34c4d20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f535e699769985e server_cookie=9aa7b88f07eef084 in_seq=0 out_seq=0 2024-04-25T00:29:11.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.898+0000 7f8c1d600640 10 client.26015 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.899+0000 7f8c1d600640 1 -- 172.21.15.42:0/2731269695 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3d341cff0 con 0x55f3d34c2930 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.899+0000 7f8c1d600640 10 client.26015 handle_command_reply: tid=0 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.899+0000 7f8c391e0740 1 client.26015 shutdown 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 1 -- 172.21.15.42:0/2731269695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34acca0 msgr2=0x55f3d34b6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34acca0 0x55f3d34b6f10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8c28009de0 tx=0x7f8c28000f40 comp rx=0 tx=0).stop 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 1 -- 172.21.15.42:0/2731269695 shutdown_connections 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3d34c2930 0x55f3d34c4d20 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34b2800 0x55f3d34be490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34ad690 0x55f3d34b7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c1d600640 0 client.26015 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 1 --2- 172.21.15.42:0/2731269695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34acca0 0x55f3d34b6f10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 1 -- 172.21.15.42:0/2731269695 >> 172.21.15.42:0/2731269695 conn(0x55f3d34a8310 msgr2=0x55f3d34a7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:11.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 1 -- 172.21.15.42:0/2731269695 shutdown_connections 2024-04-25T00:29:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 1 -- 172.21.15.42:0/2731269695 wait complete. 2024-04-25T00:29:11.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.900+0000 7f8c391e0740 20 client.26015 trim_cache size 0 max 16384 2024-04-25T00:29:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.917+0000 7f8c391e0740 1 Processor -- start 2024-04-25T00:29:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.917+0000 7f8c391e0740 1 -- start start 2024-04-25T00:29:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 0x55f3d34abaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34ad690 0x55f3d34be450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34b2800 0x55f3d34a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c391e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301ad600 con 0x55f3d34acca0 2024-04-25T00:29:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c391e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301aaed0 con 0x55f3d34b2800 2024-04-25T00:29:11.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c391e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301a8d20 con 0x55f3d34ad690 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c26a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34ad690 0x55f3d34be450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 0x55f3d34abaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c26a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34ad690 0x55f3d34be450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:36692/0 (socket says 172.21.15.42:36692) 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c26a00640 1 -- 172.21.15.42:0/3393768466 learned_addr learned my addr 172.21.15.42:0/3393768466 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.918+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/3393768466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34b2800 0x55f3d34a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.919+0000 7f8c26a00640 1 -- 172.21.15.42:0/3393768466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34b2800 msgr2=0x55f3d34a7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.919+0000 7f8c26a00640 1 --2- 172.21.15.42:0/3393768466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34b2800 0x55f3d34a7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.919+0000 7f8c26a00640 1 -- 172.21.15.42:0/3393768466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 msgr2=0x55f3d34abaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.919+0000 7f8c26a00640 1 --2- 172.21.15.42:0/3393768466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 0x55f3d34abaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.919+0000 7f8c26a00640 1 -- 172.21.15.42:0/3393768466 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3d34ac8e0 con 0x55f3d34ad690 2024-04-25T00:29:11.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.919+0000 7f8c26a00640 1 --2- 172.21.15.42:0/3393768466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34ad690 0x55f3d34be450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c180045c0 tx=0x7f8c1800bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=895f158cdd85ba36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.919+0000 7f8c1fe00640 1 -- 172.21.15.42:0/3393768466 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18023070 con 0x55f3d34ad690 2024-04-25T00:29:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.919+0000 7f8c1fe00640 1 -- 172.21.15.42:0/3393768466 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1801d910 con 0x55f3d34ad690 2024-04-25T00:29:11.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.919+0000 7f8c1fe00640 1 -- 172.21.15.42:0/3393768466 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c18011cb0 con 0x55f3d34ad690 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.920+0000 7f8c391e0740 1 -- 172.21.15.42:0/3393768466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34ad690 msgr2=0x55f3d34be450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.920+0000 7f8c391e0740 1 --2- 172.21.15.42:0/3393768466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34ad690 0x55f3d34be450 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8c180045c0 tx=0x7f8c1800bf20 comp rx=0 tx=0).stop 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.920+0000 7f8c391e0740 1 -- 172.21.15.42:0/3393768466 shutdown_connections 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.920+0000 7f8c391e0740 1 --2- 172.21.15.42:0/3393768466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34b2800 0x55f3d34a7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.920+0000 7f8c391e0740 1 --2- 172.21.15.42:0/3393768466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34ad690 0x55f3d34be450 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.920+0000 7f8c391e0740 1 --2- 172.21.15.42:0/3393768466 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 0x55f3d34abaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.920+0000 7f8c391e0740 1 -- 172.21.15.42:0/3393768466 >> 172.21.15.42:0/3393768466 conn(0x55f3d34a8310 msgr2=0x55f3d34b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.920+0000 7f8c391e0740 1 -- 172.21.15.42:0/3393768466 shutdown_connections 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.920+0000 7f8c391e0740 1 -- 172.21.15.42:0/3393768466 wait complete. 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.921+0000 7f8c391e0740 1 -- start start 2024-04-25T00:29:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.921+0000 7f8c391e0740 1 Processor -- start 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 0x55f3d34bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 0x55f3d34bc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c391e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34b2800 0x55f3d34bd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c391e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301a9390 con 0x55f3d34acca0 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c391e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301aa800 con 0x55f3d34ad690 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c391e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8c301a9a60 con 0x55f3d34b2800 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c391e0740 10 client.0 _ll_register_callbacks cb 0x7f8c200f2ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c26a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 0x55f3d34bc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c26a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 0x55f3d34bc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39218/0 (socket says 172.21.15.42:39218) 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c2fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 0x55f3d34bbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c26a00640 1 -- 172.21.15.42:0/1398485268 learned_addr learned my addr 172.21.15.42:0/1398485268 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34b2800 0x55f3d34bd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c26a00640 1 -- 172.21.15.42:0/1398485268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34b2800 msgr2=0x55f3d34bd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c26a00640 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34b2800 0x55f3d34bd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c26a00640 1 -- 172.21.15.42:0/1398485268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 msgr2=0x55f3d34bbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.922+0000 7f8c26a00640 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 0x55f3d34bbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.923+0000 7f8c26a00640 1 -- 172.21.15.42:0/1398485268 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3d34a8140 con 0x55f3d34ad690 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.923+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 0x55f3d34bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.923+0000 7f8c26a00640 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 0x55f3d34bc1f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8c18021040 tx=0x7f8c1800b920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd1973eb2fb0da54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.923+0000 7f8c1d600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.923+0000 7f8c1d600640 1 -- 172.21.15.42:0/1398485268 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c1802d060 con 0x55f3d34ad690 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.923+0000 7f8c391e0740 1 -- 172.21.15.42:0/1398485268 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c1801d910 con 0x55f3d34ad690 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.923+0000 7f8c1d600640 1 -- 172.21.15.42:0/1398485268 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f8c18003e40 con 0x55f3d34ad690 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.923+0000 7f8c1d600640 1 -- 172.21.15.42:0/1398485268 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8c18015070 con 0x55f3d34ad690 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.923+0000 7f8c391e0740 10 client.26021 fetch_fsmap learned FSMap version 31 2024-04-25T00:29:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.923+0000 7f8c391e0740 1 -- 172.21.15.42:0/1398485268 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f3d3531670 con 0x55f3d34ad690 2024-04-25T00:29:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.924+0000 7f8c1d600640 1 -- 172.21.15.42:0/1398485268 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c1801d910 con 0x55f3d34ad690 2024-04-25T00:29:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.924+0000 7f8c1d600640 1 -- 172.21.15.42:0/1398485268 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f8c18005220 con 0x55f3d34ad690 2024-04-25T00:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.924+0000 7f8c391e0740 10 client.26021 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.924+0000 7f8c391e0740 10 client.26021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.924+0000 7f8c391e0740 20 client.26021 populate_metadata read hostname 'smithi042' 2024-04-25T00:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.924+0000 7f8c391e0740 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3d3531f30 0x55f3d3534350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.924+0000 7f8c391e0740 4 client.26021 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.924+0000 7f8c391e0740 1 -- 172.21.15.42:0/1398485268 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3d3417340 con 0x55f3d3531f30 2024-04-25T00:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.924+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3d3531f30 0x55f3d3534350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.925+0000 7f8c2fe00640 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3d3531f30 0x55f3d3534350 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6161a7331ecb19e0 server_cookie=9dc31939f0506622 in_seq=0 out_seq=0 2024-04-25T00:29:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:11.925+0000 7f8c1d600640 10 client.26021 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c1d600640 1 -- 172.21.15.42:0/1398485268 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3d3417340 con 0x55f3d3531f30 2024-04-25T00:29:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c1d600640 10 client.26021 handle_command_reply: tid=0 2024-04-25T00:29:12.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c391e0740 1 client.26021 shutdown 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c391e0740 1 -- 172.21.15.42:0/1398485268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 msgr2=0x55f3d34bc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c391e0740 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 0x55f3d34bc1f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8c18021040 tx=0x7f8c1800b920 comp rx=0 tx=0).stop 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c391e0740 1 -- 172.21.15.42:0/1398485268 shutdown_connections 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c391e0740 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3d3531f30 0x55f3d3534350 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c391e0740 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3d34b2800 0x55f3d34bd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c391e0740 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3d34ad690 0x55f3d34bc1f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c391e0740 1 --2- 172.21.15.42:0/1398485268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3d34acca0 0x55f3d34bbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c1d600640 0 client.26021 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.001+0000 7f8c391e0740 1 -- 172.21.15.42:0/1398485268 >> 172.21.15.42:0/1398485268 conn(0x55f3d34a8310 msgr2=0x55f3d34b7190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.002+0000 7f8c391e0740 1 -- 172.21.15.42:0/1398485268 shutdown_connections 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.002+0000 7f8c391e0740 1 -- 172.21.15.42:0/1398485268 wait complete. 2024-04-25T00:29:12.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.002+0000 7f8c391e0740 20 client.26021 trim_cache size 0 max 16384 2024-04-25T00:29:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:29:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:29:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:29:12.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:29:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 -- 172.21.15.42:0/17187549 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8c100a2c30 msgr2=0x7f8c100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8c100a2c30 0x7f8c100a50f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c200b8310 tx=0x7f8c200f2f90 comp rx=0 tx=0).stop 2024-04-25T00:29:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 -- 172.21.15.42:0/17187549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c30074140 msgr2=0x7f8c30199be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c30074140 0x7f8c30199be0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f8c20031440 tx=0x7f8c20032ce0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 -- 172.21.15.42:0/17187549 shutdown_connections 2024-04-25T00:29:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8c100a2c30 0x7f8c100a50f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8c30109630 0x7f8c301a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8c30075720 0x7f8c3019a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 --2- 172.21.15.42:0/17187549 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c30074140 0x7f8c30199be0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 -- 172.21.15.42:0/17187549 >> 172.21.15.42:0/17187549 conn(0x7f8c300fcd10 msgr2=0x7f8c301055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 -- 172.21.15.42:0/17187549 shutdown_connections 2024-04-25T00:29:12.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:12.003+0000 7f8c36400640 1 -- 172.21.15.42:0/17187549 wait complete. 2024-04-25T00:29:12.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:11 smithi042 ceph-mon[28130]: pgmap v701: 97 pgs: 97 active+clean; 120 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T00:29:12.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:11 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:29:12.137 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:11 smithi136 ceph-mon[32923]: pgmap v701: 97 pgs: 97 active+clean; 120 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T00:29:12.137 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:11 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:29:12.193 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:11 smithi106 ceph-mon[32510]: pgmap v701: 97 pgs: 97 active+clean; 120 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 481 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T00:29:12.193 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:11 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:29:12.268 INFO:tasks.workunit.client.0.smithi042.stdout: 1 642628 6.29 MB/sec execute 422 sec latency 36.618 ms 2024-04-25T00:29:13.268 INFO:tasks.workunit.client.0.smithi042.stdout: 1 643395 6.29 MB/sec execute 423 sec latency 17.128 ms 2024-04-25T00:29:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:13 smithi136 ceph-mon[32923]: pgmap v702: 97 pgs: 97 active+clean; 121 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-04-25T00:29:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:13 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:13 smithi106 ceph-mon[32510]: pgmap v702: 97 pgs: 97 active+clean; 121 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-04-25T00:29:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:13 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:13 smithi042 ceph-mon[28130]: pgmap v702: 97 pgs: 97 active+clean; 121 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-04-25T00:29:13.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.630 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:13.631 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:13 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:14.268 INFO:tasks.workunit.client.0.smithi042.stdout: 1 644778 6.29 MB/sec execute 424 sec latency 51.422 ms 2024-04-25T00:29:15.270 INFO:tasks.workunit.client.0.smithi042.stdout: 1 646609 6.30 MB/sec execute 425 sec latency 16.232 ms 2024-04-25T00:29:15.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:15 smithi042 ceph-mon[28130]: pgmap v703: 97 pgs: 97 active+clean; 121 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-25T00:29:15.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:15.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:15 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:15 smithi136 ceph-mon[32923]: pgmap v703: 97 pgs: 97 active+clean; 121 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-25T00:29:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:15 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:15 smithi106 ceph-mon[32510]: pgmap v703: 97 pgs: 97 active+clean; 121 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 1.4 MiB/s wr, 147 op/s 2024-04-25T00:29:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:15 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:16.268 INFO:tasks.workunit.client.0.smithi042.stdout: 1 647293 6.29 MB/sec execute 426 sec latency 23.767 ms 2024-04-25T00:29:17.268 INFO:tasks.workunit.client.0.smithi042.stdout: 1 648768 6.29 MB/sec execute 427 sec latency 25.445 ms 2024-04-25T00:29:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:17 smithi042 ceph-mon[28130]: pgmap v704: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.9 MiB/s wr, 232 op/s 2024-04-25T00:29:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:29:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:29:17.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:17.640 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:17 smithi106 ceph-mon[32510]: pgmap v704: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.9 MiB/s wr, 232 op/s 2024-04-25T00:29:17.640 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:29:17.640 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:29:17.640 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:17 smithi136 ceph-mon[32923]: pgmap v704: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 1.9 MiB/s wr, 232 op/s 2024-04-25T00:29:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:29:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:29:17.764 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:29:18.268 INFO:tasks.workunit.client.0.smithi042.stdout: 1 650136 6.30 MB/sec execute 428 sec latency 32.080 ms 2024-04-25T00:29:19.268 INFO:tasks.workunit.client.0.smithi042.stdout: 1 650680 6.29 MB/sec execute 429 sec latency 32.772 ms 2024-04-25T00:29:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:19 smithi042 ceph-mon[28130]: pgmap v705: 97 pgs: 97 active+clean; 124 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-25T00:29:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:19 smithi136 ceph-mon[32923]: pgmap v705: 97 pgs: 97 active+clean; 124 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-25T00:29:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:19 smithi106 ceph-mon[32510]: pgmap v705: 97 pgs: 97 active+clean; 124 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 459 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-25T00:29:20.268 INFO:tasks.workunit.client.0.smithi042.stdout: 1 651738 6.29 MB/sec execute 430 sec latency 34.982 ms 2024-04-25T00:29:21.269 INFO:tasks.workunit.client.0.smithi042.stdout: 1 653500 6.30 MB/sec execute 431 sec latency 19.641 ms 2024-04-25T00:29:21.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:21 smithi042 ceph-mon[28130]: pgmap v706: 97 pgs: 97 active+clean; 124 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:29:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:21 smithi136 ceph-mon[32923]: pgmap v706: 97 pgs: 97 active+clean; 124 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:29:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:21 smithi106 ceph-mon[32510]: pgmap v706: 97 pgs: 97 active+clean; 124 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:29:22.269 INFO:tasks.workunit.client.0.smithi042.stdout: 1 653909 6.29 MB/sec execute 432 sec latency 21.498 ms 2024-04-25T00:29:23.269 INFO:tasks.workunit.client.0.smithi042.stdout: 1 654893 6.29 MB/sec execute 433 sec latency 26.729 ms 2024-04-25T00:29:23.882 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:23 smithi042 ceph-mon[28130]: pgmap v707: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T00:29:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:23 smithi136 ceph-mon[32923]: pgmap v707: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T00:29:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:23 smithi106 ceph-mon[32510]: pgmap v707: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 526 KiB/s rd, 1.7 MiB/s wr, 206 op/s 2024-04-25T00:29:24.269 INFO:tasks.workunit.client.0.smithi042.stdout: 1 656337 6.29 MB/sec execute 434 sec latency 37.849 ms 2024-04-25T00:29:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.160+0000 7f570bc00640 1 -- 172.21.15.42:0/1395670278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5704108fc0 msgr2=0x7f5704109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.160+0000 7f570bc00640 1 --2- 172.21.15.42:0/1395670278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5704108fc0 0x7f5704109440 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009900 tx=0x7f56f002a410 comp rx=0 tx=0).stop 2024-04-25T00:29:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.160+0000 7f570bc00640 1 -- 172.21.15.42:0/1395670278 shutdown_connections 2024-04-25T00:29:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.160+0000 7f570bc00640 1 --2- 172.21.15.42:0/1395670278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704109980 0x7f5704110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.160+0000 7f570bc00640 1 --2- 172.21.15.42:0/1395670278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5704108fc0 0x7f5704109440 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.160+0000 7f570bc00640 1 --2- 172.21.15.42:0/1395670278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5704107dd0 0x7f57041081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.160+0000 7f570bc00640 1 -- 172.21.15.42:0/1395670278 >> 172.21.15.42:0/1395670278 conn(0x7f5704076d90 msgr2=0x7f57040791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.160+0000 7f570bc00640 1 -- 172.21.15.42:0/1395670278 shutdown_connections 2024-04-25T00:29:25.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.160+0000 7f570bc00640 1 -- 172.21.15.42:0/1395670278 wait complete. 2024-04-25T00:29:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.161+0000 7f570bc00640 1 Processor -- start 2024-04-25T00:29:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.161+0000 7f570bc00640 1 -- start start 2024-04-25T00:29:25.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f570bc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704107dd0 0x7f570419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f570bc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5704108fc0 0x7f570419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f570bc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5704109980 0x7f57041a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f570bc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5704113a80 con 0x7f5704108fc0 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f570bc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5704113900 con 0x7f5704109980 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f570bc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5704113c00 con 0x7f5704107dd0 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5709200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704107dd0 0x7f570419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5709200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704107dd0 0x7f570419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:58916/0 (socket says 172.21.15.42:58916) 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5709200640 1 -- 172.21.15.42:0/1090378569 learned_addr learned my addr 172.21.15.42:0/1090378569 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5703e00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5704108fc0 0x7f570419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5709c00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5704109980 0x7f57041a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5709200640 1 -- 172.21.15.42:0/1090378569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5704109980 msgr2=0x7f57041a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:25.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5709200640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5704109980 0x7f57041a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5709200640 1 -- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5704108fc0 msgr2=0x7f570419e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5709200640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5704108fc0 0x7f570419e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5709200640 1 -- 172.21.15.42:0/1090378569 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57041a5c00 con 0x7f5704107dd0 2024-04-25T00:29:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.162+0000 7f5703e00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5704108fc0 0x7f570419e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.163+0000 7f5709200640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704107dd0 0x7f570419dfe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f56f800e9c0 tx=0x7f56f800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.163+0000 7f5701600640 1 -- 172.21.15.42:0/1090378569 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f800cd80 con 0x7f5704107dd0 2024-04-25T00:29:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.163+0000 7f5701600640 1 -- 172.21.15.42:0/1090378569 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f80035f0 con 0x7f5704107dd0 2024-04-25T00:29:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.163+0000 7f570bc00640 1 -- 172.21.15.42:0/1090378569 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57041a5ef0 con 0x7f5704107dd0 2024-04-25T00:29:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.163+0000 7f5701600640 1 -- 172.21.15.42:0/1090378569 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f8010690 con 0x7f5704107dd0 2024-04-25T00:29:25.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.163+0000 7f570bc00640 1 -- 172.21.15.42:0/1090378569 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57041a6350 con 0x7f5704107dd0 2024-04-25T00:29:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.168+0000 7f5701600640 1 -- 172.21.15.42:0/1090378569 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f56f80037a0 con 0x7f5704107dd0 2024-04-25T00:29:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.168+0000 7f570ebe7740 1 Processor -- start 2024-04-25T00:29:25.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.168+0000 7f570ebe7740 1 -- start start 2024-04-25T00:29:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e5c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e61190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 0x55d6c3e63c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f570ebe7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d6c3dbeab0 con 0x55d6c3e5c690 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f570ebe7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d6c3d67640 con 0x55d6c3e61800 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f570ebe7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d6c3dbaa70 con 0x55d6c3e5bca0 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f5703e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 0x55d6c3e63c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f5703e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 0x55d6c3e63c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39250/0 (socket says 172.21.15.42:39250) 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f5703e00640 1 -- 172.21.15.42:0/2622553275 learned_addr learned my addr 172.21.15.42:0/2622553275 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f5709c00640 1 --2- 172.21.15.42:0/2622553275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e5c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f5709200640 1 --2- 172.21.15.42:0/2622553275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e61190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f5703e00640 1 -- 172.21.15.42:0/2622553275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5bca0 msgr2=0x55d6c3e5c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f5703e00640 1 --2- 172.21.15.42:0/2622553275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e5c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f5703e00640 1 -- 172.21.15.42:0/2622553275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 msgr2=0x55d6c3e61190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:25.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f5703e00640 1 --2- 172.21.15.42:0/2622553275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e61190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.169+0000 7f5703e00640 1 -- 172.21.15.42:0/2622553275 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c3ad3210 con 0x55d6c3e61800 2024-04-25T00:29:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.170+0000 7f5703e00640 1 --2- 172.21.15.42:0/2622553275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 0x55d6c3e63c90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f56f0031440 tx=0x7f56f00040f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5baaf2acf9d0f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.170+0000 7f5701600640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56ec0a2c30 0x7f56ec0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.170+0000 7f56f6000640 1 -- 172.21.15.42:0/2622553275 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0035030 con 0x55d6c3e61800 2024-04-25T00:29:25.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.170+0000 7f56f6000640 1 -- 172.21.15.42:0/2622553275 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f0005520 con 0x55d6c3e61800 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.170+0000 7f56f6000640 1 -- 172.21.15.42:0/2622553275 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f00316b0 con 0x55d6c3e61800 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.170+0000 7f5701600640 1 -- 172.21.15.42:0/1090378569 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f56f80cf4e0 con 0x7f5704107dd0 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.170+0000 7f5709c00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56ec0a2c30 0x7f56ec0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.171+0000 7f570ebe7740 1 -- 172.21.15.42:0/2622553275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 msgr2=0x55d6c3e63c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.171+0000 7f570ebe7740 1 --2- 172.21.15.42:0/2622553275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 0x55d6c3e63c90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f56f0031440 tx=0x7f56f00040f0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.171+0000 7f5709c00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56ec0a2c30 0x7f56ec0a50f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55d6c3e5cef0 tx=0x7f56fc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.171+0000 7f570ebe7740 1 -- 172.21.15.42:0/2622553275 shutdown_connections 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.171+0000 7f570ebe7740 1 --2- 172.21.15.42:0/2622553275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 0x55d6c3e63c90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.171+0000 7f570ebe7740 1 --2- 172.21.15.42:0/2622553275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e61190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.171+0000 7f570ebe7740 1 --2- 172.21.15.42:0/2622553275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e5c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.171+0000 7f570ebe7740 1 -- 172.21.15.42:0/2622553275 >> 172.21.15.42:0/2622553275 conn(0x55d6c3e57310 msgr2=0x55d6c3e57710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.171+0000 7f570ebe7740 1 -- 172.21.15.42:0/2622553275 shutdown_connections 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.171+0000 7f570ebe7740 1 -- 172.21.15.42:0/2622553275 wait complete. 2024-04-25T00:29:25.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.172+0000 7f570ebe7740 1 -- start start 2024-04-25T00:29:25.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.172+0000 7f570ebe7740 1 Processor -- start 2024-04-25T00:29:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e65e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e663d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e61800 0x55d6c3e6d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f570ebe7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d6c3e64750 con 0x55d6c3e5c690 2024-04-25T00:29:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f570ebe7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d6c3d7a990 con 0x55d6c3e5bca0 2024-04-25T00:29:25.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f570ebe7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d6c3e648d0 con 0x55d6c3e61800 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f570ebe7740 10 client.0 _ll_register_callbacks cb 0x55d6c3d7c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f5709200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e65e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f5709c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e61800 0x55d6c3e6d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f5709200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e65e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39266/0 (socket says 172.21.15.42:39266) 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f5709200640 1 -- 172.21.15.42:0/4286237243 learned_addr learned my addr 172.21.15.42:0/4286237243 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f5703e00640 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e663d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f5709200640 1 -- 172.21.15.42:0/4286237243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e61800 msgr2=0x55d6c3e6d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f5709200640 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e61800 0x55d6c3e6d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f5709200640 1 -- 172.21.15.42:0/4286237243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 msgr2=0x55d6c3e663d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f5709200640 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e663d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.173+0000 7f5709200640 1 -- 172.21.15.42:0/4286237243 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c3b9f620 con 0x55d6c3e5bca0 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f5709200640 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e65e90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f56f800c6a0 tx=0x7f56f80980b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=795d1ca7ba96054c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:25.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f56eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f56eaa00640 1 -- 172.21.15.42:0/4286237243 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f809c410 con 0x55d6c3e5bca0 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f56eaa00640 1 -- 172.21.15.42:0/4286237243 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f56f8003b30 con 0x55d6c3e5bca0 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f570ebe7740 1 -- 172.21.15.42:0/4286237243 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56f0005520 con 0x55d6c3e5bca0 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f56eaa00640 1 -- 172.21.15.42:0/4286237243 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f56f809bd10 con 0x55d6c3e5bca0 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f570ebe7740 10 client.26033 fetch_fsmap learned FSMap version 31 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f570ebe7740 1 -- 172.21.15.42:0/4286237243 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d6c3e704b0 con 0x55d6c3e5bca0 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f56eaa00640 1 -- 172.21.15.42:0/4286237243 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f0005520 con 0x55d6c3e5bca0 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f56eaa00640 1 -- 172.21.15.42:0/4286237243 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f56f809b160 con 0x55d6c3e5bca0 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f570ebe7740 10 client.26033 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.174+0000 7f570ebe7740 10 client.26033 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.175+0000 7f570ebe7740 20 client.26033 populate_metadata read hostname 'smithi042' 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.175+0000 7f570ebe7740 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d6c3e71b70 0x55d6c3e73fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.175+0000 7f570ebe7740 4 client.26033 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:25.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.175+0000 7f570ebe7740 1 -- 172.21.15.42:0/4286237243 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6c3dcbff0 con 0x55d6c3e71b70 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.175+0000 7f5703e00640 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d6c3e71b70 0x55d6c3e73fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.175+0000 7f5703e00640 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d6c3e71b70 0x55d6c3e73fc0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c174958895f4bf server_cookie=52f033fe640b9ebd in_seq=0 out_seq=0 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.175+0000 7f56eaa00640 10 client.26033 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.176+0000 7f56eaa00640 1 -- 172.21.15.42:0/4286237243 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d6c3dcbff0 con 0x55d6c3e71b70 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.176+0000 7f56eaa00640 10 client.26033 handle_command_reply: tid=0 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.176+0000 7f570ebe7740 1 client.26033 shutdown 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.176+0000 7f570ebe7740 1 -- 172.21.15.42:0/4286237243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 msgr2=0x55d6c3e65e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.176+0000 7f570ebe7740 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e65e90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f56f800c6a0 tx=0x7f56f80980b0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.177+0000 7f570ebe7740 1 -- 172.21.15.42:0/4286237243 shutdown_connections 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.177+0000 7f570ebe7740 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d6c3e71b70 0x55d6c3e73fc0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.177+0000 7f570ebe7740 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e61800 0x55d6c3e6d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.177+0000 7f570ebe7740 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e663d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.177+0000 7f56eaa00640 0 client.26033 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.177+0000 7f570ebe7740 1 --2- 172.21.15.42:0/4286237243 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e65e90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.177+0000 7f570ebe7740 1 -- 172.21.15.42:0/4286237243 >> 172.21.15.42:0/4286237243 conn(0x55d6c3e57310 msgr2=0x55d6c3e56480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.177+0000 7f570ebe7740 1 -- 172.21.15.42:0/4286237243 shutdown_connections 2024-04-25T00:29:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.177+0000 7f570ebe7740 1 -- 172.21.15.42:0/4286237243 wait complete. 2024-04-25T00:29:25.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.177+0000 7f570ebe7740 20 client.26033 trim_cache size 0 max 16384 2024-04-25T00:29:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.195+0000 7f570ebe7740 1 Processor -- start 2024-04-25T00:29:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.195+0000 7f570ebe7740 1 -- start start 2024-04-25T00:29:25.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e6e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e5a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e61800 0x55d6c3e5a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f570ebe7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f57041b18e0 con 0x55d6c3e5c690 2024-04-25T00:29:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f570ebe7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f57041af1b0 con 0x55d6c3e5bca0 2024-04-25T00:29:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f570ebe7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57041ad000 con 0x55d6c3e61800 2024-04-25T00:29:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f5709200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e6e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f5709200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e6e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39274/0 (socket says 172.21.15.42:39274) 2024-04-25T00:29:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f5709200640 1 -- 172.21.15.42:0/1957819561 learned_addr learned my addr 172.21.15.42:0/1957819561 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f5703e00640 1 --2- 172.21.15.42:0/1957819561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e5a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.196+0000 7f5709c00640 1 --2- 172.21.15.42:0/1957819561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e61800 0x55d6c3e5a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.197+0000 7f5709200640 1 -- 172.21.15.42:0/1957819561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e61800 msgr2=0x55d6c3e5a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.197+0000 7f5709200640 1 --2- 172.21.15.42:0/1957819561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e61800 0x55d6c3e5a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.197+0000 7f5709200640 1 -- 172.21.15.42:0/1957819561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 msgr2=0x55d6c3e5a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.197+0000 7f5709200640 1 --2- 172.21.15.42:0/1957819561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e5a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.197+0000 7f5709200640 1 -- 172.21.15.42:0/1957819561 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6c3e57140 con 0x55d6c3e5bca0 2024-04-25T00:29:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.197+0000 7f5709200640 1 --2- 172.21.15.42:0/1957819561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e6e1b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f56f80986e0 tx=0x7f56f80d2c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae4ca8f79ca23e05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:25.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.198+0000 7f56f5600640 1 -- 172.21.15.42:0/1957819561 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f809c410 con 0x55d6c3e5bca0 2024-04-25T00:29:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.198+0000 7f56f5600640 1 -- 172.21.15.42:0/1957819561 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f80a4980 con 0x55d6c3e5bca0 2024-04-25T00:29:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.198+0000 7f56f5600640 1 -- 172.21.15.42:0/1957819561 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f80cf0f0 con 0x55d6c3e5bca0 2024-04-25T00:29:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.199+0000 7f570ebe7740 1 -- 172.21.15.42:0/1957819561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 msgr2=0x55d6c3e6e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:25.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.199+0000 7f570ebe7740 1 --2- 172.21.15.42:0/1957819561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e6e1b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f56f80986e0 tx=0x7f56f80d2c10 comp rx=0 tx=0).stop 2024-04-25T00:29:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.199+0000 7f570ebe7740 1 -- 172.21.15.42:0/1957819561 shutdown_connections 2024-04-25T00:29:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.199+0000 7f570ebe7740 1 --2- 172.21.15.42:0/1957819561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e61800 0x55d6c3e5a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.199+0000 7f570ebe7740 1 --2- 172.21.15.42:0/1957819561 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5c690 0x55d6c3e5a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.199+0000 7f570ebe7740 1 --2- 172.21.15.42:0/1957819561 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e6e1b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.199+0000 7f570ebe7740 1 -- 172.21.15.42:0/1957819561 >> 172.21.15.42:0/1957819561 conn(0x55d6c3e57310 msgr2=0x55d6c3e6daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.199+0000 7f570ebe7740 1 -- 172.21.15.42:0/1957819561 shutdown_connections 2024-04-25T00:29:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.199+0000 7f570ebe7740 1 -- 172.21.15.42:0/1957819561 wait complete. 2024-04-25T00:29:25.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.199+0000 7f570ebe7740 1 -- start start 2024-04-25T00:29:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f570ebe7740 1 Processor -- start 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e6a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5c690 0x55d6c3e6a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f570ebe7740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 0x55d6c3e6b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f570ebe7740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f57041ad670 con 0x55d6c3e5bca0 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f570ebe7740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f57041aeae0 con 0x55d6c3e61800 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f570ebe7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f57041add40 con 0x55d6c3e5c690 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f570ebe7740 10 client.0 _ll_register_callbacks cb 0x7f56f8003e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f5703e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5c690 0x55d6c3e6a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f5703e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5c690 0x55d6c3e6a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:58938/0 (socket says 172.21.15.42:58938) 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f5703e00640 1 -- 172.21.15.42:0/3507034697 learned_addr learned my addr 172.21.15.42:0/3507034697 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f5709200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e6a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.200+0000 7f5709c00640 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 0x55d6c3e6b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.201+0000 7f5703e00640 1 -- 172.21.15.42:0/3507034697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 msgr2=0x55d6c3e6b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.201+0000 7f5703e00640 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 0x55d6c3e6b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.201+0000 7f5703e00640 1 -- 172.21.15.42:0/3507034697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5bca0 msgr2=0x55d6c3e6a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.201+0000 7f5703e00640 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e6a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.201+0000 7f5703e00640 1 -- 172.21.15.42:0/3507034697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6c3e6bec0 con 0x55d6c3e5c690 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.201+0000 7f5703e00640 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5c690 0x55d6c3e6a910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009900 tx=0x7f56f0004590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d104749db313182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.201+0000 7f56eaa00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.201+0000 7f56eaa00640 1 -- 172.21.15.42:0/3507034697 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56f0035040 con 0x55d6c3e5c690 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.201+0000 7f570ebe7740 1 -- 172.21.15.42:0/3507034697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56f80a4980 con 0x55d6c3e5c690 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.201+0000 7f56eaa00640 1 -- 172.21.15.42:0/3507034697 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f56f0031b80 con 0x55d6c3e5c690 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f56eaa00640 1 -- 172.21.15.42:0/3507034697 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f56f0060050 con 0x55d6c3e5c690 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f570ebe7740 10 client.26044 fetch_fsmap learned FSMap version 31 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f570ebe7740 1 -- 172.21.15.42:0/3507034697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d6c3ed8500 con 0x55d6c3e5c690 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f56eaa00640 1 -- 172.21.15.42:0/3507034697 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56f80a4980 con 0x55d6c3e5c690 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f56eaa00640 1 -- 172.21.15.42:0/3507034697 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f56f0003660 con 0x55d6c3e5c690 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f570ebe7740 10 client.26044 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:29:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f570ebe7740 10 client.26044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f570ebe7740 20 client.26044 populate_metadata read hostname 'smithi042' 2024-04-25T00:29:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f570ebe7740 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d6c3ed8ad0 0x55d6c3edaec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f570ebe7740 4 client.26044 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:29:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.202+0000 7f570ebe7740 1 -- 172.21.15.42:0/3507034697 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d6c3dc6340 con 0x55d6c3ed8ad0 2024-04-25T00:29:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.203+0000 7f5709200640 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d6c3ed8ad0 0x55d6c3edaec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.203+0000 7f5709200640 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d6c3ed8ad0 0x55d6c3edaec0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d949f359a85571 server_cookie=b9314d69c2e1707 in_seq=0 out_seq=0 2024-04-25T00:29:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.203+0000 7f56eaa00640 10 client.26044 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:25.269 INFO:tasks.workunit.client.0.smithi042.stdout: 1 657299 6.29 MB/sec execute 435 sec latency 40.172 ms 2024-04-25T00:29:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.269+0000 7f56eaa00640 1 -- 172.21.15.42:0/3507034697 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d6c3dc6340 con 0x55d6c3ed8ad0 2024-04-25T00:29:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.269+0000 7f56eaa00640 10 client.26044 handle_command_reply: tid=0 2024-04-25T00:29:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.269+0000 7f570ebe7740 1 client.26044 shutdown 2024-04-25T00:29:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 1 -- 172.21.15.42:0/3507034697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5c690 msgr2=0x55d6c3e6a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5c690 0x55d6c3e6a910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f56f0009900 tx=0x7f56f0004590 comp rx=0 tx=0).stop 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 1 -- 172.21.15.42:0/3507034697 shutdown_connections 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d6c3ed8ad0 0x55d6c3edaec0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d6c3e61800 0x55d6c3e6b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d6c3e5c690 0x55d6c3e6a910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 1 --2- 172.21.15.42:0/3507034697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d6c3e5bca0 0x55d6c3e6a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f56eaa00640 0 client.26044 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 1 -- 172.21.15.42:0/3507034697 >> 172.21.15.42:0/3507034697 conn(0x55d6c3e57310 msgr2=0x55d6c3e5a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 1 -- 172.21.15.42:0/3507034697 shutdown_connections 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 1 -- 172.21.15.42:0/3507034697 wait complete. 2024-04-25T00:29:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.270+0000 7f570ebe7740 20 client.26044 trim_cache size 0 max 16384 2024-04-25T00:29:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:29:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:29:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:29:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 -- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56ec0a2c30 msgr2=0x7f56ec0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56ec0a2c30 0x7f56ec0a50f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55d6c3e5cef0 tx=0x7f56fc009500 comp rx=0 tx=0).stop 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 -- 172.21.15.42:0/1090378569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704107dd0 msgr2=0x7f570419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704107dd0 0x7f570419dfe0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f56f800e9c0 tx=0x7f56f800ee80 comp rx=0 tx=0).stop 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 -- 172.21.15.42:0/1090378569 shutdown_connections 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f56ec0a2c30 0x7f56ec0a50f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5704109980 0x7f57041a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5704108fc0 0x7f570419e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 --2- 172.21.15.42:0/1090378569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5704107dd0 0x7f570419dfe0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 -- 172.21.15.42:0/1090378569 >> 172.21.15.42:0/1090378569 conn(0x7f5704076d90 msgr2=0x7f570410c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 -- 172.21.15.42:0/1090378569 shutdown_connections 2024-04-25T00:29:25.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:25.271+0000 7f570bc00640 1 -- 172.21.15.42:0/1090378569 wait complete. 2024-04-25T00:29:25.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:25 smithi042 ceph-mon[28130]: pgmap v708: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-25T00:29:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:25 smithi136 ceph-mon[32923]: pgmap v708: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-25T00:29:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:25 smithi106 ceph-mon[32510]: pgmap v708: 97 pgs: 97 active+clean; 125 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-04-25T00:29:26.269 INFO:tasks.workunit.client.0.smithi042.stdout: 1 657896 6.28 MB/sec execute 436 sec latency 27.274 ms 2024-04-25T00:29:27.269 INFO:tasks.workunit.client.0.smithi042.stdout: 1 659075 6.27 MB/sec execute 437 sec latency 28.736 ms 2024-04-25T00:29:27.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:27 smithi136 ceph-mon[32923]: pgmap v709: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-04-25T00:29:27.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:27 smithi042 ceph-mon[28130]: pgmap v709: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-04-25T00:29:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:27 smithi106 ceph-mon[32510]: pgmap v709: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-04-25T00:29:28.269 INFO:tasks.workunit.client.0.smithi042.stdout: 1 660666 6.29 MB/sec execute 438 sec latency 23.916 ms 2024-04-25T00:29:29.269 INFO:tasks.workunit.client.0.smithi042.stdout: 1 661104 6.28 MB/sec execute 439 sec latency 58.096 ms 2024-04-25T00:29:29.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:29 smithi042 ceph-mon[28130]: pgmap v710: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-25T00:29:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:29 smithi136 ceph-mon[32923]: pgmap v710: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-25T00:29:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:29 smithi106 ceph-mon[32510]: pgmap v710: 97 pgs: 97 active+clean; 123 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 423 KiB/s rd, 1.3 MiB/s wr, 163 op/s 2024-04-25T00:29:30.270 INFO:tasks.workunit.client.0.smithi042.stdout: 1 662049 6.27 MB/sec execute 440 sec latency 29.087 ms 2024-04-25T00:29:31.270 INFO:tasks.workunit.client.0.smithi042.stdout: 1 663463 6.28 MB/sec execute 441 sec latency 24.668 ms 2024-04-25T00:29:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:31 smithi136 ceph-mon[32923]: pgmap v711: 97 pgs: 97 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-04-25T00:29:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:31 smithi106 ceph-mon[32510]: pgmap v711: 97 pgs: 97 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-04-25T00:29:32.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:31 smithi042 ceph-mon[28130]: pgmap v711: 97 pgs: 97 active+clean; 126 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.7 MiB/s wr, 173 op/s 2024-04-25T00:29:32.270 INFO:tasks.workunit.client.0.smithi042.stdout: 1 664395 6.28 MB/sec execute 442 sec latency 21.745 ms 2024-04-25T00:29:33.270 INFO:tasks.workunit.client.0.smithi042.stdout: 1 665132 6.27 MB/sec execute 443 sec latency 19.968 ms 2024-04-25T00:29:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:33 smithi136 ceph-mon[32923]: pgmap v712: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-04-25T00:29:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:33 smithi106 ceph-mon[32510]: pgmap v712: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-04-25T00:29:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:33 smithi042 ceph-mon[28130]: pgmap v712: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 1.8 MiB/s wr, 206 op/s 2024-04-25T00:29:34.270 INFO:tasks.workunit.client.0.smithi042.stdout: 1 666486 6.26 MB/sec execute 444 sec latency 49.529 ms 2024-04-25T00:29:35.270 INFO:tasks.workunit.client.0.smithi042.stdout: 1 667922 6.28 MB/sec execute 445 sec latency 20.696 ms 2024-04-25T00:29:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:35 smithi136 ceph-mon[32923]: pgmap v713: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-04-25T00:29:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:35 smithi106 ceph-mon[32510]: pgmap v713: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-04-25T00:29:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:35 smithi042 ceph-mon[28130]: pgmap v713: 97 pgs: 97 active+clean; 127 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-04-25T00:29:36.271 INFO:tasks.workunit.client.0.smithi042.stdout: 1 668754 6.27 MB/sec execute 446 sec latency 30.176 ms 2024-04-25T00:29:37.270 INFO:tasks.workunit.client.0.smithi042.stdout: 1 670058 6.26 MB/sec execute 447 sec latency 26.153 ms 2024-04-25T00:29:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:37 smithi136 ceph-mon[32923]: pgmap v714: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-04-25T00:29:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:37 smithi106 ceph-mon[32510]: pgmap v714: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-04-25T00:29:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:37 smithi042 ceph-mon[28130]: pgmap v714: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.8 MiB/s wr, 198 op/s 2024-04-25T00:29:38.271 INFO:tasks.workunit.client.0.smithi042.stdout: 1 671491 6.28 MB/sec execute 448 sec latency 15.095 ms 2024-04-25T00:29:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.423+0000 7f4702000640 1 -- 172.21.15.42:0/637578901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc1027c0 msgr2=0x7f46fc102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.423+0000 7f4702000640 1 --2- 172.21.15.42:0/637578901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc1027c0 0x7f46fc102c40 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f46e8009900 tx=0x7f46e802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.423+0000 7f4702000640 1 -- 172.21.15.42:0/637578901 shutdown_connections 2024-04-25T00:29:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.423+0000 7f4702000640 1 --2- 172.21.15.42:0/637578901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46fc103180 0x7f46fc109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.423+0000 7f4702000640 1 --2- 172.21.15.42:0/637578901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc1027c0 0x7f46fc102c40 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.423+0000 7f4702000640 1 --2- 172.21.15.42:0/637578901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc1015d0 0x7f46fc1019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.423+0000 7f4702000640 1 -- 172.21.15.42:0/637578901 >> 172.21.15.42:0/637578901 conn(0x7f46fc0fcb40 msgr2=0x7f46fc0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.423+0000 7f4702000640 1 -- 172.21.15.42:0/637578901 shutdown_connections 2024-04-25T00:29:38.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.423+0000 7f4702000640 1 -- 172.21.15.42:0/637578901 wait complete. 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.424+0000 7f4702000640 1 Processor -- start 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.424+0000 7f4702000640 1 -- start start 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f4702000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46fc1015d0 0x7f46fc197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f4702000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc1027c0 0x7f46fc197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f4702000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc103180 0x7f46fc19eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f4702000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc10d290 con 0x7f46fc1027c0 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f4702000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc10d110 con 0x7f46fc1015d0 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f4702000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc10d410 con 0x7f46fc103180 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f4700c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46fc1015d0 0x7f46fc197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f4701600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc103180 0x7f46fc19eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f4700c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46fc1015d0 0x7f46fc197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40204/0 (socket says 172.21.15.42:40204) 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f4700c00640 1 -- 172.21.15.42:0/3490998680 learned_addr learned my addr 172.21.15.42:0/3490998680 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.425+0000 7f46fbe00640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc1027c0 0x7f46fc197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f4700c00640 1 -- 172.21.15.42:0/3490998680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc103180 msgr2=0x7f46fc19eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f4700c00640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc103180 0x7f46fc19eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f4700c00640 1 -- 172.21.15.42:0/3490998680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc1027c0 msgr2=0x7f46fc197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f4700c00640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc1027c0 0x7f46fc197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f4700c00640 1 -- 172.21.15.42:0/3490998680 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46fc19f520 con 0x7f46fc1015d0 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f4700c00640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46fc1015d0 0x7f46fc197900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e9c0 tx=0x7f46f000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f46f9600640 1 -- 172.21.15.42:0/3490998680 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f000cd80 con 0x7f46fc1015d0 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f46f9600640 1 -- 172.21.15.42:0/3490998680 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46f00035f0 con 0x7f46fc1015d0 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f4702000640 1 -- 172.21.15.42:0/3490998680 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46fc19f810 con 0x7f46fc1015d0 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f46f9600640 1 -- 172.21.15.42:0/3490998680 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f0010690 con 0x7f46fc1015d0 2024-04-25T00:29:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.426+0000 7f4702000640 1 -- 172.21.15.42:0/3490998680 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46fc19fd30 con 0x7f46fc1015d0 2024-04-25T00:29:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.429+0000 7f46f9600640 1 -- 172.21.15.42:0/3490998680 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f46f00037a0 con 0x7f46fc1015d0 2024-04-25T00:29:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.430+0000 7f4706405740 1 Processor -- start 2024-04-25T00:29:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.430+0000 7f4706405740 1 -- start start 2024-04-25T00:29:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.430+0000 7f4706405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.430+0000 7f4706405740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9d9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.430+0000 7f4706405740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9dbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.430+0000 7f4706405740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bf3e936ab0 con 0x55bf3e9d99b0 2024-04-25T00:29:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.430+0000 7f4706405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bf3e8df640 con 0x55bf3e9d3e50 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.430+0000 7f4706405740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bf3e932a70 con 0x55bf3e9d4840 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.431+0000 7f4701600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.431+0000 7f4701600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9d4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:40220/0 (socket says 172.21.15.42:40220) 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.431+0000 7f4701600640 1 -- 172.21.15.42:0/2055946356 learned_addr learned my addr 172.21.15.42:0/2055946356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.431+0000 7f4700c00640 1 --2- 172.21.15.42:0/2055946356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9d9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.431+0000 7f46fbe00640 1 --2- 172.21.15.42:0/2055946356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9dbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.431+0000 7f4701600640 1 -- 172.21.15.42:0/2055946356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 msgr2=0x55bf3e9d9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.431+0000 7f4701600640 1 --2- 172.21.15.42:0/2055946356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9d9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.431+0000 7f4701600640 1 -- 172.21.15.42:0/2055946356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d99b0 msgr2=0x55bf3e9dbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.431+0000 7f4701600640 1 --2- 172.21.15.42:0/2055946356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9dbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.431+0000 7f4701600640 1 -- 172.21.15.42:0/2055946356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf3e64b210 con 0x55bf3e9d3e50 2024-04-25T00:29:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.432+0000 7f4701600640 1 --2- 172.21.15.42:0/2055946356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9d4270 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f46f40039f0 tx=0x7f46f4003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f2aa548f4e8d5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.432+0000 7f46edc00640 1 -- 172.21.15.42:0/2055946356 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f401a470 con 0x55bf3e9d3e50 2024-04-25T00:29:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.432+0000 7f46edc00640 1 -- 172.21.15.42:0/2055946356 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46f4015070 con 0x55bf3e9d3e50 2024-04-25T00:29:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.432+0000 7f46edc00640 1 -- 172.21.15.42:0/2055946356 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4010900 con 0x55bf3e9d3e50 2024-04-25T00:29:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.432+0000 7f46f9600640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e40a2b80 0x7f46e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4706405740 1 -- 172.21.15.42:0/2055946356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d3e50 msgr2=0x55bf3e9d4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4700c00640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e40a2b80 0x7f46e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4706405740 1 --2- 172.21.15.42:0/2055946356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9d4270 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f46f40039f0 tx=0x7f46f4003eb0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f46f9600640 1 -- 172.21.15.42:0/3490998680 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f46f00cf440 con 0x7f46fc1015d0 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4706405740 1 -- 172.21.15.42:0/2055946356 shutdown_connections 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4706405740 1 --2- 172.21.15.42:0/2055946356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9dbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4706405740 1 --2- 172.21.15.42:0/2055946356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9d9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4706405740 1 --2- 172.21.15.42:0/2055946356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9d4270 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4706405740 1 -- 172.21.15.42:0/2055946356 >> 172.21.15.42:0/2055946356 conn(0x55bf3e9cf4c0 msgr2=0x55bf3e9cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4706405740 1 -- 172.21.15.42:0/2055946356 shutdown_connections 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4706405740 1 -- 172.21.15.42:0/2055946356 wait complete. 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.433+0000 7f4700c00640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e40a2b80 0x7f46e40a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f46f0098640 tx=0x7f46f00ccde0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.434+0000 7f4706405740 1 -- start start 2024-04-25T00:29:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.434+0000 7f4706405740 1 Processor -- start 2024-04-25T00:29:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.434+0000 7f4706405740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9de0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.434+0000 7f4706405740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9de610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.434+0000 7f4706405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9e5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.434+0000 7f4706405740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bf3e9dc960 con 0x55bf3e9d3e50 2024-04-25T00:29:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.434+0000 7f4706405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bf3e8f2990 con 0x55bf3e9d99b0 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.434+0000 7f4706405740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bf3e9dcae0 con 0x55bf3e9d4840 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.434+0000 7f4706405740 10 client.0 _ll_register_callbacks cb 0x55bf3e8f4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f4701600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9de0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9de610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9de610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41306/0 (socket says 172.21.15.42:41306) 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f4701600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9de0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56484/0 (socket says 172.21.15.42:56484) 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f46fbe00640 1 -- 172.21.15.42:0/618908456 learned_addr learned my addr 172.21.15.42:0/618908456 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f4701600640 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9e5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f46fbe00640 1 -- 172.21.15.42:0/618908456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 msgr2=0x55bf3e9e5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f46fbe00640 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9e5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f46fbe00640 1 -- 172.21.15.42:0/618908456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 msgr2=0x55bf3e9de0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f46fbe00640 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9de0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f46fbe00640 1 -- 172.21.15.42:0/618908456 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf3e717620 con 0x55bf3e9d4840 2024-04-25T00:29:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f4701600640 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9de0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f46fbe00640 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9de610 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f46e80314e0 tx=0x7f46e8004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34486ae2aba185e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.435+0000 7f46e2000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.436+0000 7f46e2000640 1 -- 172.21.15.42:0/618908456 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8035040 con 0x55bf3e9d4840 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.436+0000 7f46e2000640 1 -- 172.21.15.42:0/618908456 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f46e8005510 con 0x55bf3e9d4840 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.436+0000 7f4706405740 1 -- 172.21.15.42:0/618908456 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46f4015070 con 0x55bf3e9d4840 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.436+0000 7f46e2000640 1 -- 172.21.15.42:0/618908456 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f46e804d070 con 0x55bf3e9d4840 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.436+0000 7f4706405740 10 client.26056 fetch_fsmap learned FSMap version 31 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.436+0000 7f4706405740 1 -- 172.21.15.42:0/618908456 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bf3e9e86a0 con 0x55bf3e9d4840 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.436+0000 7f46e2000640 1 -- 172.21.15.42:0/618908456 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46f4015070 con 0x55bf3e9d4840 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.436+0000 7f46e2000640 1 -- 172.21.15.42:0/618908456 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f46e8031b30 con 0x55bf3e9d4840 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.437+0000 7f4706405740 10 client.26056 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.437+0000 7f4706405740 10 client.26056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.437+0000 7f4706405740 20 client.26056 populate_metadata read hostname 'smithi042' 2024-04-25T00:29:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.437+0000 7f4706405740 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bf3e9e9d70 0x55bf3e9ec180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.437+0000 7f4706405740 4 client.26056 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.437+0000 7f4706405740 1 -- 172.21.15.42:0/618908456 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf3e943ff0 con 0x55bf3e9e9d70 2024-04-25T00:29:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.437+0000 7f4701600640 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bf3e9e9d70 0x55bf3e9ec180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.437+0000 7f4701600640 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bf3e9e9d70 0x55bf3e9ec180 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adbd7e56306f4643 server_cookie=701d09c5c466e07b in_seq=0 out_seq=0 2024-04-25T00:29:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.437+0000 7f46e2000640 10 client.26056 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.438+0000 7f46e2000640 1 -- 172.21.15.42:0/618908456 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bf3e943ff0 con 0x55bf3e9e9d70 2024-04-25T00:29:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.438+0000 7f46e2000640 10 client.26056 handle_command_reply: tid=0 2024-04-25T00:29:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.438+0000 7f4706405740 1 client.26056 shutdown 2024-04-25T00:29:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.438+0000 7f4706405740 1 -- 172.21.15.42:0/618908456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 msgr2=0x55bf3e9de610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.438+0000 7f4706405740 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9de610 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f46e80314e0 tx=0x7f46e8004440 comp rx=0 tx=0).stop 2024-04-25T00:29:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.439+0000 7f4706405740 1 -- 172.21.15.42:0/618908456 shutdown_connections 2024-04-25T00:29:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.439+0000 7f4706405740 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bf3e9e9d70 0x55bf3e9ec180 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.439+0000 7f4706405740 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9e5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.439+0000 7f4706405740 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9de610 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.439+0000 7f46e2000640 0 client.26056 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.439+0000 7f4706405740 1 --2- 172.21.15.42:0/618908456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9de0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.439+0000 7f4706405740 1 -- 172.21.15.42:0/618908456 >> 172.21.15.42:0/618908456 conn(0x55bf3e9cf4c0 msgr2=0x55bf3e9ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.439+0000 7f4706405740 1 -- 172.21.15.42:0/618908456 shutdown_connections 2024-04-25T00:29:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.439+0000 7f4706405740 1 -- 172.21.15.42:0/618908456 wait complete. 2024-04-25T00:29:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.439+0000 7f4706405740 20 client.26056 trim_cache size 0 max 16384 2024-04-25T00:29:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.456+0000 7f4706405740 1 Processor -- start 2024-04-25T00:29:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.456+0000 7f4706405740 1 -- start start 2024-04-25T00:29:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f4706405740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f4706405740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9d26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f4706405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9f0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f4706405740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1ab2a0 con 0x55bf3e9d3e50 2024-04-25T00:29:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f4706405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1a8b70 con 0x55bf3e9d99b0 2024-04-25T00:29:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f4706405740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1a69c0 con 0x55bf3e9d4840 2024-04-25T00:29:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9d26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f46fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9d26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41310/0 (socket says 172.21.15.42:41310) 2024-04-25T00:29:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f46fbe00640 1 -- 172.21.15.42:0/1083754793 learned_addr learned my addr 172.21.15.42:0/1083754793 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f4701600640 1 --2- 172.21.15.42:0/1083754793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f4701600640 1 --2- 172.21.15.42:0/1083754793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9f0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f46fbe00640 1 -- 172.21.15.42:0/1083754793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 msgr2=0x55bf3e9f0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f46fbe00640 1 --2- 172.21.15.42:0/1083754793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9f0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f46fbe00640 1 -- 172.21.15.42:0/1083754793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 msgr2=0x55bf3e9d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f46fbe00640 1 --2- 172.21.15.42:0/1083754793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f46fbe00640 1 -- 172.21.15.42:0/1083754793 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf3e9cf2f0 con 0x55bf3e9d4840 2024-04-25T00:29:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.457+0000 7f4701600640 1 --2- 172.21.15.42:0/1083754793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9f0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:29:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.458+0000 7f46fbe00640 1 --2- 172.21.15.42:0/1083754793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9d26e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f46e8031440 tx=0x7f46e8005cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=690065798e0e42d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.458+0000 7f46ed200640 1 -- 172.21.15.42:0/1083754793 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8000e60 con 0x55bf3e9d4840 2024-04-25T00:29:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.458+0000 7f46ed200640 1 -- 172.21.15.42:0/1083754793 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46e8056070 con 0x55bf3e9d4840 2024-04-25T00:29:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.458+0000 7f46ed200640 1 -- 172.21.15.42:0/1083754793 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e8035070 con 0x55bf3e9d4840 2024-04-25T00:29:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.459+0000 7f4706405740 1 -- 172.21.15.42:0/1083754793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 msgr2=0x55bf3e9d26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.459+0000 7f4706405740 1 --2- 172.21.15.42:0/1083754793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9d26e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f46e8031440 tx=0x7f46e8005cb0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.459+0000 7f4706405740 1 -- 172.21.15.42:0/1083754793 shutdown_connections 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.459+0000 7f4706405740 1 --2- 172.21.15.42:0/1083754793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3e9f0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.459+0000 7f4706405740 1 --2- 172.21.15.42:0/1083754793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3e9d26e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.459+0000 7f4706405740 1 --2- 172.21.15.42:0/1083754793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3e9d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.459+0000 7f4706405740 1 -- 172.21.15.42:0/1083754793 >> 172.21.15.42:0/1083754793 conn(0x55bf3e9cf4c0 msgr2=0x55bf3e9ca110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.459+0000 7f4706405740 1 -- 172.21.15.42:0/1083754793 shutdown_connections 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.459+0000 7f4706405740 1 -- 172.21.15.42:0/1083754793 wait complete. 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.459+0000 7f4706405740 1 -- start start 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4706405740 1 Processor -- start 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4706405740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3ea4bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4706405740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3ea4c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4706405740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3ea53300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4706405740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1a7030 con 0x55bf3e9d3e50 2024-04-25T00:29:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4706405740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1a84a0 con 0x55bf3e9d99b0 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4706405740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f46fc1a7700 con 0x55bf3e9d4840 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4706405740 10 client.0 _ll_register_callbacks cb 0x7f46e80040d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4701600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3ea4bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4701600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3ea4bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56502/0 (socket says 172.21.15.42:56502) 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f4701600640 1 -- 172.21.15.42:0/2075976804 learned_addr learned my addr 172.21.15.42:0/2075976804 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.460+0000 7f46fbe00640 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3ea4c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f4701600640 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3ea53300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f4701600640 1 -- 172.21.15.42:0/2075976804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 msgr2=0x55bf3ea4c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f4701600640 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3ea4c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f4701600640 1 -- 172.21.15.42:0/2075976804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 msgr2=0x55bf3ea53300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f4701600640 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3ea53300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f4701600640 1 -- 172.21.15.42:0/2075976804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf3ea538f0 con 0x55bf3e9d3e50 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f4701600640 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3ea4bd80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009b60 tx=0x7f46f4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dca1bc768a57b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f46e2000640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f46e2000640 1 -- 172.21.15.42:0/2075976804 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f4025040 con 0x55bf3e9d3e50 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f46e2000640 1 -- 172.21.15.42:0/2075976804 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f46f4004940 con 0x55bf3e9d3e50 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.461+0000 7f4706405740 1 -- 172.21.15.42:0/2075976804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46e8056070 con 0x55bf3e9d3e50 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.462+0000 7f46e2000640 1 -- 172.21.15.42:0/2075976804 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f46f4004b40 con 0x55bf3e9d3e50 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.462+0000 7f4706405740 10 client.15924 fetch_fsmap learned FSMap version 31 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.462+0000 7f4706405740 1 -- 172.21.15.42:0/2075976804 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55bf3ea54be0 con 0x55bf3e9d3e50 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.462+0000 7f46e2000640 1 -- 172.21.15.42:0/2075976804 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46e8056070 con 0x55bf3e9d3e50 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.462+0000 7f46e2000640 1 -- 172.21.15.42:0/2075976804 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f46f4028070 con 0x55bf3e9d3e50 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.462+0000 7f4706405740 10 client.15924 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.462+0000 7f4706405740 10 client.15924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.462+0000 7f4706405740 20 client.15924 populate_metadata read hostname 'smithi042' 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.462+0000 7f4706405740 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bf3ea55420 0x55bf3ea57840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.463+0000 7f4706405740 4 client.15924 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.463+0000 7f4706405740 1 -- 172.21.15.42:0/2075976804 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bf3e93e340 con 0x55bf3ea55420 2024-04-25T00:29:38.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.463+0000 7f46fbe00640 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bf3ea55420 0x55bf3ea57840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.463+0000 7f46fbe00640 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bf3ea55420 0x55bf3ea57840 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d6bc0168473669c server_cookie=8e3a292fdfe5d54f in_seq=0 out_seq=0 2024-04-25T00:29:38.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.463+0000 7f46e2000640 10 client.15924 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.523+0000 7f46e2000640 1 -- 172.21.15.42:0/2075976804 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bf3e93e340 con 0x55bf3ea55420 2024-04-25T00:29:38.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.523+0000 7f46e2000640 10 client.15924 handle_command_reply: tid=0 2024-04-25T00:29:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.523+0000 7f4706405740 1 client.15924 shutdown 2024-04-25T00:29:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 1 -- 172.21.15.42:0/2075976804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 msgr2=0x55bf3ea4bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3ea4bd80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f46f4009b60 tx=0x7f46f4005fd0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 1 -- 172.21.15.42:0/2075976804 shutdown_connections 2024-04-25T00:29:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bf3ea55420 0x55bf3ea57840 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bf3e9d99b0 0x55bf3ea53300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bf3e9d4840 0x55bf3ea4c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 1 --2- 172.21.15.42:0/2075976804 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bf3e9d3e50 0x55bf3ea4bd80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f46e2000640 0 client.15924 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 1 -- 172.21.15.42:0/2075976804 >> 172.21.15.42:0/2075976804 conn(0x55bf3e9cf4c0 msgr2=0x55bf3e9d21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 1 -- 172.21.15.42:0/2075976804 shutdown_connections 2024-04-25T00:29:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 1 -- 172.21.15.42:0/2075976804 wait complete. 2024-04-25T00:29:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.524+0000 7f4706405740 20 client.15924 trim_cache size 0 max 16384 2024-04-25T00:29:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:29:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:29:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:29:38.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:29:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 -- 172.21.15.42:0/3490998680 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e40a2b80 msgr2=0x7f46e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e40a2b80 0x7f46e40a5040 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f46f0098640 tx=0x7f46f00ccde0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 -- 172.21.15.42:0/3490998680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46fc1015d0 msgr2=0x7f46fc197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46fc1015d0 0x7f46fc197900 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f46f000e9c0 tx=0x7f46f000ee80 comp rx=0 tx=0).stop 2024-04-25T00:29:38.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 -- 172.21.15.42:0/3490998680 shutdown_connections 2024-04-25T00:29:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46e40a2b80 0x7f46e40a5040 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f46fc103180 0x7f46fc19eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f46fc1027c0 0x7f46fc197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 --2- 172.21.15.42:0/3490998680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46fc1015d0 0x7f46fc197900 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 -- 172.21.15.42:0/3490998680 >> 172.21.15.42:0/3490998680 conn(0x7f46fc0fcb40 msgr2=0x7f46fc105ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 -- 172.21.15.42:0/3490998680 shutdown_connections 2024-04-25T00:29:38.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:38.525+0000 7f4702000640 1 -- 172.21.15.42:0/3490998680 wait complete. 2024-04-25T00:29:39.271 INFO:tasks.workunit.client.0.smithi042.stdout: 1 671612 6.26 MB/sec execute 449 sec latency 754.656 ms 2024-04-25T00:29:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:39 smithi136 ceph-mon[32923]: pgmap v715: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-25T00:29:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:39 smithi106 ceph-mon[32510]: pgmap v715: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-25T00:29:40.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:39 smithi042 ceph-mon[28130]: pgmap v715: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 351 KiB/s rd, 1.6 MiB/s wr, 160 op/s 2024-04-25T00:29:40.271 INFO:tasks.workunit.client.0.smithi042.stdout: 1 671612 6.25 MB/sec execute 450 sec latency 1754.769 ms 2024-04-25T00:29:41.271 INFO:tasks.workunit.client.0.smithi042.stdout: 1 671612 6.24 MB/sec execute 451 sec latency 2754.935 ms 2024-04-25T00:29:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:41 smithi136 ceph-mon[32923]: pgmap v716: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-25T00:29:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:41 smithi106 ceph-mon[32510]: pgmap v716: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-25T00:29:42.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:41 smithi042 ceph-mon[28130]: pgmap v716: 97 pgs: 97 active+clean; 128 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 346 KiB/s rd, 1.8 MiB/s wr, 170 op/s 2024-04-25T00:29:42.271 INFO:tasks.workunit.client.0.smithi042.stdout: 1 671612 6.22 MB/sec execute 452 sec latency 3755.081 ms 2024-04-25T00:29:43.271 INFO:tasks.workunit.client.0.smithi042.stdout: 1 671612 6.21 MB/sec execute 453 sec latency 4755.181 ms 2024-04-25T00:29:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:43 smithi136 ceph-mon[32923]: pgmap v717: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-04-25T00:29:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:43 smithi106 ceph-mon[32510]: pgmap v717: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-04-25T00:29:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:43 smithi042 ceph-mon[28130]: pgmap v717: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.7 MiB/s wr, 194 op/s 2024-04-25T00:29:44.271 INFO:tasks.workunit.client.0.smithi042.stdout: 1 672347 6.20 MB/sec execute 454 sec latency 5011.870 ms 2024-04-25T00:29:45.271 INFO:tasks.workunit.client.0.smithi042.stdout: 1 673716 6.20 MB/sec execute 455 sec latency 74.021 ms 2024-04-25T00:29:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:45 smithi136 ceph-mon[32923]: pgmap v718: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-04-25T00:29:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:45 smithi106 ceph-mon[32510]: pgmap v718: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-04-25T00:29:46.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:45 smithi042 ceph-mon[28130]: pgmap v718: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 152 op/s 2024-04-25T00:29:46.271 INFO:tasks.workunit.client.0.smithi042.stdout: 1 675085 6.21 MB/sec execute 456 sec latency 19.939 ms 2024-04-25T00:29:47.272 INFO:tasks.workunit.client.0.smithi042.stdout: 1 675949 6.20 MB/sec execute 457 sec latency 25.185 ms 2024-04-25T00:29:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:47 smithi136 ceph-mon[32923]: pgmap v719: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-04-25T00:29:48.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:47 smithi106 ceph-mon[32510]: pgmap v719: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-04-25T00:29:48.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:47 smithi042 ceph-mon[28130]: pgmap v719: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 355 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-04-25T00:29:48.272 INFO:tasks.workunit.client.0.smithi042.stdout: 1 677209 6.20 MB/sec execute 458 sec latency 27.420 ms 2024-04-25T00:29:49.272 INFO:tasks.workunit.client.0.smithi042.stdout: 1 678588 6.21 MB/sec execute 459 sec latency 15.375 ms 2024-04-25T00:29:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:49 smithi136 ceph-mon[32923]: pgmap v720: 97 pgs: 97 active+clean; 126 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-04-25T00:29:50.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:49 smithi106 ceph-mon[32510]: pgmap v720: 97 pgs: 97 active+clean; 126 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-04-25T00:29:50.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:49 smithi042 ceph-mon[28130]: pgmap v720: 97 pgs: 97 active+clean; 126 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-04-25T00:29:50.272 INFO:tasks.workunit.client.0.smithi042.stdout: 1 679350 6.20 MB/sec execute 460 sec latency 46.361 ms 2024-04-25T00:29:51.272 INFO:tasks.workunit.client.0.smithi042.stdout: 1 680695 6.20 MB/sec execute 461 sec latency 27.850 ms 2024-04-25T00:29:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.684+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3443917591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd19c105570 msgr2=0x7fd19c10be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.684+0000 7fd1a1c00640 1 --2- 172.21.15.42:0/3443917591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd19c105570 0x7fd19c10be00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd194009be0 tx=0x7fd19402a6c0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.684+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3443917591 shutdown_connections 2024-04-25T00:29:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.684+0000 7fd1a1c00640 1 --2- 172.21.15.42:0/3443917591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd19c105570 0x7fd19c10be00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.684+0000 7fd1a1c00640 1 --2- 172.21.15.42:0/3443917591 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c104bb0 0x7fd19c105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.684+0000 7fd1a1c00640 1 --2- 172.21.15.42:0/3443917591 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd19c1039c0 0x7fd19c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.684+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3443917591 >> 172.21.15.42:0/3443917591 conn(0x7fd19c0fef30 msgr2=0x7fd19c101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.684+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3443917591 shutdown_connections 2024-04-25T00:29:51.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.684+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3443917591 wait complete. 2024-04-25T00:29:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.685+0000 7fd1a1c00640 1 Processor -- start 2024-04-25T00:29:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.685+0000 7fd1a1c00640 1 -- start start 2024-04-25T00:29:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.685+0000 7fd1a1c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c1039c0 0x7fd19c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd1a1c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd19c104bb0 0x7fd19c19a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd1a1c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd19c105570 0x7fd19c1a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd1a1c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c10f620 con 0x7fd19c104bb0 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd1a1c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c10f4a0 con 0x7fd19c105570 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd1a1c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c10f7a0 con 0x7fd19c1039c0 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd19c104bb0 0x7fd19c19a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd19c105570 0x7fd19c1a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c1039c0 0x7fd19c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd19c105570 0x7fd19c1a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59244/0 (socket says 172.21.15.42:59244) 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c1039c0 0x7fd19c199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:51902/0 (socket says 172.21.15.42:51902) 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19b400640 1 -- 172.21.15.42:0/3064600334 learned_addr learned my addr 172.21.15.42:0/3064600334 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd19c104bb0 0x7fd19c19a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:43874/0 (socket says 172.21.15.42:43874) 2024-04-25T00:29:51.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19aa00640 1 -- 172.21.15.42:0/3064600334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd19c105570 msgr2=0x7fd19c1a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19aa00640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd19c105570 0x7fd19c1a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19aa00640 1 -- 172.21.15.42:0/3064600334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd19c104bb0 msgr2=0x7fd19c19a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19aa00640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd19c104bb0 0x7fd19c19a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19aa00640 1 -- 172.21.15.42:0/3064600334 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c1a17c0 con 0x7fd19c1039c0 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.686+0000 7fd19b400640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd19c105570 0x7fd19c1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.687+0000 7fd19aa00640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c1039c0 0x7fd19c199ba0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e9c0 tx=0x7fd18c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.687+0000 7fd193400640 1 -- 172.21.15.42:0/3064600334 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00ccc0 con 0x7fd19c1039c0 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.687+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3064600334 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c1a1ab0 con 0x7fd19c1039c0 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.687+0000 7fd193400640 1 -- 172.21.15.42:0/3064600334 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd18c0035f0 con 0x7fd19c1039c0 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.687+0000 7fd193400640 1 -- 172.21.15.42:0/3064600334 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00ce60 con 0x7fd19c1039c0 2024-04-25T00:29:51.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.687+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3064600334 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c1a2050 con 0x7fd19c1039c0 2024-04-25T00:29:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.690+0000 7fd193400640 1 -- 172.21.15.42:0/3064600334 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd18c0037a0 con 0x7fd19c1039c0 2024-04-25T00:29:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.691+0000 7fd1a4b96740 1 Processor -- start 2024-04-25T00:29:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.691+0000 7fd1a4b96740 1 -- start start 2024-04-25T00:29:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.691+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c100c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.691+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 0x556776c15190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.691+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 0x556776c17c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.691+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556776b72ab0 con 0x556776c10690 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.691+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556776b1b640 con 0x556776c0fca0 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.691+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556776b6ea70 con 0x556776c15800 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c100c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c100c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59258/0 (socket says 172.21.15.42:59258) 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19b400640 1 -- 172.21.15.42:0/430522666 learned_addr learned my addr 172.21.15.42:0/430522666 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19a000640 1 --2- 172.21.15.42:0/430522666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 0x556776c17c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19aa00640 1 --2- 172.21.15.42:0/430522666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 0x556776c15190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19b400640 1 -- 172.21.15.42:0/430522666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 msgr2=0x556776c17c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19b400640 1 --2- 172.21.15.42:0/430522666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 0x556776c17c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19b400640 1 -- 172.21.15.42:0/430522666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 msgr2=0x556776c15190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19b400640 1 --2- 172.21.15.42:0/430522666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 0x556776c15190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd193400640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1800a2cb0 0x7fd1800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19b400640 1 -- 172.21.15.42:0/430522666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556776887210 con 0x556776c0fca0 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd19aa00640 1 --2- 172.21.15.42:0/430522666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 0x556776c15190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.692+0000 7fd193400640 1 -- 172.21.15.42:0/3064600334 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd18c0cf430 con 0x7fd19c1039c0 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.693+0000 7fd19a000640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1800a2cb0 0x7fd1800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.693+0000 7fd19b400640 1 --2- 172.21.15.42:0/430522666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c100c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd1940303d0 tx=0x7fd194030400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f78f6fbbf5b1d40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.693+0000 7fd187e00640 1 -- 172.21.15.42:0/430522666 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194041030 con 0x556776c0fca0 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.693+0000 7fd187e00640 1 -- 172.21.15.42:0/430522666 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd194003e50 con 0x556776c0fca0 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.693+0000 7fd187e00640 1 -- 172.21.15.42:0/430522666 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194036040 con 0x556776c0fca0 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.694+0000 7fd1a4b96740 1 -- 172.21.15.42:0/430522666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 msgr2=0x556776c100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.694+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/430522666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c100c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd1940303d0 tx=0x7fd194030400 comp rx=0 tx=0).stop 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.694+0000 7fd19a000640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1800a2cb0 0x7fd1800a5170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd19c19b0b0 tx=0x7fd188009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.694+0000 7fd1a4b96740 1 -- 172.21.15.42:0/430522666 shutdown_connections 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.694+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/430522666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 0x556776c17c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.694+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/430522666 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 0x556776c15190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.694+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/430522666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c100c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.694+0000 7fd1a4b96740 1 -- 172.21.15.42:0/430522666 >> 172.21.15.42:0/430522666 conn(0x556776c0b310 msgr2=0x556776c0b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.694+0000 7fd1a4b96740 1 -- 172.21.15.42:0/430522666 shutdown_connections 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.694+0000 7fd1a4b96740 1 -- 172.21.15.42:0/430522666 wait complete. 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd1a4b96740 1 -- start start 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd1a4b96740 1 Processor -- start 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c19ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 0x556776c1a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c15800 0x556776c21420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556776c18750 con 0x556776c15800 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556776b2e990 con 0x556776c0fca0 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556776c188d0 con 0x556776c10690 2024-04-25T00:29:51.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd1a4b96740 10 client.0 _ll_register_callbacks cb 0x556776b30950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd19aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c19ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd19aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c19ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59268/0 (socket says 172.21.15.42:59268) 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.695+0000 7fd19aa00640 1 -- 172.21.15.42:0/3464399861 learned_addr learned my addr 172.21.15.42:0/3464399861 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd19a000640 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 0x556776c1a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd19b400640 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c15800 0x556776c21420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd19aa00640 1 -- 172.21.15.42:0/3464399861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 msgr2=0x556776c1a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd19aa00640 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 0x556776c1a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd19aa00640 1 -- 172.21.15.42:0/3464399861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c15800 msgr2=0x556776c21420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd19aa00640 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c15800 0x556776c21420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd19aa00640 1 -- 172.21.15.42:0/3464399861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556776953620 con 0x556776c0fca0 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd19aa00640 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c19ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd18c098630 tx=0x7fd18c09b5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6663d53faefcd420 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd184c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd184c00640 1 -- 172.21.15.42:0/3464399861 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c0a0010 con 0x556776c0fca0 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd184c00640 1 -- 172.21.15.42:0/3464399861 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd18c0d2b80 con 0x556776c0fca0 2024-04-25T00:29:51.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.696+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3464399861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd194003e50 con 0x556776c0fca0 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd184c00640 1 -- 172.21.15.42:0/3464399861 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd18c0a48d0 con 0x556776c0fca0 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd1a4b96740 10 client.26066 fetch_fsmap learned FSMap version 31 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3464399861 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556776c24470 con 0x556776c0fca0 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd184c00640 1 -- 172.21.15.42:0/3464399861 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd194003e50 con 0x556776c0fca0 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd184c00640 1 -- 172.21.15.42:0/3464399861 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fd18c07aad0 con 0x556776c0fca0 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd1a4b96740 10 client.26066 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd1a4b96740 10 client.26066 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd1a4b96740 20 client.26066 populate_metadata read hostname 'smithi042' 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556776c25b00 0x556776c27f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd1a4b96740 4 client.26066 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.697+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3464399861 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556776b7fff0 con 0x556776c25b00 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.698+0000 7fd19a000640 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556776c25b00 0x556776c27f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.698+0000 7fd19a000640 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556776c25b00 0x556776c27f10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f929927bcbfd9d8f server_cookie=5243bcc2712d72b4 in_seq=0 out_seq=0 2024-04-25T00:29:51.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.698+0000 7fd184c00640 10 client.26066 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd184c00640 1 -- 172.21.15.42:0/3464399861 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556776b7fff0 con 0x556776c25b00 2024-04-25T00:29:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd184c00640 10 client.26066 handle_command_reply: tid=0 2024-04-25T00:29:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 client.26066 shutdown 2024-04-25T00:29:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3464399861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 msgr2=0x556776c19ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c19ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd18c098630 tx=0x7fd18c09b5c0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3464399861 shutdown_connections 2024-04-25T00:29:51.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556776c25b00 0x556776c27f10 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c15800 0x556776c21420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 0x556776c1a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/3464399861 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c19ea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd184c00640 0 client.26066 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3464399861 >> 172.21.15.42:0/3464399861 conn(0x556776c0b310 msgr2=0x556776c0a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3464399861 shutdown_connections 2024-04-25T00:29:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.699+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3464399861 wait complete. 2024-04-25T00:29:51.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.700+0000 7fd1a4b96740 20 client.26066 trim_cache size 0 max 16384 2024-04-25T00:29:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.717+0000 7fd1a4b96740 1 Processor -- start 2024-04-25T00:29:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.717+0000 7fd1a4b96740 1 -- start start 2024-04-25T00:29:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c1a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 0x556776c213b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c15800 0x556776c2c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1ad560 con 0x556776c15800 2024-04-25T00:29:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1aae30 con 0x556776c0fca0 2024-04-25T00:29:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a8c80 con 0x556776c10690 2024-04-25T00:29:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd19a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 0x556776c213b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd19aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c1a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd19a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 0x556776c213b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:51942/0 (socket says 172.21.15.42:51942) 2024-04-25T00:29:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd19a000640 1 -- 172.21.15.42:0/3542706776 learned_addr learned my addr 172.21.15.42:0/3542706776 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd19aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c1a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59276/0 (socket says 172.21.15.42:59276) 2024-04-25T00:29:51.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd19b400640 1 --2- 172.21.15.42:0/3542706776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c15800 0x556776c2c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.718+0000 7fd19a000640 1 -- 172.21.15.42:0/3542706776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 msgr2=0x556776c1a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.719+0000 7fd19a000640 1 --2- 172.21.15.42:0/3542706776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c1a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.719+0000 7fd19a000640 1 -- 172.21.15.42:0/3542706776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c15800 msgr2=0x556776c2c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.719+0000 7fd19a000640 1 --2- 172.21.15.42:0/3542706776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c15800 0x556776c2c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.719+0000 7fd19a000640 1 -- 172.21.15.42:0/3542706776 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556776c0f8e0 con 0x556776c10690 2024-04-25T00:29:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.719+0000 7fd19aa00640 1 --2- 172.21.15.42:0/3542706776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c1a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:29:51.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.719+0000 7fd19a000640 1 --2- 172.21.15.42:0/3542706776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 0x556776c213b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x556776c0c1e0 tx=0x7fd188019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e2502ca07218325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.719+0000 7fd187400640 1 -- 172.21.15.42:0/3542706776 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188004020 con 0x556776c10690 2024-04-25T00:29:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.719+0000 7fd187400640 1 -- 172.21.15.42:0/3542706776 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd188005500 con 0x556776c10690 2024-04-25T00:29:51.726 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.719+0000 7fd187400640 1 -- 172.21.15.42:0/3542706776 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd188004d00 con 0x556776c10690 2024-04-25T00:29:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.720+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3542706776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 msgr2=0x556776c213b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.720+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/3542706776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 0x556776c213b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x556776c0c1e0 tx=0x7fd188019000 comp rx=0 tx=0).stop 2024-04-25T00:29:51.729 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.720+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3542706776 shutdown_connections 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.720+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/3542706776 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c15800 0x556776c2c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.720+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/3542706776 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c10690 0x556776c213b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.720+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/3542706776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c1a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.720+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3542706776 >> 172.21.15.42:0/3542706776 conn(0x556776c0b310 msgr2=0x556776c19ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.720+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3542706776 shutdown_connections 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.720+0000 7fd1a4b96740 1 -- 172.21.15.42:0/3542706776 wait complete. 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.721+0000 7fd1a4b96740 1 -- start start 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.721+0000 7fd1a4b96740 1 Processor -- start 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.721+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c1f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.721+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 0x556776c1f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.730 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.721+0000 7fd1a4b96740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 0x556776c20840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.721+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a92f0 con 0x556776c10690 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.721+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1aa760 con 0x556776c0fca0 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.721+0000 7fd1a4b96740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a99c0 con 0x556776c15800 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.721+0000 7fd1a4b96740 10 client.0 _ll_register_callbacks cb 0x7fd18c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.722+0000 7fd19b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 0x556776c20840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.722+0000 7fd19a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 0x556776c1f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.722+0000 7fd19b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 0x556776c20840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:51952/0 (socket says 172.21.15.42:51952) 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.722+0000 7fd19b400640 1 -- 172.21.15.42:0/2933364278 learned_addr learned my addr 172.21.15.42:0/2933364278 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.722+0000 7fd19aa00640 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c1f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.722+0000 7fd19b400640 1 -- 172.21.15.42:0/2933364278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 msgr2=0x556776c1f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.722+0000 7fd19b400640 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c1f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.722+0000 7fd19b400640 1 -- 172.21.15.42:0/2933364278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 msgr2=0x556776c1f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:29:51.731 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.722+0000 7fd19b400640 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 0x556776c1f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.722+0000 7fd19b400640 1 -- 172.21.15.42:0/2933364278 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556776c0b140 con 0x556776c15800 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.723+0000 7fd19b400640 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 0x556776c20840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd19402ac80 tx=0x7fd194030600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed023d2427f2dda7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.723+0000 7fd184c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.723+0000 7fd184c00640 1 -- 172.21.15.42:0/2933364278 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194036040 con 0x556776c15800 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.723+0000 7fd1a4b96740 1 -- 172.21.15.42:0/2933364278 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd188005500 con 0x556776c15800 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.723+0000 7fd184c00640 1 -- 172.21.15.42:0/2933364278 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd194003b70 con 0x556776c15800 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.723+0000 7fd184c00640 1 -- 172.21.15.42:0/2933364278 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd194041040 con 0x556776c15800 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.723+0000 7fd1a4b96740 10 client.26080 fetch_fsmap learned FSMap version 31 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.723+0000 7fd1a4b96740 1 -- 172.21.15.42:0/2933364278 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556776c8c690 con 0x556776c15800 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.724+0000 7fd184c00640 1 -- 172.21.15.42:0/2933364278 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd188005500 con 0x556776c15800 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.724+0000 7fd184c00640 1 -- 172.21.15.42:0/2933364278 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fd19403d600 con 0x556776c15800 2024-04-25T00:29:51.732 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.724+0000 7fd1a4b96740 10 client.26080 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:29:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.724+0000 7fd1a4b96740 10 client.26080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:29:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.724+0000 7fd1a4b96740 20 client.26080 populate_metadata read hostname 'smithi042' 2024-04-25T00:29:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.724+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556776c8cbc0 0x556776c8efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:29:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.724+0000 7fd1a4b96740 4 client.26080 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:29:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.724+0000 7fd1a4b96740 1 -- 172.21.15.42:0/2933364278 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556776b7a340 con 0x556776c8cbc0 2024-04-25T00:29:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.724+0000 7fd19aa00640 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556776c8cbc0 0x556776c8efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:29:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.725+0000 7fd19aa00640 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556776c8cbc0 0x556776c8efe0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=705018f673ef1634 server_cookie=dcdf620ba7c1ee85 in_seq=0 out_seq=0 2024-04-25T00:29:51.733 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.725+0000 7fd184c00640 10 client.26080 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.771+0000 7fd184c00640 1 -- 172.21.15.42:0/2933364278 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556776b7a340 con 0x556776c8cbc0 2024-04-25T00:29:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.771+0000 7fd184c00640 10 client.26080 handle_command_reply: tid=0 2024-04-25T00:29:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.771+0000 7fd1a4b96740 1 client.26080 shutdown 2024-04-25T00:29:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.771+0000 7fd1a4b96740 1 -- 172.21.15.42:0/2933364278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 msgr2=0x556776c20840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.774 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.771+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 0x556776c20840 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd19402ac80 tx=0x7fd194030600 comp rx=0 tx=0).stop 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.772+0000 7fd1a4b96740 1 -- 172.21.15.42:0/2933364278 shutdown_connections 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.772+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556776c8cbc0 0x556776c8efe0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.772+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556776c15800 0x556776c20840 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.772+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556776c10690 0x556776c1f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.772+0000 7fd184c00640 0 client.26080 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.772+0000 7fd1a4b96740 1 --2- 172.21.15.42:0/2933364278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556776c0fca0 0x556776c1f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.772+0000 7fd1a4b96740 1 -- 172.21.15.42:0/2933364278 >> 172.21.15.42:0/2933364278 conn(0x556776c0b310 msgr2=0x556776c19ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.772+0000 7fd1a4b96740 1 -- 172.21.15.42:0/2933364278 shutdown_connections 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.772+0000 7fd1a4b96740 1 -- 172.21.15.42:0/2933364278 wait complete. 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.772+0000 7fd1a4b96740 20 client.26080 trim_cache size 0 max 16384 2024-04-25T00:29:51.775 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:29:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:29:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:29:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:29:51.776 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3064600334 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1800a2cb0 msgr2=0x7fd1800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1800a2cb0 0x7fd1800a5170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd19c19b0b0 tx=0x7fd188009260 comp rx=0 tx=0).stop 2024-04-25T00:29:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3064600334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c1039c0 msgr2=0x7fd19c199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:29:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c1039c0 0x7fd19c199ba0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd18c00e9c0 tx=0x7fd18c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:29:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3064600334 shutdown_connections 2024-04-25T00:29:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1800a2cb0 0x7fd1800a5170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd19c105570 0x7fd19c1a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.777 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd19c104bb0 0x7fd19c19a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 --2- 172.21.15.42:0/3064600334 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd19c1039c0 0x7fd19c199ba0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:29:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3064600334 >> 172.21.15.42:0/3064600334 conn(0x7fd19c0fef30 msgr2=0x7fd19c107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:29:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3064600334 shutdown_connections 2024-04-25T00:29:51.778 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:29:51.773+0000 7fd1a1c00640 1 -- 172.21.15.42:0/3064600334 wait complete. 2024-04-25T00:29:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:51 smithi136 ceph-mon[32923]: pgmap v721: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-04-25T00:29:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:51 smithi106 ceph-mon[32510]: pgmap v721: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-04-25T00:29:52.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:51 smithi042 ceph-mon[28130]: pgmap v721: 97 pgs: 97 active+clean; 125 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-04-25T00:29:52.272 INFO:tasks.workunit.client.0.smithi042.stdout: 1 682150 6.21 MB/sec execute 462 sec latency 14.207 ms 2024-04-25T00:29:53.272 INFO:tasks.workunit.client.0.smithi042.stdout: 1 683021 6.20 MB/sec execute 463 sec latency 19.160 ms 2024-04-25T00:29:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:53 smithi136 ceph-mon[32923]: pgmap v722: 97 pgs: 97 active+clean; 126 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-04-25T00:29:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:53 smithi106 ceph-mon[32510]: pgmap v722: 97 pgs: 97 active+clean; 126 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-04-25T00:29:54.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:53 smithi042 ceph-mon[28130]: pgmap v722: 97 pgs: 97 active+clean; 126 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 1.5 MiB/s wr, 167 op/s 2024-04-25T00:29:54.272 INFO:tasks.workunit.client.0.smithi042.stdout: 1 684293 6.20 MB/sec execute 464 sec latency 25.855 ms 2024-04-25T00:29:55.273 INFO:tasks.workunit.client.0.smithi042.stdout: 1 685663 6.21 MB/sec execute 465 sec latency 61.127 ms 2024-04-25T00:29:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:55 smithi136 ceph-mon[32923]: pgmap v723: 97 pgs: 97 active+clean; 126 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-04-25T00:29:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:55 smithi106 ceph-mon[32510]: pgmap v723: 97 pgs: 97 active+clean; 126 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-04-25T00:29:56.145 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:55 smithi042 ceph-mon[28130]: pgmap v723: 97 pgs: 97 active+clean; 126 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1.2 MiB/s wr, 119 op/s 2024-04-25T00:29:56.273 INFO:tasks.workunit.client.0.smithi042.stdout: 1 686378 6.20 MB/sec execute 466 sec latency 25.884 ms 2024-04-25T00:29:57.273 INFO:tasks.workunit.client.0.smithi042.stdout: 1 687697 6.20 MB/sec execute 467 sec latency 31.953 ms 2024-04-25T00:29:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:57 smithi136 ceph-mon[32923]: pgmap v724: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-04-25T00:29:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:57 smithi106 ceph-mon[32510]: pgmap v724: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-04-25T00:29:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:57 smithi042 ceph-mon[28130]: pgmap v724: 97 pgs: 97 active+clean; 127 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 421 KiB/s rd, 1.7 MiB/s wr, 168 op/s 2024-04-25T00:29:58.273 INFO:tasks.workunit.client.0.smithi042.stdout: 1 689173 6.21 MB/sec execute 468 sec latency 15.541 ms 2024-04-25T00:29:59.273 INFO:tasks.workunit.client.0.smithi042.stdout: 1 689832 6.20 MB/sec execute 469 sec latency 28.525 ms 2024-04-25T00:30:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:29:59 smithi136 ceph-mon[32923]: pgmap v725: 97 pgs: 97 active+clean; 129 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-25T00:30:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:29:59 smithi106 ceph-mon[32510]: pgmap v725: 97 pgs: 97 active+clean; 129 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-25T00:30:00.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:29:59 smithi042 ceph-mon[28130]: pgmap v725: 97 pgs: 97 active+clean; 129 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-25T00:30:00.273 INFO:tasks.workunit.client.0.smithi042.stdout: 1 691090 6.20 MB/sec execute 470 sec latency 40.208 ms 2024-04-25T00:30:01.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:00 smithi136 ceph-mon[32923]: overall HEALTH_OK 2024-04-25T00:30:01.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:00 smithi106 ceph-mon[32510]: overall HEALTH_OK 2024-04-25T00:30:01.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:00 smithi042 ceph-mon[28130]: overall HEALTH_OK 2024-04-25T00:30:01.276 INFO:tasks.workunit.client.0.smithi042.stdout: 1 692632 6.21 MB/sec execute 471 sec latency 20.681 ms 2024-04-25T00:30:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:01 smithi136 ceph-mon[32923]: pgmap v726: 97 pgs: 97 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-04-25T00:30:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:01 smithi106 ceph-mon[32510]: pgmap v726: 97 pgs: 97 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-04-25T00:30:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:01 smithi042 ceph-mon[28130]: pgmap v726: 97 pgs: 97 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 1.6 MiB/s wr, 176 op/s 2024-04-25T00:30:02.274 INFO:tasks.workunit.client.0.smithi042.stdout: 1 693173 6.20 MB/sec execute 472 sec latency 58.423 ms 2024-04-25T00:30:03.274 INFO:tasks.workunit.client.0.smithi042.stdout: 1 694183 6.20 MB/sec execute 473 sec latency 28.481 ms 2024-04-25T00:30:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:03 smithi136 ceph-mon[32923]: pgmap v727: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-04-25T00:30:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:03 smithi106 ceph-mon[32510]: pgmap v727: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-04-25T00:30:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:03 smithi042 ceph-mon[28130]: pgmap v727: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 540 KiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-04-25T00:30:04.274 INFO:tasks.workunit.client.0.smithi042.stdout: 1 695698 6.20 MB/sec execute 474 sec latency 30.993 ms 2024-04-25T00:30:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.954+0000 7f9e50c00640 1 -- 172.21.15.42:0/1210963547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e4c1017a0 msgr2=0x7f9e4c101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.954+0000 7f9e50c00640 1 --2- 172.21.15.42:0/1210963547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e4c1017a0 0x7f9e4c101bc0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c009900 tx=0x7f9e3c02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.954+0000 7f9e50c00640 1 -- 172.21.15.42:0/1210963547 shutdown_connections 2024-04-25T00:30:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.954+0000 7f9e50c00640 1 --2- 172.21.15.42:0/1210963547 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e4c103350 0x7f9e4c109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.954+0000 7f9e50c00640 1 --2- 172.21.15.42:0/1210963547 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c102990 0x7f9e4c102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.954+0000 7f9e50c00640 1 --2- 172.21.15.42:0/1210963547 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e4c1017a0 0x7f9e4c101bc0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.954+0000 7f9e50c00640 1 -- 172.21.15.42:0/1210963547 >> 172.21.15.42:0/1210963547 conn(0x7f9e4c0fcd30 msgr2=0x7f9e4c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.954+0000 7f9e50c00640 1 -- 172.21.15.42:0/1210963547 shutdown_connections 2024-04-25T00:30:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.954+0000 7f9e50c00640 1 -- 172.21.15.42:0/1210963547 wait complete. 2024-04-25T00:30:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.955+0000 7f9e50c00640 1 Processor -- start 2024-04-25T00:30:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.955+0000 7f9e50c00640 1 -- start start 2024-04-25T00:30:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e50c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c1017a0 0x7f9e4c1979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e50c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e4c102990 0x7f9e4c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e50c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e4c103350 0x7f9e4c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e50c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c10d440 con 0x7f9e4c102990 2024-04-25T00:30:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e50c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c10d2c0 con 0x7f9e4c103350 2024-04-25T00:30:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e50c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c10d5c0 con 0x7f9e4c1017a0 2024-04-25T00:30:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e49e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c1017a0 0x7f9e4c1979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e49e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c1017a0 0x7f9e4c1979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47050/0 (socket says 172.21.15.42:47050) 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e4a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e4c103350 0x7f9e4c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e49e00640 1 -- 172.21.15.42:0/1745175954 learned_addr learned my addr 172.21.15.42:0/1745175954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e43e00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e4c102990 0x7f9e4c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e49e00640 1 -- 172.21.15.42:0/1745175954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e4c103350 msgr2=0x7f9e4c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.956+0000 7f9e49e00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e4c103350 0x7f9e4c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.957+0000 7f9e49e00640 1 -- 172.21.15.42:0/1745175954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e4c102990 msgr2=0x7f9e4c197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.957+0000 7f9e49e00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e4c102990 0x7f9e4c197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.957+0000 7f9e49e00640 1 -- 172.21.15.42:0/1745175954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c19f610 con 0x7f9e4c1017a0 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.957+0000 7f9e4a800640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e4c103350 0x7f9e4c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.957+0000 7f9e49e00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c1017a0 0x7f9e4c1979f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c031440 tx=0x7f9e3c032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.957+0000 7f9e42a00640 1 -- 172.21.15.42:0/1745175954 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c03f030 con 0x7f9e4c1017a0 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.957+0000 7f9e50c00640 1 -- 172.21.15.42:0/1745175954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4c19f8a0 con 0x7f9e4c1017a0 2024-04-25T00:30:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.957+0000 7f9e42a00640 1 -- 172.21.15.42:0/1745175954 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e3c003ce0 con 0x7f9e4c1017a0 2024-04-25T00:30:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.957+0000 7f9e42a00640 1 -- 172.21.15.42:0/1745175954 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c035070 con 0x7f9e4c1017a0 2024-04-25T00:30:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.957+0000 7f9e50c00640 1 -- 172.21.15.42:0/1745175954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4c19fde0 con 0x7f9e4c1017a0 2024-04-25T00:30:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.961+0000 7f9e42a00640 1 -- 172.21.15.42:0/1745175954 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e3c042030 con 0x7f9e4c1017a0 2024-04-25T00:30:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.961+0000 7f9e539d3740 1 Processor -- start 2024-04-25T00:30:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.961+0000 7f9e539d3740 1 -- start start 2024-04-25T00:30:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.961+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc20b0 0x56524ccc24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.961+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.961+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc7c10 0x56524ccca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.961+0000 7f9e539d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56524cc20ab0 con 0x56524ccc7c10 2024-04-25T00:30:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.961+0000 7f9e539d3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56524cbc9640 con 0x56524ccc2aa0 2024-04-25T00:30:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.961+0000 7f9e539d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56524cc1ca70 con 0x56524ccc20b0 2024-04-25T00:30:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e49e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e4a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc20b0 0x56524ccc24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e4a800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc20b0 0x56524ccc24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47062/0 (socket says 172.21.15.42:47062) 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e4a800640 1 -- 172.21.15.42:0/3106161133 learned_addr learned my addr 172.21.15.42:0/3106161133 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e49e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:41714/0 (socket says 172.21.15.42:41714) 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e43e00640 1 --2- 172.21.15.42:0/3106161133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc7c10 0x56524ccca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e4a800640 1 -- 172.21.15.42:0/3106161133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 msgr2=0x56524ccc75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e4a800640 1 --2- 172.21.15.42:0/3106161133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e4a800640 1 -- 172.21.15.42:0/3106161133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc7c10 msgr2=0x56524ccca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e4a800640 1 --2- 172.21.15.42:0/3106161133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc7c10 0x56524ccca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e4a800640 1 -- 172.21.15.42:0/3106161133 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524c935210 con 0x56524ccc20b0 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e49e00640 1 --2- 172.21.15.42:0/3106161133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.962+0000 7f9e4a800640 1 --2- 172.21.15.42:0/3106161133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc20b0 0x56524ccc24d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400f760 tx=0x7f9e4400fc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27734b93e701408c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e3b000640 1 -- 172.21.15.42:0/3106161133 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e44004d20 con 0x56524ccc20b0 2024-04-25T00:30:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e3b000640 1 -- 172.21.15.42:0/3106161133 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e44003970 con 0x56524ccc20b0 2024-04-25T00:30:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e3b000640 1 -- 172.21.15.42:0/3106161133 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e440043c0 con 0x56524ccc20b0 2024-04-25T00:30:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e42a00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e300a2cb0 0x7f9e300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e42a00640 1 -- 172.21.15.42:0/1745175954 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f9e3c0ef800 con 0x7f9e4c1017a0 2024-04-25T00:30:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e49e00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e300a2cb0 0x7f9e300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e539d3740 1 -- 172.21.15.42:0/3106161133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc20b0 msgr2=0x56524ccc24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e539d3740 1 --2- 172.21.15.42:0/3106161133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc20b0 0x56524ccc24d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9e4400f760 tx=0x7f9e4400fc20 comp rx=0 tx=0).stop 2024-04-25T00:30:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e539d3740 1 -- 172.21.15.42:0/3106161133 shutdown_connections 2024-04-25T00:30:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e539d3740 1 --2- 172.21.15.42:0/3106161133 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc7c10 0x56524ccca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.963+0000 7f9e539d3740 1 --2- 172.21.15.42:0/3106161133 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.964+0000 7f9e539d3740 1 --2- 172.21.15.42:0/3106161133 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc20b0 0x56524ccc24d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.964+0000 7f9e539d3740 1 -- 172.21.15.42:0/3106161133 >> 172.21.15.42:0/3106161133 conn(0x56524ccbd720 msgr2=0x56524ccbdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.964+0000 7f9e539d3740 1 -- 172.21.15.42:0/3106161133 shutdown_connections 2024-04-25T00:30:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.964+0000 7f9e539d3740 1 -- 172.21.15.42:0/3106161133 wait complete. 2024-04-25T00:30:04.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.964+0000 7f9e49e00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e300a2cb0 0x7f9e300a5170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0bbde0 tx=0x7f9e3c0f5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.964+0000 7f9e539d3740 1 -- start start 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.964+0000 7f9e539d3740 1 Processor -- start 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 0x56524ccc50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc7c10 0x56524ccc6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e539d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56524ccb8f40 con 0x56524ccc20b0 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e539d3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56524cbdc990 con 0x56524ccc2aa0 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e539d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56524ccb90c0 con 0x56524ccc7c10 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e539d3740 10 client.0 _ll_register_callbacks cb 0x56524cbde950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e4a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 0x56524ccc50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:41722/0 (socket says 172.21.15.42:41722) 2024-04-25T00:30:04.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e43e00640 1 -- 172.21.15.42:0/701630605 learned_addr learned my addr 172.21.15.42:0/701630605 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.965+0000 7f9e4a800640 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc7c10 0x56524ccc6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e43e00640 1 -- 172.21.15.42:0/701630605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc7c10 msgr2=0x56524ccc6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e43e00640 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc7c10 0x56524ccc6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e43e00640 1 -- 172.21.15.42:0/701630605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 msgr2=0x56524ccc50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e43e00640 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 0x56524ccc50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e43e00640 1 -- 172.21.15.42:0/701630605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524ca01620 con 0x56524ccc2aa0 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e4a800640 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc7c10 0x56524ccc6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e43e00640 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc55f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9e34012700 tx=0x7f9e34012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=472caf2bf08f7653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e2f400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e2f400640 1 -- 172.21.15.42:0/701630605 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400f8c0 con 0x56524ccc2aa0 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e539d3740 1 -- 172.21.15.42:0/701630605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e44003970 con 0x56524ccc2aa0 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.966+0000 7f9e2f400640 1 -- 172.21.15.42:0/701630605 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f9e3400fa60 con 0x56524ccc2aa0 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e2f400640 1 -- 172.21.15.42:0/701630605 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e3400fc60 con 0x56524ccc2aa0 2024-04-25T00:30:04.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e539d3740 10 client.26084 fetch_fsmap learned FSMap version 31 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e539d3740 1 -- 172.21.15.42:0/701630605 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56524ccd2530 con 0x56524ccc2aa0 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e2f400640 1 -- 172.21.15.42:0/701630605 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e44003970 con 0x56524ccc2aa0 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e2f400640 1 -- 172.21.15.42:0/701630605 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f9e34027420 con 0x56524ccc2aa0 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e539d3740 10 client.26084 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e539d3740 10 client.26084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e539d3740 20 client.26084 populate_metadata read hostname 'smithi042' 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e539d3740 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56524ccd3b70 0x56524ccd5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e539d3740 4 client.26084 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.967+0000 7f9e539d3740 1 -- 172.21.15.42:0/701630605 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56524cc2dff0 con 0x56524ccd3b70 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.968+0000 7f9e4a800640 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56524ccd3b70 0x56524ccd5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.968+0000 7f9e4a800640 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56524ccd3b70 0x56524ccd5fe0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a9143c383ac0482 server_cookie=f049079053fee9e1 in_seq=0 out_seq=0 2024-04-25T00:30:04.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.968+0000 7f9e2f400640 10 client.26084 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:04.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e2f400640 1 -- 172.21.15.42:0/701630605 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56524cc2dff0 con 0x56524ccd3b70 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e2f400640 10 client.26084 handle_command_reply: tid=0 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 client.26084 shutdown 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 -- 172.21.15.42:0/701630605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 msgr2=0x56524ccc55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc55f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f9e34012700 tx=0x7f9e34012bc0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 -- 172.21.15.42:0/701630605 shutdown_connections 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56524ccd3b70 0x56524ccd5fe0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc7c10 0x56524ccc6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc2aa0 0x56524ccc55f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e2f400640 0 client.26084 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 --2- 172.21.15.42:0/701630605 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 0x56524ccc50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 -- 172.21.15.42:0/701630605 >> 172.21.15.42:0/701630605 conn(0x56524ccbd720 msgr2=0x56524ccb8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:04.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 -- 172.21.15.42:0/701630605 shutdown_connections 2024-04-25T00:30:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 1 -- 172.21.15.42:0/701630605 wait complete. 2024-04-25T00:30:04.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.969+0000 7f9e539d3740 20 client.26084 trim_cache size 0 max 16384 2024-04-25T00:30:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.987+0000 7f9e539d3740 1 Processor -- start 2024-04-25T00:30:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.987+0000 7f9e539d3740 1 -- start start 2024-04-25T00:30:04.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.987+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc20b0 0x56524ccc3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.987+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccc3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.987+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc7c10 0x56524ccc0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.987+0000 7f9e539d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c1a91a0 con 0x56524ccc7c10 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.987+0000 7f9e539d3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c1a6a70 con 0x56524ccc20b0 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.987+0000 7f9e539d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c1a48c0 con 0x56524ccc2aa0 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccc3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e4a800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc20b0 0x56524ccc3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccc3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47078/0 (socket says 172.21.15.42:47078) 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e43e00640 1 -- 172.21.15.42:0/1282725849 learned_addr learned my addr 172.21.15.42:0/1282725849 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e4a800640 1 --2- 172.21.15.42:0/1282725849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc7c10 0x56524ccc0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e43e00640 1 -- 172.21.15.42:0/1282725849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc20b0 msgr2=0x56524ccc3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e43e00640 1 --2- 172.21.15.42:0/1282725849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc20b0 0x56524ccc3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e43e00640 1 -- 172.21.15.42:0/1282725849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc7c10 msgr2=0x56524ccc0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e43e00640 1 --2- 172.21.15.42:0/1282725849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc7c10 0x56524ccc0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e43e00640 1 -- 172.21.15.42:0/1282725849 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56524ccbd550 con 0x56524ccc2aa0 2024-04-25T00:30:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.988+0000 7f9e43e00640 1 --2- 172.21.15.42:0/1282725849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccc3c50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9e340058c0 tx=0x7f9e34009410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88f5ced25fe5d861 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.989+0000 7f9e3a600640 1 -- 172.21.15.42:0/1282725849 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e34027c10 con 0x56524ccc2aa0 2024-04-25T00:30:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.989+0000 7f9e3a600640 1 -- 172.21.15.42:0/1282725849 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e3402b020 con 0x56524ccc2aa0 2024-04-25T00:30:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.989+0000 7f9e3a600640 1 -- 172.21.15.42:0/1282725849 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3400f790 con 0x56524ccc2aa0 2024-04-25T00:30:04.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.989+0000 7f9e539d3740 1 -- 172.21.15.42:0/1282725849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 msgr2=0x56524ccc3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.989+0000 7f9e539d3740 1 --2- 172.21.15.42:0/1282725849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccc3c50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9e340058c0 tx=0x7f9e34009410 comp rx=0 tx=0).stop 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.989+0000 7f9e539d3740 1 -- 172.21.15.42:0/1282725849 shutdown_connections 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.989+0000 7f9e539d3740 1 --2- 172.21.15.42:0/1282725849 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc7c10 0x56524ccc0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.989+0000 7f9e539d3740 1 --2- 172.21.15.42:0/1282725849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccc3c50 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.989+0000 7f9e539d3740 1 --2- 172.21.15.42:0/1282725849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc20b0 0x56524ccc3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.989+0000 7f9e539d3740 1 -- 172.21.15.42:0/1282725849 >> 172.21.15.42:0/1282725849 conn(0x56524ccbd720 msgr2=0x56524ccb9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.990+0000 7f9e539d3740 1 -- 172.21.15.42:0/1282725849 shutdown_connections 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.990+0000 7f9e539d3740 1 -- 172.21.15.42:0/1282725849 wait complete. 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.990+0000 7f9e539d3740 1 -- start start 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.990+0000 7f9e539d3740 1 Processor -- start 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 0x56524cccdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e539d3740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc7c10 0x56524cccf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e539d3740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c1a4f30 con 0x56524ccc20b0 2024-04-25T00:30:04.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e539d3740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c1a63a0 con 0x56524ccc7c10 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e539d3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c1a5600 con 0x56524ccc2aa0 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e539d3740 10 client.0 _ll_register_callbacks cb 0x7f9e34009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e4a800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 0x56524cccdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e43e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47086/0 (socket says 172.21.15.42:47086) 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e43e00640 1 -- 172.21.15.42:0/332272788 learned_addr learned my addr 172.21.15.42:0/332272788 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e4a800640 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc7c10 0x56524cccf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e43e00640 1 -- 172.21.15.42:0/332272788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc7c10 msgr2=0x56524cccf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e43e00640 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc7c10 0x56524cccf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e43e00640 1 -- 172.21.15.42:0/332272788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 msgr2=0x56524cccdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e43e00640 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 0x56524cccdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e43e00640 1 -- 172.21.15.42:0/332272788 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56524cccf7c0 con 0x56524ccc2aa0 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.991+0000 7f9e4a800640 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 0x56524cccdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:04.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.992+0000 7f9e43e00640 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccce210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400f9a0 tx=0x7f9e3400f9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44469432da3dfaba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.992+0000 7f9e2f400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:30:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.992+0000 7f9e2f400640 1 -- 172.21.15.42:0/332272788 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e34005300 con 0x56524ccc2aa0 2024-04-25T00:30:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.992+0000 7f9e2f400640 1 -- 172.21.15.42:0/332272788 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f9e34018040 con 0x56524ccc2aa0 2024-04-25T00:30:04.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.992+0000 7f9e539d3740 1 -- 172.21.15.42:0/332272788 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e3402b020 con 0x56524ccc2aa0 2024-04-25T00:30:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.992+0000 7f9e2f400640 1 -- 172.21.15.42:0/332272788 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9e340054a0 con 0x56524ccc2aa0 2024-04-25T00:30:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.992+0000 7f9e539d3740 10 client.26098 fetch_fsmap learned FSMap version 31 2024-04-25T00:30:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.992+0000 7f9e539d3740 1 -- 172.21.15.42:0/332272788 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56524cd3a280 con 0x56524ccc2aa0 2024-04-25T00:30:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.993+0000 7f9e2f400640 1 -- 172.21.15.42:0/332272788 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e3402b020 con 0x56524ccc2aa0 2024-04-25T00:30:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.993+0000 7f9e2f400640 1 -- 172.21.15.42:0/332272788 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f9e34030460 con 0x56524ccc2aa0 2024-04-25T00:30:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.993+0000 7f9e539d3740 10 client.26098 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:30:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.993+0000 7f9e539d3740 10 client.26098 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:04.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.993+0000 7f9e539d3740 20 client.26098 populate_metadata read hostname 'smithi042' 2024-04-25T00:30:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.993+0000 7f9e539d3740 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56524cd3ab90 0x56524cd3cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.993+0000 7f9e539d3740 4 client.26098 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:30:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.993+0000 7f9e539d3740 1 -- 172.21.15.42:0/332272788 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56524cc28340 con 0x56524cd3ab90 2024-04-25T00:30:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.993+0000 7f9e4a800640 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56524cd3ab90 0x56524cd3cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.994+0000 7f9e4a800640 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56524cd3ab90 0x56524cd3cfb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad984f8e9ebfa9f2 server_cookie=60c9c8b73c0b556c in_seq=0 out_seq=0 2024-04-25T00:30:04.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:04.994+0000 7f9e2f400640 10 client.26098 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.062+0000 7f9e2f400640 1 -- 172.21.15.42:0/332272788 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56524cc28340 con 0x56524cd3ab90 2024-04-25T00:30:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.062+0000 7f9e2f400640 10 client.26098 handle_command_reply: tid=0 2024-04-25T00:30:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.062+0000 7f9e539d3740 1 client.26098 shutdown 2024-04-25T00:30:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.063+0000 7f9e539d3740 1 -- 172.21.15.42:0/332272788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 msgr2=0x56524ccce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.063+0000 7f9e539d3740 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccce210 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9e3400f9a0 tx=0x7f9e3400f9d0 comp rx=0 tx=0).stop 2024-04-25T00:30:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.063+0000 7f9e539d3740 1 -- 172.21.15.42:0/332272788 shutdown_connections 2024-04-25T00:30:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.063+0000 7f9e539d3740 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56524cd3ab90 0x56524cd3cfb0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.063+0000 7f9e539d3740 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56524ccc7c10 0x56524cccf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.063+0000 7f9e539d3740 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56524ccc2aa0 0x56524ccce210 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.063+0000 7f9e539d3740 1 --2- 172.21.15.42:0/332272788 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56524ccc20b0 0x56524cccdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.063+0000 7f9e2f400640 0 client.26098 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.063+0000 7f9e539d3740 1 -- 172.21.15.42:0/332272788 >> 172.21.15.42:0/332272788 conn(0x56524ccbd720 msgr2=0x56524ccc0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.064+0000 7f9e539d3740 1 -- 172.21.15.42:0/332272788 shutdown_connections 2024-04-25T00:30:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.064+0000 7f9e539d3740 1 -- 172.21.15.42:0/332272788 wait complete. 2024-04-25T00:30:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.064+0000 7f9e539d3740 20 client.26098 trim_cache size 0 max 16384 2024-04-25T00:30:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:30:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:30:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:30:05.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:30:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 -- 172.21.15.42:0/1745175954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e300a2cb0 msgr2=0x7f9e300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e300a2cb0 0x7f9e300a5170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0bbde0 tx=0x7f9e3c0f5530 comp rx=0 tx=0).stop 2024-04-25T00:30:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 -- 172.21.15.42:0/1745175954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c1017a0 msgr2=0x7f9e4c1979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c1017a0 0x7f9e4c1979f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c031440 tx=0x7f9e3c032ce0 comp rx=0 tx=0).stop 2024-04-25T00:30:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 -- 172.21.15.42:0/1745175954 shutdown_connections 2024-04-25T00:30:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f9e300a2cb0 0x7f9e300a5170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e4c103350 0x7f9e4c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:05.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9e4c102990 0x7f9e4c197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 --2- 172.21.15.42:0/1745175954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9e4c1017a0 0x7f9e4c1979f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 -- 172.21.15.42:0/1745175954 >> 172.21.15.42:0/1745175954 conn(0x7f9e4c0fcd30 msgr2=0x7f9e4c105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 -- 172.21.15.42:0/1745175954 shutdown_connections 2024-04-25T00:30:05.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:05.066+0000 7f9e50c00640 1 -- 172.21.15.42:0/1745175954 wait complete. 2024-04-25T00:30:05.274 INFO:tasks.workunit.client.0.smithi042.stdout: 1 696483 6.20 MB/sec execute 475 sec latency 56.881 ms 2024-04-25T00:30:06.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:05 smithi136 ceph-mon[32923]: pgmap v728: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-04-25T00:30:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:05 smithi106 ceph-mon[32510]: pgmap v728: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-04-25T00:30:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:05 smithi042 ceph-mon[28130]: pgmap v728: 97 pgs: 97 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 372 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-04-25T00:30:06.274 INFO:tasks.workunit.client.0.smithi042.stdout: 1 697509 6.20 MB/sec execute 476 sec latency 28.321 ms 2024-04-25T00:30:07.274 INFO:tasks.workunit.client.0.smithi042.stdout: 1 698648 6.19 MB/sec execute 477 sec latency 26.890 ms 2024-04-25T00:30:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:07 smithi136 ceph-mon[32923]: pgmap v729: 97 pgs: 97 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-04-25T00:30:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:07 smithi106 ceph-mon[32510]: pgmap v729: 97 pgs: 97 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-04-25T00:30:08.130 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:07 smithi042 ceph-mon[28130]: pgmap v729: 97 pgs: 97 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 512 KiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-04-25T00:30:08.274 INFO:tasks.workunit.client.0.smithi042.stdout: 1 699892 6.20 MB/sec execute 478 sec latency 76.527 ms 2024-04-25T00:30:09.274 INFO:tasks.workunit.client.0.smithi042.stdout: 1 700655 6.19 MB/sec execute 479 sec latency 17.732 ms 2024-04-25T00:30:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:09 smithi136 ceph-mon[32923]: pgmap v730: 97 pgs: 97 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-25T00:30:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:09 smithi106 ceph-mon[32510]: pgmap v730: 97 pgs: 97 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-25T00:30:10.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:09 smithi042 ceph-mon[28130]: pgmap v730: 97 pgs: 97 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 502 KiB/s rd, 1.5 MiB/s wr, 201 op/s 2024-04-25T00:30:10.275 INFO:tasks.workunit.client.0.smithi042.stdout: 1 702019 6.19 MB/sec execute 480 sec latency 28.101 ms 2024-04-25T00:30:11.275 INFO:tasks.workunit.client.0.smithi042.stdout: 1 703387 6.20 MB/sec execute 481 sec latency 17.748 ms 2024-04-25T00:30:12.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:11 smithi136 ceph-mon[32923]: pgmap v731: 97 pgs: 97 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-25T00:30:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:11 smithi106 ceph-mon[32510]: pgmap v731: 97 pgs: 97 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-25T00:30:12.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:11 smithi042 ceph-mon[28130]: pgmap v731: 97 pgs: 97 active+clean; 128 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2024-04-25T00:30:12.275 INFO:tasks.workunit.client.0.smithi042.stdout: 1 704017 6.19 MB/sec execute 482 sec latency 27.392 ms 2024-04-25T00:30:13.275 INFO:tasks.workunit.client.0.smithi042.stdout: 1 705109 6.19 MB/sec execute 483 sec latency 27.311 ms 2024-04-25T00:30:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:13 smithi136 ceph-mon[32923]: pgmap v732: 97 pgs: 97 active+clean; 130 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.7 MiB/s wr, 208 op/s 2024-04-25T00:30:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:13 smithi106 ceph-mon[32510]: pgmap v732: 97 pgs: 97 active+clean; 130 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.7 MiB/s wr, 208 op/s 2024-04-25T00:30:14.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:13 smithi042 ceph-mon[28130]: pgmap v732: 97 pgs: 97 active+clean; 130 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 1.7 MiB/s wr, 208 op/s 2024-04-25T00:30:14.275 INFO:tasks.workunit.client.0.smithi042.stdout: 1 706747 6.20 MB/sec execute 484 sec latency 23.216 ms 2024-04-25T00:30:15.275 INFO:tasks.workunit.client.0.smithi042.stdout: 1 707226 6.19 MB/sec execute 485 sec latency 35.778 ms 2024-04-25T00:30:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:15 smithi136 ceph-mon[32923]: pgmap v733: 97 pgs: 97 active+clean; 130 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-25T00:30:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:15 smithi106 ceph-mon[32510]: pgmap v733: 97 pgs: 97 active+clean; 130 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-25T00:30:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:15 smithi042 ceph-mon[28130]: pgmap v733: 97 pgs: 97 active+clean; 130 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 361 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-04-25T00:30:16.275 INFO:tasks.workunit.client.0.smithi042.stdout: 1 708295 6.19 MB/sec execute 486 sec latency 30.948 ms 2024-04-25T00:30:17.276 INFO:tasks.workunit.client.0.smithi042.stdout: 1 709934 6.20 MB/sec execute 487 sec latency 26.324 ms 2024-04-25T00:30:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:17 smithi136 ceph-mon[32923]: pgmap v734: 97 pgs: 97 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-04-25T00:30:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:30:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:17 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:17 smithi106 ceph-mon[32510]: pgmap v734: 97 pgs: 97 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-04-25T00:30:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:30:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:17 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:17 smithi042 ceph-mon[28130]: pgmap v734: 97 pgs: 97 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 1.5 MiB/s wr, 205 op/s 2024-04-25T00:30:18.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:30:18.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:17 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:18.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.231+0000 7f323ce00640 1 -- 172.21.15.42:0/2972728424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32381027c0 msgr2=0x7f3238102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.231+0000 7f323ce00640 1 --2- 172.21.15.42:0/2972728424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32381027c0 0x7f3238102c40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f3224009900 tx=0x7f322402a400 comp rx=0 tx=0).stop 2024-04-25T00:30:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.231+0000 7f323ce00640 1 -- 172.21.15.42:0/2972728424 shutdown_connections 2024-04-25T00:30:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.231+0000 7f323ce00640 1 --2- 172.21.15.42:0/2972728424 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3238103180 0x7f3238109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.231+0000 7f323ce00640 1 --2- 172.21.15.42:0/2972728424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32381027c0 0x7f3238102c40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.231+0000 7f323ce00640 1 --2- 172.21.15.42:0/2972728424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32381015d0 0x7f32381019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.231+0000 7f323ce00640 1 -- 172.21.15.42:0/2972728424 >> 172.21.15.42:0/2972728424 conn(0x7f32380fcb40 msgr2=0x7f32380fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.231+0000 7f323ce00640 1 -- 172.21.15.42:0/2972728424 shutdown_connections 2024-04-25T00:30:18.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.231+0000 7f323ce00640 1 -- 172.21.15.42:0/2972728424 wait complete. 2024-04-25T00:30:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.233+0000 7f323ce00640 1 Processor -- start 2024-04-25T00:30:18.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.233+0000 7f323ce00640 1 -- start start 2024-04-25T00:30:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.233+0000 7f323ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32381015d0 0x7f3238199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.233+0000 7f323ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32381027c0 0x7f323819a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.233+0000 7f323ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3238103180 0x7f32381a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.233+0000 7f323ce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f323810d2b0 con 0x7f32381015d0 2024-04-25T00:30:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.233+0000 7f323ce00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f323810d130 con 0x7f3238103180 2024-04-25T00:30:18.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.233+0000 7f323ce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f323810d430 con 0x7f32381027c0 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3236a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32381027c0 0x7f323819a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3237e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3238103180 0x7f32381a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3237400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32381015d0 0x7f3238199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3236a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32381027c0 0x7f323819a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:35888/0 (socket says 172.21.15.42:35888) 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3236a00640 1 -- 172.21.15.42:0/78122964 learned_addr learned my addr 172.21.15.42:0/78122964 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3237e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3238103180 0x7f32381a10c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35424/0 (socket says 172.21.15.42:35424) 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3236a00640 1 -- 172.21.15.42:0/78122964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3238103180 msgr2=0x7f32381a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3236a00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3238103180 0x7f32381a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3236a00640 1 -- 172.21.15.42:0/78122964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32381015d0 msgr2=0x7f3238199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3236a00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32381015d0 0x7f3238199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3236a00640 1 -- 172.21.15.42:0/78122964 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32381a1730 con 0x7f32381027c0 2024-04-25T00:30:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3237e00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3238103180 0x7f32381a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.234+0000 7f3237400640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32381015d0 0x7f3238199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.235+0000 7f3236a00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32381027c0 0x7f323819a050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3224031440 tx=0x7f3224032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.235+0000 7f322fe00640 1 -- 172.21.15.42:0/78122964 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322403f030 con 0x7f32381027c0 2024-04-25T00:30:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.235+0000 7f323ce00640 1 -- 172.21.15.42:0/78122964 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32381a19c0 con 0x7f32381027c0 2024-04-25T00:30:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.235+0000 7f322fe00640 1 -- 172.21.15.42:0/78122964 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3224003ce0 con 0x7f32381027c0 2024-04-25T00:30:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.236+0000 7f322fe00640 1 -- 172.21.15.42:0/78122964 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3224035070 con 0x7f32381027c0 2024-04-25T00:30:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.236+0000 7f323ce00640 1 -- 172.21.15.42:0/78122964 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32381a1f00 con 0x7f32381027c0 2024-04-25T00:30:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.239+0000 7f3241200740 1 Processor -- start 2024-04-25T00:30:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.239+0000 7f3241200740 1 -- start start 2024-04-25T00:30:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.239+0000 7f3241200740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40ba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3241200740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 0x55efb40bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3241200740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40bf9b0 0x55efb40c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3241200740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55efb401cab0 con 0x55efb40bf9b0 2024-04-25T00:30:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3241200740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55efb3fc5640 con 0x55efb40b9e50 2024-04-25T00:30:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3241200740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55efb4018a70 con 0x55efb40ba840 2024-04-25T00:30:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3237e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3237e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40ba270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35428/0 (socket says 172.21.15.42:35428) 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3237e00640 1 -- 172.21.15.42:0/111269080 learned_addr learned my addr 172.21.15.42:0/111269080 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3237400640 1 --2- 172.21.15.42:0/111269080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 0x55efb40bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3237e00640 1 -- 172.21.15.42:0/111269080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 msgr2=0x55efb40bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3237e00640 1 --2- 172.21.15.42:0/111269080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 0x55efb40bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3237e00640 1 -- 172.21.15.42:0/111269080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40bf9b0 msgr2=0x55efb40c1e40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3237e00640 1 --2- 172.21.15.42:0/111269080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40bf9b0 0x55efb40c1e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3237e00640 1 -- 172.21.15.42:0/111269080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb3d31210 con 0x55efb40b9e50 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.240+0000 7f3237e00640 1 --2- 172.21.15.42:0/111269080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40ba270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f32300107e0 tx=0x7f3230010ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0dff1f539901908 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.241+0000 7f322cc00640 1 -- 172.21.15.42:0/111269080 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230003b70 con 0x55efb40b9e50 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.241+0000 7f322cc00640 1 -- 172.21.15.42:0/111269080 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f323000ee10 con 0x55efb40b9e50 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.241+0000 7f322cc00640 1 -- 172.21.15.42:0/111269080 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230011440 con 0x55efb40b9e50 2024-04-25T00:30:18.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.241+0000 7f322fe00640 1 -- 172.21.15.42:0/78122964 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3224042030 con 0x7f32381027c0 2024-04-25T00:30:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.242+0000 7f3241200740 1 -- 172.21.15.42:0/111269080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 msgr2=0x55efb40ba270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.242+0000 7f3241200740 1 --2- 172.21.15.42:0/111269080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40ba270 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f32300107e0 tx=0x7f3230010ca0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.242+0000 7f3241200740 1 -- 172.21.15.42:0/111269080 shutdown_connections 2024-04-25T00:30:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.242+0000 7f3241200740 1 --2- 172.21.15.42:0/111269080 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40bf9b0 0x55efb40c1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.242+0000 7f3241200740 1 --2- 172.21.15.42:0/111269080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 0x55efb40bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.242+0000 7f3241200740 1 --2- 172.21.15.42:0/111269080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40ba270 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.242+0000 7f3241200740 1 -- 172.21.15.42:0/111269080 >> 172.21.15.42:0/111269080 conn(0x55efb40b54c0 msgr2=0x55efb40b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.242+0000 7f3241200740 1 -- 172.21.15.42:0/111269080 shutdown_connections 2024-04-25T00:30:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.242+0000 7f3241200740 1 -- 172.21.15.42:0/111269080 wait complete. 2024-04-25T00:30:18.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.243+0000 7f322fe00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f321c0a2a90 0x7f321c0a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.243+0000 7f322fe00640 1 -- 172.21.15.42:0/78122964 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f32240ef210 con 0x7f32381027c0 2024-04-25T00:30:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.243+0000 7f3241200740 1 -- start start 2024-04-25T00:30:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.243+0000 7f3237400640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f321c0a2a90 0x7f321c0a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.243+0000 7f3241200740 1 Processor -- start 2024-04-25T00:30:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.244+0000 7f3241200740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.244+0000 7f3241200740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 0x55efb40cce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.244+0000 7f3241200740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40bf9b0 0x55efb40d3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.244+0000 7f3241200740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55efb40c2960 con 0x55efb40bf9b0 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.244+0000 7f3237400640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f321c0a2a90 0x7f321c0a4f50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55efb40b6390 tx=0x7f3228005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.244+0000 7f3241200740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55efb3fd8990 con 0x55efb40b9e50 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.244+0000 7f3241200740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55efb40c2ae0 con 0x55efb40ba840 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.244+0000 7f3241200740 10 client.0 _ll_register_callbacks cb 0x55efb3fda950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.244+0000 7f3237400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.245+0000 7f3237400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35430/0 (socket says 172.21.15.42:35430) 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.245+0000 7f3237400640 1 -- 172.21.15.42:0/3242942976 learned_addr learned my addr 172.21.15.42:0/3242942976 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.245+0000 7f3237e00640 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40bf9b0 0x55efb40d3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.245+0000 7f3236a00640 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 0x55efb40cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.245+0000 7f3236a00640 1 -- 172.21.15.42:0/3242942976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 msgr2=0x55efb40cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.245+0000 7f3236a00640 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.245+0000 7f3236a00640 1 -- 172.21.15.42:0/3242942976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40bf9b0 msgr2=0x55efb40d3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.245+0000 7f3236a00640 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40bf9b0 0x55efb40d3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.245+0000 7f3236a00640 1 -- 172.21.15.42:0/3242942976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb3dfd620 con 0x55efb40ba840 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.245+0000 7f3237400640 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.246+0000 7f3236a00640 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 0x55efb40cce60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3224000c00 tx=0x7f32240bb710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3fe917bec0b582 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.246+0000 7f3221600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.246+0000 7f3221600640 1 -- 172.21.15.42:0/3242942976 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32240d4060 con 0x55efb40ba840 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.246+0000 7f3221600640 1 -- 172.21.15.42:0/3242942976 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f322403bd40 con 0x55efb40ba840 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.246+0000 7f3241200740 1 -- 172.21.15.42:0/3242942976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f323000ee10 con 0x55efb40ba840 2024-04-25T00:30:18.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.246+0000 7f3221600640 1 -- 172.21.15.42:0/3242942976 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3224049040 con 0x55efb40ba840 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.246+0000 7f3241200740 10 client.26110 fetch_fsmap learned FSMap version 31 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.246+0000 7f3241200740 1 -- 172.21.15.42:0/3242942976 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55efb40c2a20 con 0x55efb40ba840 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.246+0000 7f3221600640 1 -- 172.21.15.42:0/3242942976 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f323000ee10 con 0x55efb40ba840 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.246+0000 7f3221600640 1 -- 172.21.15.42:0/3242942976 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f322404c060 con 0x55efb40ba840 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.247+0000 7f3241200740 10 client.26110 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.247+0000 7f3241200740 10 client.26110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.247+0000 7f3241200740 20 client.26110 populate_metadata read hostname 'smithi042' 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.247+0000 7f3241200740 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55efb40d84f0 0x55efb40da8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.247+0000 7f3241200740 4 client.26110 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.247+0000 7f3241200740 1 -- 172.21.15.42:0/3242942976 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efb4029ff0 con 0x55efb40d84f0 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.247+0000 7f3237400640 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55efb40d84f0 0x55efb40da8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.248+0000 7f3237400640 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55efb40d84f0 0x55efb40da8e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7b870433cc66bc7 server_cookie=33a480f81b8ce7f5 in_seq=0 out_seq=0 2024-04-25T00:30:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.248+0000 7f3221600640 10 client.26110 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.248+0000 7f3221600640 1 -- 172.21.15.42:0/3242942976 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55efb4029ff0 con 0x55efb40d84f0 2024-04-25T00:30:18.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.248+0000 7f3221600640 10 client.26110 handle_command_reply: tid=0 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.249+0000 7f3241200740 1 client.26110 shutdown 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.249+0000 7f3241200740 1 -- 172.21.15.42:0/3242942976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 msgr2=0x55efb40cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.249+0000 7f3241200740 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 0x55efb40cce60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3224000c00 tx=0x7f32240bb710 comp rx=0 tx=0).stop 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.250+0000 7f3241200740 1 -- 172.21.15.42:0/3242942976 shutdown_connections 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.250+0000 7f3241200740 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55efb40d84f0 0x55efb40da8e0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.250+0000 7f3241200740 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40bf9b0 0x55efb40d3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.250+0000 7f3241200740 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40ba840 0x55efb40cce60 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.250+0000 7f3241200740 1 --2- 172.21.15.42:0/3242942976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40b9e50 0x55efb40cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.250+0000 7f3221600640 0 client.26110 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.250+0000 7f3241200740 1 -- 172.21.15.42:0/3242942976 >> 172.21.15.42:0/3242942976 conn(0x55efb40b54c0 msgr2=0x55efb40b4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.250+0000 7f3241200740 1 -- 172.21.15.42:0/3242942976 shutdown_connections 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.250+0000 7f3241200740 1 -- 172.21.15.42:0/3242942976 wait complete. 2024-04-25T00:30:18.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.250+0000 7f3241200740 20 client.26110 trim_cache size 0 max 16384 2024-04-25T00:30:18.275 INFO:tasks.workunit.client.0.smithi042.stdout: 1 710697 6.19 MB/sec execute 488 sec latency 19.440 ms 2024-04-25T00:30:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.286+0000 7f3241200740 1 Processor -- start 2024-04-25T00:30:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.286+0000 7f3241200740 1 -- start start 2024-04-25T00:30:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3241200740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40bbb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3241200740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 0x55efb40bc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3241200740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40bf9b0 0x55efb40b83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3241200740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32381a90a0 con 0x55efb40ba840 2024-04-25T00:30:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3241200740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f32381a6970 con 0x55efb40bf9b0 2024-04-25T00:30:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3241200740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32381a47e0 con 0x55efb40b9e50 2024-04-25T00:30:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3237400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40bbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3237400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40bbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:35908/0 (socket says 172.21.15.42:35908) 2024-04-25T00:30:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3237400640 1 -- 172.21.15.42:0/4094220696 learned_addr learned my addr 172.21.15.42:0/4094220696 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3237e00640 1 --2- 172.21.15.42:0/4094220696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40bf9b0 0x55efb40b83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.287+0000 7f3236a00640 1 --2- 172.21.15.42:0/4094220696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 0x55efb40bc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.288+0000 7f3237400640 1 -- 172.21.15.42:0/4094220696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40bf9b0 msgr2=0x55efb40b83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.288+0000 7f3237400640 1 --2- 172.21.15.42:0/4094220696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40bf9b0 0x55efb40b83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.288+0000 7f3237400640 1 -- 172.21.15.42:0/4094220696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 msgr2=0x55efb40bc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.288+0000 7f3237400640 1 --2- 172.21.15.42:0/4094220696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 0x55efb40bc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.288+0000 7f3237400640 1 -- 172.21.15.42:0/4094220696 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb40b9a90 con 0x55efb40b9e50 2024-04-25T00:30:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.288+0000 7f3237400640 1 --2- 172.21.15.42:0/4094220696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40bbb00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3228009530 tx=0x7f322801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56a45b8da65d8085 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.288+0000 7f3223e00640 1 -- 172.21.15.42:0/4094220696 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228031070 con 0x55efb40b9e50 2024-04-25T00:30:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.288+0000 7f3223e00640 1 -- 172.21.15.42:0/4094220696 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f322801f420 con 0x55efb40b9e50 2024-04-25T00:30:18.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.288+0000 7f3223e00640 1 -- 172.21.15.42:0/4094220696 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f322801d670 con 0x55efb40b9e50 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.289+0000 7f3241200740 1 -- 172.21.15.42:0/4094220696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 msgr2=0x55efb40bbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.289+0000 7f3241200740 1 --2- 172.21.15.42:0/4094220696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40bbb00 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3228009530 tx=0x7f322801a000 comp rx=0 tx=0).stop 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.290+0000 7f3241200740 1 -- 172.21.15.42:0/4094220696 shutdown_connections 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.290+0000 7f3241200740 1 --2- 172.21.15.42:0/4094220696 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40bf9b0 0x55efb40b83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.290+0000 7f3241200740 1 --2- 172.21.15.42:0/4094220696 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 0x55efb40bc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.290+0000 7f3241200740 1 --2- 172.21.15.42:0/4094220696 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40bbb00 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.290+0000 7f3241200740 1 -- 172.21.15.42:0/4094220696 >> 172.21.15.42:0/4094220696 conn(0x55efb40b54c0 msgr2=0x55efb40b58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.290+0000 7f3241200740 1 -- 172.21.15.42:0/4094220696 shutdown_connections 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.290+0000 7f3241200740 1 -- 172.21.15.42:0/4094220696 wait complete. 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.290+0000 7f3241200740 1 -- start start 2024-04-25T00:30:18.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.291+0000 7f3241200740 1 Processor -- start 2024-04-25T00:30:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.291+0000 7f3241200740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.291+0000 7f3241200740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 0x55efb40d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.291+0000 7f3241200740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40bf9b0 0x55efb40d2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.291+0000 7f3241200740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f32381a4e30 con 0x55efb40ba840 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.291+0000 7f3241200740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f32381a62a0 con 0x55efb40bf9b0 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.291+0000 7f3241200740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32381a5500 con 0x55efb40b9e50 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.291+0000 7f3241200740 10 client.0 _ll_register_callbacks cb 0x7f32240bb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3237400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3237400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:35920/0 (socket says 172.21.15.42:35920) 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3237400640 1 -- 172.21.15.42:0/105158053 learned_addr learned my addr 172.21.15.42:0/105158053 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3237e00640 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40bf9b0 0x55efb40d2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3236a00640 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 0x55efb40d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3237400640 1 -- 172.21.15.42:0/105158053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40bf9b0 msgr2=0x55efb40d2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3237400640 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40bf9b0 0x55efb40d2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3237400640 1 -- 172.21.15.42:0/105158053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 msgr2=0x55efb40d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3237400640 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 0x55efb40d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3237400640 1 -- 172.21.15.42:0/105158053 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb40daf60 con 0x55efb40b9e50 2024-04-25T00:30:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.292+0000 7f3236a00640 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 0x55efb40d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.293+0000 7f3237400640 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40d0da0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f322801fd80 tx=0x7f3228003a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=deb8c04c6bd2308f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.293+0000 7f3221600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:30:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.293+0000 7f3221600640 1 -- 172.21.15.42:0/105158053 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3228031040 con 0x55efb40b9e50 2024-04-25T00:30:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.293+0000 7f3221600640 1 -- 172.21.15.42:0/105158053 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f322802b870 con 0x55efb40b9e50 2024-04-25T00:30:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.293+0000 7f3241200740 1 -- 172.21.15.42:0/105158053 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f322801f420 con 0x55efb40b9e50 2024-04-25T00:30:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.293+0000 7f3221600640 1 -- 172.21.15.42:0/105158053 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f322801f5d0 con 0x55efb40b9e50 2024-04-25T00:30:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.293+0000 7f3241200740 10 client.26116 fetch_fsmap learned FSMap version 31 2024-04-25T00:30:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.293+0000 7f3241200740 1 -- 172.21.15.42:0/105158053 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55efb413ea60 con 0x55efb40b9e50 2024-04-25T00:30:18.295 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.293+0000 7f3221600640 1 -- 172.21.15.42:0/105158053 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f322801f420 con 0x55efb40b9e50 2024-04-25T00:30:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.294+0000 7f3221600640 1 -- 172.21.15.42:0/105158053 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f3228004750 con 0x55efb40b9e50 2024-04-25T00:30:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.294+0000 7f3241200740 10 client.26116 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:30:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.294+0000 7f3241200740 10 client.26116 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.294+0000 7f3241200740 20 client.26116 populate_metadata read hostname 'smithi042' 2024-04-25T00:30:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.294+0000 7f3241200740 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55efb413f430 0x55efb4141850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.294+0000 7f3241200740 4 client.26116 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:30:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.294+0000 7f3241200740 1 -- 172.21.15.42:0/105158053 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55efb4024340 con 0x55efb413f430 2024-04-25T00:30:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.294+0000 7f3236a00640 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55efb413f430 0x55efb4141850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.295+0000 7f3236a00640 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55efb413f430 0x55efb4141850 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc32380e19366ded server_cookie=61d1ab8a9166ac1 in_seq=0 out_seq=0 2024-04-25T00:30:18.296 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.295+0000 7f3221600640 10 client.26116 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.372+0000 7f3221600640 1 -- 172.21.15.42:0/105158053 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55efb4024340 con 0x55efb413f430 2024-04-25T00:30:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.372+0000 7f3221600640 10 client.26116 handle_command_reply: tid=0 2024-04-25T00:30:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.372+0000 7f3241200740 1 client.26116 shutdown 2024-04-25T00:30:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.372+0000 7f3241200740 1 -- 172.21.15.42:0/105158053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 msgr2=0x55efb40d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.372+0000 7f3241200740 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40d0da0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f322801fd80 tx=0x7f3228003a00 comp rx=0 tx=0).stop 2024-04-25T00:30:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.372+0000 7f3241200740 1 -- 172.21.15.42:0/105158053 shutdown_connections 2024-04-25T00:30:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.372+0000 7f3241200740 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55efb413f430 0x55efb4141850 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.372+0000 7f3241200740 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55efb40bf9b0 0x55efb40d2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.372+0000 7f3241200740 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55efb40ba840 0x55efb40d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.372+0000 7f3241200740 1 --2- 172.21.15.42:0/105158053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55efb40b9e50 0x55efb40d0da0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.373+0000 7f3241200740 1 -- 172.21.15.42:0/105158053 >> 172.21.15.42:0/105158053 conn(0x55efb40b54c0 msgr2=0x55efb40cc920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.373+0000 7f3221600640 0 client.26116 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.373+0000 7f3241200740 1 -- 172.21.15.42:0/105158053 shutdown_connections 2024-04-25T00:30:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.373+0000 7f3241200740 1 -- 172.21.15.42:0/105158053 wait complete. 2024-04-25T00:30:18.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.373+0000 7f3241200740 20 client.26116 trim_cache size 0 max 16384 2024-04-25T00:30:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:30:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:30:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:30:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:30:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 -- 172.21.15.42:0/78122964 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f321c0a2a90 msgr2=0x7f321c0a4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f321c0a2a90 0x7f321c0a4f50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55efb40b6390 tx=0x7f3228005f90 comp rx=0 tx=0).stop 2024-04-25T00:30:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 -- 172.21.15.42:0/78122964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32381027c0 msgr2=0x7f323819a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32381027c0 0x7f323819a050 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3224031440 tx=0x7f3224032ce0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 -- 172.21.15.42:0/78122964 shutdown_connections 2024-04-25T00:30:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f321c0a2a90 0x7f321c0a4f50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3238103180 0x7f32381a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32381027c0 0x7f323819a050 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 --2- 172.21.15.42:0/78122964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f32381015d0 0x7f3238199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 -- 172.21.15.42:0/78122964 >> 172.21.15.42:0/78122964 conn(0x7f32380fcb40 msgr2=0x7f3238073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 -- 172.21.15.42:0/78122964 shutdown_connections 2024-04-25T00:30:18.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:18.374+0000 7f323ce00640 1 -- 172.21.15.42:0/78122964 wait complete. 2024-04-25T00:30:19.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:18 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:30:19.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:18 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:30:19.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:18 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:19.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:18 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:30:19.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:18 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:30:19.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:18 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:19.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:18 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:30:19.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:18 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:30:19.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:18 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:30:19.275 INFO:tasks.workunit.client.0.smithi042.stdout: 1 711756 6.19 MB/sec execute 489 sec latency 29.789 ms 2024-04-25T00:30:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:19 smithi106 ceph-mon[32510]: pgmap v735: 97 pgs: 97 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-04-25T00:30:20.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:19 smithi042 ceph-mon[28130]: pgmap v735: 97 pgs: 97 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-04-25T00:30:20.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:19 smithi136 ceph-mon[32923]: pgmap v735: 97 pgs: 97 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 388 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-04-25T00:30:20.275 INFO:tasks.workunit.client.0.smithi042.stdout: 1 713121 6.19 MB/sec execute 490 sec latency 32.894 ms 2024-04-25T00:30:21.276 INFO:tasks.workunit.client.0.smithi042.stdout: 1 714139 6.20 MB/sec execute 491 sec latency 29.414 ms 2024-04-25T00:30:22.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:21 smithi106 ceph-mon[32510]: pgmap v736: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-04-25T00:30:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:21 smithi042 ceph-mon[28130]: pgmap v736: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-04-25T00:30:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:21 smithi136 ceph-mon[32923]: pgmap v736: 97 pgs: 97 active+clean; 133 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-04-25T00:30:22.276 INFO:tasks.workunit.client.0.smithi042.stdout: 1 714824 6.19 MB/sec execute 492 sec latency 27.151 ms 2024-04-25T00:30:23.276 INFO:tasks.workunit.client.0.smithi042.stdout: 1 716112 6.18 MB/sec execute 493 sec latency 25.574 ms 2024-04-25T00:30:24.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:23 smithi106 ceph-mon[32510]: pgmap v737: 97 pgs: 97 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-04-25T00:30:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:23 smithi042 ceph-mon[28130]: pgmap v737: 97 pgs: 97 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-04-25T00:30:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:23 smithi136 ceph-mon[32923]: pgmap v737: 97 pgs: 97 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-04-25T00:30:24.276 INFO:tasks.workunit.client.0.smithi042.stdout: 1 717527 6.20 MB/sec execute 494 sec latency 24.187 ms 2024-04-25T00:30:25.276 INFO:tasks.workunit.client.0.smithi042.stdout: 1 718046 6.18 MB/sec execute 495 sec latency 22.742 ms 2024-04-25T00:30:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:25 smithi106 ceph-mon[32510]: pgmap v738: 97 pgs: 97 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1005 KiB/s wr, 143 op/s 2024-04-25T00:30:26.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:25 smithi042 ceph-mon[28130]: pgmap v738: 97 pgs: 97 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1005 KiB/s wr, 143 op/s 2024-04-25T00:30:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:25 smithi136 ceph-mon[32923]: pgmap v738: 97 pgs: 97 active+clean; 131 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 353 KiB/s rd, 1005 KiB/s wr, 143 op/s 2024-04-25T00:30:26.276 INFO:tasks.workunit.client.0.smithi042.stdout: 1 719269 6.18 MB/sec execute 496 sec latency 37.316 ms 2024-04-25T00:30:27.276 INFO:tasks.workunit.client.0.smithi042.stdout: 1 720974 6.19 MB/sec execute 497 sec latency 18.932 ms 2024-04-25T00:30:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:27 smithi106 ceph-mon[32510]: pgmap v739: 97 pgs: 97 active+clean; 130 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-25T00:30:28.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:27 smithi042 ceph-mon[28130]: pgmap v739: 97 pgs: 97 active+clean; 130 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-25T00:30:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:27 smithi136 ceph-mon[32923]: pgmap v739: 97 pgs: 97 active+clean; 130 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 435 KiB/s rd, 1.4 MiB/s wr, 180 op/s 2024-04-25T00:30:28.277 INFO:tasks.workunit.client.0.smithi042.stdout: 1 721450 6.18 MB/sec execute 498 sec latency 19.181 ms 2024-04-25T00:30:29.277 INFO:tasks.workunit.client.0.smithi042.stdout: 1 722496 6.18 MB/sec execute 499 sec latency 28.255 ms 2024-04-25T00:30:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:29 smithi106 ceph-mon[32510]: pgmap v740: 97 pgs: 97 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-25T00:30:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:29 smithi042 ceph-mon[28130]: pgmap v740: 97 pgs: 97 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-25T00:30:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:29 smithi136 ceph-mon[32923]: pgmap v740: 97 pgs: 97 active+clean; 132 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-25T00:30:30.277 INFO:tasks.workunit.client.0.smithi042.stdout: 1 724057 6.19 MB/sec execute 500 sec latency 25.116 ms 2024-04-25T00:30:31.277 INFO:tasks.workunit.client.0.smithi042.stdout: 1 724811 6.19 MB/sec execute 501 sec latency 80.986 ms 2024-04-25T00:30:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.531+0000 7f973ee00640 1 -- 172.21.15.42:0/180930967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9738101600 msgr2=0x7f9738101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.531+0000 7f973ee00640 1 --2- 172.21.15.42:0/180930967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9738101600 0x7f9738101a20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f972c009900 tx=0x7f972c02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.531+0000 7f973ee00640 1 -- 172.21.15.42:0/180930967 shutdown_connections 2024-04-25T00:30:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.531+0000 7f973ee00640 1 --2- 172.21.15.42:0/180930967 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97381031b0 0x7f9738109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.531+0000 7f973ee00640 1 --2- 172.21.15.42:0/180930967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97381027f0 0x7f9738102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.531+0000 7f973ee00640 1 --2- 172.21.15.42:0/180930967 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9738101600 0x7f9738101a20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.531+0000 7f973ee00640 1 -- 172.21.15.42:0/180930967 >> 172.21.15.42:0/180930967 conn(0x7f97380fcb70 msgr2=0x7f97380fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.532+0000 7f973ee00640 1 -- 172.21.15.42:0/180930967 shutdown_connections 2024-04-25T00:30:31.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.532+0000 7f973ee00640 1 -- 172.21.15.42:0/180930967 wait complete. 2024-04-25T00:30:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.532+0000 7f973ee00640 1 Processor -- start 2024-04-25T00:30:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973ee00640 1 -- start start 2024-04-25T00:30:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973ee00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9738101600 0x7f9738199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973ee00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97381027f0 0x7f973819a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973ee00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97381031b0 0x7f97381a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973ee00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97380751f0 con 0x7f97381031b0 2024-04-25T00:30:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973ee00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f9738075070 con 0x7f9738101600 2024-04-25T00:30:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973ee00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9738075370 con 0x7f97381027f0 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9738101600 0x7f9738199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9738101600 0x7f9738199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60260/0 (socket says 172.21.15.42:60260) 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973da00640 1 -- 172.21.15.42:0/2130649592 learned_addr learned my addr 172.21.15.42:0/2130649592 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973e400640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97381031b0 0x7f97381a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.533+0000 7f973d000640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97381027f0 0x7f973819a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f973da00640 1 -- 172.21.15.42:0/2130649592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97381027f0 msgr2=0x7f973819a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f973da00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97381027f0 0x7f973819a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f973da00640 1 -- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97381031b0 msgr2=0x7f97381a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f973da00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97381031b0 0x7f97381a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f973da00640 1 -- 172.21.15.42:0/2130649592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97381a1720 con 0x7f9738101600 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f973e400640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97381031b0 0x7f97381a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f973da00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9738101600 0x7f9738199b00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f972c031440 tx=0x7f972c032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f9736000640 1 -- 172.21.15.42:0/2130649592 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c03f030 con 0x7f9738101600 2024-04-25T00:30:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f9736000640 1 -- 172.21.15.42:0/2130649592 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f972c003ce0 con 0x7f9738101600 2024-04-25T00:30:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f9736000640 1 -- 172.21.15.42:0/2130649592 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f972c035070 con 0x7f9738101600 2024-04-25T00:30:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.534+0000 7f973ee00640 1 -- 172.21.15.42:0/2130649592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97381a19b0 con 0x7f9738101600 2024-04-25T00:30:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.535+0000 7f973ee00640 1 -- 172.21.15.42:0/2130649592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97381a1ef0 con 0x7f9738101600 2024-04-25T00:30:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.538+0000 7f9736000640 1 -- 172.21.15.42:0/2130649592 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f972c042030 con 0x7f9738101600 2024-04-25T00:30:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.538+0000 7f974339e740 1 Processor -- start 2024-04-25T00:30:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.538+0000 7f974339e740 1 -- start start 2024-04-25T00:30:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f974339e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c51270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f974339e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c51840 0x556174c56340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f974339e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c569b0 0x556174c58e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f974339e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556174bb3ab0 con 0x556174c569b0 2024-04-25T00:30:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f974339e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556174b5c640 con 0x556174c50e50 2024-04-25T00:30:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f974339e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556174bafa70 con 0x556174c51840 2024-04-25T00:30:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f973e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c51270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f973e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c51270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60268/0 (socket says 172.21.15.42:60268) 2024-04-25T00:30:31.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f973e400640 1 -- 172.21.15.42:0/3814838339 learned_addr learned my addr 172.21.15.42:0/3814838339 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f973da00640 1 --2- 172.21.15.42:0/3814838339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c51840 0x556174c56340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f973d000640 1 --2- 172.21.15.42:0/3814838339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c569b0 0x556174c58e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f973e400640 1 -- 172.21.15.42:0/3814838339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c51840 msgr2=0x556174c56340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f973e400640 1 --2- 172.21.15.42:0/3814838339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c51840 0x556174c56340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f973e400640 1 -- 172.21.15.42:0/3814838339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c569b0 msgr2=0x556174c58e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f973e400640 1 --2- 172.21.15.42:0/3814838339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c569b0 0x556174c58e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.539+0000 7f973e400640 1 -- 172.21.15.42:0/3814838339 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561748c8210 con 0x556174c50e50 2024-04-25T00:30:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.540+0000 7f973e400640 1 --2- 172.21.15.42:0/3814838339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c51270 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9730009f30 tx=0x7f9730010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c5ff28c0cdb884a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.540+0000 7f972aa00640 1 -- 172.21.15.42:0/3814838339 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730016070 con 0x556174c50e50 2024-04-25T00:30:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.540+0000 7f972aa00640 1 -- 172.21.15.42:0/3814838339 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f973001bb40 con 0x556174c50e50 2024-04-25T00:30:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.540+0000 7f972aa00640 1 -- 172.21.15.42:0/3814838339 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730012cb0 con 0x556174c50e50 2024-04-25T00:30:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.540+0000 7f9736000640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f97200a2c00 0x7f97200a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.540+0000 7f9736000640 1 -- 172.21.15.42:0/2130649592 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f972c0ef720 con 0x7f9738101600 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f973da00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f97200a2c00 0x7f97200a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f974339e740 1 -- 172.21.15.42:0/3814838339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 msgr2=0x556174c51270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f974339e740 1 --2- 172.21.15.42:0/3814838339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c51270 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f9730009f30 tx=0x7f9730010800 comp rx=0 tx=0).stop 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f974339e740 1 -- 172.21.15.42:0/3814838339 shutdown_connections 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f974339e740 1 --2- 172.21.15.42:0/3814838339 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c569b0 0x556174c58e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f974339e740 1 --2- 172.21.15.42:0/3814838339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c51840 0x556174c56340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f974339e740 1 --2- 172.21.15.42:0/3814838339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c51270 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f974339e740 1 -- 172.21.15.42:0/3814838339 >> 172.21.15.42:0/3814838339 conn(0x556174c4c4c0 msgr2=0x556174c4c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f974339e740 1 -- 172.21.15.42:0/3814838339 shutdown_connections 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f974339e740 1 -- 172.21.15.42:0/3814838339 wait complete. 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.541+0000 7f973da00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f97200a2c00 0x7f97200a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f972c02a910 tx=0x7f972c0f5530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.542+0000 7f974339e740 1 -- start start 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.542+0000 7f974339e740 1 Processor -- start 2024-04-25T00:30:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.542+0000 7f974339e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c5b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.542+0000 7f974339e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 0x556174c5b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.542+0000 7f974339e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c569b0 0x556174c62660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.542+0000 7f974339e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556174c59960 con 0x556174c51840 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.542+0000 7f974339e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556174b6f990 con 0x556174c50e50 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.542+0000 7f974339e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556174c59ae0 con 0x556174c569b0 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.542+0000 7f974339e740 10 client.0 _ll_register_callbacks cb 0x556174b71950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c5b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c5b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60282/0 (socket says 172.21.15.42:60282) 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973e400640 1 -- 172.21.15.42:0/1564288207 learned_addr learned my addr 172.21.15.42:0/1564288207 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973d000640 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 0x556174c5b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973e400640 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c569b0 0x556174c62660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973e400640 1 -- 172.21.15.42:0/1564288207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c569b0 msgr2=0x556174c62660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:31.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973e400640 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c569b0 0x556174c62660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973e400640 1 -- 172.21.15.42:0/1564288207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 msgr2=0x556174c5b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973e400640 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 0x556174c5b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973e400640 1 -- 172.21.15.42:0/1564288207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556174994620 con 0x556174c50e50 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f973e400640 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c5b150 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9730009cc0 tx=0x7f9730010fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a50f8840a960db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.543+0000 7f971f400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.544+0000 7f971f400640 1 -- 172.21.15.42:0/1564288207 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730023070 con 0x556174c50e50 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.544+0000 7f971f400640 1 -- 172.21.15.42:0/1564288207 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f973001b660 con 0x556174c50e50 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.544+0000 7f974339e740 1 -- 172.21.15.42:0/1564288207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f973001bb40 con 0x556174c50e50 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.544+0000 7f971f400640 1 -- 172.21.15.42:0/1564288207 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9730016040 con 0x556174c50e50 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.544+0000 7f974339e740 10 client.26120 fetch_fsmap learned FSMap version 31 2024-04-25T00:30:31.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.544+0000 7f974339e740 1 -- 172.21.15.42:0/1564288207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556174c65640 con 0x556174c50e50 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.545+0000 7f971f400640 1 -- 172.21.15.42:0/1564288207 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f973001bb40 con 0x556174c50e50 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.545+0000 7f971f400640 1 -- 172.21.15.42:0/1564288207 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f973000f040 con 0x556174c50e50 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.545+0000 7f974339e740 10 client.26120 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.545+0000 7f974339e740 10 client.26120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.545+0000 7f974339e740 20 client.26120 populate_metadata read hostname 'smithi042' 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.545+0000 7f974339e740 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556174c66b10 0x556174c68f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.545+0000 7f974339e740 4 client.26120 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.545+0000 7f974339e740 1 -- 172.21.15.42:0/1564288207 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556174bc0ff0 con 0x556174c66b10 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.545+0000 7f973d000640 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556174c66b10 0x556174c68f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.546+0000 7f973d000640 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556174c66b10 0x556174c68f00 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8ec08221c83749 server_cookie=3b7d8cb7f14d9582 in_seq=0 out_seq=0 2024-04-25T00:30:31.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.546+0000 7f971f400640 10 client.26120 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.546+0000 7f971f400640 1 -- 172.21.15.42:0/1564288207 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556174bc0ff0 con 0x556174c66b10 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.546+0000 7f971f400640 10 client.26120 handle_command_reply: tid=0 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.546+0000 7f974339e740 1 client.26120 shutdown 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 1 -- 172.21.15.42:0/1564288207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 msgr2=0x556174c5b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c5b150 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9730009cc0 tx=0x7f9730010fd0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 1 -- 172.21.15.42:0/1564288207 shutdown_connections 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556174c66b10 0x556174c68f00 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c569b0 0x556174c62660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 0x556174c5b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 1 --2- 172.21.15.42:0/1564288207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c50e50 0x556174c5b150 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f971f400640 0 client.26120 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 1 -- 172.21.15.42:0/1564288207 >> 172.21.15.42:0/1564288207 conn(0x556174c4c4c0 msgr2=0x556174c4b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 1 -- 172.21.15.42:0/1564288207 shutdown_connections 2024-04-25T00:30:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 1 -- 172.21.15.42:0/1564288207 wait complete. 2024-04-25T00:30:31.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.547+0000 7f974339e740 20 client.26120 trim_cache size 0 max 16384 2024-04-25T00:30:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.564+0000 7f974339e740 1 Processor -- start 2024-04-25T00:30:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.564+0000 7f974339e740 1 -- start start 2024-04-25T00:30:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f974339e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c4c4c0 0x556174c5ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f974339e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c50e50 0x556174c625f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f974339e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c51840 0x556174c5b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f974339e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97381ad540 con 0x556174c50e50 2024-04-25T00:30:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f974339e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f97381aae10 con 0x556174c51840 2024-04-25T00:30:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f974339e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f97381a8c60 con 0x556174c4c4c0 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f973d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c50e50 0x556174c625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f973e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c51840 0x556174c5b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f973d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c50e50 0x556174c625f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37170/0 (socket says 172.21.15.42:37170) 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f973d000640 1 -- 172.21.15.42:0/787887220 learned_addr learned my addr 172.21.15.42:0/787887220 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.565+0000 7f973e400640 1 --2- 172.21.15.42:0/787887220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c4c4c0 0x556174c5ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.566+0000 7f973d000640 1 -- 172.21.15.42:0/787887220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c4c4c0 msgr2=0x556174c5ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.566+0000 7f973d000640 1 --2- 172.21.15.42:0/787887220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c4c4c0 0x556174c5ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.566+0000 7f973d000640 1 -- 172.21.15.42:0/787887220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c51840 msgr2=0x556174c5b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.566+0000 7f973d000640 1 --2- 172.21.15.42:0/787887220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c51840 0x556174c5b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.566+0000 7f973d000640 1 -- 172.21.15.42:0/787887220 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556174c50a90 con 0x556174c50e50 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.566+0000 7f973d000640 1 --2- 172.21.15.42:0/787887220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c50e50 0x556174c625f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x556174c4d390 tx=0x7f972400de30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a4bab0652642001 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.566+0000 7f972a000640 1 -- 172.21.15.42:0/787887220 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724003de0 con 0x556174c50e50 2024-04-25T00:30:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.566+0000 7f972a000640 1 -- 172.21.15.42:0/787887220 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f972401e530 con 0x556174c50e50 2024-04-25T00:30:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.566+0000 7f972a000640 1 -- 172.21.15.42:0/787887220 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9724012b30 con 0x556174c50e50 2024-04-25T00:30:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.567+0000 7f974339e740 1 -- 172.21.15.42:0/787887220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c50e50 msgr2=0x556174c625f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.567+0000 7f974339e740 1 --2- 172.21.15.42:0/787887220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c50e50 0x556174c625f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x556174c4d390 tx=0x7f972400de30 comp rx=0 tx=0).stop 2024-04-25T00:30:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.567+0000 7f974339e740 1 -- 172.21.15.42:0/787887220 shutdown_connections 2024-04-25T00:30:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.567+0000 7f974339e740 1 --2- 172.21.15.42:0/787887220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c51840 0x556174c5b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.567+0000 7f974339e740 1 --2- 172.21.15.42:0/787887220 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c50e50 0x556174c625f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.567+0000 7f974339e740 1 --2- 172.21.15.42:0/787887220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c4c4c0 0x556174c5ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.567+0000 7f974339e740 1 -- 172.21.15.42:0/787887220 >> 172.21.15.42:0/787887220 conn(0x556174c56480 msgr2=0x556174c47110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.567+0000 7f974339e740 1 -- 172.21.15.42:0/787887220 shutdown_connections 2024-04-25T00:30:31.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.567+0000 7f974339e740 1 -- 172.21.15.42:0/787887220 wait complete. 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.568+0000 7f974339e740 1 -- start start 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.568+0000 7f974339e740 1 Processor -- start 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.568+0000 7f974339e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c4c4c0 0x556174c60220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.568+0000 7f974339e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 0x556174c60760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.568+0000 7f974339e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c569b0 0x556174c61770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.568+0000 7f974339e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f97381a92d0 con 0x556174c51840 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.568+0000 7f974339e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f97381aa740 con 0x556174c569b0 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.568+0000 7f974339e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f97381a99a0 con 0x556174c4c4c0 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.568+0000 7f974339e740 10 client.0 _ll_register_callbacks cb 0x7f972c0f5400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 0x556174c60760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c569b0 0x556174c61770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973e400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c569b0 0x556174c61770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60298/0 (socket says 172.21.15.42:60298) 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973d000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 0x556174c60760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37174/0 (socket says 172.21.15.42:37174) 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973e400640 1 -- 172.21.15.42:0/3123800721 learned_addr learned my addr 172.21.15.42:0/3123800721 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973e400640 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c4c4c0 0x556174c60220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973e400640 1 -- 172.21.15.42:0/3123800721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c4c4c0 msgr2=0x556174c60220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973e400640 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c4c4c0 0x556174c60220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973e400640 1 -- 172.21.15.42:0/3123800721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 msgr2=0x556174c60760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973e400640 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 0x556174c60760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973e400640 1 -- 172.21.15.42:0/3123800721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556174c5a210 con 0x556174c569b0 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973d000640 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 0x556174c60760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f973e400640 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c569b0 0x556174c61770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9730036fd0 tx=0x7f9730005a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df30ee508e21f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.569+0000 7f971f400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f971f400640 1 -- 172.21.15.42:0/3123800721 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9730023030 con 0x556174c569b0 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f971f400640 1 -- 172.21.15.42:0/3123800721 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f9730036a60 con 0x556174c569b0 2024-04-25T00:30:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f974339e740 1 -- 172.21.15.42:0/3123800721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f972401e530 con 0x556174c569b0 2024-04-25T00:30:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f971f400640 1 -- 172.21.15.42:0/3123800721 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f973004b070 con 0x556174c569b0 2024-04-25T00:30:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f974339e740 10 client.26126 fetch_fsmap learned FSMap version 31 2024-04-25T00:30:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f974339e740 1 -- 172.21.15.42:0/3123800721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x556174c4c2f0 con 0x556174c569b0 2024-04-25T00:30:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f971f400640 1 -- 172.21.15.42:0/3123800721 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f972401e530 con 0x556174c569b0 2024-04-25T00:30:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f971f400640 1 -- 172.21.15.42:0/3123800721 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f97300444b0 con 0x556174c569b0 2024-04-25T00:30:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f974339e740 10 client.26126 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:30:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f974339e740 10 client.26126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f974339e740 20 client.26126 populate_metadata read hostname 'smithi042' 2024-04-25T00:30:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f974339e740 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556174ccd990 0x556174ccfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f974339e740 4 client.26126 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:30:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.570+0000 7f974339e740 1 -- 172.21.15.42:0/3123800721 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556174bbb340 con 0x556174ccd990 2024-04-25T00:30:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.571+0000 7f973d000640 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556174ccd990 0x556174ccfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.571+0000 7f973d000640 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556174ccd990 0x556174ccfdb0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46ad895d3b759afa server_cookie=ed2b1c299de325e8 in_seq=0 out_seq=0 2024-04-25T00:30:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.571+0000 7f971f400640 10 client.26126 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f971f400640 1 -- 172.21.15.42:0/3123800721 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556174bbb340 con 0x556174ccd990 2024-04-25T00:30:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f971f400640 10 client.26126 handle_command_reply: tid=0 2024-04-25T00:30:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 client.26126 shutdown 2024-04-25T00:30:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 -- 172.21.15.42:0/3123800721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c569b0 msgr2=0x556174c61770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c569b0 0x556174c61770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9730036fd0 tx=0x7f9730005a50 comp rx=0 tx=0).stop 2024-04-25T00:30:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 -- 172.21.15.42:0/3123800721 shutdown_connections 2024-04-25T00:30:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556174ccd990 0x556174ccfdb0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556174c569b0 0x556174c61770 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556174c51840 0x556174c60760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 --2- 172.21.15.42:0/3123800721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556174c4c4c0 0x556174c60220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f971f400640 0 client.26126 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 -- 172.21.15.42:0/3123800721 >> 172.21.15.42:0/3123800721 conn(0x556174c56480 msgr2=0x556174c47110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 -- 172.21.15.42:0/3123800721 shutdown_connections 2024-04-25T00:30:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.630+0000 7f974339e740 1 -- 172.21.15.42:0/3123800721 wait complete. 2024-04-25T00:30:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.631+0000 7f974339e740 20 client.26126 trim_cache size 0 max 16384 2024-04-25T00:30:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:30:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:30:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:30:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:30:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.631+0000 7f973ee00640 1 -- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f97200a2c00 msgr2=0x7f97200a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.631+0000 7f973ee00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f97200a2c00 0x7f97200a50c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f972c02a910 tx=0x7f972c0f5530 comp rx=0 tx=0).stop 2024-04-25T00:30:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.631+0000 7f973ee00640 1 -- 172.21.15.42:0/2130649592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9738101600 msgr2=0x7f9738199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.631+0000 7f973ee00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9738101600 0x7f9738199b00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f972c031440 tx=0x7f972c032e20 comp rx=0 tx=0).stop 2024-04-25T00:30:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.632+0000 7f973ee00640 1 -- 172.21.15.42:0/2130649592 shutdown_connections 2024-04-25T00:30:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.632+0000 7f973ee00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f97200a2c00 0x7f97200a50c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.632+0000 7f973ee00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f97381031b0 0x7f97381a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.632+0000 7f973ee00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f97381027f0 0x7f973819a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.632+0000 7f973ee00640 1 --2- 172.21.15.42:0/2130649592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9738101600 0x7f9738199b00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.632+0000 7f973ee00640 1 -- 172.21.15.42:0/2130649592 >> 172.21.15.42:0/2130649592 conn(0x7f97380fcb70 msgr2=0x7f9738105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.632+0000 7f973ee00640 1 -- 172.21.15.42:0/2130649592 shutdown_connections 2024-04-25T00:30:31.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:31.632+0000 7f973ee00640 1 -- 172.21.15.42:0/2130649592 wait complete. 2024-04-25T00:30:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:31 smithi106 ceph-mon[32510]: pgmap v741: 97 pgs: 97 active+clean; 133 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-25T00:30:32.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:31 smithi042 ceph-mon[28130]: pgmap v741: 97 pgs: 97 active+clean; 133 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-25T00:30:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:31 smithi136 ceph-mon[32923]: pgmap v741: 97 pgs: 97 active+clean; 133 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-25T00:30:32.277 INFO:tasks.workunit.client.0.smithi042.stdout: 1 725814 6.18 MB/sec execute 502 sec latency 50.261 ms 2024-04-25T00:30:33.277 INFO:tasks.workunit.client.0.smithi042.stdout: 1 726904 6.18 MB/sec execute 503 sec latency 26.000 ms 2024-04-25T00:30:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:33 smithi106 ceph-mon[32510]: pgmap v742: 97 pgs: 97 active+clean; 134 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-04-25T00:30:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:33 smithi042 ceph-mon[28130]: pgmap v742: 97 pgs: 97 active+clean; 134 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-04-25T00:30:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:33 smithi136 ceph-mon[32923]: pgmap v742: 97 pgs: 97 active+clean; 134 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 499 KiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-04-25T00:30:34.277 INFO:tasks.workunit.client.0.smithi042.stdout: 1 728218 6.19 MB/sec execute 504 sec latency 32.994 ms 2024-04-25T00:30:35.277 INFO:tasks.workunit.client.0.smithi042.stdout: 1 728820 6.18 MB/sec execute 505 sec latency 21.709 ms 2024-04-25T00:30:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:35 smithi106 ceph-mon[32510]: pgmap v743: 97 pgs: 97 active+clean; 134 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-04-25T00:30:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:35 smithi042 ceph-mon[28130]: pgmap v743: 97 pgs: 97 active+clean; 134 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-04-25T00:30:36.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:35 smithi136 ceph-mon[32923]: pgmap v743: 97 pgs: 97 active+clean; 134 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.3 MiB/s wr, 140 op/s 2024-04-25T00:30:36.277 INFO:tasks.workunit.client.0.smithi042.stdout: 1 730151 6.17 MB/sec execute 506 sec latency 23.607 ms 2024-04-25T00:30:37.277 INFO:tasks.workunit.client.0.smithi042.stdout: 1 731720 6.19 MB/sec execute 507 sec latency 18.757 ms 2024-04-25T00:30:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:37 smithi106 ceph-mon[32510]: pgmap v744: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-04-25T00:30:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:37 smithi042 ceph-mon[28130]: pgmap v744: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-04-25T00:30:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:37 smithi136 ceph-mon[32923]: pgmap v744: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 1.7 MiB/s wr, 195 op/s 2024-04-25T00:30:38.278 INFO:tasks.workunit.client.0.smithi042.stdout: 1 732315 6.18 MB/sec execute 508 sec latency 17.461 ms 2024-04-25T00:30:39.278 INFO:tasks.workunit.client.0.smithi042.stdout: 1 733587 6.18 MB/sec execute 509 sec latency 34.826 ms 2024-04-25T00:30:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:39 smithi106 ceph-mon[32510]: pgmap v745: 97 pgs: 97 active+clean; 136 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-25T00:30:40.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:39 smithi042 ceph-mon[28130]: pgmap v745: 97 pgs: 97 active+clean; 136 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-25T00:30:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:39 smithi136 ceph-mon[32923]: pgmap v745: 97 pgs: 97 active+clean; 136 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-04-25T00:30:40.278 INFO:tasks.workunit.client.0.smithi042.stdout: 1 735252 6.19 MB/sec execute 510 sec latency 25.690 ms 2024-04-25T00:30:41.278 INFO:tasks.workunit.client.0.smithi042.stdout: 1 735770 6.18 MB/sec execute 511 sec latency 91.027 ms 2024-04-25T00:30:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:41 smithi106 ceph-mon[32510]: pgmap v746: 97 pgs: 97 active+clean; 137 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-04-25T00:30:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:41 smithi042 ceph-mon[28130]: pgmap v746: 97 pgs: 97 active+clean; 137 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-04-25T00:30:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:41 smithi136 ceph-mon[32923]: pgmap v746: 97 pgs: 97 active+clean; 137 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 416 KiB/s rd, 1.7 MiB/s wr, 191 op/s 2024-04-25T00:30:42.278 INFO:tasks.workunit.client.0.smithi042.stdout: 1 736813 6.18 MB/sec execute 512 sec latency 36.095 ms 2024-04-25T00:30:43.278 INFO:tasks.workunit.client.0.smithi042.stdout: 1 738645 6.19 MB/sec execute 513 sec latency 20.331 ms 2024-04-25T00:30:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:43 smithi106 ceph-mon[32510]: pgmap v747: 97 pgs: 97 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-04-25T00:30:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:43 smithi042 ceph-mon[28130]: pgmap v747: 97 pgs: 97 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-04-25T00:30:44.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:43 smithi136 ceph-mon[32923]: pgmap v747: 97 pgs: 97 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 383 KiB/s rd, 1.7 MiB/s wr, 176 op/s 2024-04-25T00:30:44.278 INFO:tasks.workunit.client.0.smithi042.stdout: 1 738810 6.18 MB/sec execute 514 sec latency 760.293 ms 2024-04-25T00:30:44.779 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.779+0000 7ff73d400640 1 -- 172.21.15.42:0/1776811057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7381015c0 msgr2=0x7ff7381019e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.779+0000 7ff73d400640 1 --2- 172.21.15.42:0/1776811057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7381015c0 0x7ff7381019e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff728009860 tx=0x7ff72802a340 comp rx=0 tx=0).stop 2024-04-25T00:30:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.779+0000 7ff73d400640 1 -- 172.21.15.42:0/1776811057 shutdown_connections 2024-04-25T00:30:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.779+0000 7ff73d400640 1 --2- 172.21.15.42:0/1776811057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738103170 0x7ff738109a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.779+0000 7ff73d400640 1 --2- 172.21.15.42:0/1776811057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7381027b0 0x7ff738102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.779+0000 7ff73d400640 1 --2- 172.21.15.42:0/1776811057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7381015c0 0x7ff7381019e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.779+0000 7ff73d400640 1 -- 172.21.15.42:0/1776811057 >> 172.21.15.42:0/1776811057 conn(0x7ff7380fcb50 msgr2=0x7ff7380fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.779+0000 7ff73d400640 1 -- 172.21.15.42:0/1776811057 shutdown_connections 2024-04-25T00:30:44.780 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.779+0000 7ff73d400640 1 -- 172.21.15.42:0/1776811057 wait complete. 2024-04-25T00:30:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.780+0000 7ff73d400640 1 Processor -- start 2024-04-25T00:30:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.780+0000 7ff73d400640 1 -- start start 2024-04-25T00:30:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.780+0000 7ff73d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7381015c0 0x7ff738199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.780+0000 7ff73d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7381027b0 0x7ff73819a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.782 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.780+0000 7ff73d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738103170 0x7ff7381a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.780+0000 7ff73d400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff73810d210 con 0x7ff7381027b0 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.780+0000 7ff73d400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff73810d090 con 0x7ff7381015c0 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.780+0000 7ff73d400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff73810d390 con 0x7ff738103170 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff737e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7381015c0 0x7ff738199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff72fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7381027b0 0x7ff73819a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff737e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7381015c0 0x7ff738199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:47102/0 (socket says 172.21.15.42:47102) 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff737e00640 1 -- 172.21.15.42:0/95955826 learned_addr learned my addr 172.21.15.42:0/95955826 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff73ca00640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738103170 0x7ff7381a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff737e00640 1 -- 172.21.15.42:0/95955826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738103170 msgr2=0x7ff7381a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff737e00640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738103170 0x7ff7381a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff737e00640 1 -- 172.21.15.42:0/95955826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7381027b0 msgr2=0x7ff73819a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff737e00640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7381027b0 0x7ff73819a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff737e00640 1 -- 172.21.15.42:0/95955826 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7381a1710 con 0x7ff7381015c0 2024-04-25T00:30:44.783 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff73ca00640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738103170 0x7ff7381a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.781+0000 7ff737e00640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7381015c0 0x7ff738199af0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff728035040 tx=0x7ff728033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.782+0000 7ff735600640 1 -- 172.21.15.42:0/95955826 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728037040 con 0x7ff7381015c0 2024-04-25T00:30:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.782+0000 7ff735600640 1 -- 172.21.15.42:0/95955826 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff728030850 con 0x7ff7381015c0 2024-04-25T00:30:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.782+0000 7ff73d400640 1 -- 172.21.15.42:0/95955826 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7381a19a0 con 0x7ff7381015c0 2024-04-25T00:30:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.782+0000 7ff735600640 1 -- 172.21.15.42:0/95955826 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff728004920 con 0x7ff7381015c0 2024-04-25T00:30:44.784 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.782+0000 7ff73d400640 1 -- 172.21.15.42:0/95955826 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7381a1e90 con 0x7ff7381015c0 2024-04-25T00:30:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.785+0000 7ff735600640 1 -- 172.21.15.42:0/95955826 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff728004050 con 0x7ff7381015c0 2024-04-25T00:30:44.785 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.785+0000 7ff741965740 1 Processor -- start 2024-04-25T00:30:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.785+0000 7ff741965740 1 -- start start 2024-04-25T00:30:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff741965740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cc8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff741965740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357ccd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff741965740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357ccd9b0 0x560357ccfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff741965740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560357c2aab0 con 0x560357cc7e50 2024-04-25T00:30:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff741965740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560357bd3640 con 0x560357ccd9b0 2024-04-25T00:30:44.786 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff741965740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560357c26a70 con 0x560357cc8840 2024-04-25T00:30:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff73ca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cc8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff72fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357ccd9b0 0x560357ccfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff72fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357ccd9b0 0x560357ccfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:47112/0 (socket says 172.21.15.42:47112) 2024-04-25T00:30:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff72fe00640 1 -- 172.21.15.42:0/2569674035 learned_addr learned my addr 172.21.15.42:0/2569674035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.786+0000 7ff737e00640 1 --2- 172.21.15.42:0/2569674035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357ccd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.788 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.787+0000 7ff72fe00640 1 -- 172.21.15.42:0/2569674035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 msgr2=0x560357ccd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.787+0000 7ff72fe00640 1 --2- 172.21.15.42:0/2569674035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357ccd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.787+0000 7ff72fe00640 1 -- 172.21.15.42:0/2569674035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 msgr2=0x560357cc8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.787+0000 7ff72fe00640 1 --2- 172.21.15.42:0/2569674035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cc8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.787+0000 7ff72fe00640 1 -- 172.21.15.42:0/2569674035 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035793f210 con 0x560357ccd9b0 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.787+0000 7ff72fe00640 1 --2- 172.21.15.42:0/2569674035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357ccd9b0 0x560357ccfe40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff720009c80 tx=0x7ff720011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dcb5163a76c131f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.787+0000 7ff735600640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff71c0a2b80 0x7ff71c0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.787+0000 7ff72d200640 1 -- 172.21.15.42:0/2569674035 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720004220 con 0x560357ccd9b0 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.787+0000 7ff72d200640 1 -- 172.21.15.42:0/2569674035 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff720009e30 con 0x560357ccd9b0 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.787+0000 7ff72d200640 1 -- 172.21.15.42:0/2569674035 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720019850 con 0x560357ccd9b0 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff735600640 1 -- 172.21.15.42:0/95955826 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff7280eef70 con 0x7ff7381015c0 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff73ca00640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff71c0a2b80 0x7ff71c0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff741965740 1 -- 172.21.15.42:0/2569674035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357ccd9b0 msgr2=0x560357ccfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.789 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff741965740 1 --2- 172.21.15.42:0/2569674035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357ccd9b0 0x560357ccfe40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff720009c80 tx=0x7ff720011810 comp rx=0 tx=0).stop 2024-04-25T00:30:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff741965740 1 -- 172.21.15.42:0/2569674035 shutdown_connections 2024-04-25T00:30:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff741965740 1 --2- 172.21.15.42:0/2569674035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357ccd9b0 0x560357ccfe40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff741965740 1 --2- 172.21.15.42:0/2569674035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357ccd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff741965740 1 --2- 172.21.15.42:0/2569674035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cc8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff741965740 1 -- 172.21.15.42:0/2569674035 >> 172.21.15.42:0/2569674035 conn(0x560357cc34c0 msgr2=0x560357cc38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff73ca00640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff71c0a2b80 0x7ff71c0a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x560357cc90a0 tx=0x7ff730009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.788+0000 7ff741965740 1 -- 172.21.15.42:0/2569674035 shutdown_connections 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.789+0000 7ff741965740 1 -- 172.21.15.42:0/2569674035 wait complete. 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.789+0000 7ff741965740 1 -- start start 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.789+0000 7ff741965740 1 Processor -- start 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff741965740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc7e50 0x560357cd20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff741965740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc8840 0x560357cd2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff741965740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357ccd9b0 0x560357cd9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff741965740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560357cd0960 con 0x560357cc8840 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff741965740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560357be6990 con 0x560357cc7e50 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff741965740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560357cd0ae0 con 0x560357ccd9b0 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff741965740 10 client.0 _ll_register_callbacks cb 0x560357be8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff737e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc7e50 0x560357cd20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff737e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc7e50 0x560357cd20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:47120/0 (socket says 172.21.15.42:47120) 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff737e00640 1 -- 172.21.15.42:0/3904782955 learned_addr learned my addr 172.21.15.42:0/3904782955 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff73ca00640 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357ccd9b0 0x560357cd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff72fe00640 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc8840 0x560357cd2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff737e00640 1 -- 172.21.15.42:0/3904782955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357ccd9b0 msgr2=0x560357cd9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff737e00640 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357ccd9b0 0x560357cd9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff737e00640 1 -- 172.21.15.42:0/3904782955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc8840 msgr2=0x560357cd2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff737e00640 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc8840 0x560357cd2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff737e00640 1 -- 172.21.15.42:0/3904782955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560357a0b620 con 0x560357cc7e50 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.790+0000 7ff72fe00640 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc8840 0x560357cd2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.791+0000 7ff737e00640 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc7e50 0x560357cd20f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff7280b7c00 tx=0x7ff7280b7400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faec75c688a27266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.791+0000 7ff725600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.791+0000 7ff725600640 1 -- 172.21.15.42:0/3904782955 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7280bb2e0 con 0x560357cc7e50 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.791+0000 7ff725600640 1 -- 172.21.15.42:0/3904782955 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff7280a60b0 con 0x560357cc7e50 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.791+0000 7ff741965740 1 -- 172.21.15.42:0/3904782955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff720009e30 con 0x560357cc7e50 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.791+0000 7ff725600640 1 -- 172.21.15.42:0/3904782955 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff7280bb100 con 0x560357cc7e50 2024-04-25T00:30:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.791+0000 7ff741965740 10 client.26138 fetch_fsmap learned FSMap version 31 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.791+0000 7ff741965740 1 -- 172.21.15.42:0/3904782955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560357cdc5e0 con 0x560357cc7e50 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.791+0000 7ff725600640 1 -- 172.21.15.42:0/3904782955 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff720009e30 con 0x560357cc7e50 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.791+0000 7ff725600640 1 -- 172.21.15.42:0/3904782955 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7ff72803a070 con 0x560357cc7e50 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.792+0000 7ff741965740 10 client.26138 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.792+0000 7ff741965740 10 client.26138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.792+0000 7ff741965740 20 client.26138 populate_metadata read hostname 'smithi042' 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.792+0000 7ff741965740 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560357cddd10 0x560357ce01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.792+0000 7ff741965740 4 client.26138 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.792+0000 7ff741965740 1 -- 172.21.15.42:0/3904782955 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560357c37ff0 con 0x560357cddd10 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.792+0000 7ff72fe00640 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560357cddd10 0x560357ce01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.792+0000 7ff72fe00640 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560357cddd10 0x560357ce01b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d162421f4e40a2df server_cookie=f94820da77ad207f in_seq=0 out_seq=0 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.792+0000 7ff725600640 10 client.26138 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.793+0000 7ff725600640 1 -- 172.21.15.42:0/3904782955 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560357c37ff0 con 0x560357cddd10 2024-04-25T00:30:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.793+0000 7ff725600640 10 client.26138 handle_command_reply: tid=0 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.793+0000 7ff741965740 1 client.26138 shutdown 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 1 -- 172.21.15.42:0/3904782955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc7e50 msgr2=0x560357cd20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc7e50 0x560357cd20f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff7280b7c00 tx=0x7ff7280b7400 comp rx=0 tx=0).stop 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 1 -- 172.21.15.42:0/3904782955 shutdown_connections 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560357cddd10 0x560357ce01b0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357ccd9b0 0x560357cd9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff725600640 0 client.26138 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc8840 0x560357cd2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 1 --2- 172.21.15.42:0/3904782955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc7e50 0x560357cd20f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 1 -- 172.21.15.42:0/3904782955 >> 172.21.15.42:0/3904782955 conn(0x560357cc34c0 msgr2=0x560357cc2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 1 -- 172.21.15.42:0/3904782955 shutdown_connections 2024-04-25T00:30:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 1 -- 172.21.15.42:0/3904782955 wait complete. 2024-04-25T00:30:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.794+0000 7ff741965740 20 client.26138 trim_cache size 0 max 16384 2024-04-25T00:30:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.810+0000 7ff741965740 1 Processor -- start 2024-04-25T00:30:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff741965740 1 -- start start 2024-04-25T00:30:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff741965740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc34c0 0x560357cc6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff741965740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cc69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff741965740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357cd1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff741965740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381ad4a0 con 0x560357cc7e50 2024-04-25T00:30:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff741965740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381aad70 con 0x560357cc34c0 2024-04-25T00:30:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff741965740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381a8bc0 con 0x560357cc8840 2024-04-25T00:30:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff72fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cc69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff72fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cc69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59230/0 (socket says 172.21.15.42:59230) 2024-04-25T00:30:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff72fe00640 1 -- 172.21.15.42:0/66947744 learned_addr learned my addr 172.21.15.42:0/66947744 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.811+0000 7ff737e00640 1 --2- 172.21.15.42:0/66947744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc34c0 0x560357cc6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.812+0000 7ff73ca00640 1 --2- 172.21.15.42:0/66947744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357cd1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.812+0000 7ff72fe00640 1 -- 172.21.15.42:0/66947744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 msgr2=0x560357cd1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.812+0000 7ff72fe00640 1 --2- 172.21.15.42:0/66947744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357cd1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.812+0000 7ff72fe00640 1 -- 172.21.15.42:0/66947744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc34c0 msgr2=0x560357cc6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.812+0000 7ff72fe00640 1 --2- 172.21.15.42:0/66947744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc34c0 0x560357cc6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.812+0000 7ff72fe00640 1 -- 172.21.15.42:0/66947744 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560357cc32f0 con 0x560357cc7e50 2024-04-25T00:30:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.812+0000 7ff72fe00640 1 --2- 172.21.15.42:0/66947744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cc69c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff720009ba0 tx=0x7ff720009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ca76744f26a6f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.812+0000 7ff727e00640 1 -- 172.21.15.42:0/66947744 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720004220 con 0x560357cc7e50 2024-04-25T00:30:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.812+0000 7ff727e00640 1 -- 172.21.15.42:0/66947744 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff72002f030 con 0x560357cc7e50 2024-04-25T00:30:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.812+0000 7ff727e00640 1 -- 172.21.15.42:0/66947744 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720004880 con 0x560357cc7e50 2024-04-25T00:30:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.813+0000 7ff741965740 1 -- 172.21.15.42:0/66947744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 msgr2=0x560357cc69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.813+0000 7ff741965740 1 --2- 172.21.15.42:0/66947744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cc69c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7ff720009ba0 tx=0x7ff720009640 comp rx=0 tx=0).stop 2024-04-25T00:30:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.813+0000 7ff741965740 1 -- 172.21.15.42:0/66947744 shutdown_connections 2024-04-25T00:30:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.813+0000 7ff741965740 1 --2- 172.21.15.42:0/66947744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357cd1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.813+0000 7ff741965740 1 --2- 172.21.15.42:0/66947744 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cc69c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.813+0000 7ff741965740 1 --2- 172.21.15.42:0/66947744 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc34c0 0x560357cc6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.813+0000 7ff741965740 1 -- 172.21.15.42:0/66947744 >> 172.21.15.42:0/66947744 conn(0x560357ccd480 msgr2=0x560357cc38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.813+0000 7ff741965740 1 -- 172.21.15.42:0/66947744 shutdown_connections 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.813+0000 7ff741965740 1 -- 172.21.15.42:0/66947744 wait complete. 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.814+0000 7ff741965740 1 -- start start 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.814+0000 7ff741965740 1 Processor -- start 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.814+0000 7ff741965740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc34c0 0x560357cd6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.814+0000 7ff741965740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cd6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.814+0000 7ff741965740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357cd7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.814+0000 7ff741965740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381a9230 con 0x560357cc7e50 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.814+0000 7ff741965740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381aa6a0 con 0x560357cc34c0 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.814+0000 7ff741965740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381a9900 con 0x560357cc8840 2024-04-25T00:30:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.814+0000 7ff741965740 10 client.0 _ll_register_callbacks cb 0x7ff7280097b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff72fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cd6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff72fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cd6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59234/0 (socket says 172.21.15.42:59234) 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff72fe00640 1 -- 172.21.15.42:0/304338532 learned_addr learned my addr 172.21.15.42:0/304338532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff73ca00640 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357cd7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff737e00640 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc34c0 0x560357cd6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff72fe00640 1 -- 172.21.15.42:0/304338532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 msgr2=0x560357cd7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff72fe00640 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357cd7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff72fe00640 1 -- 172.21.15.42:0/304338532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc34c0 msgr2=0x560357cd6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff72fe00640 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc34c0 0x560357cd6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff72fe00640 1 -- 172.21.15.42:0/304338532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560357cd81f0 con 0x560357cc7e50 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff72fe00640 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cd6c40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff720009530 tx=0x7ff720011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9068164704168e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.815+0000 7ff725600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.816+0000 7ff725600640 1 -- 172.21.15.42:0/304338532 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720038020 con 0x560357cc7e50 2024-04-25T00:30:44.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.816+0000 7ff741965740 1 -- 172.21.15.42:0/304338532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff72002f030 con 0x560357cc7e50 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.816+0000 7ff725600640 1 -- 172.21.15.42:0/304338532 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7ff7200035f0 con 0x560357cc7e50 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.816+0000 7ff725600640 1 -- 172.21.15.42:0/304338532 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff720004420 con 0x560357cc7e50 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.816+0000 7ff741965740 10 client.16011 fetch_fsmap learned FSMap version 31 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.816+0000 7ff741965740 1 -- 172.21.15.42:0/304338532 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560357d447a0 con 0x560357cc7e50 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.816+0000 7ff725600640 1 -- 172.21.15.42:0/304338532 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff72002f030 con 0x560357cc7e50 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.816+0000 7ff725600640 1 -- 172.21.15.42:0/304338532 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7ff7200198f0 con 0x560357cc7e50 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.816+0000 7ff741965740 10 client.16011 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.816+0000 7ff741965740 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.817+0000 7ff741965740 20 client.16011 populate_metadata read hostname 'smithi042' 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.817+0000 7ff741965740 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560357ccd9b0 0x560357d46f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.817+0000 7ff741965740 4 client.16011 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.817+0000 7ff741965740 1 -- 172.21.15.42:0/304338532 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560357c32340 con 0x560357ccd9b0 2024-04-25T00:30:44.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.817+0000 7ff737e00640 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560357ccd9b0 0x560357d46f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.817+0000 7ff737e00640 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560357ccd9b0 0x560357d46f50 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae19e1fa922141af server_cookie=80b62c43ed51a594 in_seq=0 out_seq=0 2024-04-25T00:30:44.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.817+0000 7ff725600640 10 client.16011 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff725600640 1 -- 172.21.15.42:0/304338532 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560357c32340 con 0x560357ccd9b0 2024-04-25T00:30:44.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff725600640 10 client.16011 handle_command_reply: tid=0 2024-04-25T00:30:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff741965740 1 client.16011 shutdown 2024-04-25T00:30:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff741965740 1 -- 172.21.15.42:0/304338532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 msgr2=0x560357cd6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff741965740 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cd6c40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff720009530 tx=0x7ff720011d10 comp rx=0 tx=0).stop 2024-04-25T00:30:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff741965740 1 -- 172.21.15.42:0/304338532 shutdown_connections 2024-04-25T00:30:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff741965740 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560357ccd9b0 0x560357d46f50 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff741965740 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560357cc8840 0x560357cd7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff741965740 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560357cc7e50 0x560357cd6c40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff725600640 0 client.16011 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff741965740 1 --2- 172.21.15.42:0/304338532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560357cc34c0 0x560357cd6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.873+0000 7ff741965740 1 -- 172.21.15.42:0/304338532 >> 172.21.15.42:0/304338532 conn(0x560357ccd480 msgr2=0x560357cc38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.874+0000 7ff741965740 1 -- 172.21.15.42:0/304338532 shutdown_connections 2024-04-25T00:30:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.874+0000 7ff741965740 1 -- 172.21.15.42:0/304338532 wait complete. 2024-04-25T00:30:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.874+0000 7ff741965740 20 client.16011 trim_cache size 0 max 16384 2024-04-25T00:30:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:30:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:30:44.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:30:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:30:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 -- 172.21.15.42:0/95955826 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff71c0a2b80 msgr2=0x7ff71c0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff71c0a2b80 0x7ff71c0a5040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x560357cc90a0 tx=0x7ff730009500 comp rx=0 tx=0).stop 2024-04-25T00:30:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 -- 172.21.15.42:0/95955826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7381015c0 msgr2=0x7ff738199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:44.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7381015c0 0x7ff738199af0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff728035040 tx=0x7ff728033d90 comp rx=0 tx=0).stop 2024-04-25T00:30:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 -- 172.21.15.42:0/95955826 shutdown_connections 2024-04-25T00:30:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff71c0a2b80 0x7ff71c0a5040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738103170 0x7ff7381a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff7381027b0 0x7ff73819a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 --2- 172.21.15.42:0/95955826 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff7381015c0 0x7ff738199af0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 -- 172.21.15.42:0/95955826 >> 172.21.15.42:0/95955826 conn(0x7ff7380fcb50 msgr2=0x7ff738073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 -- 172.21.15.42:0/95955826 shutdown_connections 2024-04-25T00:30:44.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:44.875+0000 7ff73d400640 1 -- 172.21.15.42:0/95955826 wait complete. 2024-04-25T00:30:45.278 INFO:tasks.workunit.client.0.smithi042.stdout: 1 738810 6.17 MB/sec execute 515 sec latency 1760.402 ms 2024-04-25T00:30:46.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:45 smithi042 ceph-mon[28130]: pgmap v748: 97 pgs: 97 active+clean; 140 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-04-25T00:30:46.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:45 smithi136 ceph-mon[32923]: pgmap v748: 97 pgs: 97 active+clean; 140 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-04-25T00:30:46.279 INFO:tasks.workunit.client.0.smithi042.stdout: 1 738810 6.15 MB/sec execute 516 sec latency 2760.538 ms 2024-04-25T00:30:46.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:45 smithi106 ceph-mon[32510]: pgmap v748: 97 pgs: 97 active+clean; 140 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 363 KiB/s rd, 1.5 MiB/s wr, 157 op/s 2024-04-25T00:30:47.279 INFO:tasks.workunit.client.0.smithi042.stdout: 1 738810 6.14 MB/sec execute 517 sec latency 3760.682 ms 2024-04-25T00:30:48.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:47 smithi042 ceph-mon[28130]: pgmap v749: 97 pgs: 97 active+clean; 138 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-04-25T00:30:48.140 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:47 smithi106 ceph-mon[32510]: pgmap v749: 97 pgs: 97 active+clean; 138 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-04-25T00:30:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:47 smithi136 ceph-mon[32923]: pgmap v749: 97 pgs: 97 active+clean; 138 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 1.7 MiB/s wr, 205 op/s 2024-04-25T00:30:48.279 INFO:tasks.workunit.client.0.smithi042.stdout: 1 738810 6.13 MB/sec execute 518 sec latency 4760.847 ms 2024-04-25T00:30:49.279 INFO:tasks.workunit.client.0.smithi042.stdout: 1 739177 6.12 MB/sec execute 519 sec latency 5019.802 ms 2024-04-25T00:30:50.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:49 smithi042 ceph-mon[28130]: pgmap v750: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-04-25T00:30:50.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:49 smithi136 ceph-mon[32923]: pgmap v750: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-04-25T00:30:50.282 INFO:tasks.workunit.client.0.smithi042.stdout: 1 740195 6.12 MB/sec execute 520 sec latency 30.266 ms 2024-04-25T00:30:50.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:49 smithi106 ceph-mon[32510]: pgmap v750: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 382 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-04-25T00:30:51.279 INFO:tasks.workunit.client.0.smithi042.stdout: 1 741885 6.12 MB/sec execute 521 sec latency 35.395 ms 2024-04-25T00:30:52.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:51 smithi042 ceph-mon[28130]: pgmap v751: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-04-25T00:30:52.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:51 smithi136 ceph-mon[32923]: pgmap v751: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-04-25T00:30:52.280 INFO:tasks.workunit.client.0.smithi042.stdout: 1 742570 6.12 MB/sec execute 522 sec latency 20.109 ms 2024-04-25T00:30:52.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:51 smithi106 ceph-mon[32510]: pgmap v751: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-04-25T00:30:53.280 INFO:tasks.workunit.client.0.smithi042.stdout: 1 743581 6.12 MB/sec execute 523 sec latency 40.585 ms 2024-04-25T00:30:54.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:53 smithi042 ceph-mon[28130]: pgmap v752: 97 pgs: 97 active+clean; 134 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-04-25T00:30:54.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:53 smithi136 ceph-mon[32923]: pgmap v752: 97 pgs: 97 active+clean; 134 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-04-25T00:30:54.280 INFO:tasks.workunit.client.0.smithi042.stdout: 1 745051 6.12 MB/sec execute 524 sec latency 25.300 ms 2024-04-25T00:30:54.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:53 smithi106 ceph-mon[32510]: pgmap v752: 97 pgs: 97 active+clean; 134 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.3 MiB/s wr, 134 op/s 2024-04-25T00:30:55.280 INFO:tasks.workunit.client.0.smithi042.stdout: 1 746123 6.12 MB/sec execute 525 sec latency 14.600 ms 2024-04-25T00:30:56.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:55 smithi042 ceph-mon[28130]: pgmap v753: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-04-25T00:30:56.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:55 smithi136 ceph-mon[32923]: pgmap v753: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-04-25T00:30:56.280 INFO:tasks.workunit.client.0.smithi042.stdout: 1 747234 6.12 MB/sec execute 526 sec latency 29.147 ms 2024-04-25T00:30:56.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:55 smithi106 ceph-mon[32510]: pgmap v753: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-04-25T00:30:57.281 INFO:tasks.workunit.client.0.smithi042.stdout: 1 748697 6.12 MB/sec execute 527 sec latency 27.436 ms 2024-04-25T00:30:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.021+0000 7f8934a00640 1 -- 172.21.15.42:0/4019868078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8930103800 msgr2=0x7f8930103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.021+0000 7f8934a00640 1 --2- 172.21.15.42:0/4019868078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8930103800 0x7f8930103c20 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8920009860 tx=0x7f892002a340 comp rx=0 tx=0).stop 2024-04-25T00:30:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.022+0000 7f8934a00640 1 -- 172.21.15.42:0/4019868078 shutdown_connections 2024-04-25T00:30:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.022+0000 7f8934a00640 1 --2- 172.21.15.42:0/4019868078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89301053b0 0x7f893010bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.022+0000 7f8934a00640 1 --2- 172.21.15.42:0/4019868078 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89301049f0 0x7f8930104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.022+0000 7f8934a00640 1 --2- 172.21.15.42:0/4019868078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8930103800 0x7f8930103c20 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.022+0000 7f8934a00640 1 -- 172.21.15.42:0/4019868078 >> 172.21.15.42:0/4019868078 conn(0x7f89300fed70 msgr2=0x7f89301011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.022+0000 7f8934a00640 1 -- 172.21.15.42:0/4019868078 shutdown_connections 2024-04-25T00:30:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.022+0000 7f8934a00640 1 -- 172.21.15.42:0/4019868078 wait complete. 2024-04-25T00:30:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.023+0000 7f8934a00640 1 Processor -- start 2024-04-25T00:30:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.023+0000 7f8934a00640 1 -- start start 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.023+0000 7f8934a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8930103800 0x7f8930199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.023+0000 7f8934a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89301049f0 0x7f893019a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.023+0000 7f8934a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89301053b0 0x7f89301a1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.023+0000 7f8934a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f893010f480 con 0x7f89301049f0 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.023+0000 7f8934a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f893010f300 con 0x7f8930103800 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.023+0000 7f8934a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f893010f600 con 0x7f89301053b0 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89301053b0 0x7f89301a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8930103800 0x7f8930199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89301053b0 0x7f89301a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38558/0 (socket says 172.21.15.42:38558) 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892fe00640 1 -- 172.21.15.42:0/3102863048 learned_addr learned my addr 172.21.15.42:0/3102863048 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892ea00640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89301049f0 0x7f893019a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892f400640 1 -- 172.21.15.42:0/3102863048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89301053b0 msgr2=0x7f89301a1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892f400640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89301053b0 0x7f89301a1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892f400640 1 -- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89301049f0 msgr2=0x7f893019a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892f400640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89301049f0 0x7f893019a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892f400640 1 -- 172.21.15.42:0/3102863048 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89301a16f0 con 0x7f8930103800 2024-04-25T00:30:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892ea00640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89301049f0 0x7f893019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.024+0000 7f892f400640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8930103800 0x7f8930199ad0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f892002a860 tx=0x7f8920033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:58.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.025+0000 7f8927e00640 1 -- 172.21.15.42:0/3102863048 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920037040 con 0x7f8930103800 2024-04-25T00:30:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.025+0000 7f8927e00640 1 -- 172.21.15.42:0/3102863048 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8920030850 con 0x7f8930103800 2024-04-25T00:30:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.025+0000 7f8934a00640 1 -- 172.21.15.42:0/3102863048 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89301a1980 con 0x7f8930103800 2024-04-25T00:30:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.025+0000 7f8927e00640 1 -- 172.21.15.42:0/3102863048 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8920004920 con 0x7f8930103800 2024-04-25T00:30:58.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.025+0000 7f8934a00640 1 -- 172.21.15.42:0/3102863048 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89301a1e90 con 0x7f8930103800 2024-04-25T00:30:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.029+0000 7f8938d80740 1 Processor -- start 2024-04-25T00:30:58.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.029+0000 7f8938d80740 1 -- start start 2024-04-25T00:30:58.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f8938d80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f29f4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f8938d80740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f4840 0x5647f29f9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f8938d80740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f99b0 0x5647f29fbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f8927e00640 1 -- 172.21.15.42:0/3102863048 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8920004050 con 0x7f8930103800 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f8938d80740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5647f2956ab0 con 0x5647f29f99b0 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f8938d80740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5647f28ff640 con 0x5647f29f4840 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f8938d80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5647f2952a70 con 0x5647f29f3e50 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f892fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f29f4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f892fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f29f4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38560/0 (socket says 172.21.15.42:38560) 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f892fe00640 1 -- 172.21.15.42:0/1869382888 learned_addr learned my addr 172.21.15.42:0/1869382888 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f892f400640 1 --2- 172.21.15.42:0/1869382888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f4840 0x5647f29f9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f892ea00640 1 --2- 172.21.15.42:0/1869382888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f99b0 0x5647f29fbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.030+0000 7f892fe00640 1 -- 172.21.15.42:0/1869382888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f4840 msgr2=0x5647f29f9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.031+0000 7f892fe00640 1 --2- 172.21.15.42:0/1869382888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f4840 0x5647f29f9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.031+0000 7f892fe00640 1 -- 172.21.15.42:0/1869382888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f99b0 msgr2=0x5647f29fbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:58.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.031+0000 7f892fe00640 1 --2- 172.21.15.42:0/1869382888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f99b0 0x5647f29fbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.031+0000 7f892fe00640 1 -- 172.21.15.42:0/1869382888 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647f266b210 con 0x5647f29f3e50 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.031+0000 7f892f400640 1 --2- 172.21.15.42:0/1869382888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f4840 0x5647f29f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.031+0000 7f892fe00640 1 --2- 172.21.15.42:0/1869382888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f29f4270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f89280114c0 tx=0x7f8928011980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=475159b4be6c9b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.031+0000 7f891be00640 1 -- 172.21.15.42:0/1869382888 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928004b20 con 0x5647f29f3e50 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.031+0000 7f891be00640 1 -- 172.21.15.42:0/1869382888 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8928004cc0 con 0x5647f29f3e50 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.031+0000 7f891be00640 1 -- 172.21.15.42:0/1869382888 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928005500 con 0x5647f29f3e50 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8927e00640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f89140a2c00 0x7f89140a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8938d80740 1 -- 172.21.15.42:0/1869382888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 msgr2=0x5647f29f4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8938d80740 1 --2- 172.21.15.42:0/1869382888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f29f4270 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f89280114c0 tx=0x7f8928011980 comp rx=0 tx=0).stop 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8927e00640 1 -- 172.21.15.42:0/3102863048 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f89200eee60 con 0x7f8930103800 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8938d80740 1 -- 172.21.15.42:0/1869382888 shutdown_connections 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8938d80740 1 --2- 172.21.15.42:0/1869382888 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f99b0 0x5647f29fbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8938d80740 1 --2- 172.21.15.42:0/1869382888 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f4840 0x5647f29f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8938d80740 1 --2- 172.21.15.42:0/1869382888 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f29f4270 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8938d80740 1 -- 172.21.15.42:0/1869382888 >> 172.21.15.42:0/1869382888 conn(0x5647f29ef4c0 msgr2=0x5647f29ef8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f892f400640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f89140a2c00 0x7f89140a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8938d80740 1 -- 172.21.15.42:0/1869382888 shutdown_connections 2024-04-25T00:30:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.032+0000 7f8938d80740 1 -- 172.21.15.42:0/1869382888 wait complete. 2024-04-25T00:30:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.033+0000 7f8938d80740 1 -- start start 2024-04-25T00:30:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.033+0000 7f892f400640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f89140a2c00 0x7f89140a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f892002a930 tx=0x7f89200f28a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:58.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.033+0000 7f8938d80740 1 Processor -- start 2024-04-25T00:30:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f8938d80740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f3e50 0x5647f29fe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f8938d80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f29fe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f8938d80740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f99b0 0x5647f2a05590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f8938d80740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5647f29fc960 con 0x5647f29f3e50 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f8938d80740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5647f2912990 con 0x5647f29f99b0 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f8938d80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5647f29fcae0 con 0x5647f29f4840 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f8938d80740 10 client.0 _ll_register_callbacks cb 0x5647f2914950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f29fe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f29fe550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38572/0 (socket says 172.21.15.42:38572) 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892ea00640 1 -- 172.21.15.42:0/2274275127 learned_addr learned my addr 172.21.15.42:0/2274275127 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892fe00640 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f3e50 0x5647f29fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892fe00640 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f99b0 0x5647f2a05590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892ea00640 1 -- 172.21.15.42:0/2274275127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f99b0 msgr2=0x5647f2a05590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892ea00640 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f99b0 0x5647f2a05590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892ea00640 1 -- 172.21.15.42:0/2274275127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f3e50 msgr2=0x5647f29fe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892ea00640 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f3e50 0x5647f29fe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892ea00640 1 -- 172.21.15.42:0/2274275127 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647f2737620 con 0x5647f29f4840 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.034+0000 7f892fe00640 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f3e50 0x5647f29fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.035+0000 7f892ea00640 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f29fe550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f891c009d70 tx=0x7f891c0039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eadd23690d0eba66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.035+0000 7f8918c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.035+0000 7f8918c00640 1 -- 172.21.15.42:0/2274275127 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c012860 con 0x5647f29f4840 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.035+0000 7f8918c00640 1 -- 172.21.15.42:0/2274275127 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f891c012a00 con 0x5647f29f4840 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.035+0000 7f8938d80740 1 -- 172.21.15.42:0/2274275127 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8928004cc0 con 0x5647f29f4840 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.035+0000 7f8918c00640 1 -- 172.21.15.42:0/2274275127 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f891c023070 con 0x5647f29f4840 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.035+0000 7f8938d80740 10 client.26164 fetch_fsmap learned FSMap version 31 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.035+0000 7f8938d80740 1 -- 172.21.15.42:0/2274275127 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5647f2a08690 con 0x5647f29f4840 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.036+0000 7f8918c00640 1 -- 172.21.15.42:0/2274275127 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8928004cc0 con 0x5647f29f4840 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.036+0000 7f8918c00640 1 -- 172.21.15.42:0/2274275127 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f891c01a020 con 0x5647f29f4840 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.036+0000 7f8938d80740 10 client.26164 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:30:58.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.036+0000 7f8938d80740 10 client.26164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.036+0000 7f8938d80740 20 client.26164 populate_metadata read hostname 'smithi042' 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.036+0000 7f8938d80740 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5647f2a09e80 0x5647f2a0c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.036+0000 7f8938d80740 4 client.26164 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.036+0000 7f8938d80740 1 -- 172.21.15.42:0/2274275127 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647f2963ff0 con 0x5647f2a09e80 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.036+0000 7f892fe00640 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5647f2a09e80 0x5647f2a0c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.037+0000 7f892fe00640 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5647f2a09e80 0x5647f2a0c270 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd81ce33744384f server_cookie=305302d6f2738c04 in_seq=0 out_seq=0 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.037+0000 7f8918c00640 10 client.26164 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8918c00640 1 -- 172.21.15.42:0/2274275127 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647f2963ff0 con 0x5647f2a09e80 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8918c00640 10 client.26164 handle_command_reply: tid=0 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 client.26164 shutdown 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 -- 172.21.15.42:0/2274275127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 msgr2=0x5647f29fe550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f29fe550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f891c009d70 tx=0x7f891c0039c0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 -- 172.21.15.42:0/2274275127 shutdown_connections 2024-04-25T00:30:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5647f2a09e80 0x5647f2a0c270 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f99b0 0x5647f2a05590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f29fe550 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 --2- 172.21.15.42:0/2274275127 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f3e50 0x5647f29fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 -- 172.21.15.42:0/2274275127 >> 172.21.15.42:0/2274275127 conn(0x5647f29ef4c0 msgr2=0x5647f29ee620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8918c00640 0 client.26164 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 -- 172.21.15.42:0/2274275127 shutdown_connections 2024-04-25T00:30:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 1 -- 172.21.15.42:0/2274275127 wait complete. 2024-04-25T00:30:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.038+0000 7f8938d80740 20 client.26164 trim_cache size 0 max 16384 2024-04-25T00:30:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.055+0000 7f8938d80740 1 Processor -- start 2024-04-25T00:30:58.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.055+0000 7f8938d80740 1 -- start start 2024-04-25T00:30:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.055+0000 7f8938d80740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f3e50 0x5647f29fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.055+0000 7f8938d80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f2a05550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.055+0000 7f8938d80740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f99b0 0x5647f29f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.055+0000 7f8938d80740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f89301ad360 con 0x5647f29f99b0 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.055+0000 7f8938d80740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f89301aac30 con 0x5647f29f3e50 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.055+0000 7f8938d80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f89301a8a80 con 0x5647f29f4840 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f2a05550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f3e50 0x5647f29fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f2a05550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38580/0 (socket says 172.21.15.42:38580) 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892ea00640 1 -- 172.21.15.42:0/3882108966 learned_addr learned my addr 172.21.15.42:0/3882108966 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892fe00640 1 --2- 172.21.15.42:0/3882108966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f99b0 0x5647f29f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892ea00640 1 -- 172.21.15.42:0/3882108966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f3e50 msgr2=0x5647f29fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892ea00640 1 --2- 172.21.15.42:0/3882108966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f3e50 0x5647f29fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892ea00640 1 -- 172.21.15.42:0/3882108966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f99b0 msgr2=0x5647f29f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892ea00640 1 --2- 172.21.15.42:0/3882108966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f99b0 0x5647f29f21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892ea00640 1 -- 172.21.15.42:0/3882108966 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647f29ef2f0 con 0x5647f29f4840 2024-04-25T00:30:58.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.056+0000 7f892fe00640 1 --2- 172.21.15.42:0/3882108966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f3e50 0x5647f29fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:30:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.057+0000 7f892ea00640 1 --2- 172.21.15.42:0/3882108966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f2a05550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f891c0114a0 tx=0x7f891c011950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdf15fe1ebda33f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:58.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.057+0000 7f891b400640 1 -- 172.21.15.42:0/3882108966 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c011c70 con 0x5647f29f4840 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.057+0000 7f891b400640 1 -- 172.21.15.42:0/3882108966 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f891c011e10 con 0x5647f29f4840 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.057+0000 7f891b400640 1 -- 172.21.15.42:0/3882108966 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c012d80 con 0x5647f29f4840 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.057+0000 7f8938d80740 1 -- 172.21.15.42:0/3882108966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 msgr2=0x5647f2a05550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.057+0000 7f8938d80740 1 --2- 172.21.15.42:0/3882108966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f2a05550 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f891c0114a0 tx=0x7f891c011950 comp rx=0 tx=0).stop 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.057+0000 7f8938d80740 1 -- 172.21.15.42:0/3882108966 shutdown_connections 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.057+0000 7f8938d80740 1 --2- 172.21.15.42:0/3882108966 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f99b0 0x5647f29f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.058+0000 7f8938d80740 1 --2- 172.21.15.42:0/3882108966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f4840 0x5647f2a05550 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.058+0000 7f8938d80740 1 --2- 172.21.15.42:0/3882108966 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f3e50 0x5647f29fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.058+0000 7f8938d80740 1 -- 172.21.15.42:0/3882108966 >> 172.21.15.42:0/3882108966 conn(0x5647f29ef4c0 msgr2=0x5647f29fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.058+0000 7f8938d80740 1 -- 172.21.15.42:0/3882108966 shutdown_connections 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.058+0000 7f8938d80740 1 -- 172.21.15.42:0/3882108966 wait complete. 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.058+0000 7f8938d80740 1 -- start start 2024-04-25T00:30:58.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.058+0000 7f8938d80740 1 Processor -- start 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f8938d80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f2a02f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f8938d80740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f4840 0x5647f2a034b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f8938d80740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f99b0 0x5647f2a044c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f8938d80740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f89301a90f0 con 0x5647f29f4840 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f8938d80740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f89301aa560 con 0x5647f29f99b0 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f8938d80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f89301a97c0 con 0x5647f29f3e50 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f8938d80740 10 client.0 _ll_register_callbacks cb 0x7f891c009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f892fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f2a02f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f892fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f2a02f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38590/0 (socket says 172.21.15.42:38590) 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f892fe00640 1 -- 172.21.15.42:0/3782620731 learned_addr learned my addr 172.21.15.42:0/3782620731 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f892fe00640 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f99b0 0x5647f2a044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f892ea00640 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f4840 0x5647f2a034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f892fe00640 1 -- 172.21.15.42:0/3782620731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f99b0 msgr2=0x5647f2a044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:58.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f892fe00640 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f99b0 0x5647f2a044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f892fe00640 1 -- 172.21.15.42:0/3782620731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f4840 msgr2=0x5647f2a034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f892fe00640 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f4840 0x5647f2a034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.059+0000 7f892fe00640 1 -- 172.21.15.42:0/3782620731 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647f29f5510 con 0x5647f29f3e50 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.060+0000 7f892fe00640 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f2a02f70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f89280114c0 tx=0x7f8928013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f426df607794e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.060+0000 7f8918c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.060+0000 7f8918c00640 1 -- 172.21.15.42:0/3782620731 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8928031040 con 0x5647f29f3e50 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.060+0000 7f8938d80740 1 -- 172.21.15.42:0/3782620731 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f891c011e10 con 0x5647f29f3e50 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.060+0000 7f8918c00640 1 -- 172.21.15.42:0/3782620731 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f892800fd10 con 0x5647f29f3e50 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.060+0000 7f8918c00640 1 -- 172.21.15.42:0/3782620731 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f892803e050 con 0x5647f29f3e50 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.060+0000 7f8938d80740 10 client.26170 fetch_fsmap learned FSMap version 31 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.060+0000 7f8938d80740 1 -- 172.21.15.42:0/3782620731 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5647f29f3a90 con 0x5647f29f3e50 2024-04-25T00:30:58.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.060+0000 7f8918c00640 1 -- 172.21.15.42:0/3782620731 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f891c011e10 con 0x5647f29f3e50 2024-04-25T00:30:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.060+0000 7f8918c00640 1 -- 172.21.15.42:0/3782620731 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f89280183b0 con 0x5647f29f3e50 2024-04-25T00:30:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.061+0000 7f8938d80740 10 client.26170 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:30:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.061+0000 7f8938d80740 10 client.26170 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:30:58.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.061+0000 7f8938d80740 20 client.26170 populate_metadata read hostname 'smithi042' 2024-04-25T00:30:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.061+0000 7f8938d80740 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5647f2a79260 0x5647f2a7b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:30:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.061+0000 7f8938d80740 4 client.26170 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:30:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.061+0000 7f8938d80740 1 -- 172.21.15.42:0/3782620731 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5647f295e340 con 0x5647f2a79260 2024-04-25T00:30:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.061+0000 7f892ea00640 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5647f2a79260 0x5647f2a7b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:30:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.061+0000 7f892ea00640 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5647f2a79260 0x5647f2a7b650 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b945762e19907e87 server_cookie=a646b83cc0baeebc in_seq=0 out_seq=0 2024-04-25T00:30:58.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.061+0000 7f8918c00640 10 client.26170 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8918c00640 1 -- 172.21.15.42:0/3782620731 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5647f295e340 con 0x5647f2a79260 2024-04-25T00:30:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8918c00640 10 client.26170 handle_command_reply: tid=0 2024-04-25T00:30:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8938d80740 1 client.26170 shutdown 2024-04-25T00:30:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8938d80740 1 -- 172.21.15.42:0/3782620731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 msgr2=0x5647f2a02f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8938d80740 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f2a02f70 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f89280114c0 tx=0x7f8928013040 comp rx=0 tx=0).stop 2024-04-25T00:30:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8938d80740 1 -- 172.21.15.42:0/3782620731 shutdown_connections 2024-04-25T00:30:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8938d80740 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5647f2a79260 0x5647f2a7b650 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8938d80740 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5647f29f99b0 0x5647f2a044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8938d80740 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5647f29f4840 0x5647f2a034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8918c00640 0 client.26170 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:30:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8938d80740 1 --2- 172.21.15.42:0/3782620731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5647f29f3e50 0x5647f2a02f70 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8938d80740 1 -- 172.21.15.42:0/3782620731 >> 172.21.15.42:0/3782620731 conn(0x5647f29ef4c0 msgr2=0x5647f29f21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.113+0000 7f8938d80740 1 -- 172.21.15.42:0/3782620731 shutdown_connections 2024-04-25T00:30:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8938d80740 1 -- 172.21.15.42:0/3782620731 wait complete. 2024-04-25T00:30:58.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8938d80740 20 client.26170 trim_cache size 0 max 16384 2024-04-25T00:30:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:30:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:30:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:30:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:30:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8934a00640 1 -- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f89140a2c00 msgr2=0x7f89140a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8934a00640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f89140a2c00 0x7f89140a50c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f892002a930 tx=0x7f89200f28a0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8934a00640 1 -- 172.21.15.42:0/3102863048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8930103800 msgr2=0x7f8930199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:30:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8934a00640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8930103800 0x7f8930199ad0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f892002a860 tx=0x7f8920033d90 comp rx=0 tx=0).stop 2024-04-25T00:30:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8934a00640 1 -- 172.21.15.42:0/3102863048 shutdown_connections 2024-04-25T00:30:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8934a00640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f89140a2c00 0x7f89140a50c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8934a00640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89301053b0 0x7f89301a1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8934a00640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f89301049f0 0x7f893019a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8934a00640 1 --2- 172.21.15.42:0/3102863048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8930103800 0x7f8930199ad0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:30:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.114+0000 7f8934a00640 1 -- 172.21.15.42:0/3102863048 >> 172.21.15.42:0/3102863048 conn(0x7f89300fed70 msgr2=0x7f8930107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:30:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.115+0000 7f8934a00640 1 -- 172.21.15.42:0/3102863048 shutdown_connections 2024-04-25T00:30:58.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:30:58.115+0000 7f8934a00640 1 -- 172.21.15.42:0/3102863048 wait complete. 2024-04-25T00:30:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:57 smithi042 ceph-mon[28130]: pgmap v754: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-04-25T00:30:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:57 smithi136 ceph-mon[32923]: pgmap v754: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-04-25T00:30:58.281 INFO:tasks.workunit.client.0.smithi042.stdout: 1 749771 6.12 MB/sec execute 528 sec latency 33.475 ms 2024-04-25T00:30:58.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:57 smithi106 ceph-mon[32510]: pgmap v754: 97 pgs: 97 active+clean; 135 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.4 MiB/s wr, 156 op/s 2024-04-25T00:30:59.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:30:58 smithi136 ceph-mon[32923]: pgmap v755: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-04-25T00:30:59.281 INFO:tasks.workunit.client.0.smithi042.stdout: 1 750905 6.12 MB/sec execute 529 sec latency 25.965 ms 2024-04-25T00:30:59.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:30:58 smithi106 ceph-mon[32510]: pgmap v755: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-04-25T00:30:59.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:30:58 smithi042 ceph-mon[28130]: pgmap v755: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-04-25T00:31:00.281 INFO:tasks.workunit.client.0.smithi042.stdout: 1 752795 6.13 MB/sec execute 530 sec latency 22.018 ms 2024-04-25T00:31:01.281 INFO:tasks.workunit.client.0.smithi042.stdout: 1 753552 6.13 MB/sec execute 531 sec latency 23.936 ms 2024-04-25T00:31:01.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:01 smithi136 ceph-mon[32923]: pgmap v756: 97 pgs: 97 active+clean; 138 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-04-25T00:31:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:01 smithi106 ceph-mon[32510]: pgmap v756: 97 pgs: 97 active+clean; 138 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-04-25T00:31:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:01 smithi042 ceph-mon[28130]: pgmap v756: 97 pgs: 97 active+clean; 138 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 151 op/s 2024-04-25T00:31:02.281 INFO:tasks.workunit.client.0.smithi042.stdout: 1 754578 6.12 MB/sec execute 532 sec latency 176.093 ms 2024-04-25T00:31:03.281 INFO:tasks.workunit.client.0.smithi042.stdout: 1 756537 6.14 MB/sec execute 533 sec latency 37.221 ms 2024-04-25T00:31:03.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:03 smithi136 ceph-mon[32923]: pgmap v757: 97 pgs: 97 active+clean; 140 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-25T00:31:03.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:03 smithi106 ceph-mon[32510]: pgmap v757: 97 pgs: 97 active+clean; 140 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-25T00:31:03.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:03 smithi042 ceph-mon[28130]: pgmap v757: 97 pgs: 97 active+clean; 140 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 409 KiB/s rd, 1.6 MiB/s wr, 186 op/s 2024-04-25T00:31:04.282 INFO:tasks.workunit.client.0.smithi042.stdout: 1 757261 6.13 MB/sec execute 534 sec latency 29.014 ms 2024-04-25T00:31:05.282 INFO:tasks.workunit.client.0.smithi042.stdout: 1 758665 6.13 MB/sec execute 535 sec latency 26.826 ms 2024-04-25T00:31:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:05 smithi136 ceph-mon[32923]: pgmap v758: 97 pgs: 97 active+clean; 136 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-04-25T00:31:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:05 smithi106 ceph-mon[32510]: pgmap v758: 97 pgs: 97 active+clean; 136 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-04-25T00:31:05.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:05 smithi042 ceph-mon[28130]: pgmap v758: 97 pgs: 97 active+clean; 136 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 429 KiB/s rd, 1.3 MiB/s wr, 175 op/s 2024-04-25T00:31:06.283 INFO:tasks.workunit.client.0.smithi042.stdout: 1 760163 6.14 MB/sec execute 536 sec latency 17.861 ms 2024-04-25T00:31:07.283 INFO:tasks.workunit.client.0.smithi042.stdout: 1 760925 6.13 MB/sec execute 537 sec latency 86.372 ms 2024-04-25T00:31:07.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:07 smithi136 ceph-mon[32923]: pgmap v759: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-04-25T00:31:07.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:07 smithi106 ceph-mon[32510]: pgmap v759: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-04-25T00:31:07.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:07 smithi042 ceph-mon[28130]: pgmap v759: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 565 KiB/s rd, 1.8 MiB/s wr, 215 op/s 2024-04-25T00:31:08.283 INFO:tasks.workunit.client.0.smithi042.stdout: 1 762307 6.13 MB/sec execute 538 sec latency 27.362 ms 2024-04-25T00:31:09.283 INFO:tasks.workunit.client.0.smithi042.stdout: 1 763791 6.14 MB/sec execute 539 sec latency 13.039 ms 2024-04-25T00:31:09.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:09 smithi136 ceph-mon[32923]: pgmap v760: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-04-25T00:31:09.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:09 smithi106 ceph-mon[32510]: pgmap v760: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-04-25T00:31:09.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:09 smithi042 ceph-mon[28130]: pgmap v760: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 1.5 MiB/s wr, 185 op/s 2024-04-25T00:31:10.283 INFO:tasks.workunit.client.0.smithi042.stdout: 1 764844 6.14 MB/sec execute 540 sec latency 25.404 ms 2024-04-25T00:31:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.266+0000 7fadfd800640 1 -- 172.21.15.42:0/3472484705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadf81027f0 msgr2=0x7fadf8102c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.266+0000 7fadfd800640 1 --2- 172.21.15.42:0/3472484705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadf81027f0 0x7fadf8102c70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fade0009900 tx=0x7fade002a400 comp rx=0 tx=0).stop 2024-04-25T00:31:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.266+0000 7fadfd800640 1 -- 172.21.15.42:0/3472484705 shutdown_connections 2024-04-25T00:31:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.266+0000 7fadfd800640 1 --2- 172.21.15.42:0/3472484705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf81031b0 0x7fadf8109a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.266+0000 7fadfd800640 1 --2- 172.21.15.42:0/3472484705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadf81027f0 0x7fadf8102c70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.266+0000 7fadfd800640 1 --2- 172.21.15.42:0/3472484705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf8101600 0x7fadf8101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.266+0000 7fadfd800640 1 -- 172.21.15.42:0/3472484705 >> 172.21.15.42:0/3472484705 conn(0x7fadf80fcb70 msgr2=0x7fadf80fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.266+0000 7fadfd800640 1 -- 172.21.15.42:0/3472484705 shutdown_connections 2024-04-25T00:31:11.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.266+0000 7fadfd800640 1 -- 172.21.15.42:0/3472484705 wait complete. 2024-04-25T00:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.267+0000 7fadfd800640 1 Processor -- start 2024-04-25T00:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.267+0000 7fadfd800640 1 -- start start 2024-04-25T00:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfd800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf8101600 0x7fadf8197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfd800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadf81027f0 0x7fadf8197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfd800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf81031b0 0x7fadf819eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfd800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810d2c0 con 0x7fadf8101600 2024-04-25T00:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfd800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810d140 con 0x7fadf81027f0 2024-04-25T00:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfd800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fadf810d440 con 0x7fadf81031b0 2024-04-25T00:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf81031b0 0x7fadf819eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadf7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf8101600 0x7fadf8197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadf81027f0 0x7fadf8197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf81031b0 0x7fadf819eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:54034/0 (socket says 172.21.15.42:54034) 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadf81027f0 0x7fadf8197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51868/0 (socket says 172.21.15.42:51868) 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfce00640 1 -- 172.21.15.42:0/2157727313 learned_addr learned my addr 172.21.15.42:0/2157727313 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfce00640 1 -- 172.21.15.42:0/2157727313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadf81027f0 msgr2=0x7fadf8197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfce00640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadf81027f0 0x7fadf8197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfce00640 1 -- 172.21.15.42:0/2157727313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf8101600 msgr2=0x7fadf8197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfce00640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf8101600 0x7fadf8197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadfce00640 1 -- 172.21.15.42:0/2157727313 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadf819f520 con 0x7fadf81031b0 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.268+0000 7fadf7e00640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf8101600 0x7fadf8197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.269+0000 7fadfce00640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf81031b0 0x7fadf819eeb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fadf000efd0 tx=0x7fadf000c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.269+0000 7fadf5600640 1 -- 172.21.15.42:0/2157727313 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0003b70 con 0x7fadf81031b0 2024-04-25T00:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.269+0000 7fadf5600640 1 -- 172.21.15.42:0/2157727313 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadf0003d10 con 0x7fadf81031b0 2024-04-25T00:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.269+0000 7fadf5600640 1 -- 172.21.15.42:0/2157727313 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf00109f0 con 0x7fadf81031b0 2024-04-25T00:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.269+0000 7fadfd800640 1 -- 172.21.15.42:0/2157727313 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadf819f810 con 0x7fadf81031b0 2024-04-25T00:31:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.269+0000 7fadfd800640 1 -- 172.21.15.42:0/2157727313 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadf819fd30 con 0x7fadf81031b0 2024-04-25T00:31:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.272+0000 7fadf5600640 1 -- 172.21.15.42:0/2157727313 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fadf0004020 con 0x7fadf81031b0 2024-04-25T00:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.272+0000 7fae01d66740 1 Processor -- start 2024-04-25T00:31:11.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fae01d66740 1 -- start start 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fae01d66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x564357058680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fae01d66740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 0x56435705d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fae01d66740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x564357060250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fae01d66740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564356fb6ab0 con 0x564357058c50 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fae01d66740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564356f5f640 con 0x56435705ddc0 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fae01d66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564356fb2a70 con 0x564357058260 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fadf7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 0x56435705d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fadefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x564357060250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fadf7e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 0x56435705d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:56102/0 (socket says 172.21.15.42:56102) 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fadf7e00640 1 -- 172.21.15.42:0/3587243665 learned_addr learned my addr 172.21.15.42:0/3587243665 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:11.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.273+0000 7fadefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x564357060250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51878/0 (socket says 172.21.15.42:51878) 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.274+0000 7fadefe00640 1 --2- 172.21.15.42:0/3587243665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x564357058680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.274+0000 7fadf7e00640 1 -- 172.21.15.42:0/3587243665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 msgr2=0x564357058680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.274+0000 7fadf7e00640 1 --2- 172.21.15.42:0/3587243665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x564357058680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.274+0000 7fadf7e00640 1 -- 172.21.15.42:0/3587243665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 msgr2=0x564357060250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.274+0000 7fadf7e00640 1 --2- 172.21.15.42:0/3587243665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x564357060250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.274+0000 7fadf7e00640 1 -- 172.21.15.42:0/3587243665 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564356ccb210 con 0x564357058c50 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.274+0000 7fadefe00640 1 --2- 172.21.15.42:0/3587243665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x564357060250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.274+0000 7fadf5600640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faddc0a2c00 0x7faddc0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.274+0000 7fadf7e00640 1 --2- 172.21.15.42:0/3587243665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 0x56435705d750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fade80114c0 tx=0x7fade8011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d80e5078c7549c16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.275+0000 7fadf5600640 1 -- 172.21.15.42:0/2157727313 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fadf00cf430 con 0x7fadf81031b0 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.275+0000 7fadefe00640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faddc0a2c00 0x7faddc0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.275+0000 7faded600640 1 -- 172.21.15.42:0/3587243665 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade8011250 con 0x564357058c50 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.275+0000 7faded600640 1 -- 172.21.15.42:0/3587243665 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade8005500 con 0x564357058c50 2024-04-25T00:31:11.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.275+0000 7faded600640 1 -- 172.21.15.42:0/3587243665 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade800f820 con 0x564357058c50 2024-04-25T00:31:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 -- 172.21.15.42:0/3587243665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 msgr2=0x56435705d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fadefe00640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faddc0a2c00 0x7faddc0a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fade0031440 tx=0x7fade0005780 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 --2- 172.21.15.42:0/3587243665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 0x56435705d750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fade80114c0 tx=0x7fade8011980 comp rx=0 tx=0).stop 2024-04-25T00:31:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 -- 172.21.15.42:0/3587243665 shutdown_connections 2024-04-25T00:31:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 --2- 172.21.15.42:0/3587243665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x564357060250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 --2- 172.21.15.42:0/3587243665 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 0x56435705d750 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 --2- 172.21.15.42:0/3587243665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x564357058680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 -- 172.21.15.42:0/3587243665 >> 172.21.15.42:0/3587243665 conn(0x56435704f4c0 msgr2=0x56435704f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 -- 172.21.15.42:0/3587243665 shutdown_connections 2024-04-25T00:31:11.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 -- 172.21.15.42:0/3587243665 wait complete. 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 -- start start 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.276+0000 7fae01d66740 1 Processor -- start 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fae01d66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x564357062480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fae01d66740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x5643570629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fae01d66740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56435705ddc0 0x564357069a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fae01d66740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564357060d70 con 0x56435705ddc0 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fae01d66740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564356f72990 con 0x564357058c50 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fae01d66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564357060ef0 con 0x564357058260 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fae01d66740 10 client.0 _ll_register_callbacks cb 0x564356f74950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fadefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x5643570629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fadefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x5643570629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51892/0 (socket says 172.21.15.42:51892) 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fadefe00640 1 -- 172.21.15.42:0/2185281246 learned_addr learned my addr 172.21.15.42:0/2185281246 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:11.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fadfce00640 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56435705ddc0 0x564357069a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.277+0000 7fadf7e00640 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x564357062480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.278+0000 7fadefe00640 1 -- 172.21.15.42:0/2185281246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 msgr2=0x564357062480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.278+0000 7fadefe00640 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x564357062480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.278+0000 7fadefe00640 1 -- 172.21.15.42:0/2185281246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56435705ddc0 msgr2=0x564357069a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.278+0000 7fadefe00640 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56435705ddc0 0x564357069a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.278+0000 7fadefe00640 1 -- 172.21.15.42:0/2185281246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564356d97620 con 0x564357058c50 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.278+0000 7fadefe00640 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x5643570629c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fade0004650 tx=0x7fade0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e389b766beac32a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.278+0000 7fade6000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.278+0000 7fade6000640 1 -- 172.21.15.42:0/2185281246 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade0003b80 con 0x564357058c50 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.278+0000 7fade6000640 1 -- 172.21.15.42:0/2185281246 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fade0003d20 con 0x564357058c50 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.278+0000 7fae01d66740 1 -- 172.21.15.42:0/2185281246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fade8005500 con 0x564357058c50 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fade6000640 1 -- 172.21.15.42:0/2185281246 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fade005c070 con 0x564357058c50 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fae01d66740 10 client.26174 fetch_fsmap learned FSMap version 31 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fae01d66740 1 -- 172.21.15.42:0/2185281246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56435706ca50 con 0x564357058c50 2024-04-25T00:31:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fade6000640 1 -- 172.21.15.42:0/2185281246 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fade8005500 con 0x564357058c50 2024-04-25T00:31:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fade6000640 1 -- 172.21.15.42:0/2185281246 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fade003c590 con 0x564357058c50 2024-04-25T00:31:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fae01d66740 10 client.26174 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:31:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fae01d66740 10 client.26174 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fae01d66740 20 client.26174 populate_metadata read hostname 'smithi042' 2024-04-25T00:31:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fae01d66740 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56435706e130 0x564357070580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fae01d66740 4 client.26174 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.279+0000 7fae01d66740 1 -- 172.21.15.42:0/2185281246 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564356fc3ff0 con 0x56435706e130 2024-04-25T00:31:11.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.280+0000 7fadf7e00640 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56435706e130 0x564357070580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.280+0000 7fadf7e00640 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56435706e130 0x564357070580 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15fce200c2c02f7f server_cookie=d97ca87b284355c7 in_seq=0 out_seq=0 2024-04-25T00:31:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.280+0000 7fade6000640 10 client.26174 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fade6000640 1 -- 172.21.15.42:0/2185281246 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564356fc3ff0 con 0x56435706e130 2024-04-25T00:31:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fade6000640 10 client.26174 handle_command_reply: tid=0 2024-04-25T00:31:11.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fae01d66740 1 client.26174 shutdown 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fae01d66740 1 -- 172.21.15.42:0/2185281246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 msgr2=0x5643570629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fae01d66740 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x5643570629c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fade0004650 tx=0x7fade0009260 comp rx=0 tx=0).stop 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fae01d66740 1 -- 172.21.15.42:0/2185281246 shutdown_connections 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fae01d66740 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56435706e130 0x564357070580 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fae01d66740 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56435705ddc0 0x564357069a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fae01d66740 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x5643570629c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fade6000640 0 client.26174 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fae01d66740 1 --2- 172.21.15.42:0/2185281246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x564357062480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.281+0000 7fae01d66740 1 -- 172.21.15.42:0/2185281246 >> 172.21.15.42:0/2185281246 conn(0x56435704f4c0 msgr2=0x56435704e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.282+0000 7fae01d66740 1 -- 172.21.15.42:0/2185281246 shutdown_connections 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.282+0000 7fae01d66740 1 -- 172.21.15.42:0/2185281246 wait complete. 2024-04-25T00:31:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.282+0000 7fae01d66740 20 client.26174 trim_cache size 0 max 16384 2024-04-25T00:31:11.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 766388 6.14 MB/sec execute 541 sec latency 21.168 ms 2024-04-25T00:31:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.300+0000 7fae01d66740 1 Processor -- start 2024-04-25T00:31:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.300+0000 7fae01d66740 1 -- start start 2024-04-25T00:31:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.300+0000 7fae01d66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x56435704a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.300+0000 7fae01d66740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 0x5643570699c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.300+0000 7fae01d66740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x56435704e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.300+0000 7fae01d66740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81a90a0 con 0x564357058c50 2024-04-25T00:31:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.300+0000 7fae01d66740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81a6970 con 0x56435705ddc0 2024-04-25T00:31:11.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.300+0000 7fae01d66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81a47c0 con 0x564357058260 2024-04-25T00:31:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadfce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x56435704e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadfce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x56435704e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51900/0 (socket says 172.21.15.42:51900) 2024-04-25T00:31:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadf7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x56435704a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadfce00640 1 -- 172.21.15.42:0/2989553132 learned_addr learned my addr 172.21.15.42:0/2989553132 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadefe00640 1 --2- 172.21.15.42:0/2989553132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 0x5643570699c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadfce00640 1 -- 172.21.15.42:0/2989553132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 msgr2=0x56435704a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadfce00640 1 --2- 172.21.15.42:0/2989553132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x56435704a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadfce00640 1 -- 172.21.15.42:0/2989553132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 msgr2=0x5643570699c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadfce00640 1 --2- 172.21.15.42:0/2989553132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 0x5643570699c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadfce00640 1 -- 172.21.15.42:0/2989553132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56435705d110 con 0x56435705ddc0 2024-04-25T00:31:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadf7e00640 1 --2- 172.21.15.42:0/2989553132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x56435704a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.301+0000 7fadfce00640 1 --2- 172.21.15.42:0/2989553132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x56435704e4a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fadf00ccf00 tx=0x7fadf00104c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8aec6aef23459ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.302+0000 7fadecc00640 1 -- 172.21.15.42:0/2989553132 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf00b4e60 con 0x56435705ddc0 2024-04-25T00:31:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.302+0000 7fadecc00640 1 -- 172.21.15.42:0/2989553132 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadf00b9070 con 0x56435705ddc0 2024-04-25T00:31:11.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.302+0000 7fadecc00640 1 -- 172.21.15.42:0/2989553132 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf0004310 con 0x56435705ddc0 2024-04-25T00:31:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 -- 172.21.15.42:0/2989553132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 msgr2=0x56435704e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:11.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 --2- 172.21.15.42:0/2989553132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x56435704e4a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fadf00ccf00 tx=0x7fadf00104c0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 -- 172.21.15.42:0/2989553132 shutdown_connections 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 --2- 172.21.15.42:0/2989553132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56435705ddc0 0x56435704e4a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 --2- 172.21.15.42:0/2989553132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564357058c50 0x5643570699c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 --2- 172.21.15.42:0/2989553132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x56435704a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 -- 172.21.15.42:0/2989553132 >> 172.21.15.42:0/2989553132 conn(0x56435704f4c0 msgr2=0x56435704dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 -- 172.21.15.42:0/2989553132 shutdown_connections 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 -- 172.21.15.42:0/2989553132 wait complete. 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 -- start start 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.303+0000 7fae01d66740 1 Processor -- start 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fae01d66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x5643570697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fae01d66740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x564357067df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fae01d66740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56435705ddc0 0x564357068330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fae01d66740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81a4e30 con 0x56435705ddc0 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fae01d66740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81a62a0 con 0x564357058c50 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fae01d66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fadf81a5500 con 0x564357058260 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fae01d66740 10 client.0 _ll_register_callbacks cb 0x7fade0004300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x564357067df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadf7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x5643570697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadefe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x564357067df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51902/0 (socket says 172.21.15.42:51902) 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadefe00640 1 -- 172.21.15.42:0/3128382961 learned_addr learned my addr 172.21.15.42:0/3128382961 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadfce00640 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56435705ddc0 0x564357068330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadf7e00640 1 -- 172.21.15.42:0/3128382961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 msgr2=0x564357067df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadf7e00640 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x564357067df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadf7e00640 1 -- 172.21.15.42:0/3128382961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56435705ddc0 msgr2=0x564357068330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadf7e00640 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56435705ddc0 0x564357068330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadf7e00640 1 -- 172.21.15.42:0/3128382961 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564357068a70 con 0x564357058260 2024-04-25T00:31:11.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.304+0000 7fadefe00640 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x564357067df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.305+0000 7fadf7e00640 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x5643570697e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x564357050390 tx=0x7fade8000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6099aeaa91022e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.305+0000 7fade6000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:31:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.305+0000 7fade6000640 1 -- 172.21.15.42:0/3128382961 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fade802f020 con 0x564357058260 2024-04-25T00:31:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.305+0000 7fae01d66740 1 -- 172.21.15.42:0/3128382961 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadf00b9070 con 0x564357058260 2024-04-25T00:31:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.305+0000 7fade6000640 1 -- 172.21.15.42:0/3128382961 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fade80050b0 con 0x564357058260 2024-04-25T00:31:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.305+0000 7fade6000640 1 -- 172.21.15.42:0/3128382961 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fade801d040 con 0x564357058260 2024-04-25T00:31:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.305+0000 7fae01d66740 10 client.26188 fetch_fsmap learned FSMap version 31 2024-04-25T00:31:11.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.305+0000 7fae01d66740 1 -- 172.21.15.42:0/3128382961 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5643570d56f0 con 0x564357058260 2024-04-25T00:31:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.306+0000 7fade6000640 1 -- 172.21.15.42:0/3128382961 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fadf00b9070 con 0x564357058260 2024-04-25T00:31:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.306+0000 7fade6000640 1 -- 172.21.15.42:0/3128382961 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7fade802b5d0 con 0x564357058260 2024-04-25T00:31:11.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.306+0000 7fae01d66740 10 client.26188 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:31:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.306+0000 7fae01d66740 10 client.26188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.306+0000 7fae01d66740 20 client.26188 populate_metadata read hostname 'smithi042' 2024-04-25T00:31:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.306+0000 7fae01d66740 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643570d5e30 0x5643570d8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.306+0000 7fae01d66740 4 client.26188 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:31:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.306+0000 7fae01d66740 1 -- 172.21.15.42:0/3128382961 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564356fbe340 con 0x5643570d5e30 2024-04-25T00:31:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.307+0000 7fadefe00640 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643570d5e30 0x5643570d8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.307+0000 7fadefe00640 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643570d5e30 0x5643570d8250 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182393ddaef3b296 server_cookie=e4389fc472237889 in_seq=0 out_seq=0 2024-04-25T00:31:11.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.307+0000 7fade6000640 10 client.26188 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.376+0000 7fade6000640 1 -- 172.21.15.42:0/3128382961 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564356fbe340 con 0x5643570d5e30 2024-04-25T00:31:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.376+0000 7fade6000640 10 client.26188 handle_command_reply: tid=0 2024-04-25T00:31:11.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.376+0000 7fae01d66740 1 client.26188 shutdown 2024-04-25T00:31:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.376+0000 7fae01d66740 1 -- 172.21.15.42:0/3128382961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 msgr2=0x5643570697e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.376+0000 7fae01d66740 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x5643570697e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x564357050390 tx=0x7fade8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.377+0000 7fae01d66740 1 -- 172.21.15.42:0/3128382961 shutdown_connections 2024-04-25T00:31:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.377+0000 7fae01d66740 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5643570d5e30 0x5643570d8250 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.377+0000 7fae01d66740 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56435705ddc0 0x564357068330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.377+0000 7fade6000640 0 client.26188 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:11.378 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.377+0000 7fae01d66740 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564357058c50 0x564357067df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.377+0000 7fae01d66740 1 --2- 172.21.15.42:0/3128382961 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564357058260 0x5643570697e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.379 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.377+0000 7fae01d66740 1 -- 172.21.15.42:0/3128382961 >> 172.21.15.42:0/3128382961 conn(0x56435704f4c0 msgr2=0x564357052c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:31:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:31:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:31:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:31:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.377+0000 7fae01d66740 1 -- 172.21.15.42:0/3128382961 shutdown_connections 2024-04-25T00:31:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.377+0000 7fae01d66740 1 -- 172.21.15.42:0/3128382961 wait complete. 2024-04-25T00:31:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.378+0000 7fae01d66740 20 client.26188 trim_cache size 0 max 16384 2024-04-25T00:31:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.379+0000 7fadfd800640 1 -- 172.21.15.42:0/2157727313 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faddc0a2c00 msgr2=0x7faddc0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.379+0000 7fadfd800640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faddc0a2c00 0x7faddc0a50c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fade0031440 tx=0x7fade0005780 comp rx=0 tx=0).stop 2024-04-25T00:31:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.379+0000 7fadfd800640 1 -- 172.21.15.42:0/2157727313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf81031b0 msgr2=0x7fadf819eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.379+0000 7fadfd800640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf81031b0 0x7fadf819eeb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fadf000efd0 tx=0x7fadf000c6a0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.380+0000 7fadfd800640 1 -- 172.21.15.42:0/2157727313 shutdown_connections 2024-04-25T00:31:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.380+0000 7fadfd800640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7faddc0a2c00 0x7faddc0a50c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.380+0000 7fadfd800640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fadf81031b0 0x7fadf819eeb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.380+0000 7fadfd800640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadf81027f0 0x7fadf8197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.380+0000 7fadfd800640 1 --2- 172.21.15.42:0/2157727313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fadf8101600 0x7fadf8197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.380+0000 7fadfd800640 1 -- 172.21.15.42:0/2157727313 >> 172.21.15.42:0/2157727313 conn(0x7fadf80fcb70 msgr2=0x7fadf8105b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.381+0000 7fadfd800640 1 -- 172.21.15.42:0/2157727313 shutdown_connections 2024-04-25T00:31:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:11.381+0000 7fadfd800640 1 -- 172.21.15.42:0/2157727313 wait complete. 2024-04-25T00:31:11.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:11 smithi136 ceph-mon[32923]: pgmap v761: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-25T00:31:11.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:11 smithi106 ceph-mon[32510]: pgmap v761: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-25T00:31:11.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:11 smithi042 ceph-mon[28130]: pgmap v761: 97 pgs: 97 active+clean; 137 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 398 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-04-25T00:31:12.283 INFO:tasks.workunit.client.0.smithi042.stdout: 1 767418 6.14 MB/sec execute 542 sec latency 34.252 ms 2024-04-25T00:31:13.284 INFO:tasks.workunit.client.0.smithi042.stdout: 1 768595 6.14 MB/sec execute 543 sec latency 28.443 ms 2024-04-25T00:31:13.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:13 smithi136 ceph-mon[32923]: pgmap v762: 97 pgs: 97 active+clean; 139 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-04-25T00:31:13.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:13 smithi106 ceph-mon[32510]: pgmap v762: 97 pgs: 97 active+clean; 139 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-04-25T00:31:13.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:13 smithi042 ceph-mon[28130]: pgmap v762: 97 pgs: 97 active+clean; 139 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-04-25T00:31:14.283 INFO:tasks.workunit.client.0.smithi042.stdout: 1 770483 6.15 MB/sec execute 544 sec latency 21.267 ms 2024-04-25T00:31:15.284 INFO:tasks.workunit.client.0.smithi042.stdout: 1 771221 6.14 MB/sec execute 545 sec latency 21.838 ms 2024-04-25T00:31:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:15 smithi136 ceph-mon[32923]: pgmap v763: 97 pgs: 97 active+clean; 139 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-04-25T00:31:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:15 smithi106 ceph-mon[32510]: pgmap v763: 97 pgs: 97 active+clean; 139 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-04-25T00:31:15.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:15 smithi042 ceph-mon[28130]: pgmap v763: 97 pgs: 97 active+clean; 139 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-04-25T00:31:16.284 INFO:tasks.workunit.client.0.smithi042.stdout: 1 772194 6.14 MB/sec execute 546 sec latency 30.576 ms 2024-04-25T00:31:17.284 INFO:tasks.workunit.client.0.smithi042.stdout: 1 773884 6.15 MB/sec execute 547 sec latency 50.199 ms 2024-04-25T00:31:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:17 smithi136 ceph-mon[32923]: pgmap v764: 97 pgs: 97 active+clean; 141 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-04-25T00:31:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:17 smithi106 ceph-mon[32510]: pgmap v764: 97 pgs: 97 active+clean; 141 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-04-25T00:31:17.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:17 smithi042 ceph-mon[28130]: pgmap v764: 97 pgs: 97 active+clean; 141 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 635 KiB/s rd, 2.0 MiB/s wr, 245 op/s 2024-04-25T00:31:18.284 INFO:tasks.workunit.client.0.smithi042.stdout: 1 774606 6.14 MB/sec execute 548 sec latency 19.729 ms 2024-04-25T00:31:19.284 INFO:tasks.workunit.client.0.smithi042.stdout: 1 775697 6.14 MB/sec execute 549 sec latency 25.277 ms 2024-04-25T00:31:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:19 smithi136 ceph-mon[32923]: pgmap v765: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-04-25T00:31:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:19 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:31:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:19 smithi106 ceph-mon[32510]: pgmap v765: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-04-25T00:31:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:19 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:31:19.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:19 smithi042 ceph-mon[28130]: pgmap v765: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 1.4 MiB/s wr, 193 op/s 2024-04-25T00:31:19.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:19 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:31:20.284 INFO:tasks.workunit.client.0.smithi042.stdout: 1 777559 6.15 MB/sec execute 550 sec latency 26.771 ms 2024-04-25T00:31:21.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:31:21.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:31:21.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:31:21.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:31:21.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:31:21.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:31:21.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:31:21.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:31:21.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:31:21.284 INFO:tasks.workunit.client.0.smithi042.stdout: 1 778183 6.15 MB/sec execute 551 sec latency 24.682 ms 2024-04-25T00:31:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:21 smithi136 ceph-mon[32923]: pgmap v766: 97 pgs: 97 active+clean; 143 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-04-25T00:31:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:21 smithi106 ceph-mon[32510]: pgmap v766: 97 pgs: 97 active+clean; 143 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-04-25T00:31:22.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:21 smithi042 ceph-mon[28130]: pgmap v766: 97 pgs: 97 active+clean; 143 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 496 KiB/s rd, 1.6 MiB/s wr, 215 op/s 2024-04-25T00:31:22.284 INFO:tasks.workunit.client.0.smithi042.stdout: 1 779282 6.14 MB/sec execute 552 sec latency 42.610 ms 2024-04-25T00:31:23.284 INFO:tasks.workunit.client.0.smithi042.stdout: 1 780889 6.15 MB/sec execute 553 sec latency 23.521 ms 2024-04-25T00:31:23.930 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:23 smithi042 ceph-mon[28130]: pgmap v767: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2024-04-25T00:31:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:23 smithi136 ceph-mon[32923]: pgmap v767: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2024-04-25T00:31:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:23 smithi106 ceph-mon[32510]: pgmap v767: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.7 MiB/s wr, 228 op/s 2024-04-25T00:31:24.285 INFO:tasks.workunit.client.0.smithi042.stdout: 1 781644 6.15 MB/sec execute 554 sec latency 16.453 ms 2024-04-25T00:31:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.548+0000 7f8709000640 1 -- 172.21.15.42:0/2572607056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8704107dc0 msgr2=0x7f87041081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.548+0000 7f8709000640 1 --2- 172.21.15.42:0/2572607056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8704107dc0 0x7f87041081e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009900 tx=0x7f86f402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.548+0000 7f8709000640 1 -- 172.21.15.42:0/2572607056 shutdown_connections 2024-04-25T00:31:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.548+0000 7f8709000640 1 --2- 172.21.15.42:0/2572607056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8704109970 0x7f8704110200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.548+0000 7f8709000640 1 --2- 172.21.15.42:0/2572607056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8704108fb0 0x7f8704109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.548+0000 7f8709000640 1 --2- 172.21.15.42:0/2572607056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8704107dc0 0x7f87041081e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.548+0000 7f8709000640 1 -- 172.21.15.42:0/2572607056 >> 172.21.15.42:0/2572607056 conn(0x7f8704076d60 msgr2=0x7f87040791a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.548+0000 7f8709000640 1 -- 172.21.15.42:0/2572607056 shutdown_connections 2024-04-25T00:31:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.548+0000 7f8709000640 1 -- 172.21.15.42:0/2572607056 wait complete. 2024-04-25T00:31:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.549+0000 7f8709000640 1 Processor -- start 2024-04-25T00:31:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8709000640 1 -- start start 2024-04-25T00:31:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8709000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8704107dc0 0x7f870419e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8709000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8704108fb0 0x7f870419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8709000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8704109970 0x7f87041a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8709000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8704113a60 con 0x7f8704107dc0 2024-04-25T00:31:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8709000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87041138e0 con 0x7f8704108fb0 2024-04-25T00:31:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8709000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8704113be0 con 0x7f8704109970 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8702800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8704109970 0x7f87041a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8701400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8704108fb0 0x7f870419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8701e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8704107dc0 0x7f870419e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8702800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8704109970 0x7f87041a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47428/0 (socket says 172.21.15.42:47428) 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.550+0000 7f8702800640 1 -- 172.21.15.42:0/3379864404 learned_addr learned my addr 172.21.15.42:0/3379864404 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.551+0000 7f8702800640 1 -- 172.21.15.42:0/3379864404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8704108fb0 msgr2=0x7f870419e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.551+0000 7f8702800640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8704108fb0 0x7f870419e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.551+0000 7f8702800640 1 -- 172.21.15.42:0/3379864404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8704107dc0 msgr2=0x7f870419e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.551+0000 7f8702800640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8704107dc0 0x7f870419e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.551+0000 7f8702800640 1 -- 172.21.15.42:0/3379864404 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87041a5c60 con 0x7f8704109970 2024-04-25T00:31:24.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.551+0000 7f8702800640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8704109970 0x7f87041a55f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00eb10 tx=0x7f86fc00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.551+0000 7f86faa00640 1 -- 172.21.15.42:0/3379864404 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc004b20 con 0x7f8704109970 2024-04-25T00:31:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.551+0000 7f86faa00640 1 -- 172.21.15.42:0/3379864404 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86fc004cc0 con 0x7f8704109970 2024-04-25T00:31:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.551+0000 7f8709000640 1 -- 172.21.15.42:0/3379864404 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87041a5f50 con 0x7f8704109970 2024-04-25T00:31:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.551+0000 7f86faa00640 1 -- 172.21.15.42:0/3379864404 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc010760 con 0x7f8704109970 2024-04-25T00:31:24.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.552+0000 7f8709000640 1 -- 172.21.15.42:0/3379864404 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87041a6410 con 0x7f8704109970 2024-04-25T00:31:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.555+0000 7f870bfcf740 1 Processor -- start 2024-04-25T00:31:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.555+0000 7f86faa00640 1 -- 172.21.15.42:0/3379864404 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86fc005020 con 0x7f8704109970 2024-04-25T00:31:24.556 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f870bfcf740 1 -- start start 2024-04-25T00:31:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b96b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 0x55914b970190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b972c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f870bfcf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55914b8cdab0 con 0x55914b96b690 2024-04-25T00:31:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f870bfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55914b876640 con 0x55914b96aca0 2024-04-25T00:31:24.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f870bfcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55914b8c9a70 con 0x55914b970800 2024-04-25T00:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f8701400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b96b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f8701400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b96b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:57210/0 (socket says 172.21.15.42:57210) 2024-04-25T00:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f8701e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 0x55914b970190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f8701400640 1 -- 172.21.15.42:0/3338576478 learned_addr learned my addr 172.21.15.42:0/3338576478 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:24.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.556+0000 7f8701400640 1 --2- 172.21.15.42:0/3338576478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b972c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.557+0000 7f8701400640 1 -- 172.21.15.42:0/3338576478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 msgr2=0x55914b972c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.557+0000 7f8701400640 1 --2- 172.21.15.42:0/3338576478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b972c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.557+0000 7f8701400640 1 -- 172.21.15.42:0/3338576478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 msgr2=0x55914b970190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.557+0000 7f8701400640 1 --2- 172.21.15.42:0/3338576478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 0x55914b970190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.557+0000 7f8701400640 1 -- 172.21.15.42:0/3338576478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914b5e2210 con 0x55914b96aca0 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.557+0000 7f8701e00640 1 --2- 172.21.15.42:0/3338576478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 0x55914b970190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.557+0000 7f8701400640 1 --2- 172.21.15.42:0/3338576478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b96b0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0137e0 tx=0x7f86ec009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b456f1ba1ed6d3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.557+0000 7f86faa00640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f86e80a2c30 0x7f86e80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.558+0000 7f86faa00640 1 -- 172.21.15.42:0/3379864404 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f86fc0cf5c0 con 0x7f8704109970 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.558+0000 7f86f3400640 1 -- 172.21.15.42:0/3338576478 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec013e20 con 0x55914b96aca0 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.558+0000 7f86f3400640 1 -- 172.21.15.42:0/3338576478 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ec011cc0 con 0x55914b96aca0 2024-04-25T00:31:24.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.558+0000 7f86f3400640 1 -- 172.21.15.42:0/3338576478 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec01fcc0 con 0x55914b96aca0 2024-04-25T00:31:24.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.558+0000 7f8701e00640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f86e80a2c30 0x7f86e80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f870bfcf740 1 -- 172.21.15.42:0/3338576478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 msgr2=0x55914b96b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f870bfcf740 1 --2- 172.21.15.42:0/3338576478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b96b0c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0137e0 tx=0x7f86ec009370 comp rx=0 tx=0).stop 2024-04-25T00:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f8701e00640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f86e80a2c30 0x7f86e80a50f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f86f402a9e0 tx=0x7f86f40315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f870bfcf740 1 -- 172.21.15.42:0/3338576478 shutdown_connections 2024-04-25T00:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f870bfcf740 1 --2- 172.21.15.42:0/3338576478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b972c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f870bfcf740 1 --2- 172.21.15.42:0/3338576478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 0x55914b970190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f870bfcf740 1 --2- 172.21.15.42:0/3338576478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b96b0c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f870bfcf740 1 -- 172.21.15.42:0/3338576478 >> 172.21.15.42:0/3338576478 conn(0x55914b966310 msgr2=0x55914b966710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f870bfcf740 1 -- 172.21.15.42:0/3338576478 shutdown_connections 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f870bfcf740 1 -- 172.21.15.42:0/3338576478 wait complete. 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.559+0000 7f870bfcf740 1 -- start start 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f870bfcf740 1 Processor -- start 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96aca0 0x55914b974f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b975490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b97c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f870bfcf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55914b9737b0 con 0x55914b96aca0 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f870bfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55914b889990 con 0x55914b96b690 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f870bfcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55914b973930 con 0x55914b970800 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f870bfcf740 10 client.0 _ll_register_callbacks cb 0x55914b88b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f8701400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b975490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f8701e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96aca0 0x55914b974f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f8702800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b97c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.560+0000 7f8701400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b975490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:57212/0 (socket says 172.21.15.42:57212) 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.561+0000 7f8701400640 1 -- 172.21.15.42:0/3889461940 learned_addr learned my addr 172.21.15.42:0/3889461940 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.561+0000 7f8701400640 1 -- 172.21.15.42:0/3889461940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 msgr2=0x55914b97c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.561+0000 7f8701400640 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b97c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.561+0000 7f8701400640 1 -- 172.21.15.42:0/3889461940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96aca0 msgr2=0x55914b974f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.561+0000 7f8701400640 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96aca0 0x55914b974f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.561+0000 7f8701400640 1 -- 172.21.15.42:0/3889461940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914b6ae620 con 0x55914b96b690 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.561+0000 7f8702800640 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b97c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.561+0000 7f8701400640 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b975490 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86ec011a60 tx=0x7f86ec011a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b955a092653210 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.561+0000 7f86e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.562+0000 7f86e7e00640 1 -- 172.21.15.42:0/3889461940 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec01fe50 con 0x55914b96b690 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.562+0000 7f86e7e00640 1 -- 172.21.15.42:0/3889461940 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f86ec004040 con 0x55914b96b690 2024-04-25T00:31:24.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.562+0000 7f870bfcf740 1 -- 172.21.15.42:0/3889461940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ec011cc0 con 0x55914b96b690 2024-04-25T00:31:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.562+0000 7f86e7e00640 1 -- 172.21.15.42:0/3889461940 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f86ec004240 con 0x55914b96b690 2024-04-25T00:31:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.562+0000 7f870bfcf740 10 client.26192 fetch_fsmap learned FSMap version 31 2024-04-25T00:31:24.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.562+0000 7f870bfcf740 1 -- 172.21.15.42:0/3889461940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55914b97f4b0 con 0x55914b96b690 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.562+0000 7f86e7e00640 1 -- 172.21.15.42:0/3889461940 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ec011cc0 con 0x55914b96b690 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.563+0000 7f86e7e00640 1 -- 172.21.15.42:0/3889461940 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f86ec0046c0 con 0x55914b96b690 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.563+0000 7f870bfcf740 10 client.26192 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.563+0000 7f870bfcf740 10 client.26192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.563+0000 7f870bfcf740 20 client.26192 populate_metadata read hostname 'smithi042' 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.563+0000 7f870bfcf740 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55914b9809e0 0x55914b982dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.563+0000 7f870bfcf740 4 client.26192 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.563+0000 7f870bfcf740 1 -- 172.21.15.42:0/3889461940 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55914b8daff0 con 0x55914b9809e0 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.563+0000 7f8701e00640 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55914b9809e0 0x55914b982dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.564+0000 7f8701e00640 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55914b9809e0 0x55914b982dd0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93873b04d84d0151 server_cookie=5e176ed0b07c6287 in_seq=0 out_seq=0 2024-04-25T00:31:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.564+0000 7f86e7e00640 10 client.26192 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f86e7e00640 1 -- 172.21.15.42:0/3889461940 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55914b8daff0 con 0x55914b9809e0 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f86e7e00640 10 client.26192 handle_command_reply: tid=0 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 client.26192 shutdown 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 -- 172.21.15.42:0/3889461940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 msgr2=0x55914b975490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b975490 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f86ec011a60 tx=0x7f86ec011a90 comp rx=0 tx=0).stop 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 -- 172.21.15.42:0/3889461940 shutdown_connections 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55914b9809e0 0x55914b982dd0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b97c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b975490 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f86e7e00640 0 client.26192 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 --2- 172.21.15.42:0/3889461940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96aca0 0x55914b974f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 -- 172.21.15.42:0/3889461940 >> 172.21.15.42:0/3889461940 conn(0x55914b966310 msgr2=0x55914b965480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 -- 172.21.15.42:0/3889461940 shutdown_connections 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 1 -- 172.21.15.42:0/3889461940 wait complete. 2024-04-25T00:31:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.565+0000 7f870bfcf740 20 client.26192 trim_cache size 0 max 16384 2024-04-25T00:31:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.583+0000 7f870bfcf740 1 Processor -- start 2024-04-25T00:31:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.583+0000 7f870bfcf740 1 -- start start 2024-04-25T00:31:24.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.583+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b96aca0 0x55914b974240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.583+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b97c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.583+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b970800 0x55914b974780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.583+0000 7f870bfcf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f87041b1960 con 0x55914b970800 2024-04-25T00:31:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.583+0000 7f870bfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87041af230 con 0x55914b96b690 2024-04-25T00:31:24.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.583+0000 7f870bfcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87041ad080 con 0x55914b96aca0 2024-04-25T00:31:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b97c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b97c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:57218/0 (socket says 172.21.15.42:57218) 2024-04-25T00:31:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b96aca0 0x55914b974240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701400640 1 -- 172.21.15.42:0/2254389132 learned_addr learned my addr 172.21.15.42:0/2254389132 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8702800640 1 --2- 172.21.15.42:0/2254389132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b970800 0x55914b974780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701400640 1 -- 172.21.15.42:0/2254389132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b96aca0 msgr2=0x55914b974240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701400640 1 --2- 172.21.15.42:0/2254389132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b96aca0 0x55914b974240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701400640 1 -- 172.21.15.42:0/2254389132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b970800 msgr2=0x55914b974780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:24.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701400640 1 --2- 172.21.15.42:0/2254389132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b970800 0x55914b974780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701400640 1 -- 172.21.15.42:0/2254389132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914b96a8e0 con 0x55914b96b690 2024-04-25T00:31:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701e00640 1 --2- 172.21.15.42:0/2254389132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b96aca0 0x55914b974240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:31:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.584+0000 7f8701400640 1 --2- 172.21.15.42:0/2254389132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b97c490 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0097e0 tx=0x7f86ec009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b33aa9f1d3eebafd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.585+0000 7f86f2a00640 1 -- 172.21.15.42:0/2254389132 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec009cd0 con 0x55914b96b690 2024-04-25T00:31:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.585+0000 7f86f2a00640 1 -- 172.21.15.42:0/2254389132 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ec01f780 con 0x55914b96b690 2024-04-25T00:31:24.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.585+0000 7f86f2a00640 1 -- 172.21.15.42:0/2254389132 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec011b00 con 0x55914b96b690 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.586+0000 7f870bfcf740 1 -- 172.21.15.42:0/2254389132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 msgr2=0x55914b97c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.586+0000 7f870bfcf740 1 --2- 172.21.15.42:0/2254389132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b97c490 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0097e0 tx=0x7f86ec009370 comp rx=0 tx=0).stop 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.586+0000 7f870bfcf740 1 -- 172.21.15.42:0/2254389132 shutdown_connections 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.586+0000 7f870bfcf740 1 --2- 172.21.15.42:0/2254389132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b970800 0x55914b974780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.586+0000 7f870bfcf740 1 --2- 172.21.15.42:0/2254389132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96b690 0x55914b97c490 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.586+0000 7f870bfcf740 1 --2- 172.21.15.42:0/2254389132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b96aca0 0x55914b974240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.586+0000 7f870bfcf740 1 -- 172.21.15.42:0/2254389132 >> 172.21.15.42:0/2254389132 conn(0x55914b966310 msgr2=0x55914b96ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.586+0000 7f870bfcf740 1 -- 172.21.15.42:0/2254389132 shutdown_connections 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.586+0000 7f870bfcf740 1 -- 172.21.15.42:0/2254389132 wait complete. 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.586+0000 7f870bfcf740 1 -- start start 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f870bfcf740 1 Processor -- start 2024-04-25T00:31:24.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b9797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 0x55914b979cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f870bfcf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b97acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f870bfcf740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f87041ad6f0 con 0x55914b96b690 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f870bfcf740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f87041aeb60 con 0x55914b96aca0 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f870bfcf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f87041addc0 con 0x55914b970800 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f870bfcf740 10 client.0 _ll_register_callbacks cb 0x7f86ec005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f8701e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b9797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f8701e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b9797b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:57226/0 (socket says 172.21.15.42:57226) 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f8701400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 0x55914b979cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f8701e00640 1 -- 172.21.15.42:0/2370309693 learned_addr learned my addr 172.21.15.42:0/2370309693 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.587+0000 7f8702800640 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b97acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.588+0000 7f8701e00640 1 -- 172.21.15.42:0/2370309693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 msgr2=0x55914b97acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.588+0000 7f8701e00640 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b97acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.588+0000 7f8701e00640 1 -- 172.21.15.42:0/2370309693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 msgr2=0x55914b979cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:24.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.588+0000 7f8701e00640 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 0x55914b979cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.588+0000 7f8701e00640 1 -- 172.21.15.42:0/2370309693 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914b966fe0 con 0x55914b96aca0 2024-04-25T00:31:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.588+0000 7f8701e00640 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b9797b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86f4004650 tx=0x7f86f4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1282a65f34e70919 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.588+0000 7f86e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:31:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.588+0000 7f86e7e00640 1 -- 172.21.15.42:0/2370309693 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4035040 con 0x55914b96aca0 2024-04-25T00:31:24.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.588+0000 7f86e7e00640 1 -- 172.21.15.42:0/2370309693 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f86f4054500 con 0x55914b96aca0 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.588+0000 7f870bfcf740 1 -- 172.21.15.42:0/2370309693 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ec01f780 con 0x55914b96aca0 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f86e7e00640 1 -- 172.21.15.42:0/2370309693 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f86f4064070 con 0x55914b96aca0 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f870bfcf740 10 client.26198 fetch_fsmap learned FSMap version 31 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f870bfcf740 1 -- 172.21.15.42:0/2370309693 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55914b97c450 con 0x55914b96aca0 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f86e7e00640 1 -- 172.21.15.42:0/2370309693 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ec01f780 con 0x55914b96aca0 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f86e7e00640 1 -- 172.21.15.42:0/2370309693 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f86f40326c0 con 0x55914b96aca0 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f870bfcf740 10 client.26198 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f870bfcf740 10 client.26198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f870bfcf740 20 client.26198 populate_metadata read hostname 'smithi042' 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f870bfcf740 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55914b9e7320 0x55914b9e9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f870bfcf740 4 client.26198 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:31:24.593 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f870bfcf740 1 -- 172.21.15.42:0/2370309693 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55914b8d5340 con 0x55914b9e7320 2024-04-25T00:31:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.589+0000 7f8701400640 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55914b9e7320 0x55914b9e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.590+0000 7f8701400640 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55914b9e7320 0x55914b9e9740 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c592b26fadb9fde1 server_cookie=20404c8839c05bb1 in_seq=0 out_seq=0 2024-04-25T00:31:24.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.590+0000 7f86e7e00640 10 client.26198 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f86e7e00640 1 -- 172.21.15.42:0/2370309693 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55914b8d5340 con 0x55914b9e7320 2024-04-25T00:31:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f86e7e00640 10 client.26198 handle_command_reply: tid=0 2024-04-25T00:31:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f870bfcf740 1 client.26198 shutdown 2024-04-25T00:31:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f870bfcf740 1 -- 172.21.15.42:0/2370309693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 msgr2=0x55914b9797b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f870bfcf740 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b9797b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f86f4004650 tx=0x7f86f4009260 comp rx=0 tx=0).stop 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f870bfcf740 1 -- 172.21.15.42:0/2370309693 shutdown_connections 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f870bfcf740 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55914b9e7320 0x55914b9e9740 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f870bfcf740 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55914b970800 0x55914b97acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f86e7e00640 0 client.26198 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f870bfcf740 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55914b96b690 0x55914b979cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f870bfcf740 1 --2- 172.21.15.42:0/2370309693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55914b96aca0 0x55914b9797b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.650+0000 7f870bfcf740 1 -- 172.21.15.42:0/2370309693 >> 172.21.15.42:0/2370309693 conn(0x55914b966310 msgr2=0x55914b974240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.651+0000 7f870bfcf740 1 -- 172.21.15.42:0/2370309693 shutdown_connections 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.651+0000 7f870bfcf740 1 -- 172.21.15.42:0/2370309693 wait complete. 2024-04-25T00:31:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.651+0000 7f870bfcf740 20 client.26198 trim_cache size 0 max 16384 2024-04-25T00:31:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:31:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:31:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:31:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:31:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 -- 172.21.15.42:0/3379864404 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f86e80a2c30 msgr2=0x7f86e80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f86e80a2c30 0x7f86e80a50f0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f86f402a9e0 tx=0x7f86f40315b0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 -- 172.21.15.42:0/3379864404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8704109970 msgr2=0x7f87041a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8704109970 0x7f87041a55f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00eb10 tx=0x7f86fc00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 -- 172.21.15.42:0/3379864404 shutdown_connections 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f86e80a2c30 0x7f86e80a50f0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8704109970 0x7f87041a55f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8704108fb0 0x7f870419e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 --2- 172.21.15.42:0/3379864404 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8704107dc0 0x7f870419e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 -- 172.21.15.42:0/3379864404 >> 172.21.15.42:0/3379864404 conn(0x7f8704076d60 msgr2=0x7f870410c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 -- 172.21.15.42:0/3379864404 shutdown_connections 2024-04-25T00:31:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:24.652+0000 7f8709000640 1 -- 172.21.15.42:0/3379864404 wait complete. 2024-04-25T00:31:25.285 INFO:tasks.workunit.client.0.smithi042.stdout: 1 782658 6.14 MB/sec execute 555 sec latency 52.242 ms 2024-04-25T00:31:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:25 smithi136 ceph-mon[32923]: pgmap v768: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-25T00:31:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:25 smithi106 ceph-mon[32510]: pgmap v768: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-25T00:31:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:25 smithi042 ceph-mon[28130]: pgmap v768: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 1.3 MiB/s wr, 164 op/s 2024-04-25T00:31:26.285 INFO:tasks.workunit.client.0.smithi042.stdout: 1 784032 6.15 MB/sec execute 556 sec latency 26.933 ms 2024-04-25T00:31:27.285 INFO:tasks.workunit.client.0.smithi042.stdout: 1 785051 6.15 MB/sec execute 557 sec latency 32.261 ms 2024-04-25T00:31:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:27 smithi136 ceph-mon[32923]: pgmap v769: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-04-25T00:31:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:27 smithi106 ceph-mon[32510]: pgmap v769: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-04-25T00:31:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:27 smithi042 ceph-mon[28130]: pgmap v769: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 1.7 MiB/s wr, 235 op/s 2024-04-25T00:31:28.285 INFO:tasks.workunit.client.0.smithi042.stdout: 1 785868 6.14 MB/sec execute 558 sec latency 18.072 ms 2024-04-25T00:31:29.285 INFO:tasks.workunit.client.0.smithi042.stdout: 1 787071 6.14 MB/sec execute 559 sec latency 39.176 ms 2024-04-25T00:31:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:29 smithi136 ceph-mon[32923]: pgmap v770: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-25T00:31:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:29 smithi106 ceph-mon[32510]: pgmap v770: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-25T00:31:30.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:29 smithi042 ceph-mon[28130]: pgmap v770: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-04-25T00:31:30.285 INFO:tasks.workunit.client.0.smithi042.stdout: 1 788458 6.15 MB/sec execute 560 sec latency 35.499 ms 2024-04-25T00:31:31.286 INFO:tasks.workunit.client.0.smithi042.stdout: 1 789039 6.14 MB/sec execute 561 sec latency 24.917 ms 2024-04-25T00:31:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:31 smithi136 ceph-mon[32923]: pgmap v771: 97 pgs: 97 active+clean; 142 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-04-25T00:31:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:31 smithi106 ceph-mon[32510]: pgmap v771: 97 pgs: 97 active+clean; 142 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-04-25T00:31:32.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:31 smithi042 ceph-mon[28130]: pgmap v771: 97 pgs: 97 active+clean; 142 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-04-25T00:31:32.286 INFO:tasks.workunit.client.0.smithi042.stdout: 1 790369 6.14 MB/sec execute 562 sec latency 49.059 ms 2024-04-25T00:31:33.286 INFO:tasks.workunit.client.0.smithi042.stdout: 1 791919 6.15 MB/sec execute 563 sec latency 21.229 ms 2024-04-25T00:31:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:33 smithi136 ceph-mon[32923]: pgmap v772: 97 pgs: 97 active+clean; 143 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-04-25T00:31:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:33 smithi106 ceph-mon[32510]: pgmap v772: 97 pgs: 97 active+clean; 143 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-04-25T00:31:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:33 smithi042 ceph-mon[28130]: pgmap v772: 97 pgs: 97 active+clean; 143 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 503 KiB/s rd, 1.4 MiB/s wr, 185 op/s 2024-04-25T00:31:34.286 INFO:tasks.workunit.client.0.smithi042.stdout: 1 792392 6.14 MB/sec execute 564 sec latency 87.362 ms 2024-04-25T00:31:35.286 INFO:tasks.workunit.client.0.smithi042.stdout: 1 793646 6.14 MB/sec execute 565 sec latency 24.927 ms 2024-04-25T00:31:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:35 smithi136 ceph-mon[32923]: pgmap v773: 97 pgs: 97 active+clean; 143 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-04-25T00:31:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:35 smithi106 ceph-mon[32510]: pgmap v773: 97 pgs: 97 active+clean; 143 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-04-25T00:31:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:35 smithi042 ceph-mon[28130]: pgmap v773: 97 pgs: 97 active+clean; 143 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 394 KiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-04-25T00:31:36.286 INFO:tasks.workunit.client.0.smithi042.stdout: 1 795456 6.15 MB/sec execute 566 sec latency 21.977 ms 2024-04-25T00:31:37.286 INFO:tasks.workunit.client.0.smithi042.stdout: 1 796133 6.14 MB/sec execute 567 sec latency 18.669 ms 2024-04-25T00:31:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.810+0000 7efc8e600640 1 -- 172.21.15.42:0/3575706857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc88101130 msgr2=0x7efc881015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.810+0000 7efc8e600640 1 --2- 172.21.15.42:0/3575706857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc88101130 0x7efc881015b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efc80009ca0 tx=0x7efc8002a8b0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.810+0000 7efc8e600640 1 -- 172.21.15.42:0/3575706857 shutdown_connections 2024-04-25T00:31:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.810+0000 7efc8e600640 1 --2- 172.21.15.42:0/3575706857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc88101130 0x7efc881015b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.810+0000 7efc8e600640 1 --2- 172.21.15.42:0/3575706857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc88100790 0x7efc88100bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.810+0000 7efc8e600640 1 --2- 172.21.15.42:0/3575706857 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc880ffe30 0x7efc88100250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.810+0000 7efc8e600640 1 -- 172.21.15.42:0/3575706857 >> 172.21.15.42:0/3575706857 conn(0x7efc880fba30 msgr2=0x7efc880fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.811+0000 7efc8e600640 1 -- 172.21.15.42:0/3575706857 shutdown_connections 2024-04-25T00:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.811+0000 7efc8e600640 1 -- 172.21.15.42:0/3575706857 wait complete. 2024-04-25T00:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.812+0000 7efc8e600640 1 Processor -- start 2024-04-25T00:31:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.812+0000 7efc8e600640 1 -- start start 2024-04-25T00:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.812+0000 7efc8e600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc880ffe30 0x7efc88199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.812+0000 7efc8e600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc88100790 0x7efc88199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.812+0000 7efc8e600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc88101130 0x7efc881a1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8e600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efc8810f480 con 0x7efc88100790 2024-04-25T00:31:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8e600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efc8810f300 con 0x7efc88101130 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8e600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc8810f600 con 0x7efc880ffe30 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc880ffe30 0x7efc88199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc880ffe30 0x7efc88199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33166/0 (socket says 172.21.15.42:33166) 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8d200640 1 -- 172.21.15.42:0/1580662463 learned_addr learned my addr 172.21.15.42:0/1580662463 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc87e00640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc88100790 0x7efc88199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8dc00640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc88101130 0x7efc881a1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8d200640 1 -- 172.21.15.42:0/1580662463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc88101130 msgr2=0x7efc881a1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8d200640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc88101130 0x7efc881a1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8d200640 1 -- 172.21.15.42:0/1580662463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc88100790 msgr2=0x7efc88199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8d200640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc88100790 0x7efc88199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.813+0000 7efc8d200640 1 -- 172.21.15.42:0/1580662463 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc881a16b0 con 0x7efc880ffe30 2024-04-25T00:31:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.814+0000 7efc8d200640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc880ffe30 0x7efc88199a90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00e9f0 tx=0x7efc7c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.814+0000 7efc85600640 1 -- 172.21.15.42:0/1580662463 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c00cd80 con 0x7efc880ffe30 2024-04-25T00:31:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.814+0000 7efc85600640 1 -- 172.21.15.42:0/1580662463 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7c010910 con 0x7efc880ffe30 2024-04-25T00:31:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.814+0000 7efc8e600640 1 -- 172.21.15.42:0/1580662463 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc881a19a0 con 0x7efc880ffe30 2024-04-25T00:31:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.814+0000 7efc85600640 1 -- 172.21.15.42:0/1580662463 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c018990 con 0x7efc880ffe30 2024-04-25T00:31:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.814+0000 7efc8e600640 1 -- 172.21.15.42:0/1580662463 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc881a1ef0 con 0x7efc880ffe30 2024-04-25T00:31:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.817+0000 7efc85600640 1 -- 172.21.15.42:0/1580662463 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc7c018b30 con 0x7efc880ffe30 2024-04-25T00:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.818+0000 7efc92a45740 1 Processor -- start 2024-04-25T00:31:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.818+0000 7efc92a45740 1 -- start start 2024-04-25T00:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.818+0000 7efc92a45740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232ce50 0x56285232d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.818+0000 7efc92a45740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232d840 0x562852332340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.818+0000 7efc92a45740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 0x562852334e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.818+0000 7efc92a45740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56285228fab0 con 0x56285232d840 2024-04-25T00:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.818+0000 7efc92a45740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x562852238640 con 0x5628523329b0 2024-04-25T00:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.818+0000 7efc92a45740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56285228ba70 con 0x56285232ce50 2024-04-25T00:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc8dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232ce50 0x56285232d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc8dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232ce50 0x56285232d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33172/0 (socket says 172.21.15.42:33172) 2024-04-25T00:31:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc8dc00640 1 -- 172.21.15.42:0/2667581109 learned_addr learned my addr 172.21.15.42:0/2667581109 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:37.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc87e00640 1 --2- 172.21.15.42:0/2667581109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 0x562852334e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc8d200640 1 --2- 172.21.15.42:0/2667581109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232d840 0x562852332340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc8dc00640 1 -- 172.21.15.42:0/2667581109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 msgr2=0x562852334e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc8dc00640 1 --2- 172.21.15.42:0/2667581109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 0x562852334e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc8dc00640 1 -- 172.21.15.42:0/2667581109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232d840 msgr2=0x562852332340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc8dc00640 1 --2- 172.21.15.42:0/2667581109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232d840 0x562852332340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc8dc00640 1 -- 172.21.15.42:0/2667581109 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562851fa4210 con 0x56285232ce50 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.819+0000 7efc8d200640 1 --2- 172.21.15.42:0/2667581109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232d840 0x562852332340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.820+0000 7efc85600640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc700a2b80 0x7efc700a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.820+0000 7efc85600640 1 -- 172.21.15.42:0/1580662463 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7efc7c0cf450 con 0x7efc880ffe30 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.820+0000 7efc8dc00640 1 --2- 172.21.15.42:0/2667581109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232ce50 0x56285232d270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efc80030990 tx=0x7efc80005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa8843fc267250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.820+0000 7efc8d200640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc700a2b80 0x7efc700a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.820+0000 7efc7a000640 1 -- 172.21.15.42:0/2667581109 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80030a30 con 0x56285232ce50 2024-04-25T00:31:37.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.821+0000 7efc7a000640 1 -- 172.21.15.42:0/2667581109 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc80036040 con 0x56285232ce50 2024-04-25T00:31:37.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.821+0000 7efc7a000640 1 -- 172.21.15.42:0/2667581109 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc80004460 con 0x56285232ce50 2024-04-25T00:31:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.821+0000 7efc92a45740 1 -- 172.21.15.42:0/2667581109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232ce50 msgr2=0x56285232d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.821+0000 7efc92a45740 1 --2- 172.21.15.42:0/2667581109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232ce50 0x56285232d270 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7efc80030990 tx=0x7efc80005ef0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.821+0000 7efc8d200640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc700a2b80 0x7efc700a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc7c09bc10 tx=0x7efc7c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.822+0000 7efc92a45740 1 -- 172.21.15.42:0/2667581109 shutdown_connections 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.822+0000 7efc92a45740 1 --2- 172.21.15.42:0/2667581109 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 0x562852334e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.822+0000 7efc92a45740 1 --2- 172.21.15.42:0/2667581109 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232d840 0x562852332340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.822+0000 7efc92a45740 1 --2- 172.21.15.42:0/2667581109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232ce50 0x56285232d270 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.822+0000 7efc92a45740 1 -- 172.21.15.42:0/2667581109 >> 172.21.15.42:0/2667581109 conn(0x5628523284c0 msgr2=0x5628523288c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.822+0000 7efc92a45740 1 -- 172.21.15.42:0/2667581109 shutdown_connections 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.822+0000 7efc92a45740 1 -- 172.21.15.42:0/2667581109 wait complete. 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.822+0000 7efc92a45740 1 -- start start 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.822+0000 7efc92a45740 1 Processor -- start 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc92a45740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x562852337090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc92a45740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x5628523375d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc92a45740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628523329b0 0x56285233e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc92a45740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x562852335900 con 0x56285232ce50 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc92a45740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56285224b990 con 0x56285232d840 2024-04-25T00:31:37.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc92a45740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562852335a80 con 0x5628523329b0 2024-04-25T00:31:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc92a45740 10 client.0 _ll_register_callbacks cb 0x56285224d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc87e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x5628523375d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc87e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x5628523375d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:52848/0 (socket says 172.21.15.42:52848) 2024-04-25T00:31:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc87e00640 1 -- 172.21.15.42:0/2175936136 learned_addr learned my addr 172.21.15.42:0/2175936136 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:37.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.823+0000 7efc8dc00640 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x562852337090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.824+0000 7efc87e00640 1 -- 172.21.15.42:0/2175936136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628523329b0 msgr2=0x56285233e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.824+0000 7efc87e00640 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628523329b0 0x56285233e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.824+0000 7efc87e00640 1 -- 172.21.15.42:0/2175936136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 msgr2=0x562852337090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.824+0000 7efc87e00640 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x562852337090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.824+0000 7efc87e00640 1 -- 172.21.15.42:0/2175936136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562852070620 con 0x56285232d840 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.824+0000 7efc87e00640 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x5628523375d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc74012720 tx=0x7efc74012be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d266fc7804f9caff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.824+0000 7efc6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.824+0000 7efc6ea00640 1 -- 172.21.15.42:0/2175936136 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7400f920 con 0x56285232d840 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.824+0000 7efc92a45740 1 -- 172.21.15.42:0/2175936136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc80036040 con 0x56285232d840 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.824+0000 7efc6ea00640 1 -- 172.21.15.42:0/2175936136 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7efc7400fac0 con 0x56285232d840 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.825+0000 7efc6ea00640 1 -- 172.21.15.42:0/2175936136 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efc7400fcc0 con 0x56285232d840 2024-04-25T00:31:37.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.825+0000 7efc92a45740 10 client.26210 fetch_fsmap learned FSMap version 31 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.825+0000 7efc92a45740 1 -- 172.21.15.42:0/2175936136 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x562852341660 con 0x56285232d840 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.825+0000 7efc6ea00640 1 -- 172.21.15.42:0/2175936136 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc80036040 con 0x56285232d840 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.825+0000 7efc6ea00640 1 -- 172.21.15.42:0/2175936136 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7efc74027440 con 0x56285232d840 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.825+0000 7efc92a45740 10 client.26210 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.825+0000 7efc92a45740 10 client.26210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.825+0000 7efc92a45740 20 client.26210 populate_metadata read hostname 'smithi042' 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.826+0000 7efc92a45740 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x562852342d90 0x562852345210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.826+0000 7efc92a45740 4 client.26210 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.826+0000 7efc92a45740 1 -- 172.21.15.42:0/2175936136 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56285229cff0 con 0x562852342d90 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.826+0000 7efc8dc00640 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x562852342d90 0x562852345210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.826+0000 7efc8dc00640 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x562852342d90 0x562852345210 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b96a5b8f05bed34 server_cookie=32c47c13384a8f83 in_seq=0 out_seq=0 2024-04-25T00:31:37.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.826+0000 7efc6ea00640 10 client.26210 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.827+0000 7efc6ea00640 1 -- 172.21.15.42:0/2175936136 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56285229cff0 con 0x562852342d90 2024-04-25T00:31:37.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.827+0000 7efc6ea00640 10 client.26210 handle_command_reply: tid=0 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.827+0000 7efc92a45740 1 client.26210 shutdown 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 1 -- 172.21.15.42:0/2175936136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 msgr2=0x5628523375d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x5628523375d0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc74012720 tx=0x7efc74012be0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 1 -- 172.21.15.42:0/2175936136 shutdown_connections 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x562852342d90 0x562852345210 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628523329b0 0x56285233e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x5628523375d0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 1 --2- 172.21.15.42:0/2175936136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x562852337090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 1 -- 172.21.15.42:0/2175936136 >> 172.21.15.42:0/2175936136 conn(0x5628523284c0 msgr2=0x562852327620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc6ea00640 0 client.26210 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 1 -- 172.21.15.42:0/2175936136 shutdown_connections 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 1 -- 172.21.15.42:0/2175936136 wait complete. 2024-04-25T00:31:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.828+0000 7efc92a45740 20 client.26210 trim_cache size 0 max 16384 2024-04-25T00:31:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.846+0000 7efc92a45740 1 Processor -- start 2024-04-25T00:31:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.846+0000 7efc92a45740 1 -- start start 2024-04-25T00:31:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.846+0000 7efc92a45740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x562852335e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.846+0000 7efc92a45740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232d840 0x56285233e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.846+0000 7efc92a45740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 0x562852336390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.846+0000 7efc92a45740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efc881ad590 con 0x56285232ce50 2024-04-25T00:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.846+0000 7efc92a45740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efc881aae60 con 0x5628523329b0 2024-04-25T00:31:37.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.846+0000 7efc92a45740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a8cb0 con 0x56285232d840 2024-04-25T00:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.846+0000 7efc87e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232d840 0x56285233e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc8dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x562852335e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc87e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232d840 0x56285233e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33188/0 (socket says 172.21.15.42:33188) 2024-04-25T00:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc87e00640 1 -- 172.21.15.42:0/1731810475 learned_addr learned my addr 172.21.15.42:0/1731810475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc8dc00640 1 --2- 172.21.15.42:0/1731810475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 0x562852336390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc87e00640 1 -- 172.21.15.42:0/1731810475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 msgr2=0x562852336390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc87e00640 1 --2- 172.21.15.42:0/1731810475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 0x562852336390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc87e00640 1 -- 172.21.15.42:0/1731810475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 msgr2=0x562852335e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:37.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc87e00640 1 --2- 172.21.15.42:0/1731810475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x562852335e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc87e00640 1 -- 172.21.15.42:0/1731810475 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628523282f0 con 0x56285232d840 2024-04-25T00:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc8dc00640 1 --2- 172.21.15.42:0/1731810475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 0x562852336390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.847+0000 7efc87e00640 1 --2- 172.21.15.42:0/1731810475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232d840 0x56285233e5d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc74009e20 tx=0x7efc74009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8777f50b8b4aef2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.848+0000 7efc79600640 1 -- 172.21.15.42:0/1731810475 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7402b030 con 0x56285232d840 2024-04-25T00:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.848+0000 7efc79600640 1 -- 172.21.15.42:0/1731810475 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7400fe60 con 0x56285232d840 2024-04-25T00:31:37.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.848+0000 7efc79600640 1 -- 172.21.15.42:0/1731810475 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc74027d70 con 0x56285232d840 2024-04-25T00:31:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.848+0000 7efc92a45740 1 -- 172.21.15.42:0/1731810475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232d840 msgr2=0x56285233e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:37.852 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.848+0000 7efc92a45740 1 --2- 172.21.15.42:0/1731810475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232d840 0x56285233e5d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7efc74009e20 tx=0x7efc74009ec0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.849+0000 7efc92a45740 1 -- 172.21.15.42:0/1731810475 shutdown_connections 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.849+0000 7efc92a45740 1 --2- 172.21.15.42:0/1731810475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5628523329b0 0x562852336390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.849+0000 7efc92a45740 1 --2- 172.21.15.42:0/1731810475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56285232d840 0x56285233e5d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.849+0000 7efc92a45740 1 --2- 172.21.15.42:0/1731810475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x562852335e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.849+0000 7efc92a45740 1 -- 172.21.15.42:0/1731810475 >> 172.21.15.42:0/1731810475 conn(0x5628523284c0 msgr2=0x562852337090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.849+0000 7efc92a45740 1 -- 172.21.15.42:0/1731810475 shutdown_connections 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.849+0000 7efc92a45740 1 -- 172.21.15.42:0/1731810475 wait complete. 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.849+0000 7efc92a45740 1 -- start start 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.849+0000 7efc92a45740 1 Processor -- start 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc92a45740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x56285233bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc92a45740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x56285233c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc92a45740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628523329b0 0x56285233d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc92a45740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a9320 con 0x56285232ce50 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc92a45740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efc881aa790 con 0x56285232d840 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc92a45740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc881a99f0 con 0x5628523329b0 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc92a45740 10 client.0 _ll_register_callbacks cb 0x7efc74009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc8dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x56285233bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc87e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x56285233c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc87e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x56285233c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:52858/0 (socket says 172.21.15.42:52858) 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc87e00640 1 -- 172.21.15.42:0/3053017532 learned_addr learned my addr 172.21.15.42:0/3053017532 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc8dc00640 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628523329b0 0x56285233d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc87e00640 1 -- 172.21.15.42:0/3053017532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628523329b0 msgr2=0x56285233d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc87e00640 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628523329b0 0x56285233d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc87e00640 1 -- 172.21.15.42:0/3053017532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 msgr2=0x56285233bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.850+0000 7efc87e00640 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x56285233bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.851+0000 7efc87e00640 1 -- 172.21.15.42:0/3053017532 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285233d7d0 con 0x56285232d840 2024-04-25T00:31:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.851+0000 7efc87e00640 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x56285233c290 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efc74027800 tx=0x7efc74027830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=333472a1a3d27f01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.851+0000 7efc6ea00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:31:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.851+0000 7efc6ea00640 1 -- 172.21.15.42:0/3053017532 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7402b070 con 0x56285232d840 2024-04-25T00:31:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.851+0000 7efc6ea00640 1 -- 172.21.15.42:0/3053017532 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7efc740050b0 con 0x56285232d840 2024-04-25T00:31:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.851+0000 7efc92a45740 1 -- 172.21.15.42:0/3053017532 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc7400fe60 con 0x56285232d840 2024-04-25T00:31:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.851+0000 7efc6ea00640 1 -- 172.21.15.42:0/3053017532 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efc740052b0 con 0x56285232d840 2024-04-25T00:31:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.851+0000 7efc92a45740 10 client.26216 fetch_fsmap learned FSMap version 31 2024-04-25T00:31:37.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.851+0000 7efc92a45740 1 -- 172.21.15.42:0/3053017532 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5628523a9390 con 0x56285232d840 2024-04-25T00:31:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.852+0000 7efc6ea00640 1 -- 172.21.15.42:0/3053017532 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc7400fe60 con 0x56285232d840 2024-04-25T00:31:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.852+0000 7efc6ea00640 1 -- 172.21.15.42:0/3053017532 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7efc740104c0 con 0x56285232d840 2024-04-25T00:31:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.852+0000 7efc92a45740 10 client.26216 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:31:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.852+0000 7efc92a45740 10 client.26216 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.852+0000 7efc92a45740 20 client.26216 populate_metadata read hostname 'smithi042' 2024-04-25T00:31:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.852+0000 7efc92a45740 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5628523a9b80 0x5628523abfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.852+0000 7efc92a45740 4 client.26216 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:31:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.852+0000 7efc92a45740 1 -- 172.21.15.42:0/3053017532 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562852297340 con 0x5628523a9b80 2024-04-25T00:31:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.853+0000 7efc8dc00640 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5628523a9b80 0x5628523abfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.853+0000 7efc8dc00640 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5628523a9b80 0x5628523abfa0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c15a3a6b1022eec server_cookie=54eea9e4adeea8ea in_seq=0 out_seq=0 2024-04-25T00:31:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.853+0000 7efc6ea00640 10 client.26216 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:37.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc6ea00640 1 -- 172.21.15.42:0/3053017532 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562852297340 con 0x5628523a9b80 2024-04-25T00:31:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc6ea00640 10 client.26216 handle_command_reply: tid=0 2024-04-25T00:31:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 client.26216 shutdown 2024-04-25T00:31:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 -- 172.21.15.42:0/3053017532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 msgr2=0x56285233c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x56285233c290 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efc74027800 tx=0x7efc74027830 comp rx=0 tx=0).stop 2024-04-25T00:31:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 -- 172.21.15.42:0/3053017532 shutdown_connections 2024-04-25T00:31:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5628523a9b80 0x5628523abfa0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5628523329b0 0x56285233d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56285232d840 0x56285233c290 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 --2- 172.21.15.42:0/3053017532 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56285232ce50 0x56285233bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc6ea00640 0 client.26216 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 -- 172.21.15.42:0/3053017532 >> 172.21.15.42:0/3053017532 conn(0x5628523284c0 msgr2=0x562852335e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 -- 172.21.15.42:0/3053017532 shutdown_connections 2024-04-25T00:31:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.906+0000 7efc92a45740 1 -- 172.21.15.42:0/3053017532 wait complete. 2024-04-25T00:31:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.907+0000 7efc92a45740 20 client.26216 trim_cache size 0 max 16384 2024-04-25T00:31:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:31:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:31:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:31:37.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:31:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.907+0000 7efc8e600640 1 -- 172.21.15.42:0/1580662463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc700a2b80 msgr2=0x7efc700a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.907+0000 7efc8e600640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc700a2b80 0x7efc700a5040 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7efc7c09bc10 tx=0x7efc7c000bc0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.907+0000 7efc8e600640 1 -- 172.21.15.42:0/1580662463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc880ffe30 msgr2=0x7efc88199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.907+0000 7efc8e600640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc880ffe30 0x7efc88199a90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7efc7c00e9f0 tx=0x7efc7c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.907+0000 7efc8e600640 1 -- 172.21.15.42:0/1580662463 shutdown_connections 2024-04-25T00:31:37.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.907+0000 7efc8e600640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc700a2b80 0x7efc700a5040 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.907+0000 7efc8e600640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc88101130 0x7efc881a1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.907+0000 7efc8e600640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc88100790 0x7efc88199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.907+0000 7efc8e600640 1 --2- 172.21.15.42:0/1580662463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc880ffe30 0x7efc88199a90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.908+0000 7efc8e600640 1 -- 172.21.15.42:0/1580662463 >> 172.21.15.42:0/1580662463 conn(0x7efc880fba30 msgr2=0x7efc880fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.908+0000 7efc8e600640 1 -- 172.21.15.42:0/1580662463 shutdown_connections 2024-04-25T00:31:37.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:37.908+0000 7efc8e600640 1 -- 172.21.15.42:0/1580662463 wait complete. 2024-04-25T00:31:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:37 smithi136 ceph-mon[32923]: pgmap v774: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-25T00:31:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:37 smithi106 ceph-mon[32510]: pgmap v774: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-25T00:31:38.130 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:37 smithi042 ceph-mon[28130]: pgmap v774: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 1.7 MiB/s wr, 214 op/s 2024-04-25T00:31:38.286 INFO:tasks.workunit.client.0.smithi042.stdout: 1 797723 6.14 MB/sec execute 568 sec latency 36.870 ms 2024-04-25T00:31:39.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 799180 6.15 MB/sec execute 569 sec latency 13.509 ms 2024-04-25T00:31:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:39 smithi136 ceph-mon[32923]: pgmap v775: 97 pgs: 97 active+clean; 145 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-04-25T00:31:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:39 smithi106 ceph-mon[32510]: pgmap v775: 97 pgs: 97 active+clean; 145 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-04-25T00:31:40.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:39 smithi042 ceph-mon[28130]: pgmap v775: 97 pgs: 97 active+clean; 145 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 490 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-04-25T00:31:40.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 800094 6.15 MB/sec execute 570 sec latency 24.012 ms 2024-04-25T00:31:41.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 801366 6.14 MB/sec execute 571 sec latency 24.628 ms 2024-04-25T00:31:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:41 smithi136 ceph-mon[32923]: pgmap v776: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-04-25T00:31:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:41 smithi106 ceph-mon[32510]: pgmap v776: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-04-25T00:31:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:41 smithi042 ceph-mon[28130]: pgmap v776: 97 pgs: 97 active+clean; 141 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.7 MiB/s wr, 172 op/s 2024-04-25T00:31:42.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 802774 6.15 MB/sec execute 572 sec latency 47.198 ms 2024-04-25T00:31:43.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 803669 6.15 MB/sec execute 573 sec latency 16.036 ms 2024-04-25T00:31:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:43 smithi136 ceph-mon[32923]: pgmap v777: 97 pgs: 97 active+clean; 142 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-04-25T00:31:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:43 smithi106 ceph-mon[32510]: pgmap v777: 97 pgs: 97 active+clean; 142 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-04-25T00:31:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:43 smithi042 ceph-mon[28130]: pgmap v777: 97 pgs: 97 active+clean; 142 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 445 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-04-25T00:31:44.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 804997 6.15 MB/sec execute 574 sec latency 25.349 ms 2024-04-25T00:31:45.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 806339 6.15 MB/sec execute 575 sec latency 20.131 ms 2024-04-25T00:31:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:45 smithi136 ceph-mon[32923]: pgmap v778: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-04-25T00:31:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:45 smithi106 ceph-mon[32510]: pgmap v778: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-04-25T00:31:46.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:45 smithi042 ceph-mon[28130]: pgmap v778: 97 pgs: 97 active+clean; 144 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 1.7 MiB/s wr, 188 op/s 2024-04-25T00:31:46.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 807389 6.15 MB/sec execute 576 sec latency 26.580 ms 2024-04-25T00:31:47.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 808556 6.15 MB/sec execute 577 sec latency 25.981 ms 2024-04-25T00:31:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:47 smithi136 ceph-mon[32923]: pgmap v779: 97 pgs: 97 active+clean; 145 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 2.1 MiB/s wr, 224 op/s 2024-04-25T00:31:48.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:47 smithi106 ceph-mon[32510]: pgmap v779: 97 pgs: 97 active+clean; 145 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 2.1 MiB/s wr, 224 op/s 2024-04-25T00:31:48.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:47 smithi042 ceph-mon[28130]: pgmap v779: 97 pgs: 97 active+clean; 145 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 498 KiB/s rd, 2.1 MiB/s wr, 224 op/s 2024-04-25T00:31:48.287 INFO:tasks.workunit.client.0.smithi042.stdout: 1 809890 6.16 MB/sec execute 578 sec latency 18.143 ms 2024-04-25T00:31:49.289 INFO:tasks.workunit.client.0.smithi042.stdout: 1 810937 6.15 MB/sec execute 579 sec latency 41.515 ms 2024-04-25T00:31:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:49 smithi136 ceph-mon[32923]: pgmap v780: 97 pgs: 97 active+clean; 146 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-04-25T00:31:50.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:49 smithi106 ceph-mon[32510]: pgmap v780: 97 pgs: 97 active+clean; 146 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-04-25T00:31:50.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:49 smithi042 ceph-mon[28130]: pgmap v780: 97 pgs: 97 active+clean; 146 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 441 KiB/s rd, 1.8 MiB/s wr, 203 op/s 2024-04-25T00:31:50.288 INFO:tasks.workunit.client.0.smithi042.stdout: 1 812285 6.15 MB/sec execute 580 sec latency 27.407 ms 2024-04-25T00:31:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.058+0000 7f1e10e00640 1 -- 172.21.15.42:0/3850025442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c1053b0 msgr2=0x7f1e0c10bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.058+0000 7f1e10e00640 1 --2- 172.21.15.42:0/3850025442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c1053b0 0x7f1e0c10bc40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009ca0 tx=0x7f1e0402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.058+0000 7f1e10e00640 1 -- 172.21.15.42:0/3850025442 shutdown_connections 2024-04-25T00:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.058+0000 7f1e10e00640 1 --2- 172.21.15.42:0/3850025442 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c1053b0 0x7f1e0c10bc40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.058+0000 7f1e10e00640 1 --2- 172.21.15.42:0/3850025442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c1049f0 0x7f1e0c104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.058+0000 7f1e10e00640 1 --2- 172.21.15.42:0/3850025442 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e0c103800 0x7f1e0c103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.058+0000 7f1e10e00640 1 -- 172.21.15.42:0/3850025442 >> 172.21.15.42:0/3850025442 conn(0x7f1e0c0fedb0 msgr2=0x7f1e0c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.059+0000 7f1e10e00640 1 -- 172.21.15.42:0/3850025442 shutdown_connections 2024-04-25T00:31:51.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.059+0000 7f1e10e00640 1 -- 172.21.15.42:0/3850025442 wait complete. 2024-04-25T00:31:51.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e10e00640 1 Processor -- start 2024-04-25T00:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e10e00640 1 -- start start 2024-04-25T00:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e10e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e0c103800 0x7f1e0c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e10e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c1049f0 0x7f1e0c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e10e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c1053b0 0x7f1e0c1a1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e10e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c10f480 con 0x7f1e0c103800 2024-04-25T00:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e10e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c10f300 con 0x7f1e0c1049f0 2024-04-25T00:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e10e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c10f600 con 0x7f1e0c1053b0 2024-04-25T00:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e0b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e0c103800 0x7f1e0c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e0aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c1049f0 0x7f1e0c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.060+0000 7f1e0be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c1053b0 0x7f1e0c1a1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.061+0000 7f1e0aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c1049f0 0x7f1e0c199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:37874/0 (socket says 172.21.15.42:37874) 2024-04-25T00:31:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.061+0000 7f1e0aa00640 1 -- 172.21.15.42:0/584648043 learned_addr learned my addr 172.21.15.42:0/584648043 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.061+0000 7f1e0aa00640 1 -- 172.21.15.42:0/584648043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c1053b0 msgr2=0x7f1e0c1a1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.061+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c1053b0 0x7f1e0c1a1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.061+0000 7f1e0aa00640 1 -- 172.21.15.42:0/584648043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e0c103800 msgr2=0x7f1e0c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.061+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e0c103800 0x7f1e0c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.061+0000 7f1e0aa00640 1 -- 172.21.15.42:0/584648043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e0c1a1690 con 0x7f1e0c1049f0 2024-04-25T00:31:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.061+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c1049f0 0x7f1e0c199fb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9f0 tx=0x7f1df800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.062+0000 7f1e03e00640 1 -- 172.21.15.42:0/584648043 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df800cd80 con 0x7f1e0c1049f0 2024-04-25T00:31:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.062+0000 7f1e10e00640 1 -- 172.21.15.42:0/584648043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e0c1a1980 con 0x7f1e0c1049f0 2024-04-25T00:31:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.062+0000 7f1e03e00640 1 -- 172.21.15.42:0/584648043 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1df8005500 con 0x7f1e0c1049f0 2024-04-25T00:31:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.062+0000 7f1e10e00640 1 -- 172.21.15.42:0/584648043 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e0c1a1ed0 con 0x7f1e0c1049f0 2024-04-25T00:31:51.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.062+0000 7f1e03e00640 1 -- 172.21.15.42:0/584648043 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df8010690 con 0x7f1e0c1049f0 2024-04-25T00:31:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.066+0000 7f1e15208740 1 Processor -- start 2024-04-25T00:31:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.066+0000 7f1e15208740 1 -- start start 2024-04-25T00:31:51.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.066+0000 7f1e03e00640 1 -- 172.21.15.42:0/584648043 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1df80046c0 con 0x7f1e0c1049f0 2024-04-25T00:31:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.066+0000 7f1e15208740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2df270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.066+0000 7f1e15208740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2df840 0x55658e2e4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.066+0000 7f1e15208740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2e49b0 0x55658e2e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.066+0000 7f1e15208740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55658e241ab0 con 0x55658e2df840 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.066+0000 7f1e15208740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55658e1ea640 con 0x55658e2dee50 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.066+0000 7f1e15208740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55658e23da70 con 0x55658e2e49b0 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2df270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2e49b0 0x55658e2e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2df840 0x55658e2e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2df270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:37878/0 (socket says 172.21.15.42:37878) 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0be00640 1 -- 172.21.15.42:0/2744552960 learned_addr learned my addr 172.21.15.42:0/2744552960 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0be00640 1 -- 172.21.15.42:0/2744552960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2e49b0 msgr2=0x55658e2e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0be00640 1 --2- 172.21.15.42:0/2744552960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2e49b0 0x55658e2e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0be00640 1 -- 172.21.15.42:0/2744552960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2df840 msgr2=0x55658e2e4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0be00640 1 --2- 172.21.15.42:0/2744552960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2df840 0x55658e2e4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0be00640 1 -- 172.21.15.42:0/2744552960 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658df56210 con 0x55658e2dee50 2024-04-25T00:31:51.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.067+0000 7f1e0be00640 1 --2- 172.21.15.42:0/2744552960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2df270 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009ca0 tx=0x7f1e04004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef3e9b8a31b45b6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.068+0000 7f1e00c00640 1 -- 172.21.15.42:0/2744552960 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e040098c0 con 0x55658e2dee50 2024-04-25T00:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.068+0000 7f1e00c00640 1 -- 172.21.15.42:0/2744552960 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e04036040 con 0x55658e2dee50 2024-04-25T00:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.068+0000 7f1e00c00640 1 -- 172.21.15.42:0/2744552960 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04004290 con 0x55658e2dee50 2024-04-25T00:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.068+0000 7f1e03e00640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1df00a2c00 0x7f1df00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.068+0000 7f1e03e00640 1 -- 172.21.15.42:0/584648043 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1df80cf300 con 0x7f1e0c1049f0 2024-04-25T00:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.068+0000 7f1e0b400640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1df00a2c00 0x7f1df00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.068+0000 7f1e15208740 1 -- 172.21.15.42:0/2744552960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 msgr2=0x55658e2df270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.068+0000 7f1e15208740 1 --2- 172.21.15.42:0/2744552960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2df270 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1e04009ca0 tx=0x7f1e04004a40 comp rx=0 tx=0).stop 2024-04-25T00:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.069+0000 7f1e15208740 1 -- 172.21.15.42:0/2744552960 shutdown_connections 2024-04-25T00:31:51.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.069+0000 7f1e15208740 1 --2- 172.21.15.42:0/2744552960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2e49b0 0x55658e2e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.069+0000 7f1e15208740 1 --2- 172.21.15.42:0/2744552960 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2df840 0x55658e2e4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.069+0000 7f1e15208740 1 --2- 172.21.15.42:0/2744552960 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2df270 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.069+0000 7f1e15208740 1 -- 172.21.15.42:0/2744552960 >> 172.21.15.42:0/2744552960 conn(0x55658e2da4c0 msgr2=0x55658e2da8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.069+0000 7f1e15208740 1 -- 172.21.15.42:0/2744552960 shutdown_connections 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.069+0000 7f1e15208740 1 -- 172.21.15.42:0/2744552960 wait complete. 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.069+0000 7f1e0b400640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1df00a2c00 0x7f1df00a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55658e2dfc40 tx=0x7f1dfc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.069+0000 7f1e15208740 1 -- start start 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.069+0000 7f1e15208740 1 Processor -- start 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e15208740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2e8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e15208740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2e9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e15208740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2e49b0 0x55658e2f0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e15208740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55658e2e7720 con 0x55658e2e49b0 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e15208740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55658e1fd990 con 0x55658e2dee50 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e15208740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55658e2e78a0 con 0x55658e2df840 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e15208740 10 client.0 _ll_register_callbacks cb 0x55658e1ff950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e0aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2e9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e0b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2e8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e0aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2e9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33792/0 (socket says 172.21.15.42:33792) 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e0aa00640 1 -- 172.21.15.42:0/2337536406 learned_addr learned my addr 172.21.15.42:0/2337536406 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e0b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2e8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:37886/0 (socket says 172.21.15.42:37886) 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.070+0000 7f1e0be00640 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2e49b0 0x55658e2f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1e0aa00640 1 -- 172.21.15.42:0/2337536406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 msgr2=0x55658e2e8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2e8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1e0aa00640 1 -- 172.21.15.42:0/2337536406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2e49b0 msgr2=0x55658e2f0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2e49b0 0x55658e2f0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1e0aa00640 1 -- 172.21.15.42:0/2337536406 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658e022620 con 0x55658e2df840 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2e9450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9c0 tx=0x7f1df80049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=103ab91f3db4b242 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1df5600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1df5600640 1 -- 172.21.15.42:0/2337536406 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df809b6d0 con 0x55658e2df840 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1df5600640 1 -- 172.21.15.42:0/2337536406 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1df80a4570 con 0x55658e2df840 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1e15208740 1 -- 172.21.15.42:0/2337536406 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e04036040 con 0x55658e2df840 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1df5600640 1 -- 172.21.15.42:0/2337536406 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1df80b9050 con 0x55658e2df840 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1e15208740 10 client.26233 fetch_fsmap learned FSMap version 31 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.071+0000 7f1e15208740 1 -- 172.21.15.42:0/2337536406 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55658e2f34d0 con 0x55658e2df840 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.072+0000 7f1df5600640 1 -- 172.21.15.42:0/2337536406 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e04036040 con 0x55658e2df840 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.072+0000 7f1df5600640 1 -- 172.21.15.42:0/2337536406 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f1df80ce030 con 0x55658e2df840 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.072+0000 7f1e15208740 10 client.26233 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.072+0000 7f1e15208740 10 client.26233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:51.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.072+0000 7f1e15208740 20 client.26233 populate_metadata read hostname 'smithi042' 2024-04-25T00:31:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.072+0000 7f1e15208740 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55658e2f4ba0 0x55658e2f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.072+0000 7f1e15208740 4 client.26233 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:31:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.072+0000 7f1e15208740 1 -- 172.21.15.42:0/2337536406 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55658e24eff0 con 0x55658e2f4ba0 2024-04-25T00:31:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.072+0000 7f1e0b400640 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55658e2f4ba0 0x55658e2f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.073+0000 7f1e0b400640 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55658e2f4ba0 0x55658e2f6fd0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38d008f560b8f50a server_cookie=2cc3182345bf9454 in_seq=0 out_seq=0 2024-04-25T00:31:51.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.073+0000 7f1df5600640 10 client.26233 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1df5600640 1 -- 172.21.15.42:0/2337536406 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55658e24eff0 con 0x55658e2f4ba0 2024-04-25T00:31:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1df5600640 10 client.26233 handle_command_reply: tid=0 2024-04-25T00:31:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 client.26233 shutdown 2024-04-25T00:31:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 -- 172.21.15.42:0/2337536406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 msgr2=0x55658e2e9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2e9450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9c0 tx=0x7f1df80049b0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 -- 172.21.15.42:0/2337536406 shutdown_connections 2024-04-25T00:31:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55658e2f4ba0 0x55658e2f6fd0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2e49b0 0x55658e2f0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2e9450 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 --2- 172.21.15.42:0/2337536406 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2dee50 0x55658e2e8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1df5600640 0 client.26233 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 -- 172.21.15.42:0/2337536406 >> 172.21.15.42:0/2337536406 conn(0x55658e2da4c0 msgr2=0x55658e2d9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 -- 172.21.15.42:0/2337536406 shutdown_connections 2024-04-25T00:31:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.074+0000 7f1e15208740 1 -- 172.21.15.42:0/2337536406 wait complete. 2024-04-25T00:31:51.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.075+0000 7f1e15208740 20 client.26233 trim_cache size 0 max 16384 2024-04-25T00:31:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.092+0000 7f1e15208740 1 Processor -- start 2024-04-25T00:31:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.092+0000 7f1e15208740 1 -- start start 2024-04-25T00:31:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e15208740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2da4c0 0x55658e2dd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e15208740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 0x55658e2f03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e15208740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2dd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e15208740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1ad400 con 0x55658e2dee50 2024-04-25T00:31:51.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e15208740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1aacd0 con 0x55658e2da4c0 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e15208740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1a8b20 con 0x55658e2df840 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2da4c0 0x55658e2dd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2dd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2dd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33806/0 (socket says 172.21.15.42:33806) 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0be00640 1 -- 172.21.15.42:0/101615513 learned_addr learned my addr 172.21.15.42:0/101615513 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2da4c0 0x55658e2dd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:37894/0 (socket says 172.21.15.42:37894) 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/101615513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 0x55658e2f03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0be00640 1 -- 172.21.15.42:0/101615513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2da4c0 msgr2=0x55658e2dd470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0be00640 1 --2- 172.21.15.42:0/101615513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2da4c0 0x55658e2dd470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0be00640 1 -- 172.21.15.42:0/101615513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 msgr2=0x55658e2f03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0be00640 1 --2- 172.21.15.42:0/101615513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 0x55658e2f03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.093+0000 7f1e0be00640 1 -- 172.21.15.42:0/101615513 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658e2f7650 con 0x55658e2df840 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.094+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/101615513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 0x55658e2f03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:31:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.094+0000 7f1e0be00640 1 --2- 172.21.15.42:0/101615513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2dd9b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005fe0 tx=0x7f1e04005f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5cad28c2e5eaf707 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.094+0000 7f1df7e00640 1 -- 172.21.15.42:0/101615513 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04053070 con 0x55658e2df840 2024-04-25T00:31:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.094+0000 7f1df7e00640 1 -- 172.21.15.42:0/101615513 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e04036040 con 0x55658e2df840 2024-04-25T00:31:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.094+0000 7f1df7e00640 1 -- 172.21.15.42:0/101615513 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e04004440 con 0x55658e2df840 2024-04-25T00:31:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.095+0000 7f1e15208740 1 -- 172.21.15.42:0/101615513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 msgr2=0x55658e2dd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.095+0000 7f1e15208740 1 --2- 172.21.15.42:0/101615513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2dd9b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1e04005fe0 tx=0x7f1e04005f70 comp rx=0 tx=0).stop 2024-04-25T00:31:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.095+0000 7f1e15208740 1 -- 172.21.15.42:0/101615513 shutdown_connections 2024-04-25T00:31:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.095+0000 7f1e15208740 1 --2- 172.21.15.42:0/101615513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2df840 0x55658e2dd9b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.095+0000 7f1e15208740 1 --2- 172.21.15.42:0/101615513 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 0x55658e2f03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.095+0000 7f1e15208740 1 --2- 172.21.15.42:0/101615513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2da4c0 0x55658e2dd470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.095+0000 7f1e15208740 1 -- 172.21.15.42:0/101615513 >> 172.21.15.42:0/101615513 conn(0x55658e2e4480 msgr2=0x55658e2da8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.095+0000 7f1e15208740 1 -- 172.21.15.42:0/101615513 shutdown_connections 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.095+0000 7f1e15208740 1 -- 172.21.15.42:0/101615513 wait complete. 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.095+0000 7f1e15208740 1 -- start start 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e15208740 1 Processor -- start 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e15208740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2da4c0 0x55658e2ee230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e15208740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 0x55658e2ee770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e15208740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2df840 0x55658e2ef780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e15208740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1a9190 con 0x55658e2dee50 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e15208740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1aa600 con 0x55658e2df840 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e15208740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1e0c1a9860 con 0x55658e2da4c0 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e15208740 10 client.0 _ll_register_callbacks cb 0x7f1df80d2ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:31:51.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e0b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2da4c0 0x55658e2ee230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e0b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2da4c0 0x55658e2ee230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33808/0 (socket says 172.21.15.42:33808) 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e0be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2df840 0x55658e2ef780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e0b400640 1 -- 172.21.15.42:0/1374240692 learned_addr learned my addr 172.21.15.42:0/1374240692 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.096+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 0x55658e2ee770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.097+0000 7f1e0b400640 1 -- 172.21.15.42:0/1374240692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2df840 msgr2=0x55658e2ef780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.097+0000 7f1e0b400640 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2df840 0x55658e2ef780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.097+0000 7f1e0b400640 1 -- 172.21.15.42:0/1374240692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 msgr2=0x55658e2ee770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.097+0000 7f1e0b400640 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 0x55658e2ee770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.097+0000 7f1e0b400640 1 -- 172.21.15.42:0/1374240692 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658e2efd70 con 0x55658e2da4c0 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.097+0000 7f1e0b400640 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2da4c0 0x55658e2ee230 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0041e0 tx=0x7f1dfc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7aacfee1fca4218 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:31:51.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.097+0000 7f1df5600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.097+0000 7f1df5600640 1 -- 172.21.15.42:0/1374240692 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc003ac0 con 0x55658e2da4c0 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.097+0000 7f1df5600640 1 -- 172.21.15.42:0/1374240692 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f1dfc003c60 con 0x55658e2da4c0 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.097+0000 7f1e15208740 1 -- 172.21.15.42:0/1374240692 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1e04036040 con 0x55658e2da4c0 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1df5600640 1 -- 172.21.15.42:0/1374240692 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f1dfc003e60 con 0x55658e2da4c0 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1e15208740 10 client.26239 fetch_fsmap learned FSMap version 31 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1e15208740 1 -- 172.21.15.42:0/1374240692 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55658e35b990 con 0x55658e2da4c0 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1df5600640 1 -- 172.21.15.42:0/1374240692 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e04036040 con 0x55658e2da4c0 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1df5600640 1 -- 172.21.15.42:0/1374240692 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f1dfc01f9b0 con 0x55658e2da4c0 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1e15208740 10 client.26239 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1e15208740 10 client.26239 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1e15208740 20 client.26239 populate_metadata read hostname 'smithi042' 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1e15208740 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55658e2e49b0 0x55658e35e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1e15208740 4 client.26239 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:31:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1e15208740 1 -- 172.21.15.42:0/1374240692 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55658e249340 con 0x55658e2e49b0 2024-04-25T00:31:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.098+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55658e2e49b0 0x55658e35e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:31:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.099+0000 7f1e0aa00640 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55658e2e49b0 0x55658e35e400 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67efea67504e29a1 server_cookie=292633fc71a5e9ff in_seq=0 out_seq=0 2024-04-25T00:31:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.099+0000 7f1df5600640 10 client.26239 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.157+0000 7f1df5600640 1 -- 172.21.15.42:0/1374240692 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55658e249340 con 0x55658e2e49b0 2024-04-25T00:31:51.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.157+0000 7f1df5600640 10 client.26239 handle_command_reply: tid=0 2024-04-25T00:31:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.157+0000 7f1e15208740 1 client.26239 shutdown 2024-04-25T00:31:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 1 -- 172.21.15.42:0/1374240692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2da4c0 msgr2=0x55658e2ee230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2da4c0 0x55658e2ee230 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc0041e0 tx=0x7f1dfc0046c0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 1 -- 172.21.15.42:0/1374240692 shutdown_connections 2024-04-25T00:31:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55658e2e49b0 0x55658e35e400 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55658e2df840 0x55658e2ef780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55658e2dee50 0x55658e2ee770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 1 --2- 172.21.15.42:0/1374240692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55658e2da4c0 0x55658e2ee230 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1df5600640 0 client.26239 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:31:51.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 1 -- 172.21.15.42:0/1374240692 >> 172.21.15.42:0/1374240692 conn(0x55658e2e4480 msgr2=0x55658e2dd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 1 -- 172.21.15.42:0/1374240692 shutdown_connections 2024-04-25T00:31:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 1 -- 172.21.15.42:0/1374240692 wait complete. 2024-04-25T00:31:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.158+0000 7f1e15208740 20 client.26239 trim_cache size 0 max 16384 2024-04-25T00:31:51.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:31:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:31:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:31:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:31:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 -- 172.21.15.42:0/584648043 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1df00a2c00 msgr2=0x7f1df00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1df00a2c00 0x7f1df00a50c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x55658e2dfc40 tx=0x7f1dfc005f90 comp rx=0 tx=0).stop 2024-04-25T00:31:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 -- 172.21.15.42:0/584648043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c1049f0 msgr2=0x7f1e0c199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:31:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c1049f0 0x7f1e0c199fb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1df800e9f0 tx=0x7f1df800edf0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 -- 172.21.15.42:0/584648043 shutdown_connections 2024-04-25T00:31:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1df00a2c00 0x7f1df00a50c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1e0c1053b0 0x7f1e0c1a1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0c1049f0 0x7f1e0c199fb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 --2- 172.21.15.42:0/584648043 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1e0c103800 0x7f1e0c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:31:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 -- 172.21.15.42:0/584648043 >> 172.21.15.42:0/584648043 conn(0x7f1e0c0fedb0 msgr2=0x7f1e0c107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:31:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 -- 172.21.15.42:0/584648043 shutdown_connections 2024-04-25T00:31:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:31:51.159+0000 7f1e10e00640 1 -- 172.21.15.42:0/584648043 wait complete. 2024-04-25T00:31:51.288 INFO:tasks.workunit.client.0.smithi042.stdout: 1 813443 6.16 MB/sec execute 581 sec latency 33.201 ms 2024-04-25T00:31:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:51 smithi136 ceph-mon[32923]: pgmap v781: 97 pgs: 97 active+clean; 146 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-04-25T00:31:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:51 smithi106 ceph-mon[32510]: pgmap v781: 97 pgs: 97 active+clean; 146 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-04-25T00:31:52.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:51 smithi042 ceph-mon[28130]: pgmap v781: 97 pgs: 97 active+clean; 146 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 352 KiB/s rd, 1.8 MiB/s wr, 187 op/s 2024-04-25T00:31:52.288 INFO:tasks.workunit.client.0.smithi042.stdout: 1 814532 6.16 MB/sec execute 582 sec latency 21.706 ms 2024-04-25T00:31:53.289 INFO:tasks.workunit.client.0.smithi042.stdout: 1 816042 6.16 MB/sec execute 583 sec latency 44.889 ms 2024-04-25T00:31:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:53 smithi136 ceph-mon[32923]: pgmap v782: 97 pgs: 97 active+clean; 146 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-04-25T00:31:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:53 smithi106 ceph-mon[32510]: pgmap v782: 97 pgs: 97 active+clean; 146 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-04-25T00:31:54.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:53 smithi042 ceph-mon[28130]: pgmap v782: 97 pgs: 97 active+clean; 146 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 425 KiB/s rd, 1.9 MiB/s wr, 203 op/s 2024-04-25T00:31:54.289 INFO:tasks.workunit.client.0.smithi042.stdout: 1 817008 6.16 MB/sec execute 584 sec latency 23.124 ms 2024-04-25T00:31:55.289 INFO:tasks.workunit.client.0.smithi042.stdout: 1 817996 6.16 MB/sec execute 585 sec latency 28.978 ms 2024-04-25T00:31:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:55 smithi136 ceph-mon[32923]: pgmap v783: 97 pgs: 97 active+clean; 147 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-04-25T00:31:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:55 smithi106 ceph-mon[32510]: pgmap v783: 97 pgs: 97 active+clean; 147 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-04-25T00:31:56.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:55 smithi042 ceph-mon[28130]: pgmap v783: 97 pgs: 97 active+clean; 147 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 177 op/s 2024-04-25T00:31:56.289 INFO:tasks.workunit.client.0.smithi042.stdout: 1 819072 6.15 MB/sec execute 586 sec latency 25.899 ms 2024-04-25T00:31:57.289 INFO:tasks.workunit.client.0.smithi042.stdout: 1 820395 6.16 MB/sec execute 587 sec latency 23.472 ms 2024-04-25T00:31:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:57 smithi136 ceph-mon[32923]: pgmap v784: 97 pgs: 97 active+clean; 148 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.0 MiB/s wr, 221 op/s 2024-04-25T00:31:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:57 smithi106 ceph-mon[32510]: pgmap v784: 97 pgs: 97 active+clean; 148 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.0 MiB/s wr, 221 op/s 2024-04-25T00:31:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:57 smithi042 ceph-mon[28130]: pgmap v784: 97 pgs: 97 active+clean; 148 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.0 MiB/s wr, 221 op/s 2024-04-25T00:31:58.289 INFO:tasks.workunit.client.0.smithi042.stdout: 1 821010 6.15 MB/sec execute 588 sec latency 41.003 ms 2024-04-25T00:31:59.290 INFO:tasks.workunit.client.0.smithi042.stdout: 1 822114 6.15 MB/sec execute 589 sec latency 38.485 ms 2024-04-25T00:32:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:31:59 smithi136 ceph-mon[32923]: pgmap v785: 97 pgs: 97 active+clean; 149 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:32:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:31:59 smithi106 ceph-mon[32510]: pgmap v785: 97 pgs: 97 active+clean; 149 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:32:00.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:31:59 smithi042 ceph-mon[28130]: pgmap v785: 97 pgs: 97 active+clean; 149 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 1.6 MiB/s wr, 195 op/s 2024-04-25T00:32:00.289 INFO:tasks.workunit.client.0.smithi042.stdout: 1 823753 6.16 MB/sec execute 590 sec latency 39.081 ms 2024-04-25T00:32:01.289 INFO:tasks.workunit.client.0.smithi042.stdout: 1 824276 6.15 MB/sec execute 591 sec latency 25.324 ms 2024-04-25T00:32:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:01 smithi136 ceph-mon[32923]: pgmap v786: 97 pgs: 97 active+clean; 150 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-04-25T00:32:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:01 smithi106 ceph-mon[32510]: pgmap v786: 97 pgs: 97 active+clean; 150 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-04-25T00:32:02.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:01 smithi042 ceph-mon[28130]: pgmap v786: 97 pgs: 97 active+clean; 150 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 1.7 MiB/s wr, 193 op/s 2024-04-25T00:32:02.290 INFO:tasks.workunit.client.0.smithi042.stdout: 1 825293 6.15 MB/sec execute 592 sec latency 40.800 ms 2024-04-25T00:32:03.290 INFO:tasks.workunit.client.0.smithi042.stdout: 1 826872 6.15 MB/sec execute 593 sec latency 27.496 ms 2024-04-25T00:32:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:03 smithi136 ceph-mon[32923]: pgmap v787: 97 pgs: 97 active+clean; 151 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-04-25T00:32:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:03 smithi106 ceph-mon[32510]: pgmap v787: 97 pgs: 97 active+clean; 151 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-04-25T00:32:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:03 smithi042 ceph-mon[28130]: pgmap v787: 97 pgs: 97 active+clean; 151 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 452 KiB/s rd, 1.8 MiB/s wr, 213 op/s 2024-04-25T00:32:04.290 INFO:tasks.workunit.client.0.smithi042.stdout: 1 827609 6.15 MB/sec execute 594 sec latency 29.935 ms 2024-04-25T00:32:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.315+0000 7f672fe00640 1 -- 172.21.15.42:0/2297695811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6730104a30 msgr2=0x7f6730106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.315+0000 7f672fe00640 1 --2- 172.21.15.42:0/2297695811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6730104a30 0x7f6730106e20 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f671c009900 tx=0x7f671c02a410 comp rx=0 tx=0).stop 2024-04-25T00:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.315+0000 7f672fe00640 1 -- 172.21.15.42:0/2297695811 shutdown_connections 2024-04-25T00:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.315+0000 7f672fe00640 1 --2- 172.21.15.42:0/2297695811 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730107360 0x7f6730109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.315+0000 7f672fe00640 1 --2- 172.21.15.42:0/2297695811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6730104a30 0x7f6730106e20 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.315+0000 7f672fe00640 1 --2- 172.21.15.42:0/2297695811 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6730102100 0x7f67301044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.315+0000 7f672fe00640 1 -- 172.21.15.42:0/2297695811 >> 172.21.15.42:0/2297695811 conn(0x7f67300fba30 msgr2=0x7f67300fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.315+0000 7f672fe00640 1 -- 172.21.15.42:0/2297695811 shutdown_connections 2024-04-25T00:32:04.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.315+0000 7f672fe00640 1 -- 172.21.15.42:0/2297695811 wait complete. 2024-04-25T00:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.316+0000 7f672fe00640 1 Processor -- start 2024-04-25T00:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.316+0000 7f672fe00640 1 -- start start 2024-04-25T00:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.316+0000 7f672fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6730102100 0x7f6730101a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.316+0000 7f672fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730104a30 0x7f67301000b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6730107360 0x7f67301005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f673010d2a0 con 0x7f6730104a30 2024-04-25T00:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f673010d120 con 0x7f6730102100 2024-04-25T00:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f673010d420 con 0x7f6730107360 2024-04-25T00:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6730102100 0x7f6730101a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6730102100 0x7f6730101a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34266/0 (socket says 172.21.15.42:34266) 2024-04-25T00:32:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672ea00640 1 -- 172.21.15.42:0/1392589759 learned_addr learned my addr 172.21.15.42:0/1392589759 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672e000640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730104a30 0x7f67301000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672f400640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6730107360 0x7f67301005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672ea00640 1 -- 172.21.15.42:0/1392589759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6730107360 msgr2=0x7f67301005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672ea00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6730107360 0x7f67301005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672ea00640 1 -- 172.21.15.42:0/1392589759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730104a30 msgr2=0x7f67301000b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672ea00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730104a30 0x7f67301000b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.317+0000 7f672ea00640 1 -- 172.21.15.42:0/1392589759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6730100e80 con 0x7f6730102100 2024-04-25T00:32:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.318+0000 7f672f400640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6730107360 0x7f67301005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:04.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.318+0000 7f672ea00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6730102100 0x7f6730101a70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f672000e9f0 tx=0x7f672000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.318+0000 7f6734a00640 1 -- 172.21.15.42:0/1392589759 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672000cd80 con 0x7f6730102100 2024-04-25T00:32:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.318+0000 7f672fe00640 1 -- 172.21.15.42:0/1392589759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67301a1a30 con 0x7f6730102100 2024-04-25T00:32:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.318+0000 7f6734a00640 1 -- 172.21.15.42:0/1392589759 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6720005500 con 0x7f6730102100 2024-04-25T00:32:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.318+0000 7f6734a00640 1 -- 172.21.15.42:0/1392589759 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6720010690 con 0x7f6730102100 2024-04-25T00:32:04.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.318+0000 7f672fe00640 1 -- 172.21.15.42:0/1392589759 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67301a1f00 con 0x7f6730102100 2024-04-25T00:32:04.322 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.322+0000 7f6734a00640 1 -- 172.21.15.42:0/1392589759 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f67200046c0 con 0x7f6730102100 2024-04-25T00:32:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.322+0000 7f6738b65740 1 Processor -- start 2024-04-25T00:32:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f6738b65740 1 -- start start 2024-04-25T00:32:04.323 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f6738b65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e659e50 0x564c2e65a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f6738b65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e65a840 0x564c2e65f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f6738b65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e661e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f6738b65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564c2e5bcab0 con 0x564c2e65f9b0 2024-04-25T00:32:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f6738b65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564c2e565640 con 0x564c2e65a840 2024-04-25T00:32:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f6738b65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564c2e5b8a70 con 0x564c2e659e50 2024-04-25T00:32:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f672f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e659e50 0x564c2e65a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f672ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e65a840 0x564c2e65f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f672f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e659e50 0x564c2e65a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:37998/0 (socket says 172.21.15.42:37998) 2024-04-25T00:32:04.324 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.323+0000 7f672f400640 1 -- 172.21.15.42:0/1811835915 learned_addr learned my addr 172.21.15.42:0/1811835915 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f672e000640 1 --2- 172.21.15.42:0/1811835915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e661e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f672f400640 1 -- 172.21.15.42:0/1811835915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e65a840 msgr2=0x564c2e65f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f672f400640 1 --2- 172.21.15.42:0/1811835915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e65a840 0x564c2e65f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.325 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f672f400640 1 -- 172.21.15.42:0/1811835915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 msgr2=0x564c2e661e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f672f400640 1 --2- 172.21.15.42:0/1811835915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e661e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f672f400640 1 -- 172.21.15.42:0/1811835915 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c2e2d1210 con 0x564c2e659e50 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f6734a00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67140a2b80 0x7f67140a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f672f400640 1 --2- 172.21.15.42:0/1811835915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e659e50 0x564c2e65a270 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6728009fd0 tx=0x7f67280108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f078c3ca4f1d00d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f6734a00640 1 -- 172.21.15.42:0/1392589759 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f67200cf5a0 con 0x7f6730102100 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f672ea00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67140a2b80 0x7f67140a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.324+0000 7f671be00640 1 -- 172.21.15.42:0/1811835915 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728011760 con 0x564c2e659e50 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.325+0000 7f671be00640 1 -- 172.21.15.42:0/1811835915 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6728011de0 con 0x564c2e659e50 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.325+0000 7f671be00640 1 -- 172.21.15.42:0/1811835915 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672800e3c0 con 0x564c2e659e50 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.325+0000 7f6738b65740 1 -- 172.21.15.42:0/1811835915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e659e50 msgr2=0x564c2e65a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.325+0000 7f672ea00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67140a2b80 0x7f67140a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6720098230 tx=0x7f67200d5060 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.326 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.325+0000 7f6738b65740 1 --2- 172.21.15.42:0/1811835915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e659e50 0x564c2e65a270 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6728009fd0 tx=0x7f67280108e0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.326+0000 7f6738b65740 1 -- 172.21.15.42:0/1811835915 shutdown_connections 2024-04-25T00:32:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.326+0000 7f6738b65740 1 --2- 172.21.15.42:0/1811835915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e661e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.326+0000 7f6738b65740 1 --2- 172.21.15.42:0/1811835915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e65a840 0x564c2e65f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.330 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.326+0000 7f6738b65740 1 --2- 172.21.15.42:0/1811835915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e659e50 0x564c2e65a270 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.326+0000 7f6738b65740 1 -- 172.21.15.42:0/1811835915 >> 172.21.15.42:0/1811835915 conn(0x564c2e6554c0 msgr2=0x564c2e6558c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.326+0000 7f6738b65740 1 -- 172.21.15.42:0/1811835915 shutdown_connections 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.326+0000 7f6738b65740 1 -- 172.21.15.42:0/1811835915 wait complete. 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.326+0000 7f6738b65740 1 -- start start 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.326+0000 7f6738b65740 1 Processor -- start 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f6738b65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e664100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f6738b65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e664640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f6738b65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e66b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f6738b65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564c2e662960 con 0x564c2e65f9b0 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f6738b65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564c2e578990 con 0x564c2e659e50 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f6738b65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564c2e662ae0 con 0x564c2e65a840 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f6738b65740 10 client.0 _ll_register_callbacks cb 0x564c2e57a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:04.331 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f672e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e664640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f672e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e664640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38002/0 (socket says 172.21.15.42:38002) 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f672e000640 1 -- 172.21.15.42:0/3735834132 learned_addr learned my addr 172.21.15.42:0/3735834132 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f672f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e664100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.327+0000 7f672f400640 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e66b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f672e000640 1 -- 172.21.15.42:0/3735834132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 msgr2=0x564c2e664100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f672e000640 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e664100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f672e000640 1 -- 172.21.15.42:0/3735834132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 msgr2=0x564c2e66b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f672e000640 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e66b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f672e000640 1 -- 172.21.15.42:0/3735834132 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c2e39d620 con 0x564c2e65a840 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f672f400640 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e66b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f672e000640 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e664640 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f671c000c00 tx=0x7f671c0042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be111a6ab1793415 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f6718c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f6718c00640 1 -- 172.21.15.42:0/3735834132 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c035040 con 0x564c2e65a840 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f6718c00640 1 -- 172.21.15.42:0/3735834132 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f671c005450 con 0x564c2e65a840 2024-04-25T00:32:04.332 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.328+0000 7f6738b65740 1 -- 172.21.15.42:0/3735834132 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6728011de0 con 0x564c2e65a840 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6718c00640 1 -- 172.21.15.42:0/3735834132 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f671c04c070 con 0x564c2e65a840 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6738b65740 10 client.26251 fetch_fsmap learned FSMap version 31 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6738b65740 1 -- 172.21.15.42:0/3735834132 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564c2e66e6d0 con 0x564c2e65a840 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6718c00640 1 -- 172.21.15.42:0/3735834132 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6728011de0 con 0x564c2e65a840 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6718c00640 1 -- 172.21.15.42:0/3735834132 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f671c031980 con 0x564c2e65a840 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6738b65740 10 client.26251 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6738b65740 10 client.26251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6738b65740 20 client.26251 populate_metadata read hostname 'smithi042' 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6738b65740 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564c2e66fe10 0x564c2e672220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6738b65740 4 client.26251 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.329+0000 7f6738b65740 1 -- 172.21.15.42:0/3735834132 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c2e5c9ff0 con 0x564c2e66fe10 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.330+0000 7f672f400640 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564c2e66fe10 0x564c2e672220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.330+0000 7f672f400640 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564c2e66fe10 0x564c2e672220 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c09f65fe3f47cb7e server_cookie=54026475821a873c in_seq=0 out_seq=0 2024-04-25T00:32:04.333 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.330+0000 7f6718c00640 10 client.26251 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6718c00640 1 -- 172.21.15.42:0/3735834132 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564c2e5c9ff0 con 0x564c2e66fe10 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6718c00640 10 client.26251 handle_command_reply: tid=0 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6738b65740 1 client.26251 shutdown 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6738b65740 1 -- 172.21.15.42:0/3735834132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 msgr2=0x564c2e664640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6738b65740 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e664640 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f671c000c00 tx=0x7f671c0042e0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6738b65740 1 -- 172.21.15.42:0/3735834132 shutdown_connections 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6738b65740 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564c2e66fe10 0x564c2e672220 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6738b65740 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e66b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6738b65740 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e664640 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6738b65740 1 --2- 172.21.15.42:0/3735834132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e664100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6718c00640 0 client.26251 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:04.335 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6738b65740 1 -- 172.21.15.42:0/3735834132 >> 172.21.15.42:0/3735834132 conn(0x564c2e6554c0 msgr2=0x564c2e654620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.331+0000 7f6738b65740 1 -- 172.21.15.42:0/3735834132 shutdown_connections 2024-04-25T00:32:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.332+0000 7f6738b65740 1 -- 172.21.15.42:0/3735834132 wait complete. 2024-04-25T00:32:04.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.332+0000 7f6738b65740 20 client.26251 trim_cache size 0 max 16384 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.350+0000 7f6738b65740 1 Processor -- start 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.350+0000 7f6738b65740 1 -- start start 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.350+0000 7f6738b65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e6581a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.350+0000 7f6738b65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65a840 0x564c2e66b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f6738b65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65f9b0 0x564c2e6586e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f6738b65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67301ad4e0 con 0x564c2e65a840 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f6738b65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f67301aadb0 con 0x564c2e659e50 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f6738b65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f67301a8c00 con 0x564c2e65f9b0 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f672f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e6581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f672e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65a840 0x564c2e66b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f672f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e6581a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34294/0 (socket says 172.21.15.42:34294) 2024-04-25T00:32:04.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f672f400640 1 -- 172.21.15.42:0/1380794317 learned_addr learned my addr 172.21.15.42:0/1380794317 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f672f400640 1 --2- 172.21.15.42:0/1380794317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65f9b0 0x564c2e6586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f672f400640 1 -- 172.21.15.42:0/1380794317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65f9b0 msgr2=0x564c2e6586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f672f400640 1 --2- 172.21.15.42:0/1380794317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65f9b0 0x564c2e6586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f672f400640 1 -- 172.21.15.42:0/1380794317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65a840 msgr2=0x564c2e66b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f672f400640 1 --2- 172.21.15.42:0/1380794317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65a840 0x564c2e66b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.351+0000 7f672f400640 1 -- 172.21.15.42:0/1380794317 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c2e6552f0 con 0x564c2e659e50 2024-04-25T00:32:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.352+0000 7f672f400640 1 --2- 172.21.15.42:0/1380794317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e6581a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6728009fa0 tx=0x7f6728010f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e3016b915884136 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.352+0000 7f671b400640 1 -- 172.21.15.42:0/1380794317 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728024040 con 0x564c2e659e50 2024-04-25T00:32:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.352+0000 7f671b400640 1 -- 172.21.15.42:0/1380794317 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672802c3e0 con 0x564c2e659e50 2024-04-25T00:32:04.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.352+0000 7f671b400640 1 -- 172.21.15.42:0/1380794317 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67280118e0 con 0x564c2e659e50 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.353+0000 7f6738b65740 1 -- 172.21.15.42:0/1380794317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 msgr2=0x564c2e6581a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.353+0000 7f6738b65740 1 --2- 172.21.15.42:0/1380794317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e6581a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6728009fa0 tx=0x7f6728010f90 comp rx=0 tx=0).stop 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.353+0000 7f6738b65740 1 -- 172.21.15.42:0/1380794317 shutdown_connections 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.353+0000 7f6738b65740 1 --2- 172.21.15.42:0/1380794317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65f9b0 0x564c2e6586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.353+0000 7f6738b65740 1 --2- 172.21.15.42:0/1380794317 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65a840 0x564c2e66b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.353+0000 7f6738b65740 1 --2- 172.21.15.42:0/1380794317 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e6581a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.353+0000 7f6738b65740 1 -- 172.21.15.42:0/1380794317 >> 172.21.15.42:0/1380794317 conn(0x564c2e6554c0 msgr2=0x564c2e6558a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.353+0000 7f6738b65740 1 -- 172.21.15.42:0/1380794317 shutdown_connections 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.353+0000 7f6738b65740 1 -- 172.21.15.42:0/1380794317 wait complete. 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.354+0000 7f6738b65740 1 -- start start 2024-04-25T00:32:04.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.354+0000 7f6738b65740 1 Processor -- start 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.354+0000 7f6738b65740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e669540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.354+0000 7f6738b65740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e669a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.354+0000 7f6738b65740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e66aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.354+0000 7f6738b65740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f67301a9270 con 0x564c2e65f9b0 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.354+0000 7f6738b65740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f67301aa6e0 con 0x564c2e659e50 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.354+0000 7f6738b65740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f67301a9940 con 0x564c2e65a840 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.354+0000 7f6738b65740 10 client.0 _ll_register_callbacks cb 0x7f6720005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e669540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e669a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e66aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e669a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38024/0 (socket says 172.21.15.42:38024) 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672e000640 1 -- 172.21.15.42:0/1106039727 learned_addr learned my addr 172.21.15.42:0/1106039727 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e66aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:36824/0 (socket says 172.21.15.42:36824) 2024-04-25T00:32:04.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672e000640 1 -- 172.21.15.42:0/1106039727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 msgr2=0x564c2e669540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672e000640 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e669540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672e000640 1 -- 172.21.15.42:0/1106039727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 msgr2=0x564c2e66aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672e000640 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e66aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672e000640 1 -- 172.21.15.42:0/1106039727 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c2e66b030 con 0x564c2e65a840 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672f400640 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e669540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f672e000640 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e669a80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f671c031d20 tx=0x7f671c004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84cde4723f6cb2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.355+0000 7f6718c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.356+0000 7f6718c00640 1 -- 172.21.15.42:0/1106039727 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f671c035070 con 0x564c2e65a840 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.356+0000 7f6718c00640 1 -- 172.21.15.42:0/1106039727 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f671c005250 con 0x564c2e65a840 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.356+0000 7f6738b65740 1 -- 172.21.15.42:0/1106039727 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f672802c3e0 con 0x564c2e65a840 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.356+0000 7f6718c00640 1 -- 172.21.15.42:0/1106039727 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f671c005450 con 0x564c2e65a840 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.356+0000 7f6738b65740 10 client.26257 fetch_fsmap learned FSMap version 31 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.356+0000 7f6738b65740 1 -- 172.21.15.42:0/1106039727 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564c2e6d6b70 con 0x564c2e65a840 2024-04-25T00:32:04.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.357+0000 7f6718c00640 1 -- 172.21.15.42:0/1106039727 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f672802c3e0 con 0x564c2e65a840 2024-04-25T00:32:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.357+0000 7f6718c00640 1 -- 172.21.15.42:0/1106039727 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 31) ==== 8338+0+0 (secure 0 0 0) 0x7f671c059060 con 0x564c2e65a840 2024-04-25T00:32:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.357+0000 7f6738b65740 10 client.26257 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T00:32:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.357+0000 7f6738b65740 10 client.26257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.357+0000 7f6738b65740 20 client.26257 populate_metadata read hostname 'smithi042' 2024-04-25T00:32:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.357+0000 7f6738b65740 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564c2e6d7510 0x564c2e6d9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.357+0000 7f6738b65740 4 client.26257 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:32:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.357+0000 7f6738b65740 1 -- 172.21.15.42:0/1106039727 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564c2e5c4340 con 0x564c2e6d7510 2024-04-25T00:32:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.357+0000 7f672f400640 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564c2e6d7510 0x564c2e6d9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.358+0000 7f672f400640 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564c2e6d7510 0x564c2e6d9930 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cba003f5be5e628 server_cookie=b26e4b7d8485cefe in_seq=0 out_seq=0 2024-04-25T00:32:04.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.358+0000 7f6718c00640 10 client.26257 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.422+0000 7f6718c00640 1 -- 172.21.15.42:0/1106039727 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564c2e5c4340 con 0x564c2e6d7510 2024-04-25T00:32:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.422+0000 7f6718c00640 10 client.26257 handle_command_reply: tid=0 2024-04-25T00:32:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.422+0000 7f6738b65740 1 client.26257 shutdown 2024-04-25T00:32:04.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.422+0000 7f6738b65740 1 -- 172.21.15.42:0/1106039727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 msgr2=0x564c2e669a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.422+0000 7f6738b65740 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e669a80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f671c031d20 tx=0x7f671c004580 comp rx=0 tx=0).stop 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.423+0000 7f6738b65740 1 -- 172.21.15.42:0/1106039727 shutdown_connections 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.423+0000 7f6738b65740 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564c2e6d7510 0x564c2e6d9930 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.423+0000 7f6738b65740 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564c2e65f9b0 0x564c2e66aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.423+0000 7f6738b65740 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564c2e65a840 0x564c2e669a80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.423+0000 7f6738b65740 1 --2- 172.21.15.42:0/1106039727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564c2e659e50 0x564c2e669540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.423+0000 7f6738b65740 1 -- 172.21.15.42:0/1106039727 >> 172.21.15.42:0/1106039727 conn(0x564c2e6554c0 msgr2=0x564c2e653d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.423+0000 7f6718c00640 0 client.26257 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.423+0000 7f6738b65740 1 -- 172.21.15.42:0/1106039727 shutdown_connections 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.423+0000 7f6738b65740 1 -- 172.21.15.42:0/1106039727 wait complete. 2024-04-25T00:32:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.423+0000 7f6738b65740 20 client.26257 trim_cache size 0 max 16384 2024-04-25T00:32:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:32:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:32:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:32:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:32:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 -- 172.21.15.42:0/1392589759 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67140a2b80 msgr2=0x7f67140a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67140a2b80 0x7f67140a5040 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6720098230 tx=0x7f67200d5060 comp rx=0 tx=0).stop 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 -- 172.21.15.42:0/1392589759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6730102100 msgr2=0x7f6730101a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6730102100 0x7f6730101a70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f672000e9f0 tx=0x7f672000edf0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 -- 172.21.15.42:0/1392589759 shutdown_connections 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f67140a2b80 0x7f67140a5040 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6730107360 0x7f67301005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f6730104a30 0x7f67301000b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 --2- 172.21.15.42:0/1392589759 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6730102100 0x7f6730101a70 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 -- 172.21.15.42:0/1392589759 >> 172.21.15.42:0/1392589759 conn(0x7f67300fba30 msgr2=0x7f6730102ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 -- 172.21.15.42:0/1392589759 shutdown_connections 2024-04-25T00:32:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:04.424+0000 7f672fe00640 1 -- 172.21.15.42:0/1392589759 wait complete. 2024-04-25T00:32:05.290 INFO:tasks.workunit.client.0.smithi042.stdout: 1 828320 6.14 MB/sec execute 595 sec latency 57.317 ms 2024-04-25T00:32:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:05 smithi136 ceph-mon[32923]: pgmap v788: 97 pgs: 97 active+clean; 151 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:32:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:05 smithi106 ceph-mon[32510]: pgmap v788: 97 pgs: 97 active+clean; 151 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:32:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:05 smithi042 ceph-mon[28130]: pgmap v788: 97 pgs: 97 active+clean; 151 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 377 KiB/s rd, 1.5 MiB/s wr, 177 op/s 2024-04-25T00:32:06.290 INFO:tasks.workunit.client.0.smithi042.stdout: 1 829606 6.14 MB/sec execute 596 sec latency 30.879 ms 2024-04-25T00:32:07.290 INFO:tasks.workunit.client.0.smithi042.stdout: 1 830923 6.15 MB/sec execute 597 sec latency 19.872 ms 2024-04-25T00:32:07.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:07 smithi106 ceph-mon[32510]: pgmap v789: 97 pgs: 97 active+clean; 151 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-04-25T00:32:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:07 smithi136 ceph-mon[32923]: pgmap v789: 97 pgs: 97 active+clean; 151 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-04-25T00:32:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:07 smithi042 ceph-mon[28130]: pgmap v789: 97 pgs: 97 active+clean; 151 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 1.7 MiB/s wr, 217 op/s 2024-04-25T00:32:08.290 INFO:tasks.workunit.client.0.smithi042.stdout: 1 831368 6.14 MB/sec execute 598 sec latency 65.736 ms 2024-04-25T00:32:09.290 INFO:tasks.workunit.client.0.smithi042.stdout: 1 832294 6.14 MB/sec execute 599 sec latency 38.658 ms 2024-04-25T00:32:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:09 smithi136 ceph-mon[32923]: pgmap v790: 97 pgs: 97 active+clean; 148 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-25T00:32:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:09 smithi106 ceph-mon[32510]: pgmap v790: 97 pgs: 97 active+clean; 148 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-25T00:32:10.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:09 smithi042 ceph-mon[28130]: pgmap v790: 97 pgs: 97 active+clean; 148 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 374 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-04-25T00:32:10.290 INFO:tasks.workunit.client.0.smithi042.stdout: 1 cleanup 600 sec 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: 0 cleanup 600 sec 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: Operation Count AvgLat MaxLat 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: ---------------------------------------- 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: NTCreateX 117610 2.044 97.665 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: Close 86417 0.010 0.723 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: Rename 4984 7.086 229.262 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: Unlink 23728 3.145 5019.729 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: Deltree 2 135.270 227.576 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: Mkdir 1 0.061 0.061 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: Qpathinfo 106654 1.467 4169.441 2024-04-25T00:32:10.625 INFO:tasks.workunit.client.0.smithi042.stdout: Qfileinfo 18672 0.005 0.170 2024-04-25T00:32:10.626 INFO:tasks.workunit.client.0.smithi042.stdout: Qfsinfo 19534 0.544 24.254 2024-04-25T00:32:10.626 INFO:tasks.workunit.client.0.smithi042.stdout: Sfileinfo 9597 0.088 2.030 2024-04-25T00:32:10.626 INFO:tasks.workunit.client.0.smithi042.stdout: Find 41213 0.430 131.115 2024-04-25T00:32:10.626 INFO:tasks.workunit.client.0.smithi042.stdout: WriteX 58499 0.076 50.768 2024-04-25T00:32:10.626 INFO:tasks.workunit.client.0.smithi042.stdout: ReadX 184439 0.023 6.579 2024-04-25T00:32:10.626 INFO:tasks.workunit.client.0.smithi042.stdout: LockX 384 0.896 3.308 2024-04-25T00:32:10.626 INFO:tasks.workunit.client.0.smithi042.stdout: UnlockX 384 0.809 3.996 2024-04-25T00:32:10.626 INFO:tasks.workunit.client.0.smithi042.stdout: Flush 8242 5.777 149.473 2024-04-25T00:32:10.626 INFO:tasks.workunit.client.0.smithi042.stdout: 2024-04-25T00:32:10.626 INFO:tasks.workunit.client.0.smithi042.stdout:Throughput 6.13997 MB/sec 1 clients 1 procs max_latency=5019.802 ms 2024-04-25T00:32:10.627 INFO:tasks.workunit.client.0.smithi042.stderr:+ dbench 10 2024-04-25T00:32:10.628 INFO:tasks.workunit.client.0.smithi042.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-04-25T00:32:10.628 INFO:tasks.workunit.client.0.smithi042.stdout: 2024-04-25T00:32:10.628 INFO:tasks.workunit.client.0.smithi042.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-04-25T00:32:10.631 INFO:tasks.workunit.client.0.smithi042.stdout:8 of 10 processes prepared for launch 0 sec 2024-04-25T00:32:10.732 INFO:tasks.workunit.client.0.smithi042.stdout:10 of 10 processes prepared for launch 0 sec 2024-04-25T00:32:10.732 INFO:tasks.workunit.client.0.smithi042.stdout:releasing clients 2024-04-25T00:32:11.732 INFO:tasks.workunit.client.0.smithi042.stdout: 10 365 130.69 MB/sec warmup 1 sec latency 54.134 ms 2024-04-25T00:32:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:11 smithi106 ceph-mon[32510]: pgmap v791: 97 pgs: 97 active+clean; 149 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-04-25T00:32:12.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:11 smithi042 ceph-mon[28130]: pgmap v791: 97 pgs: 97 active+clean; 149 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-04-25T00:32:12.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:11 smithi136 ceph-mon[32923]: pgmap v791: 97 pgs: 97 active+clean; 149 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 427 KiB/s rd, 1.6 MiB/s wr, 188 op/s 2024-04-25T00:32:12.732 INFO:tasks.workunit.client.0.smithi042.stdout: 10 793 105.12 MB/sec warmup 2 sec latency 70.049 ms 2024-04-25T00:32:13.732 INFO:tasks.workunit.client.0.smithi042.stdout: 10 1194 77.77 MB/sec warmup 3 sec latency 931.048 ms 2024-04-25T00:32:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:13 smithi106 ceph-mon[32510]: pgmap v792: 97 pgs: 97 active+clean; 150 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-04-25T00:32:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:13 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:14.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:13 smithi042 ceph-mon[28130]: pgmap v792: 97 pgs: 97 active+clean; 150 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-04-25T00:32:14.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:13 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:14.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:13 smithi136 ceph-mon[32923]: pgmap v792: 97 pgs: 97 active+clean; 150 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 431 KiB/s rd, 1.6 MiB/s wr, 183 op/s 2024-04-25T00:32:14.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:13 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:14.732 INFO:tasks.workunit.client.0.smithi042.stdout: 10 1762 63.67 MB/sec warmup 4 sec latency 1931.167 ms 2024-04-25T00:32:15.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:14 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:15.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:14 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:15.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:14 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:15.732 INFO:tasks.workunit.client.0.smithi042.stdout: 10 2408 56.72 MB/sec warmup 5 sec latency 2931.281 ms 2024-04-25T00:32:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:15 smithi106 ceph-mon[32510]: pgmap v793: 97 pgs: 97 active+clean; 149 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-25T00:32:16.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:15 smithi042 ceph-mon[28130]: pgmap v793: 97 pgs: 97 active+clean; 149 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-25T00:32:16.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:15 smithi136 ceph-mon[32923]: pgmap v793: 97 pgs: 97 active+clean; 149 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 358 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-04-25T00:32:16.732 INFO:tasks.workunit.client.0.smithi042.stdout: 10 3102 56.62 MB/sec warmup 6 sec latency 3931.415 ms 2024-04-25T00:32:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.574+0000 7fd16f200640 1 -- 172.21.15.42:0/330252726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd168107bf0 msgr2=0x7fd168108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.574+0000 7fd16f200640 1 --2- 172.21.15.42:0/330252726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd168107bf0 0x7fd168108010 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd15c009900 tx=0x7fd15c02a410 comp rx=0 tx=0).stop 2024-04-25T00:32:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.574+0000 7fd16f200640 1 -- 172.21.15.42:0/330252726 shutdown_connections 2024-04-25T00:32:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.574+0000 7fd16f200640 1 --2- 172.21.15.42:0/330252726 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd1681097a0 0x7fd168110030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.574+0000 7fd16f200640 1 --2- 172.21.15.42:0/330252726 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168108de0 0x7fd168109260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.574+0000 7fd16f200640 1 --2- 172.21.15.42:0/330252726 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd168107bf0 0x7fd168108010 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.574+0000 7fd16f200640 1 -- 172.21.15.42:0/330252726 >> 172.21.15.42:0/330252726 conn(0x7fd168076bd0 msgr2=0x7fd168078ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.574+0000 7fd16f200640 1 -- 172.21.15.42:0/330252726 shutdown_connections 2024-04-25T00:32:17.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.574+0000 7fd16f200640 1 -- 172.21.15.42:0/330252726 wait complete. 2024-04-25T00:32:17.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.575+0000 7fd16f200640 1 Processor -- start 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.575+0000 7fd16f200640 1 -- start start 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.575+0000 7fd16f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168107bf0 0x7fd16819ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.575+0000 7fd16f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168108de0 0x7fd16819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.575+0000 7fd16f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1681097a0 0x7fd1681a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.575+0000 7fd16f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd168113890 con 0x7fd168108de0 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd168113710 con 0x7fd1681097a0 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd168113a10 con 0x7fd168107bf0 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168107bf0 0x7fd16819ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1681097a0 0x7fd1681a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16d400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168108de0 0x7fd16819e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168107bf0 0x7fd16819ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57134/0 (socket says 172.21.15.42:57134) 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1681097a0 0x7fd1681a5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35768/0 (socket says 172.21.15.42:35768) 2024-04-25T00:32:17.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16de00640 1 -- 172.21.15.42:0/1935711911 learned_addr learned my addr 172.21.15.42:0/1935711911 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16e800640 1 -- 172.21.15.42:0/1935711911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168107bf0 msgr2=0x7fd16819ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16e800640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168107bf0 0x7fd16819ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16e800640 1 -- 172.21.15.42:0/1935711911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168108de0 msgr2=0x7fd16819e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16e800640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168108de0 0x7fd16819e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16e800640 1 -- 172.21.15.42:0/1935711911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1681a5af0 con 0x7fd1681097a0 2024-04-25T00:32:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.576+0000 7fd16d400640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168108de0 0x7fd16819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:17.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.577+0000 7fd16e800640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1681097a0 0x7fd1681a5480 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd16400ec50 tx=0x7fd16400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.577+0000 7fd162a00640 1 -- 172.21.15.42:0/1935711911 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164003b70 con 0x7fd1681097a0 2024-04-25T00:32:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.577+0000 7fd162a00640 1 -- 172.21.15.42:0/1935711911 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd164003d10 con 0x7fd1681097a0 2024-04-25T00:32:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.577+0000 7fd162a00640 1 -- 172.21.15.42:0/1935711911 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd164010890 con 0x7fd1681097a0 2024-04-25T00:32:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.577+0000 7fd16f200640 1 -- 172.21.15.42:0/1935711911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1681a5de0 con 0x7fd1681097a0 2024-04-25T00:32:17.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.577+0000 7fd16f200640 1 -- 172.21.15.42:0/1935711911 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a6250 con 0x7fd1681097a0 2024-04-25T00:32:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.580+0000 7fd1735ef740 1 Processor -- start 2024-04-25T00:32:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.580+0000 7fd162a00640 1 -- 172.21.15.42:0/1935711911 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd164003fc0 con 0x7fd1681097a0 2024-04-25T00:32:17.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.581+0000 7fd1735ef740 1 -- start start 2024-04-25T00:32:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.581+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281687270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.581+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281687840 0x55a28168c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.581+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a28168c9b0 0x55a28168ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.581+0000 7fd1735ef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a2815e9ab0 con 0x55a281687840 2024-04-25T00:32:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.581+0000 7fd1735ef740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a281592640 con 0x55a281686e50 2024-04-25T00:32:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.581+0000 7fd1735ef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a2815e5a70 con 0x55a28168c9b0 2024-04-25T00:32:17.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.581+0000 7fd16d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281687270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.581+0000 7fd16d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281687270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35780/0 (socket says 172.21.15.42:35780) 2024-04-25T00:32:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.581+0000 7fd16d400640 1 -- 172.21.15.42:0/3818516593 learned_addr learned my addr 172.21.15.42:0/3818516593 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.582+0000 7fd16de00640 1 --2- 172.21.15.42:0/3818516593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281687840 0x55a28168c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.582+0000 7fd16d400640 1 --2- 172.21.15.42:0/3818516593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a28168c9b0 0x55a28168ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.582+0000 7fd16d400640 1 -- 172.21.15.42:0/3818516593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a28168c9b0 msgr2=0x55a28168ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.582+0000 7fd16d400640 1 --2- 172.21.15.42:0/3818516593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a28168c9b0 0x55a28168ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.582+0000 7fd16d400640 1 -- 172.21.15.42:0/3818516593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281687840 msgr2=0x55a28168c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.582+0000 7fd16d400640 1 --2- 172.21.15.42:0/3818516593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281687840 0x55a28168c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.582+0000 7fd16d400640 1 -- 172.21.15.42:0/3818516593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2812fe210 con 0x55a281686e50 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.582+0000 7fd16d400640 1 --2- 172.21.15.42:0/3818516593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281687270 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd1580058c0 tx=0x7fd158009770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d26b1d5ea3014124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.582+0000 7fd162a00640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1500a2b80 0x7fd1500a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.583+0000 7fd157000640 1 -- 172.21.15.42:0/3818516593 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15800fc50 con 0x55a281686e50 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.583+0000 7fd157000640 1 -- 172.21.15.42:0/3818516593 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15800fdf0 con 0x55a281686e50 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.583+0000 7fd157000640 1 -- 172.21.15.42:0/3818516593 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158003e00 con 0x55a281686e50 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.583+0000 7fd162a00640 1 -- 172.21.15.42:0/1935711911 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd1640cf5c0 con 0x7fd1681097a0 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.583+0000 7fd16de00640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1500a2b80 0x7fd1500a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd1735ef740 1 -- 172.21.15.42:0/3818516593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 msgr2=0x55a281687270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd1735ef740 1 --2- 172.21.15.42:0/3818516593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281687270 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd1580058c0 tx=0x7fd158009770 comp rx=0 tx=0).stop 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd16de00640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1500a2b80 0x7fd1500a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd15c000c00 tx=0x7fd15c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd1735ef740 1 -- 172.21.15.42:0/3818516593 shutdown_connections 2024-04-25T00:32:17.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd1735ef740 1 --2- 172.21.15.42:0/3818516593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a28168c9b0 0x55a28168ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd1735ef740 1 --2- 172.21.15.42:0/3818516593 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281687840 0x55a28168c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd1735ef740 1 --2- 172.21.15.42:0/3818516593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281687270 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd1735ef740 1 -- 172.21.15.42:0/3818516593 >> 172.21.15.42:0/3818516593 conn(0x55a2816824c0 msgr2=0x55a2816828c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd1735ef740 1 -- 172.21.15.42:0/3818516593 shutdown_connections 2024-04-25T00:32:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd1735ef740 1 -- 172.21.15.42:0/3818516593 wait complete. 2024-04-25T00:32:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.584+0000 7fd1735ef740 1 -- start start 2024-04-25T00:32:17.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd1735ef740 1 Processor -- start 2024-04-25T00:32:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281691150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a281687840 0x55a281691690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 0x55a2816986d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd1735ef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a28168f960 con 0x55a28168c9b0 2024-04-25T00:32:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd1735ef740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a2815a5990 con 0x55a281686e50 2024-04-25T00:32:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd1735ef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a28168fae0 con 0x55a281687840 2024-04-25T00:32:17.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd1735ef740 10 client.0 _ll_register_callbacks cb 0x55a2815a7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd16de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281691150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd16d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a281687840 0x55a281691690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd16de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281691150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35792/0 (socket says 172.21.15.42:35792) 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd16d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a281687840 0x55a281691690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57160/0 (socket says 172.21.15.42:57160) 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.585+0000 7fd16de00640 1 -- 172.21.15.42:0/2634459342 learned_addr learned my addr 172.21.15.42:0/2634459342 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.586+0000 7fd16e800640 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 0x55a2816986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.586+0000 7fd16d400640 1 -- 172.21.15.42:0/2634459342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 msgr2=0x55a281691150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.586+0000 7fd16d400640 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281691150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.586+0000 7fd16d400640 1 -- 172.21.15.42:0/2634459342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 msgr2=0x55a2816986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.586+0000 7fd16d400640 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 0x55a2816986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.586+0000 7fd16d400640 1 -- 172.21.15.42:0/2634459342 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2813ca620 con 0x55a281687840 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.586+0000 7fd16de00640 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281691150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.586+0000 7fd16e800640 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 0x55a2816986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:17.590 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.586+0000 7fd16d400640 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a281687840 0x55a281691690 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd1580094b0 tx=0x7fd1580114c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db1d63cfe095bb5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.586+0000 7fd14b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.587+0000 7fd14b400640 1 -- 172.21.15.42:0/2634459342 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15802c060 con 0x55a281687840 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.587+0000 7fd14b400640 1 -- 172.21.15.42:0/2634459342 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd158003e10 con 0x55a281687840 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.587+0000 7fd1735ef740 1 -- 172.21.15.42:0/2634459342 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd15800fdf0 con 0x55a281687840 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.587+0000 7fd14b400640 1 -- 172.21.15.42:0/2634459342 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd158004010 con 0x55a281687840 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.588+0000 7fd1735ef740 10 client.26269 fetch_fsmap learned FSMap version 33 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.588+0000 7fd1735ef740 1 -- 172.21.15.42:0/2634459342 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a28169b6b0 con 0x55a281687840 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.588+0000 7fd14b400640 1 -- 172.21.15.42:0/2634459342 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15800fdf0 con 0x55a281687840 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.588+0000 7fd14b400640 1 -- 172.21.15.42:0/2634459342 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd158026030 con 0x55a281687840 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.588+0000 7fd1735ef740 10 client.26269 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.588+0000 7fd1735ef740 10 client.26269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.588+0000 7fd1735ef740 20 client.26269 populate_metadata read hostname 'smithi042' 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.588+0000 7fd1735ef740 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a28169cb30 0x55a28169ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.588+0000 7fd1735ef740 4 client.26269 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.588+0000 7fd1735ef740 1 -- 172.21.15.42:0/2634459342 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2815f6ff0 con 0x55a28169cb30 2024-04-25T00:32:17.591 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.589+0000 7fd16de00640 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a28169cb30 0x55a28169ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.589+0000 7fd16de00640 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a28169cb30 0x55a28169ef20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f747210fe07e8e68 server_cookie=2a9e46094f97889d in_seq=0 out_seq=0 2024-04-25T00:32:17.592 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.589+0000 7fd14b400640 10 client.26269 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd14b400640 1 -- 172.21.15.42:0/2634459342 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a2815f6ff0 con 0x55a28169cb30 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd14b400640 10 client.26269 handle_command_reply: tid=0 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 client.26269 shutdown 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 -- 172.21.15.42:0/2634459342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a281687840 msgr2=0x55a281691690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a281687840 0x55a281691690 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd1580094b0 tx=0x7fd1580114c0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 -- 172.21.15.42:0/2634459342 shutdown_connections 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a28169cb30 0x55a28169ef20 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 0x55a2816986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a281687840 0x55a281691690 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 --2- 172.21.15.42:0/2634459342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281691150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 -- 172.21.15.42:0/2634459342 >> 172.21.15.42:0/2634459342 conn(0x55a2816824c0 msgr2=0x55a281681620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:17.594 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd14b400640 0 client.26269 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 -- 172.21.15.42:0/2634459342 shutdown_connections 2024-04-25T00:32:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.593+0000 7fd1735ef740 1 -- 172.21.15.42:0/2634459342 wait complete. 2024-04-25T00:32:17.595 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.594+0000 7fd1735ef740 20 client.26269 trim_cache size 0 max 16384 2024-04-25T00:32:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.614+0000 7fd1735ef740 1 Processor -- start 2024-04-25T00:32:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.614+0000 7fd1735ef740 1 -- start start 2024-04-25T00:32:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.614+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 0x55a281688b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.614+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281689070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.614+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 0x55a281698690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.614+0000 7fd1735ef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681b1920 con 0x55a28168c9b0 2024-04-25T00:32:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.614+0000 7fd1735ef740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681af190 con 0x55a281686e50 2024-04-25T00:32:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.614+0000 7fd1735ef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681acfe0 con 0x55a2816824c0 2024-04-25T00:32:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 0x55a281698690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281689070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16d400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281689070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35798/0 (socket says 172.21.15.42:35798) 2024-04-25T00:32:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16d400640 1 -- 172.21.15.42:0/4078509420 learned_addr learned my addr 172.21.15.42:0/4078509420 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16de00640 1 --2- 172.21.15.42:0/4078509420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 0x55a281688b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16d400640 1 -- 172.21.15.42:0/4078509420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 msgr2=0x55a281688b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16d400640 1 --2- 172.21.15.42:0/4078509420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 0x55a281688b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16d400640 1 -- 172.21.15.42:0/4078509420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 msgr2=0x55a281698690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16d400640 1 --2- 172.21.15.42:0/4078509420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 0x55a281698690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16d400640 1 -- 172.21.15.42:0/4078509420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2816822f0 con 0x55a281686e50 2024-04-25T00:32:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.615+0000 7fd16e800640 1 --2- 172.21.15.42:0/4078509420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 0x55a281698690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.616+0000 7fd16d400640 1 --2- 172.21.15.42:0/4078509420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281689070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd158024040 tx=0x7fd158003fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8f18a1daf8263a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.616+0000 7fd156600640 1 -- 172.21.15.42:0/4078509420 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158037070 con 0x55a281686e50 2024-04-25T00:32:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.616+0000 7fd156600640 1 -- 172.21.15.42:0/4078509420 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15801f3f0 con 0x55a281686e50 2024-04-25T00:32:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.616+0000 7fd156600640 1 -- 172.21.15.42:0/4078509420 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15802c040 con 0x55a281686e50 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.617+0000 7fd1735ef740 1 -- 172.21.15.42:0/4078509420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 msgr2=0x55a281689070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.617+0000 7fd1735ef740 1 --2- 172.21.15.42:0/4078509420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281689070 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd158024040 tx=0x7fd158003fa0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.617+0000 7fd1735ef740 1 -- 172.21.15.42:0/4078509420 shutdown_connections 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.617+0000 7fd1735ef740 1 --2- 172.21.15.42:0/4078509420 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a28168c9b0 0x55a281698690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.617+0000 7fd1735ef740 1 --2- 172.21.15.42:0/4078509420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a281686e50 0x55a281689070 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.617+0000 7fd1735ef740 1 --2- 172.21.15.42:0/4078509420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 0x55a281688b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.617+0000 7fd1735ef740 1 -- 172.21.15.42:0/4078509420 >> 172.21.15.42:0/4078509420 conn(0x55a281688410 msgr2=0x55a28168c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.617+0000 7fd1735ef740 1 -- 172.21.15.42:0/4078509420 shutdown_connections 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.617+0000 7fd1735ef740 1 -- 172.21.15.42:0/4078509420 wait complete. 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.617+0000 7fd1735ef740 1 -- start start 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd1735ef740 1 Processor -- start 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 0x55a281695550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281686e50 0x55a281695a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.619 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd1735ef740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a28168c9b0 0x55a281701040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd1735ef740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681ad650 con 0x55a281686e50 2024-04-25T00:32:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd1735ef740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681aeac0 con 0x55a28168c9b0 2024-04-25T00:32:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd1735ef740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd1681add20 con 0x55a2816824c0 2024-04-25T00:32:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd1735ef740 10 client.0 _ll_register_callbacks cb 0x7fd158005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd16e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a28168c9b0 0x55a281701040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd16e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a28168c9b0 0x55a281701040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35814/0 (socket says 172.21.15.42:35814) 2024-04-25T00:32:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd16e800640 1 -- 172.21.15.42:0/135118718 learned_addr learned my addr 172.21.15.42:0/135118718 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.618+0000 7fd16de00640 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 0x55a281695550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd16d400640 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281686e50 0x55a281695a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd16e800640 1 -- 172.21.15.42:0/135118718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 msgr2=0x55a281695550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd16e800640 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 0x55a281695550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd16e800640 1 -- 172.21.15.42:0/135118718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281686e50 msgr2=0x55a281695a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd16e800640 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281686e50 0x55a281695a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd16e800640 1 -- 172.21.15.42:0/135118718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a281701580 con 0x55a28168c9b0 2024-04-25T00:32:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd16de00640 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 0x55a281695550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd16d400640 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281686e50 0x55a281695a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd16e800640 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a28168c9b0 0x55a281701040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd16409bde0 tx=0x7fd16409b7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74c65ead4402443 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd14b400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd14b400640 1 -- 172.21.15.42:0/135118718 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1640b9020 con 0x55a28168c9b0 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd14b400640 1 -- 172.21.15.42:0/135118718 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fd164004d40 con 0x55a28168c9b0 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.619+0000 7fd1735ef740 1 -- 172.21.15.42:0/135118718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd15801f3f0 con 0x55a28168c9b0 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd14b400640 1 -- 172.21.15.42:0/135118718 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fd164081050 con 0x55a28168c9b0 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd1735ef740 10 client.26270 fetch_fsmap learned FSMap version 33 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd1735ef740 1 -- 172.21.15.42:0/135118718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a281702890 con 0x55a28168c9b0 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd14b400640 1 -- 172.21.15.42:0/135118718 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd15801f3f0 con 0x55a28168c9b0 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd14b400640 1 -- 172.21.15.42:0/135118718 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fd164097ba0 con 0x55a28168c9b0 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd1735ef740 10 client.26270 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd1735ef740 10 client.26270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd1735ef740 20 client.26270 populate_metadata read hostname 'smithi042' 2024-04-25T00:32:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd1735ef740 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a281703210 0x55a281705630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd1735ef740 4 client.26270 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:32:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd1735ef740 1 -- 172.21.15.42:0/135118718 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2815f1340 con 0x55a281703210 2024-04-25T00:32:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.620+0000 7fd16de00640 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a281703210 0x55a281705630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.621+0000 7fd16de00640 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a281703210 0x55a281705630 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620e4a6fa83a3f8e server_cookie=e891f2cf69741f2d in_seq=0 out_seq=0 2024-04-25T00:32:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:17.621+0000 7fd14b400640 10 client.26270 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:17.733 INFO:tasks.workunit.client.0.smithi042.stdout: 10 3481 54.15 MB/sec warmup 7 sec latency 4554.246 ms 2024-04-25T00:32:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:17 smithi106 ceph-mon[32510]: pgmap v794: 97 pgs: 97 active+clean; 145 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-04-25T00:32:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.118+0000 7fd14b400640 1 -- 172.21.15.42:0/135118718 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2815f1340 con 0x55a281703210 2024-04-25T00:32:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.118+0000 7fd14b400640 10 client.26270 handle_command_reply: tid=0 2024-04-25T00:32:18.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.118+0000 7fd1735ef740 1 client.26270 shutdown 2024-04-25T00:32:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.118+0000 7fd1735ef740 1 -- 172.21.15.42:0/135118718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a28168c9b0 msgr2=0x55a281701040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.118+0000 7fd1735ef740 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a28168c9b0 0x55a281701040 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fd16409bde0 tx=0x7fd16409b7b0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.118+0000 7fd1735ef740 1 -- 172.21.15.42:0/135118718 shutdown_connections 2024-04-25T00:32:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.118+0000 7fd1735ef740 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a281703210 0x55a281705630 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.118+0000 7fd1735ef740 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a28168c9b0 0x55a281701040 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.119+0000 7fd1735ef740 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a281686e50 0x55a281695a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.119+0000 7fd1735ef740 1 --2- 172.21.15.42:0/135118718 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a2816824c0 0x55a281695550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.119+0000 7fd1735ef740 1 -- 172.21.15.42:0/135118718 >> 172.21.15.42:0/135118718 conn(0x55a281688410 msgr2=0x55a28168bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.119+0000 7fd14b400640 0 client.26270 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.119+0000 7fd1735ef740 1 -- 172.21.15.42:0/135118718 shutdown_connections 2024-04-25T00:32:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.119+0000 7fd1735ef740 1 -- 172.21.15.42:0/135118718 wait complete. 2024-04-25T00:32:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.119+0000 7fd1735ef740 20 client.26270 trim_cache size 0 max 16384 2024-04-25T00:32:18.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:32:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:32:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:32:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:32:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.120+0000 7fd16f200640 1 -- 172.21.15.42:0/1935711911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1500a2b80 msgr2=0x7fd1500a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.120+0000 7fd16f200640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1500a2b80 0x7fd1500a5040 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd15c000c00 tx=0x7fd15c0315b0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.120+0000 7fd16f200640 1 -- 172.21.15.42:0/1935711911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1681097a0 msgr2=0x7fd1681a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:18.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.120+0000 7fd16f200640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1681097a0 0x7fd1681a5480 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd16400ec50 tx=0x7fd16400c6a0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.121+0000 7fd16f200640 1 -- 172.21.15.42:0/1935711911 shutdown_connections 2024-04-25T00:32:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.121+0000 7fd16f200640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd1500a2b80 0x7fd1500a5040 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.121+0000 7fd16f200640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1681097a0 0x7fd1681a5480 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.121+0000 7fd16f200640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd168108de0 0x7fd16819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.121+0000 7fd16f200640 1 --2- 172.21.15.42:0/1935711911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd168107bf0 0x7fd16819ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.121+0000 7fd16f200640 1 -- 172.21.15.42:0/1935711911 >> 172.21.15.42:0/1935711911 conn(0x7fd168076bd0 msgr2=0x7fd16810c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.122+0000 7fd16f200640 1 -- 172.21.15.42:0/1935711911 shutdown_connections 2024-04-25T00:32:18.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:18.122+0000 7fd16f200640 1 -- 172.21.15.42:0/1935711911 wait complete. 2024-04-25T00:32:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:17 smithi042 ceph-mon[28130]: pgmap v794: 97 pgs: 97 active+clean; 145 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-04-25T00:32:18.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:17 smithi136 ceph-mon[32923]: pgmap v794: 97 pgs: 97 active+clean; 145 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 2.5 MiB/s wr, 241 op/s 2024-04-25T00:32:18.733 INFO:tasks.workunit.client.0.smithi042.stdout: 10 3615 47.68 MB/sec warmup 8 sec latency 370.149 ms 2024-04-25T00:32:19.733 INFO:tasks.workunit.client.0.smithi042.stdout: 10 3829 42.97 MB/sec warmup 9 sec latency 61.759 ms 2024-04-25T00:32:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:19 smithi106 ceph-mon[32510]: pgmap v795: 97 pgs: 97 active+clean; 148 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.6 MiB/s wr, 203 op/s 2024-04-25T00:32:20.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:19 smithi042 ceph-mon[28130]: pgmap v795: 97 pgs: 97 active+clean; 148 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.6 MiB/s wr, 203 op/s 2024-04-25T00:32:20.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:19 smithi136 ceph-mon[32923]: pgmap v795: 97 pgs: 97 active+clean; 148 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 728 KiB/s rd, 2.6 MiB/s wr, 203 op/s 2024-04-25T00:32:20.733 INFO:tasks.workunit.client.0.smithi042.stdout: 10 4064 39.04 MB/sec warmup 10 sec latency 110.291 ms 2024-04-25T00:32:21.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:20 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:32:21.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:20 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:32:21.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:20 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:32:21.733 INFO:tasks.workunit.client.0.smithi042.stdout: 10 4522 37.02 MB/sec warmup 11 sec latency 61.308 ms 2024-04-25T00:32:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:21 smithi106 ceph-mon[32510]: pgmap v796: 97 pgs: 97 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.8 MiB/s wr, 299 op/s 2024-04-25T00:32:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:32:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:32:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:32:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:21 smithi042 ceph-mon[28130]: pgmap v796: 97 pgs: 97 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.8 MiB/s wr, 299 op/s 2024-04-25T00:32:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:32:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:32:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:32:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:21 smithi136 ceph-mon[32923]: pgmap v796: 97 pgs: 97 active+clean; 146 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 2.8 MiB/s wr, 299 op/s 2024-04-25T00:32:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:32:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:32:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:32:22.733 INFO:tasks.workunit.client.0.smithi042.stdout: 10 5165 36.78 MB/sec warmup 12 sec latency 41.039 ms 2024-04-25T00:32:23.733 INFO:tasks.workunit.client.0.smithi042.stdout: 10 5762 35.66 MB/sec warmup 13 sec latency 63.446 ms 2024-04-25T00:32:24.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:23 smithi042 ceph-mon[28130]: pgmap v797: 97 pgs: 97 active+clean; 145 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 3.3 MiB/s wr, 465 op/s 2024-04-25T00:32:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:23 smithi136 ceph-mon[32923]: pgmap v797: 97 pgs: 97 active+clean; 145 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 3.3 MiB/s wr, 465 op/s 2024-04-25T00:32:24.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:23 smithi106 ceph-mon[32510]: pgmap v797: 97 pgs: 97 active+clean; 145 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 901 KiB/s rd, 3.3 MiB/s wr, 465 op/s 2024-04-25T00:32:24.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 6280 36.68 MB/sec warmup 14 sec latency 113.531 ms 2024-04-25T00:32:25.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 6879 37.35 MB/sec warmup 15 sec latency 51.214 ms 2024-04-25T00:32:26.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:25 smithi042 ceph-mon[28130]: pgmap v798: 97 pgs: 97 active+clean; 148 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 474 op/s 2024-04-25T00:32:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:25 smithi136 ceph-mon[32923]: pgmap v798: 97 pgs: 97 active+clean; 148 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 474 op/s 2024-04-25T00:32:26.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:25 smithi106 ceph-mon[32510]: pgmap v798: 97 pgs: 97 active+clean; 148 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 474 op/s 2024-04-25T00:32:26.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 7078 35.62 MB/sec warmup 16 sec latency 75.629 ms 2024-04-25T00:32:27.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 7267 33.72 MB/sec warmup 17 sec latency 102.383 ms 2024-04-25T00:32:28.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:27 smithi042 ceph-mon[28130]: pgmap v799: 97 pgs: 97 active+clean; 151 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 545 op/s 2024-04-25T00:32:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:27 smithi136 ceph-mon[32923]: pgmap v799: 97 pgs: 97 active+clean; 151 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 545 op/s 2024-04-25T00:32:28.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:27 smithi106 ceph-mon[32510]: pgmap v799: 97 pgs: 97 active+clean; 151 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 545 op/s 2024-04-25T00:32:28.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 7468 32.02 MB/sec warmup 18 sec latency 202.656 ms 2024-04-25T00:32:29.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 7885 31.35 MB/sec warmup 19 sec latency 76.165 ms 2024-04-25T00:32:30.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:29 smithi042 ceph-mon[28130]: pgmap v800: 97 pgs: 97 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 465 op/s 2024-04-25T00:32:30.286 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:29 smithi136 ceph-mon[32923]: pgmap v800: 97 pgs: 97 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 465 op/s 2024-04-25T00:32:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:29 smithi106 ceph-mon[32510]: pgmap v800: 97 pgs: 97 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 465 op/s 2024-04-25T00:32:30.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 8387 31.26 MB/sec warmup 20 sec latency 67.880 ms 2024-04-25T00:32:31.297 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.297+0000 7fa673e00640 1 -- 172.21.15.42:0/3930313873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c100a10 msgr2=0x7fa66c100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.297+0000 7fa673e00640 1 --2- 172.21.15.42:0/3930313873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c100a10 0x7fa66c100e70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa658009900 tx=0x7fa65802a480 comp rx=0 tx=0).stop 2024-04-25T00:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.297+0000 7fa673e00640 1 -- 172.21.15.42:0/3930313873 shutdown_connections 2024-04-25T00:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.297+0000 7fa673e00640 1 --2- 172.21.15.42:0/3930313873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1013b0 0x7fa66c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.297+0000 7fa673e00640 1 --2- 172.21.15.42:0/3930313873 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c100a10 0x7fa66c100e70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.297+0000 7fa673e00640 1 --2- 172.21.15.42:0/3930313873 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c1000b0 0x7fa66c1004d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.297+0000 7fa673e00640 1 -- 172.21.15.42:0/3930313873 >> 172.21.15.42:0/3930313873 conn(0x7fa66c0fbc30 msgr2=0x7fa66c0fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.297+0000 7fa673e00640 1 -- 172.21.15.42:0/3930313873 shutdown_connections 2024-04-25T00:32:31.298 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.297+0000 7fa673e00640 1 -- 172.21.15.42:0/3930313873 wait complete. 2024-04-25T00:32:31.299 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.298+0000 7fa673e00640 1 Processor -- start 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.298+0000 7fa673e00640 1 -- start start 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa673e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1000b0 0x7fa66c199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa673e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c100a10 0x7fa66c19a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa673e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c1013b0 0x7fa66c1a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa673e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c10f690 con 0x7fa66c1013b0 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa673e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c10f510 con 0x7fa66c100a10 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa673e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c10f810 con 0x7fa66c1000b0 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa670a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c100a10 0x7fa66c19a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa670a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c100a10 0x7fa66c19a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45046/0 (socket says 172.21.15.42:45046) 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa670a00640 1 -- 172.21.15.42:0/148086045 learned_addr learned my addr 172.21.15.42:0/148086045 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa671400640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1000b0 0x7fa66c199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.300 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.299+0000 7fa671e00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c1013b0 0x7fa66c1a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.300+0000 7fa670a00640 1 -- 172.21.15.42:0/148086045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1000b0 msgr2=0x7fa66c199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.300+0000 7fa670a00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1000b0 0x7fa66c199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.300+0000 7fa670a00640 1 -- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c1013b0 msgr2=0x7fa66c1a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.300+0000 7fa670a00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c1013b0 0x7fa66c1a1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.300+0000 7fa670a00640 1 -- 172.21.15.42:0/148086045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c1a17f0 con 0x7fa66c100a10 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.300+0000 7fa671400640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1000b0 0x7fa66c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.300+0000 7fa671e00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c1013b0 0x7fa66c1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.300+0000 7fa670a00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c100a10 0x7fa66c19a180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa658031440 tx=0x7fa658032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.300+0000 7fa66a000640 1 -- 172.21.15.42:0/148086045 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65803f030 con 0x7fa66c100a10 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.301+0000 7fa66a000640 1 -- 172.21.15.42:0/148086045 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa658003ce0 con 0x7fa66c100a10 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.301+0000 7fa66a000640 1 -- 172.21.15.42:0/148086045 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658035070 con 0x7fa66c100a10 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.301+0000 7fa673e00640 1 -- 172.21.15.42:0/148086045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c1a1a80 con 0x7fa66c100a10 2024-04-25T00:32:31.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.301+0000 7fa673e00640 1 -- 172.21.15.42:0/148086045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c1a1fc0 con 0x7fa66c100a10 2024-04-25T00:32:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.306+0000 7fa66a000640 1 -- 172.21.15.42:0/148086045 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa658042030 con 0x7fa66c100a10 2024-04-25T00:32:31.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.306+0000 7fa676bad740 1 Processor -- start 2024-04-25T00:32:31.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.306+0000 7fa676bad740 1 -- start start 2024-04-25T00:32:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.307+0000 7fa676bad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e82b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.307+0000 7fa676bad740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e830190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.307+0000 7fa676bad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e830800 0x55841e832c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.307+0000 7fa676bad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55841e78dab0 con 0x55841e830800 2024-04-25T00:32:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.307+0000 7fa676bad740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55841e736640 con 0x55841e82b690 2024-04-25T00:32:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.307+0000 7fa676bad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55841e789a70 con 0x55841e82aca0 2024-04-25T00:32:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.307+0000 7fa671e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e82b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.307+0000 7fa671e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e82b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:40546/0 (socket says 172.21.15.42:40546) 2024-04-25T00:32:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.307+0000 7fa671e00640 1 -- 172.21.15.42:0/733044318 learned_addr learned my addr 172.21.15.42:0/733044318 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:31.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.307+0000 7fa671400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e830190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.308+0000 7fa671e00640 1 -- 172.21.15.42:0/733044318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 msgr2=0x55841e830190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.308+0000 7fa671e00640 1 --2- 172.21.15.42:0/733044318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e830190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.308+0000 7fa671e00640 1 -- 172.21.15.42:0/733044318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e830800 msgr2=0x55841e832c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:32:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.308+0000 7fa671e00640 1 --2- 172.21.15.42:0/733044318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e830800 0x55841e832c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.308+0000 7fa671e00640 1 -- 172.21.15.42:0/733044318 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841e4a2210 con 0x55841e82aca0 2024-04-25T00:32:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.308+0000 7fa66a000640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa6540a2d30 0x7fa6540a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.308+0000 7fa66a000640 1 -- 172.21.15.42:0/148086045 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa6580ef5e0 con 0x7fa66c100a10 2024-04-25T00:32:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.309+0000 7fa671e00640 1 --2- 172.21.15.42:0/733044318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e82b0c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa664009fd0 tx=0x7fa6640108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d07c984606073d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.309+0000 7fa671400640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa6540a2d30 0x7fa6540a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.309+0000 7fa65e600640 1 -- 172.21.15.42:0/733044318 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664011760 con 0x55841e82aca0 2024-04-25T00:32:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.309+0000 7fa65e600640 1 -- 172.21.15.42:0/733044318 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa664011de0 con 0x55841e82aca0 2024-04-25T00:32:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.309+0000 7fa65e600640 1 -- 172.21.15.42:0/733044318 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa66400e3c0 con 0x55841e82aca0 2024-04-25T00:32:31.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa676bad740 1 -- 172.21.15.42:0/733044318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 msgr2=0x55841e82b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa671400640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa6540a2d30 0x7fa6540a51f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55841e82ba90 tx=0x7fa66000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa676bad740 1 --2- 172.21.15.42:0/733044318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e82b0c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa664009fd0 tx=0x7fa6640108e0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa676bad740 1 -- 172.21.15.42:0/733044318 shutdown_connections 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa676bad740 1 --2- 172.21.15.42:0/733044318 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e830800 0x55841e832c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa676bad740 1 --2- 172.21.15.42:0/733044318 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e830190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa676bad740 1 --2- 172.21.15.42:0/733044318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e82b0c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa676bad740 1 -- 172.21.15.42:0/733044318 >> 172.21.15.42:0/733044318 conn(0x55841e826310 msgr2=0x55841e826710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa676bad740 1 -- 172.21.15.42:0/733044318 shutdown_connections 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa676bad740 1 -- 172.21.15.42:0/733044318 wait complete. 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.310+0000 7fa676bad740 1 -- start start 2024-04-25T00:32:31.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.311+0000 7fa676bad740 1 Processor -- start 2024-04-25T00:32:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.311+0000 7fa676bad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e835010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.311+0000 7fa676bad740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e835550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.311+0000 7fa676bad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e830800 0x55841e83c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.311+0000 7fa676bad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55841e8337b0 con 0x55841e830800 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.311+0000 7fa676bad740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55841e749990 con 0x55841e82b690 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.311+0000 7fa676bad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55841e833930 con 0x55841e82aca0 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.311+0000 7fa676bad740 10 client.0 _ll_register_callbacks cb 0x55841e74b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.311+0000 7fa671400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e835010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa670a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e835550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa670a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e835550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45062/0 (socket says 172.21.15.42:45062) 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa671400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e835010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:40548/0 (socket says 172.21.15.42:40548) 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa670a00640 1 -- 172.21.15.42:0/292086124 learned_addr learned my addr 172.21.15.42:0/292086124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa671e00640 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e830800 0x55841e83c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa670a00640 1 -- 172.21.15.42:0/292086124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 msgr2=0x55841e835010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa670a00640 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e835010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa670a00640 1 -- 172.21.15.42:0/292086124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e830800 msgr2=0x55841e83c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa670a00640 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e830800 0x55841e83c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa670a00640 1 -- 172.21.15.42:0/292086124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841e56e620 con 0x55841e82b690 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa671e00640 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e830800 0x55841e83c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa670a00640 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e835550 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa6580bbb10 tx=0x7fa6580bbce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=721c8cfd98683712 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.312+0000 7fa652a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa652a00640 1 -- 172.21.15.42:0/292086124 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658049040 con 0x55841e82b690 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa652a00640 1 -- 172.21.15.42:0/292086124 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa658003800 con 0x55841e82b690 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa676bad740 1 -- 172.21.15.42:0/292086124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa664011de0 con 0x55841e82b690 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa652a00640 1 -- 172.21.15.42:0/292086124 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa6580bc510 con 0x55841e82b690 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa676bad740 10 client.26282 fetch_fsmap learned FSMap version 33 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa676bad740 1 -- 172.21.15.42:0/292086124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55841e83f570 con 0x55841e82b690 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa652a00640 1 -- 172.21.15.42:0/292086124 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa664011de0 con 0x55841e82b690 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa652a00640 1 -- 172.21.15.42:0/292086124 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa65803b460 con 0x55841e82b690 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa676bad740 10 client.26282 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa676bad740 10 client.26282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa676bad740 20 client.26282 populate_metadata read hostname 'smithi042' 2024-04-25T00:32:31.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa676bad740 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55841e840d20 0x55841e843140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa676bad740 4 client.26282 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.313+0000 7fa676bad740 1 -- 172.21.15.42:0/292086124 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55841e79aff0 con 0x55841e840d20 2024-04-25T00:32:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.314+0000 7fa671400640 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55841e840d20 0x55841e843140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.314+0000 7fa671400640 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55841e840d20 0x55841e843140 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ddc383b72706ca2 server_cookie=d8dc235ed5bf4565 in_seq=0 out_seq=0 2024-04-25T00:32:31.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.314+0000 7fa652a00640 10 client.26282 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.317+0000 7fa652a00640 1 -- 172.21.15.42:0/292086124 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55841e79aff0 con 0x55841e840d20 2024-04-25T00:32:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa652a00640 10 client.26282 handle_command_reply: tid=0 2024-04-25T00:32:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 client.26282 shutdown 2024-04-25T00:32:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 -- 172.21.15.42:0/292086124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 msgr2=0x55841e835550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e835550 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa6580bbb10 tx=0x7fa6580bbce0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 -- 172.21.15.42:0/292086124 shutdown_connections 2024-04-25T00:32:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55841e840d20 0x55841e843140 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e830800 0x55841e83c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e835550 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 --2- 172.21.15.42:0/292086124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e82aca0 0x55841e835010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 -- 172.21.15.42:0/292086124 >> 172.21.15.42:0/292086124 conn(0x55841e826310 msgr2=0x55841e825480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa652a00640 0 client.26282 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 -- 172.21.15.42:0/292086124 shutdown_connections 2024-04-25T00:32:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 1 -- 172.21.15.42:0/292086124 wait complete. 2024-04-25T00:32:31.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.318+0000 7fa676bad740 20 client.26282 trim_cache size 0 max 16384 2024-04-25T00:32:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.339+0000 7fa676bad740 1 Processor -- start 2024-04-25T00:32:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.339+0000 7fa676bad740 1 -- start start 2024-04-25T00:32:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.339+0000 7fa676bad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 0x55841e82c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.339+0000 7fa676bad740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e82cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.339+0000 7fa676bad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 0x55841e829220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.339+0000 7fa676bad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1a91f0 con 0x55841e82aca0 2024-04-25T00:32:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.339+0000 7fa676bad740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1a6a60 con 0x55841e82b690 2024-04-25T00:32:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.339+0000 7fa676bad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1a48b0 con 0x55841e830800 2024-04-25T00:32:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa671e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 0x55841e829220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa670a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e82cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa671e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 0x55841e829220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:40556/0 (socket says 172.21.15.42:40556) 2024-04-25T00:32:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa671e00640 1 -- 172.21.15.42:0/2218931572 learned_addr learned my addr 172.21.15.42:0/2218931572 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa671400640 1 --2- 172.21.15.42:0/2218931572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 0x55841e82c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa670a00640 1 -- 172.21.15.42:0/2218931572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 msgr2=0x55841e829220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa670a00640 1 --2- 172.21.15.42:0/2218931572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 0x55841e829220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa670a00640 1 -- 172.21.15.42:0/2218931572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 msgr2=0x55841e82c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa670a00640 1 --2- 172.21.15.42:0/2218931572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 0x55841e82c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa670a00640 1 -- 172.21.15.42:0/2218931572 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841e826fe0 con 0x55841e82b690 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa671400640 1 --2- 172.21.15.42:0/2218931572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 0x55841e82c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa671e00640 1 --2- 172.21.15.42:0/2218931572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 0x55841e829220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.340+0000 7fa670a00640 1 --2- 172.21.15.42:0/2218931572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e82cef0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa6580b87e0 tx=0x7fa6580ba320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d5c8336d3e47943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.341+0000 7fa65dc00640 1 -- 172.21.15.42:0/2218931572 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658049070 con 0x55841e82b690 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.341+0000 7fa65dc00640 1 -- 172.21.15.42:0/2218931572 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6580c4a80 con 0x55841e82b690 2024-04-25T00:32:31.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.341+0000 7fa65dc00640 1 -- 172.21.15.42:0/2218931572 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6580c45f0 con 0x55841e82b690 2024-04-25T00:32:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.341+0000 7fa676bad740 1 -- 172.21.15.42:0/2218931572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 msgr2=0x55841e82cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.341+0000 7fa676bad740 1 --2- 172.21.15.42:0/2218931572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e82cef0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa6580b87e0 tx=0x7fa6580ba320 comp rx=0 tx=0).stop 2024-04-25T00:32:31.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.341+0000 7fa676bad740 1 -- 172.21.15.42:0/2218931572 shutdown_connections 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.341+0000 7fa676bad740 1 --2- 172.21.15.42:0/2218931572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 0x55841e829220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.341+0000 7fa676bad740 1 --2- 172.21.15.42:0/2218931572 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e82cef0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.341+0000 7fa676bad740 1 --2- 172.21.15.42:0/2218931572 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 0x55841e82c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.342+0000 7fa676bad740 1 -- 172.21.15.42:0/2218931572 >> 172.21.15.42:0/2218931572 conn(0x55841e826310 msgr2=0x55841e820fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.342+0000 7fa676bad740 1 -- 172.21.15.42:0/2218931572 shutdown_connections 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.342+0000 7fa676bad740 1 -- 172.21.15.42:0/2218931572 wait complete. 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.342+0000 7fa676bad740 1 -- start start 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.342+0000 7fa676bad740 1 Processor -- start 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa676bad740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 0x55841e839510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa676bad740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e839a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa676bad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 0x55841e83aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa676bad740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1a4f20 con 0x55841e82aca0 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa676bad740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1a6390 con 0x55841e82b690 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa676bad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1a55f0 con 0x55841e830800 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa676bad740 10 client.0 _ll_register_callbacks cb 0x7fa6580b80b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa671400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 0x55841e839510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa670a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e839a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa670a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e839a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45074/0 (socket says 172.21.15.42:45074) 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa670a00640 1 -- 172.21.15.42:0/167789403 learned_addr learned my addr 172.21.15.42:0/167789403 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa671e00640 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 0x55841e83aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa670a00640 1 -- 172.21.15.42:0/167789403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 msgr2=0x55841e83aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa670a00640 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 0x55841e83aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa670a00640 1 -- 172.21.15.42:0/167789403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 msgr2=0x55841e839510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa670a00640 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 0x55841e839510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.343+0000 7fa670a00640 1 -- 172.21.15.42:0/167789403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841e83b050 con 0x55841e82b690 2024-04-25T00:32:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.344+0000 7fa670a00640 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e839a50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa658003fb0 tx=0x7fa6580b8160 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29c56a841893044d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.344+0000 7fa652a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.344+0000 7fa652a00640 1 -- 172.21.15.42:0/167789403 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa658049070 con 0x55841e82b690 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.344+0000 7fa676bad740 1 -- 172.21.15.42:0/167789403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6580c4a80 con 0x55841e82b690 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.344+0000 7fa652a00640 1 -- 172.21.15.42:0/167789403 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fa6580c45f0 con 0x55841e82b690 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.344+0000 7fa652a00640 1 -- 172.21.15.42:0/167789403 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fa6580a5150 con 0x55841e82b690 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.344+0000 7fa676bad740 10 client.26288 fetch_fsmap learned FSMap version 33 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.344+0000 7fa676bad740 1 -- 172.21.15.42:0/167789403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55841e83b050 con 0x55841e82b690 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.344+0000 7fa652a00640 1 -- 172.21.15.42:0/167789403 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6580c4a80 con 0x55841e82b690 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.344+0000 7fa652a00640 1 -- 172.21.15.42:0/167789403 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 33) ==== 8346+0+0 (secure 0 0 0) 0x7fa6580c6ce0 con 0x55841e82b690 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.345+0000 7fa676bad740 10 client.26288 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.345+0000 7fa676bad740 10 client.26288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.345+0000 7fa676bad740 20 client.26288 populate_metadata read hostname 'smithi042' 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.345+0000 7fa676bad740 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55841e8a7ac0 0x55841e8a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.345+0000 7fa676bad740 4 client.26288 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:32:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.345+0000 7fa676bad740 1 -- 172.21.15.42:0/167789403 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55841e795340 con 0x55841e8a7ac0 2024-04-25T00:32:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.345+0000 7fa671400640 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55841e8a7ac0 0x55841e8a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.345+0000 7fa671400640 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55841e8a7ac0 0x55841e8a9ee0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd72cad0a0fab4ff server_cookie=688710a1548ee500 in_seq=0 out_seq=0 2024-04-25T00:32:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.345+0000 7fa652a00640 10 client.26288 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:31.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 8955 30.20 MB/sec warmup 21 sec latency 232.820 ms 2024-04-25T00:32:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.760+0000 7fa652a00640 1 -- 172.21.15.42:0/167789403 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55841e795340 con 0x55841e8a7ac0 2024-04-25T00:32:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.760+0000 7fa652a00640 10 client.26288 handle_command_reply: tid=0 2024-04-25T00:32:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.760+0000 7fa676bad740 1 client.26288 shutdown 2024-04-25T00:32:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.760+0000 7fa676bad740 1 -- 172.21.15.42:0/167789403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 msgr2=0x55841e839a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.760+0000 7fa676bad740 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e839a50 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa658003fb0 tx=0x7fa6580b8160 comp rx=0 tx=0).stop 2024-04-25T00:32:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.760+0000 7fa676bad740 1 -- 172.21.15.42:0/167789403 shutdown_connections 2024-04-25T00:32:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.760+0000 7fa676bad740 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55841e8a7ac0 0x55841e8a9ee0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.760+0000 7fa676bad740 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55841e830800 0x55841e83aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.761+0000 7fa676bad740 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55841e82b690 0x55841e839a50 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.761+0000 7fa652a00640 0 client.26288 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.761+0000 7fa676bad740 1 --2- 172.21.15.42:0/167789403 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55841e82aca0 0x55841e839510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.761+0000 7fa676bad740 1 -- 172.21.15.42:0/167789403 >> 172.21.15.42:0/167789403 conn(0x55841e826310 msgr2=0x55841e834fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.761+0000 7fa676bad740 1 -- 172.21.15.42:0/167789403 shutdown_connections 2024-04-25T00:32:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.761+0000 7fa676bad740 1 -- 172.21.15.42:0/167789403 wait complete. 2024-04-25T00:32:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.761+0000 7fa676bad740 20 client.26288 trim_cache size 0 max 16384 2024-04-25T00:32:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:32:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:32:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:32:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.762+0000 7fa673e00640 1 -- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa6540a2d30 msgr2=0x7fa6540a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.762+0000 7fa673e00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa6540a2d30 0x7fa6540a51f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x55841e82ba90 tx=0x7fa66000b390 comp rx=0 tx=0).stop 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.762+0000 7fa673e00640 1 -- 172.21.15.42:0/148086045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c100a10 msgr2=0x7fa66c19a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.762+0000 7fa673e00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c100a10 0x7fa66c19a180 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa658031440 tx=0x7fa658032ce0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.763+0000 7fa673e00640 1 -- 172.21.15.42:0/148086045 shutdown_connections 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.763+0000 7fa673e00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa6540a2d30 0x7fa6540a51f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.763+0000 7fa673e00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa66c1013b0 0x7fa66c1a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.763+0000 7fa673e00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa66c100a10 0x7fa66c19a180 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.763+0000 7fa673e00640 1 --2- 172.21.15.42:0/148086045 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa66c1000b0 0x7fa66c199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.763+0000 7fa673e00640 1 -- 172.21.15.42:0/148086045 >> 172.21.15.42:0/148086045 conn(0x7fa66c0fbc30 msgr2=0x7fa66c0fd280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.763+0000 7fa673e00640 1 -- 172.21.15.42:0/148086045 shutdown_connections 2024-04-25T00:32:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:31.763+0000 7fa673e00640 1 -- 172.21.15.42:0/148086045 wait complete. 2024-04-25T00:32:32.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:31 smithi042 ceph-mon[28130]: pgmap v801: 97 pgs: 97 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.2 MiB/s wr, 454 op/s 2024-04-25T00:32:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:31 smithi136 ceph-mon[32923]: pgmap v801: 97 pgs: 97 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.2 MiB/s wr, 454 op/s 2024-04-25T00:32:32.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:31 smithi106 ceph-mon[32510]: pgmap v801: 97 pgs: 97 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 970 KiB/s rd, 3.2 MiB/s wr, 454 op/s 2024-04-25T00:32:32.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 9415 30.16 MB/sec warmup 22 sec latency 201.430 ms 2024-04-25T00:32:33.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 10005 30.95 MB/sec warmup 23 sec latency 66.451 ms 2024-04-25T00:32:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:33 smithi042 ceph-mon[28130]: pgmap v802: 97 pgs: 97 active+clean; 160 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 463 op/s 2024-04-25T00:32:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:33 smithi136 ceph-mon[32923]: pgmap v802: 97 pgs: 97 active+clean; 160 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 463 op/s 2024-04-25T00:32:34.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:33 smithi106 ceph-mon[32510]: pgmap v802: 97 pgs: 97 active+clean; 160 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 463 op/s 2024-04-25T00:32:34.734 INFO:tasks.workunit.client.0.smithi042.stdout: 10 10444 31.40 MB/sec warmup 24 sec latency 75.384 ms 2024-04-25T00:32:35.735 INFO:tasks.workunit.client.0.smithi042.stdout: 10 10623 30.47 MB/sec warmup 25 sec latency 94.690 ms 2024-04-25T00:32:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:35 smithi042 ceph-mon[28130]: pgmap v803: 97 pgs: 97 active+clean; 157 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-04-25T00:32:36.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:35 smithi136 ceph-mon[32923]: pgmap v803: 97 pgs: 97 active+clean; 157 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-04-25T00:32:36.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:35 smithi106 ceph-mon[32510]: pgmap v803: 97 pgs: 97 active+clean; 157 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.2 MiB/s wr, 284 op/s 2024-04-25T00:32:36.735 INFO:tasks.workunit.client.0.smithi042.stdout: 10 10828 29.40 MB/sec warmup 26 sec latency 186.032 ms 2024-04-25T00:32:37.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:36 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:37.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:36 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:37.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:36 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:37.736 INFO:tasks.workunit.client.0.smithi042.stdout: 10 11254 28.78 MB/sec warmup 27 sec latency 92.934 ms 2024-04-25T00:32:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:37 smithi136 ceph-mon[32923]: pgmap v804: 97 pgs: 97 active+clean; 153 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 449 op/s 2024-04-25T00:32:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:37 smithi106 ceph-mon[32510]: pgmap v804: 97 pgs: 97 active+clean; 153 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 449 op/s 2024-04-25T00:32:38.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:37 smithi042 ceph-mon[28130]: pgmap v804: 97 pgs: 97 active+clean; 153 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 449 op/s 2024-04-25T00:32:38.736 INFO:tasks.workunit.client.0.smithi042.stdout: 10 12050 29.34 MB/sec warmup 28 sec latency 42.203 ms 2024-04-25T00:32:39.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:38 smithi136 ceph-mon[32923]: pgmap v805: 97 pgs: 97 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 399 op/s 2024-04-25T00:32:39.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:38 smithi106 ceph-mon[32510]: pgmap v805: 97 pgs: 97 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 399 op/s 2024-04-25T00:32:39.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:38 smithi042 ceph-mon[28130]: pgmap v805: 97 pgs: 97 active+clean; 152 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.9 MiB/s wr, 399 op/s 2024-04-25T00:32:39.736 INFO:tasks.workunit.client.0.smithi042.stdout: 10 12978 29.61 MB/sec warmup 29 sec latency 63.893 ms 2024-04-25T00:32:40.736 INFO:tasks.workunit.client.0.smithi042.stdout: 10 13762 30.60 MB/sec warmup 30 sec latency 115.838 ms 2024-04-25T00:32:41.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:40 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:41.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:40 smithi136 ceph-mon[32923]: pgmap v806: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 421 op/s 2024-04-25T00:32:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:40 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:41.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:40 smithi106 ceph-mon[32510]: pgmap v806: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 421 op/s 2024-04-25T00:32:41.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:40 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:32:41.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:40 smithi042 ceph-mon[28130]: pgmap v806: 97 pgs: 97 active+clean; 155 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 421 op/s 2024-04-25T00:32:41.736 INFO:tasks.workunit.client.0.smithi042.stdout: 10 14331 30.84 MB/sec warmup 31 sec latency 61.200 ms 2024-04-25T00:32:42.736 INFO:tasks.workunit.client.0.smithi042.stdout: 10 15003 30.56 MB/sec warmup 32 sec latency 44.258 ms 2024-04-25T00:32:43.737 INFO:tasks.workunit.client.0.smithi042.stdout: 10 15779 30.76 MB/sec warmup 33 sec latency 170.948 ms 2024-04-25T00:32:43.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:43 smithi136 ceph-mon[32923]: pgmap v807: 97 pgs: 97 active+clean; 163 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 500 op/s 2024-04-25T00:32:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:43 smithi106 ceph-mon[32510]: pgmap v807: 97 pgs: 97 active+clean; 163 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 500 op/s 2024-04-25T00:32:43.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:43 smithi042 ceph-mon[28130]: pgmap v807: 97 pgs: 97 active+clean; 163 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.0 MiB/s wr, 500 op/s 2024-04-25T00:32:44.737 INFO:tasks.workunit.client.0.smithi042.stdout: 10 16153 30.74 MB/sec warmup 34 sec latency 289.648 ms 2024-04-25T00:32:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.945+0000 7fb9fca00640 1 -- 172.21.15.42:0/2568842697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f8102990 msgr2=0x7fb9f8102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.945+0000 7fb9fca00640 1 --2- 172.21.15.42:0/2568842697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f8102990 0x7fb9f8102e10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4009900 tx=0x7fb9e402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.945+0000 7fb9fca00640 1 -- 172.21.15.42:0/2568842697 shutdown_connections 2024-04-25T00:32:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.945+0000 7fb9fca00640 1 --2- 172.21.15.42:0/2568842697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f8103350 0x7fb9f8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.945+0000 7fb9fca00640 1 --2- 172.21.15.42:0/2568842697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f8102990 0x7fb9f8102e10 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.945+0000 7fb9fca00640 1 --2- 172.21.15.42:0/2568842697 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f81017a0 0x7fb9f8101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.945+0000 7fb9fca00640 1 -- 172.21.15.42:0/2568842697 >> 172.21.15.42:0/2568842697 conn(0x7fb9f80fcd10 msgr2=0x7fb9f80ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.945+0000 7fb9fca00640 1 -- 172.21.15.42:0/2568842697 shutdown_connections 2024-04-25T00:32:44.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.945+0000 7fb9fca00640 1 -- 172.21.15.42:0/2568842697 wait complete. 2024-04-25T00:32:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.946+0000 7fb9fca00640 1 Processor -- start 2024-04-25T00:32:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.947+0000 7fb9fca00640 1 -- start start 2024-04-25T00:32:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.947+0000 7fb9fca00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f81017a0 0x7fb9f8071380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.947+0000 7fb9fca00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f8102990 0x7fb9f80718c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.947+0000 7fb9fca00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f8103350 0x7fb9f8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.947+0000 7fb9fca00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f806bb70 con 0x7fb9f81017a0 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.947+0000 7fb9fca00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f806b9f0 con 0x7fb9f8103350 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.947+0000 7fb9fca00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f806bcf0 con 0x7fb9f8102990 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f8103350 0x7fb9f8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f6800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f8103350 0x7fb9f8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42292/0 (socket says 172.21.15.42:42292) 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f8102990 0x7fb9f80718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f6800640 1 -- 172.21.15.42:0/384377871 learned_addr learned my addr 172.21.15.42:0/384377871 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f81017a0 0x7fb9f8071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f6800640 1 -- 172.21.15.42:0/384377871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f8102990 msgr2=0x7fb9f80718c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f6800640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f8102990 0x7fb9f80718c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f6800640 1 -- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f81017a0 msgr2=0x7fb9f8071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f6800640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f81017a0 0x7fb9f8071380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f6800640 1 -- 172.21.15.42:0/384377871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9f8072f70 con 0x7fb9f8103350 2024-04-25T00:32:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f81017a0 0x7fb9f8071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.948+0000 7fb9f6800640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f8103350 0x7fb9f8072900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000eb10 tx=0x7fb9f000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.949+0000 7fb9eea00640 1 -- 172.21.15.42:0/384377871 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0004b20 con 0x7fb9f8103350 2024-04-25T00:32:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.949+0000 7fb9eea00640 1 -- 172.21.15.42:0/384377871 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9f0004cc0 con 0x7fb9f8103350 2024-04-25T00:32:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.949+0000 7fb9eea00640 1 -- 172.21.15.42:0/384377871 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0010790 con 0x7fb9f8103350 2024-04-25T00:32:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.949+0000 7fb9fca00640 1 -- 172.21.15.42:0/384377871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9f8073260 con 0x7fb9f8103350 2024-04-25T00:32:44.954 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.949+0000 7fb9fca00640 1 -- 172.21.15.42:0/384377871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9f8073760 con 0x7fb9f8103350 2024-04-25T00:32:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.953+0000 7fb9ff827740 1 Processor -- start 2024-04-25T00:32:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.955+0000 7fb9eea00640 1 -- 172.21.15.42:0/384377871 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb9f0005000 con 0x7fb9f8103350 2024-04-25T00:32:44.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.955+0000 7fb9ff827740 1 -- start start 2024-04-25T00:32:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.955+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 0x55c9e5ee60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.955+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 0x55c9e5eeb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.956+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5eeb800 0x55c9e5eedc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.956+0000 7fb9ff827740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5e48ab0 con 0x55c9e5ee5ca0 2024-04-25T00:32:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.956+0000 7fb9ff827740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5df1640 con 0x55c9e5eeb800 2024-04-25T00:32:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.956+0000 7fb9ff827740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5e44a70 con 0x55c9e5ee6690 2024-04-25T00:32:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.956+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5eeb800 0x55c9e5eedc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.956+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5eeb800 0x55c9e5eedc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42298/0 (socket says 172.21.15.42:42298) 2024-04-25T00:32:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.956+0000 7fb9f5400640 1 -- 172.21.15.42:0/3450810703 learned_addr learned my addr 172.21.15.42:0/3450810703 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.956+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/3450810703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 0x55c9e5eeb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.956+0000 7fb9f5400640 1 --2- 172.21.15.42:0/3450810703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 0x55c9e5ee60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9f5400640 1 -- 172.21.15.42:0/3450810703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 msgr2=0x55c9e5eeb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9eea00640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb9dc0a2cb0 0x7fb9dc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9f5400640 1 --2- 172.21.15.42:0/3450810703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 0x55c9e5eeb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9f5400640 1 -- 172.21.15.42:0/3450810703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 msgr2=0x55c9e5ee60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9f5400640 1 --2- 172.21.15.42:0/3450810703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 0x55c9e5ee60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9f5400640 1 -- 172.21.15.42:0/3450810703 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9e5b5d210 con 0x55c9e5eeb800 2024-04-25T00:32:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9eea00640 1 -- 172.21.15.42:0/384377871 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fb9f00cf6f0 con 0x7fb9f8103350 2024-04-25T00:32:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9f5400640 1 --2- 172.21.15.42:0/3450810703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5eeb800 0x55c9e5eedc90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4005c20 tx=0x7fb9e4031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a3a6a8b796a98a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9e3000640 1 -- 172.21.15.42:0/3450810703 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4042020 con 0x55c9e5eeb800 2024-04-25T00:32:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9e3000640 1 -- 172.21.15.42:0/3450810703 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e4031b90 con 0x55c9e5eeb800 2024-04-25T00:32:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9e3000640 1 -- 172.21.15.42:0/3450810703 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4035040 con 0x55c9e5eeb800 2024-04-25T00:32:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.957+0000 7fb9f6800640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb9dc0a2cb0 0x7fb9dc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.958+0000 7fb9ff827740 1 -- 172.21.15.42:0/3450810703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5eeb800 msgr2=0x55c9e5eedc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.958+0000 7fb9ff827740 1 --2- 172.21.15.42:0/3450810703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5eeb800 0x55c9e5eedc90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb9e4005c20 tx=0x7fb9e4031400 comp rx=0 tx=0).stop 2024-04-25T00:32:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.958+0000 7fb9f6800640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb9dc0a2cb0 0x7fb9dc0a5170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00a46c0 tx=0x7fb9f000efd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.959+0000 7fb9ff827740 1 -- 172.21.15.42:0/3450810703 shutdown_connections 2024-04-25T00:32:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.959+0000 7fb9ff827740 1 --2- 172.21.15.42:0/3450810703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5eeb800 0x55c9e5eedc90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.959+0000 7fb9ff827740 1 --2- 172.21.15.42:0/3450810703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 0x55c9e5eeb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.959+0000 7fb9ff827740 1 --2- 172.21.15.42:0/3450810703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 0x55c9e5ee60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.959+0000 7fb9ff827740 1 -- 172.21.15.42:0/3450810703 >> 172.21.15.42:0/3450810703 conn(0x55c9e5ee1310 msgr2=0x55c9e5ee1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.960+0000 7fb9ff827740 1 -- 172.21.15.42:0/3450810703 shutdown_connections 2024-04-25T00:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.961+0000 7fb9ff827740 1 -- 172.21.15.42:0/3450810703 wait complete. 2024-04-25T00:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.961+0000 7fb9ff827740 1 -- start start 2024-04-25T00:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.961+0000 7fb9ff827740 1 Processor -- start 2024-04-25T00:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.963+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 0x55c9e5eeff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.963+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee6690 0x55c9e5ef04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.963+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 0x55c9e5ef74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.963+0000 7fb9ff827740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5eee750 con 0x55c9e5ee5ca0 2024-04-25T00:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.963+0000 7fb9ff827740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5e04990 con 0x55c9e5ee6690 2024-04-25T00:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.963+0000 7fb9ff827740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c9e5eee8d0 con 0x55c9e5eeb800 2024-04-25T00:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.963+0000 7fb9f5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee6690 0x55c9e5ef04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.963+0000 7fb9f5e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee6690 0x55c9e5ef04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42308/0 (socket says 172.21.15.42:42308) 2024-04-25T00:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.963+0000 7fb9f5e00640 1 -- 172.21.15.42:0/3881723040 learned_addr learned my addr 172.21.15.42:0/3881723040 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.963+0000 7fb9ff827740 10 client.0 _ll_register_callbacks cb 0x55c9e5e06950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.964+0000 7fb9f5400640 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 0x55c9e5eeff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.964+0000 7fb9f5400640 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 0x55c9e5ef74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.964+0000 7fb9f5400640 1 -- 172.21.15.42:0/3881723040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 msgr2=0x55c9e5ef74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.964+0000 7fb9f5400640 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 0x55c9e5ef74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.964+0000 7fb9f5400640 1 -- 172.21.15.42:0/3881723040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee6690 msgr2=0x55c9e5ef04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.964+0000 7fb9f5400640 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee6690 0x55c9e5ef04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.964+0000 7fb9f5400640 1 -- 172.21.15.42:0/3881723040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9e5c29620 con 0x55c9e5ee5ca0 2024-04-25T00:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.964+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee6690 0x55c9e5ef04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.965+0000 7fb9f5400640 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 0x55c9e5eeff60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9e402a9e0 tx=0x7fb9e4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881800f19f6e3173 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.965+0000 7fb9d7400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.966+0000 7fb9d7400640 1 -- 172.21.15.42:0/3881723040 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4042070 con 0x55c9e5ee5ca0 2024-04-25T00:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.966+0000 7fb9ff827740 1 -- 172.21.15.42:0/3881723040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9e4031b90 con 0x55c9e5ee5ca0 2024-04-25T00:32:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.966+0000 7fb9d7400640 1 -- 172.21.15.42:0/3881723040 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fb9e4003bc0 con 0x55c9e5ee5ca0 2024-04-25T00:32:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.966+0000 7fb9d7400640 1 -- 172.21.15.42:0/3881723040 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb9e4035040 con 0x55c9e5ee5ca0 2024-04-25T00:32:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.966+0000 7fb9ff827740 10 client.16161 fetch_fsmap learned FSMap version 35 2024-04-25T00:32:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.966+0000 7fb9ff827740 1 -- 172.21.15.42:0/3881723040 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c9e5ef7a50 con 0x55c9e5ee5ca0 2024-04-25T00:32:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.967+0000 7fb9d7400640 1 -- 172.21.15.42:0/3881723040 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e4031b90 con 0x55c9e5ee5ca0 2024-04-25T00:32:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.967+0000 7fb9d7400640 1 -- 172.21.15.42:0/3881723040 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb9e403a040 con 0x55c9e5ee5ca0 2024-04-25T00:32:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.967+0000 7fb9ff827740 10 client.16161 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T00:32:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.967+0000 7fb9ff827740 10 client.16161 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.967+0000 7fb9ff827740 20 client.16161 populate_metadata read hostname 'smithi042' 2024-04-25T00:32:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.967+0000 7fb9ff827740 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c9e5ef8ec0 0x55c9e5efb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.967+0000 7fb9ff827740 4 client.16161 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.967+0000 7fb9ff827740 1 -- 172.21.15.42:0/3881723040 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9e5e55ff0 con 0x55c9e5ef8ec0 2024-04-25T00:32:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.968+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c9e5ef8ec0 0x55c9e5efb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.969+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c9e5ef8ec0 0x55c9e5efb2d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f93366ed5b819 server_cookie=dd86bf4c7cdae555 in_seq=0 out_seq=0 2024-04-25T00:32:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.969+0000 7fb9d7400640 10 client.16161 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9d7400640 1 -- 172.21.15.42:0/3881723040 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c9e5e55ff0 con 0x55c9e5ef8ec0 2024-04-25T00:32:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9d7400640 10 client.16161 handle_command_reply: tid=0 2024-04-25T00:32:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9ff827740 1 client.16161 shutdown 2024-04-25T00:32:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9ff827740 1 -- 172.21.15.42:0/3881723040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 msgr2=0x55c9e5eeff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9ff827740 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 0x55c9e5eeff60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9e402a9e0 tx=0x7fb9e4001e70 comp rx=0 tx=0).stop 2024-04-25T00:32:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9ff827740 1 -- 172.21.15.42:0/3881723040 shutdown_connections 2024-04-25T00:32:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9ff827740 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c9e5ef8ec0 0x55c9e5efb2d0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9ff827740 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 0x55c9e5ef74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9ff827740 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee6690 0x55c9e5ef04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9ff827740 1 --2- 172.21.15.42:0/3881723040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee5ca0 0x55c9e5eeff60 secure :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9e402a9e0 tx=0x7fb9e4001e70 comp rx=0 tx=0).stop 2024-04-25T00:32:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9ff827740 1 -- 172.21.15.42:0/3881723040 >> 172.21.15.42:0/3881723040 conn(0x55c9e5ee1310 msgr2=0x55c9e5ee0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.973+0000 7fb9d7400640 0 client.16161 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.974+0000 7fb9ff827740 1 -- 172.21.15.42:0/3881723040 shutdown_connections 2024-04-25T00:32:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.974+0000 7fb9ff827740 1 -- 172.21.15.42:0/3881723040 wait complete. 2024-04-25T00:32:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.974+0000 7fb9ff827740 20 client.16161 trim_cache size 0 max 16384 2024-04-25T00:32:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.997+0000 7fb9ff827740 1 Processor -- start 2024-04-25T00:32:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.998+0000 7fb9ff827740 1 -- start start 2024-04-25T00:32:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.998+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee6690 0x55c9e5edbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.998+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 0x55c9e5ee79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.998+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee7ef0 0x55c9e5ee41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.998+0000 7fb9ff827740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81b1bc0 con 0x55c9e5ee6690 2024-04-25T00:32:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.998+0000 7fb9ff827740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81af3d0 con 0x55c9e5ee7ef0 2024-04-25T00:32:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.998+0000 7fb9ff827740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81ad240 con 0x55c9e5eeb800 2024-04-25T00:32:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.999+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee6690 0x55c9e5edbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.999+0000 7fb9f5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 0x55c9e5ee79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:44.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.999+0000 7fb9f5e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 0x55c9e5ee79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:54656/0 (socket says 172.21.15.42:54656) 2024-04-25T00:32:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.999+0000 7fb9f5e00640 1 -- 172.21.15.42:0/1764513291 learned_addr learned my addr 172.21.15.42:0/1764513291 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:45.000 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:44.999+0000 7fb9f5400640 1 --2- 172.21.15.42:0/1764513291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee7ef0 0x55c9e5ee41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.000+0000 7fb9f5e00640 1 -- 172.21.15.42:0/1764513291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee7ef0 msgr2=0x55c9e5ee41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.000+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/1764513291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee7ef0 0x55c9e5ee41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.000+0000 7fb9f5e00640 1 -- 172.21.15.42:0/1764513291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee6690 msgr2=0x55c9e5edbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.000+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/1764513291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee6690 0x55c9e5edbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.000+0000 7fb9f5e00640 1 -- 172.21.15.42:0/1764513291 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9e5ee1140 con 0x55c9e5eeb800 2024-04-25T00:32:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.000+0000 7fb9f5400640 1 --2- 172.21.15.42:0/1764513291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee6690 0x55c9e5edbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:32:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.001+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/1764513291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 0x55c9e5ee79b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb9f81027f0 tx=0x7fb9e80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fed868be6833cee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.001+0000 7fb9e2600640 1 -- 172.21.15.42:0/1764513291 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e8004220 con 0x55c9e5eeb800 2024-04-25T00:32:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.001+0000 7fb9e2600640 1 -- 172.21.15.42:0/1764513291 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e80043c0 con 0x55c9e5eeb800 2024-04-25T00:32:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.001+0000 7fb9e2600640 1 -- 172.21.15.42:0/1764513291 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e801f350 con 0x55c9e5eeb800 2024-04-25T00:32:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.002+0000 7fb9ff827740 1 -- 172.21.15.42:0/1764513291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 msgr2=0x55c9e5ee79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.002+0000 7fb9ff827740 1 --2- 172.21.15.42:0/1764513291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 0x55c9e5ee79b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fb9f81027f0 tx=0x7fb9e80046c0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.002+0000 7fb9ff827740 1 -- 172.21.15.42:0/1764513291 shutdown_connections 2024-04-25T00:32:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.002+0000 7fb9ff827740 1 --2- 172.21.15.42:0/1764513291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee7ef0 0x55c9e5ee41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.002+0000 7fb9ff827740 1 --2- 172.21.15.42:0/1764513291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5eeb800 0x55c9e5ee79b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.002+0000 7fb9ff827740 1 --2- 172.21.15.42:0/1764513291 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5ee6690 0x55c9e5edbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.002+0000 7fb9ff827740 1 -- 172.21.15.42:0/1764513291 >> 172.21.15.42:0/1764513291 conn(0x55c9e5ee1310 msgr2=0x55c9e5ee16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.002+0000 7fb9ff827740 1 -- 172.21.15.42:0/1764513291 shutdown_connections 2024-04-25T00:32:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.002+0000 7fb9ff827740 1 -- 172.21.15.42:0/1764513291 wait complete. 2024-04-25T00:32:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.003+0000 7fb9ff827740 1 -- start start 2024-04-25T00:32:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.003+0000 7fb9ff827740 1 Processor -- start 2024-04-25T00:32:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.003+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 0x55c9e5f068a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.003+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee7ef0 0x55c9e5f06de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.003+0000 7fb9ff827740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5eeb800 0x55c9e5f07320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.003+0000 7fb9ff827740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81ad890 con 0x55c9e5eeb800 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.003+0000 7fb9ff827740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81aed00 con 0x55c9e5ee7ef0 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.003+0000 7fb9ff827740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb9f81adf60 con 0x55c9e5ee6690 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.003+0000 7fb9ff827740 10 client.0 _ll_register_callbacks cb 0x7fb9e800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 0x55c9e5f068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 0x55c9e5f068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:54662/0 (socket says 172.21.15.42:54662) 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5400640 1 -- 172.21.15.42:0/1018414421 learned_addr learned my addr 172.21.15.42:0/1018414421 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5400640 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5eeb800 0x55c9e5f07320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee7ef0 0x55c9e5f06de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5400640 1 -- 172.21.15.42:0/1018414421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee7ef0 msgr2=0x55c9e5f06de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5400640 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee7ef0 0x55c9e5f06de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5400640 1 -- 172.21.15.42:0/1018414421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5eeb800 msgr2=0x55c9e5f07320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5400640 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5eeb800 0x55c9e5f07320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5400640 1 -- 172.21.15.42:0/1018414421 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9e5f60540 con 0x55c9e5ee6690 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9f5400640 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 0x55c9e5f068a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb9e402a430 tx=0x7fb9e4067040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e12ed5880ee9e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.004+0000 7fb9d7400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.005+0000 7fb9d7400640 1 -- 172.21.15.42:0/1018414421 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9e4042040 con 0x55c9e5ee6690 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.005+0000 7fb9d7400640 1 -- 172.21.15.42:0/1018414421 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fb9e4003dd0 con 0x55c9e5ee6690 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.005+0000 7fb9ff827740 1 -- 172.21.15.42:0/1018414421 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9e80043c0 con 0x55c9e5ee6690 2024-04-25T00:32:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.005+0000 7fb9d7400640 1 -- 172.21.15.42:0/1018414421 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fb9e4003fd0 con 0x55c9e5ee6690 2024-04-25T00:32:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.005+0000 7fb9ff827740 10 client.26308 fetch_fsmap learned FSMap version 35 2024-04-25T00:32:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.005+0000 7fb9ff827740 1 -- 172.21.15.42:0/1018414421 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55c9e5f616c0 con 0x55c9e5ee6690 2024-04-25T00:32:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.005+0000 7fb9d7400640 1 -- 172.21.15.42:0/1018414421 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9e80043c0 con 0x55c9e5ee6690 2024-04-25T00:32:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.006+0000 7fb9d7400640 1 -- 172.21.15.42:0/1018414421 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fb9e403a040 con 0x55c9e5ee6690 2024-04-25T00:32:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.006+0000 7fb9ff827740 10 client.26308 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T00:32:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.006+0000 7fb9ff827740 10 client.26308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.006+0000 7fb9ff827740 20 client.26308 populate_metadata read hostname 'smithi042' 2024-04-25T00:32:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.006+0000 7fb9ff827740 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c9e5f61db0 0x55c9e5f641d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.006+0000 7fb9ff827740 4 client.26308 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:32:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.006+0000 7fb9ff827740 1 -- 172.21.15.42:0/1018414421 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9e5e50340 con 0x55c9e5f61db0 2024-04-25T00:32:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.006+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c9e5f61db0 0x55c9e5f641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.007+0000 7fb9f5e00640 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c9e5f61db0 0x55c9e5f641d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f052ebb16697be75 server_cookie=bcf8ab39df84027 in_seq=0 out_seq=0 2024-04-25T00:32:45.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.007+0000 7fb9d7400640 10 client.26308 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9d7400640 1 -- 172.21.15.42:0/1018414421 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9e5e50340 con 0x55c9e5f61db0 2024-04-25T00:32:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9d7400640 10 client.26308 handle_command_reply: tid=0 2024-04-25T00:32:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 client.26308 shutdown 2024-04-25T00:32:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 -- 172.21.15.42:0/1018414421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 msgr2=0x55c9e5f068a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 0x55c9e5f068a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fb9e402a430 tx=0x7fb9e4067040 comp rx=0 tx=0).stop 2024-04-25T00:32:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 -- 172.21.15.42:0/1018414421 shutdown_connections 2024-04-25T00:32:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c9e5f61db0 0x55c9e5f641d0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c9e5eeb800 0x55c9e5f07320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c9e5ee7ef0 0x55c9e5f06de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 --2- 172.21.15.42:0/1018414421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c9e5ee6690 0x55c9e5f068a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 -- 172.21.15.42:0/1018414421 >> 172.21.15.42:0/1018414421 conn(0x55c9e5ee1310 msgr2=0x55c9e5eeabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9d7400640 0 client.26308 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 -- 172.21.15.42:0/1018414421 shutdown_connections 2024-04-25T00:32:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.242+0000 7fb9ff827740 1 -- 172.21.15.42:0/1018414421 wait complete. 2024-04-25T00:32:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.243+0000 7fb9ff827740 20 client.26308 trim_cache size 0 max 16384 2024-04-25T00:32:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:32:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:32:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:32:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:32:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.243+0000 7fb9fca00640 1 -- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb9dc0a2cb0 msgr2=0x7fb9dc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.243+0000 7fb9fca00640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb9dc0a2cb0 0x7fb9dc0a5170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00a46c0 tx=0x7fb9f000efd0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.244+0000 7fb9fca00640 1 -- 172.21.15.42:0/384377871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f8103350 msgr2=0x7fb9f8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:45.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.244+0000 7fb9fca00640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f8103350 0x7fb9f8072900 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb9f000eb10 tx=0x7fb9f000c2d0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.245+0000 7fb9fca00640 1 -- 172.21.15.42:0/384377871 shutdown_connections 2024-04-25T00:32:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.245+0000 7fb9fca00640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb9dc0a2cb0 0x7fb9dc0a5170 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.245+0000 7fb9fca00640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb9f8103350 0x7fb9f8072900 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.245+0000 7fb9fca00640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb9f8102990 0x7fb9f80718c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.245+0000 7fb9fca00640 1 --2- 172.21.15.42:0/384377871 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb9f81017a0 0x7fb9f8071380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:45.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.245+0000 7fb9fca00640 1 -- 172.21.15.42:0/384377871 >> 172.21.15.42:0/384377871 conn(0x7fb9f80fcd10 msgr2=0x7fb9f8105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.246+0000 7fb9fca00640 1 -- 172.21.15.42:0/384377871 shutdown_connections 2024-04-25T00:32:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:45.246+0000 7fb9fca00640 1 -- 172.21.15.42:0/384377871 wait complete. 2024-04-25T00:32:45.737 INFO:tasks.workunit.client.0.smithi042.stdout: 10 16795 30.71 MB/sec warmup 35 sec latency 230.741 ms 2024-04-25T00:32:45.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:45 smithi136 ceph-mon[32923]: pgmap v808: 97 pgs: 97 active+clean; 180 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 411 op/s 2024-04-25T00:32:45.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:45 smithi106 ceph-mon[32510]: pgmap v808: 97 pgs: 97 active+clean; 180 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 411 op/s 2024-04-25T00:32:45.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:45 smithi042 ceph-mon[28130]: pgmap v808: 97 pgs: 97 active+clean; 180 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 411 op/s 2024-04-25T00:32:46.737 INFO:tasks.workunit.client.0.smithi042.stdout: 10 17558 31.37 MB/sec warmup 36 sec latency 245.100 ms 2024-04-25T00:32:47.737 INFO:tasks.workunit.client.0.smithi042.stdout: 10 18230 31.41 MB/sec warmup 37 sec latency 52.327 ms 2024-04-25T00:32:47.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:47 smithi136 ceph-mon[32923]: pgmap v809: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 17 MiB/s wr, 631 op/s 2024-04-25T00:32:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:47 smithi106 ceph-mon[32510]: pgmap v809: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 17 MiB/s wr, 631 op/s 2024-04-25T00:32:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:47 smithi042 ceph-mon[28130]: pgmap v809: 97 pgs: 97 active+clean; 322 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 17 MiB/s wr, 631 op/s 2024-04-25T00:32:48.737 INFO:tasks.workunit.client.0.smithi042.stdout: 10 19077 31.37 MB/sec warmup 38 sec latency 83.855 ms 2024-04-25T00:32:49.738 INFO:tasks.workunit.client.0.smithi042.stdout: 10 19963 32.20 MB/sec warmup 39 sec latency 58.934 ms 2024-04-25T00:32:49.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:49 smithi136 ceph-mon[32923]: pgmap v810: 97 pgs: 97 active+clean; 362 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 20 MiB/s wr, 497 op/s 2024-04-25T00:32:49.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:49 smithi106 ceph-mon[32510]: pgmap v810: 97 pgs: 97 active+clean; 362 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 20 MiB/s wr, 497 op/s 2024-04-25T00:32:49.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:49 smithi042 ceph-mon[28130]: pgmap v810: 97 pgs: 97 active+clean; 362 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 20 MiB/s wr, 497 op/s 2024-04-25T00:32:50.737 INFO:tasks.workunit.client.0.smithi042.stdout: 10 20561 31.91 MB/sec warmup 40 sec latency 288.476 ms 2024-04-25T00:32:51.737 INFO:tasks.workunit.client.0.smithi042.stdout: 10 21379 32.33 MB/sec warmup 41 sec latency 71.107 ms 2024-04-25T00:32:51.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:51 smithi136 ceph-mon[32923]: pgmap v811: 97 pgs: 97 active+clean; 374 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 21 MiB/s wr, 577 op/s 2024-04-25T00:32:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:51 smithi106 ceph-mon[32510]: pgmap v811: 97 pgs: 97 active+clean; 374 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 21 MiB/s wr, 577 op/s 2024-04-25T00:32:51.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:51 smithi042 ceph-mon[28130]: pgmap v811: 97 pgs: 97 active+clean; 374 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 21 MiB/s wr, 577 op/s 2024-04-25T00:32:52.738 INFO:tasks.workunit.client.0.smithi042.stdout: 10 22234 32.60 MB/sec warmup 42 sec latency 36.110 ms 2024-04-25T00:32:53.737 INFO:tasks.workunit.client.0.smithi042.stdout: 10 23059 32.99 MB/sec warmup 43 sec latency 43.255 ms 2024-04-25T00:32:53.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:53 smithi136 ceph-mon[32923]: pgmap v812: 97 pgs: 97 active+clean; 386 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 23 MiB/s wr, 689 op/s 2024-04-25T00:32:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:53 smithi106 ceph-mon[32510]: pgmap v812: 97 pgs: 97 active+clean; 386 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 23 MiB/s wr, 689 op/s 2024-04-25T00:32:53.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:53 smithi042 ceph-mon[28130]: pgmap v812: 97 pgs: 97 active+clean; 386 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 23 MiB/s wr, 689 op/s 2024-04-25T00:32:54.737 INFO:tasks.workunit.client.0.smithi042.stdout: 10 23750 33.16 MB/sec warmup 44 sec latency 59.434 ms 2024-04-25T00:32:55.738 INFO:tasks.workunit.client.0.smithi042.stdout: 10 24322 32.78 MB/sec warmup 45 sec latency 174.487 ms 2024-04-25T00:32:55.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:55 smithi136 ceph-mon[32923]: pgmap v813: 97 pgs: 97 active+clean; 386 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 22 MiB/s wr, 604 op/s 2024-04-25T00:32:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:55 smithi106 ceph-mon[32510]: pgmap v813: 97 pgs: 97 active+clean; 386 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 22 MiB/s wr, 604 op/s 2024-04-25T00:32:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:55 smithi042 ceph-mon[28130]: pgmap v813: 97 pgs: 97 active+clean; 386 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 22 MiB/s wr, 604 op/s 2024-04-25T00:32:56.738 INFO:tasks.workunit.client.0.smithi042.stdout: 10 25220 33.30 MB/sec warmup 46 sec latency 129.734 ms 2024-04-25T00:32:57.738 INFO:tasks.workunit.client.0.smithi042.stdout: 10 26158 33.74 MB/sec warmup 47 sec latency 35.678 ms 2024-04-25T00:32:57.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:57 smithi136 ceph-mon[32923]: pgmap v814: 97 pgs: 97 active+clean; 405 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 24 MiB/s wr, 765 op/s 2024-04-25T00:32:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:57 smithi106 ceph-mon[32510]: pgmap v814: 97 pgs: 97 active+clean; 405 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 24 MiB/s wr, 765 op/s 2024-04-25T00:32:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:57 smithi042 ceph-mon[28130]: pgmap v814: 97 pgs: 97 active+clean; 405 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 24 MiB/s wr, 765 op/s 2024-04-25T00:32:58.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.434+0000 7f4916800640 1 -- 172.21.15.42:0/3000796089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4910102980 msgr2=0x7f4910104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.434+0000 7f4916800640 1 --2- 172.21.15.42:0/3000796089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4910102980 0x7f4910104df0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f48fc009900 tx=0x7f48fc02a400 comp rx=0 tx=0).stop 2024-04-25T00:32:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.437+0000 7f4916800640 1 -- 172.21.15.42:0/3000796089 shutdown_connections 2024-04-25T00:32:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.437+0000 7f4916800640 1 --2- 172.21.15.42:0/3000796089 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4910105330 0x7f49101057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.437+0000 7f4916800640 1 --2- 172.21.15.42:0/3000796089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4910102980 0x7f4910104df0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.437+0000 7f4916800640 1 --2- 172.21.15.42:0/3000796089 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4910100010 0x7f4910102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.437+0000 7f4916800640 1 -- 172.21.15.42:0/3000796089 >> 172.21.15.42:0/3000796089 conn(0x7f49100fbc10 msgr2=0x7f49100fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.438+0000 7f4916800640 1 -- 172.21.15.42:0/3000796089 shutdown_connections 2024-04-25T00:32:58.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.440+0000 7f4916800640 1 -- 172.21.15.42:0/3000796089 wait complete. 2024-04-25T00:32:58.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.442+0000 7f4916800640 1 Processor -- start 2024-04-25T00:32:58.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.442+0000 7f4916800640 1 -- start start 2024-04-25T00:32:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.443+0000 7f4916800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4910100010 0x7f491006fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.443+0000 7f4916800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4910102980 0x7f4910070410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.443+0000 7f4916800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4910105330 0x7f49101ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.443+0000 7f4916800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4910113a40 con 0x7f4910102980 2024-04-25T00:32:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.443+0000 7f4916800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49101138c0 con 0x7f4910105330 2024-04-25T00:32:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.443+0000 7f4916800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4910113bc0 con 0x7f4910100010 2024-04-25T00:32:58.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.443+0000 7f490f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4910100010 0x7f491006fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.443+0000 7f490f400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4910100010 0x7f491006fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:35038/0 (socket says 172.21.15.42:35038) 2024-04-25T00:32:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.443+0000 7f490f400640 1 -- 172.21.15.42:0/245645041 learned_addr learned my addr 172.21.15.42:0/245645041 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.444+0000 7f490fe00640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4910105330 0x7f49101ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.444+0000 7f490f400640 1 -- 172.21.15.42:0/245645041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4910105330 msgr2=0x7f49101ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.444+0000 7f490f400640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4910105330 0x7f49101ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.444+0000 7f490f400640 1 -- 172.21.15.42:0/245645041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4910102980 msgr2=0x7f4910070410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.444+0000 7f490f400640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4910102980 0x7f4910070410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.444+0000 7f490f400640 1 -- 172.21.15.42:0/245645041 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4910073590 con 0x7f4910100010 2024-04-25T00:32:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.445+0000 7f490f400640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4910100010 0x7f491006fed0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f490000c920 tx=0x7f490000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.445+0000 7f4907e00640 1 -- 172.21.15.42:0/245645041 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900007c70 con 0x7f4910100010 2024-04-25T00:32:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.445+0000 7f4907e00640 1 -- 172.21.15.42:0/245645041 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4900007e10 con 0x7f4910100010 2024-04-25T00:32:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.445+0000 7f4907e00640 1 -- 172.21.15.42:0/245645041 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4900018890 con 0x7f4910100010 2024-04-25T00:32:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.445+0000 7f4916800640 1 -- 172.21.15.42:0/245645041 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49101ae690 con 0x7f4910100010 2024-04-25T00:32:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.446+0000 7f4916800640 1 -- 172.21.15.42:0/245645041 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49101aeaa0 con 0x7f4910100010 2024-04-25T00:32:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.450+0000 7f4919600740 1 Processor -- start 2024-04-25T00:32:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.451+0000 7f4907e00640 1 -- 172.21.15.42:0/245645041 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4900018a70 con 0x7f4910100010 2024-04-25T00:32:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.451+0000 7f4919600740 1 -- start start 2024-04-25T00:32:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.452+0000 7f4919600740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd6b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.452+0000 7f4919600740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 0x55989bd70190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.452+0000 7f4919600740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd70800 0x55989bd72c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.452+0000 7f4919600740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55989bccdab0 con 0x55989bd6aca0 2024-04-25T00:32:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.452+0000 7f4919600740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55989bc76640 con 0x55989bd6b690 2024-04-25T00:32:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.452+0000 7f4919600740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55989bcc9a70 con 0x55989bd70800 2024-04-25T00:32:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.453+0000 7f490ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd70800 0x55989bd72c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.453+0000 7f490ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd70800 0x55989bd72c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:35042/0 (socket says 172.21.15.42:35042) 2024-04-25T00:32:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.453+0000 7f490ea00640 1 -- 172.21.15.42:0/3193159912 learned_addr learned my addr 172.21.15.42:0/3193159912 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.453+0000 7f490fe00640 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd6b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.454+0000 7f490f400640 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 0x55989bd70190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.454+0000 7f490ea00640 1 -- 172.21.15.42:0/3193159912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 msgr2=0x55989bd70190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.454+0000 7f4907e00640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f48f40a2d30 0x7f48f40a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.454+0000 7f4907e00640 1 -- 172.21.15.42:0/245645041 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f49000cf1c0 con 0x7f4910100010 2024-04-25T00:32:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.454+0000 7f490ea00640 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 0x55989bd70190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.454+0000 7f490ea00640 1 -- 172.21.15.42:0/3193159912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 msgr2=0x55989bd6b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.454+0000 7f490ea00640 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd6b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.454+0000 7f490ea00640 1 -- 172.21.15.42:0/3193159912 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989b9e2210 con 0x55989bd70800 2024-04-25T00:32:58.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.454+0000 7f490f400640 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 0x55989bd70190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.454+0000 7f490fe00640 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd6b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.455+0000 7f490fe00640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f48f40a2d30 0x7f48f40a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.455+0000 7f490ea00640 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd70800 0x55989bd72c90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f48fc02a920 tx=0x7f48fc000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfa2eef745648bce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.456+0000 7f4904c00640 1 -- 172.21.15.42:0/3193159912 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc03e030 con 0x55989bd70800 2024-04-25T00:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.456+0000 7f4904c00640 1 -- 172.21.15.42:0/3193159912 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc004360 con 0x55989bd70800 2024-04-25T00:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.456+0000 7f4904c00640 1 -- 172.21.15.42:0/3193159912 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc035040 con 0x55989bd70800 2024-04-25T00:32:58.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.457+0000 7f4919600740 1 -- 172.21.15.42:0/3193159912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd70800 msgr2=0x55989bd72c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.457+0000 7f4919600740 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd70800 0x55989bd72c90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f48fc02a920 tx=0x7f48fc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.457+0000 7f4919600740 1 -- 172.21.15.42:0/3193159912 shutdown_connections 2024-04-25T00:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.457+0000 7f4919600740 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd70800 0x55989bd72c90 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f48fc02a920 tx=0x7f48fc000bc0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.457+0000 7f4919600740 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 0x55989bd70190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.457+0000 7f4919600740 1 --2- 172.21.15.42:0/3193159912 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd6b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.457+0000 7f4919600740 1 -- 172.21.15.42:0/3193159912 >> 172.21.15.42:0/3193159912 conn(0x55989bd66310 msgr2=0x55989bd66710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.458+0000 7f490fe00640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f48f40a2d30 0x7f48f40a51f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4908007310 tx=0x7f4908004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.458+0000 7f4919600740 1 -- 172.21.15.42:0/3193159912 shutdown_connections 2024-04-25T00:32:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.459+0000 7f4919600740 1 -- 172.21.15.42:0/3193159912 wait complete. 2024-04-25T00:32:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.459+0000 7f4919600740 1 -- start start 2024-04-25T00:32:58.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.460+0000 7f4919600740 1 Processor -- start 2024-04-25T00:32:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.461+0000 7f4919600740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd75090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.461+0000 7f4919600740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 0x55989bd755d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.461+0000 7f4919600740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd7a600 0x55989bd7c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.461+0000 7f4919600740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55989bd73750 con 0x55989bd6aca0 2024-04-25T00:32:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.461+0000 7f4919600740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55989bc89990 con 0x55989bd6b690 2024-04-25T00:32:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.461+0000 7f4919600740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55989bd738d0 con 0x55989bd7a600 2024-04-25T00:32:58.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.461+0000 7f4919600740 10 client.0 _ll_register_callbacks cb 0x55989bc8b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.462+0000 7f490fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd7a600 0x55989bd7c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.462+0000 7f490fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd7a600 0x55989bd7c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:35046/0 (socket says 172.21.15.42:35046) 2024-04-25T00:32:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.462+0000 7f490fe00640 1 -- 172.21.15.42:0/2372116147 learned_addr learned my addr 172.21.15.42:0/2372116147 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.462+0000 7f490f400640 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd75090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.463+0000 7f490ea00640 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 0x55989bd755d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.463+0000 7f490fe00640 1 -- 172.21.15.42:0/2372116147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 msgr2=0x55989bd755d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.463+0000 7f490fe00640 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 0x55989bd755d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.463+0000 7f490fe00640 1 -- 172.21.15.42:0/2372116147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 msgr2=0x55989bd75090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.463+0000 7f490fe00640 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd75090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.463+0000 7f490fe00640 1 -- 172.21.15.42:0/2372116147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55989baae620 con 0x55989bd7a600 2024-04-25T00:32:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.464+0000 7f490f400640 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd75090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.464+0000 7f490ea00640 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 0x55989bd755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.464+0000 7f490fe00640 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd7a600 0x55989bd7c9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f49080077f0 tx=0x7f4908004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d84c5aab987b666e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.464+0000 7f48f9600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.465+0000 7f48f9600640 1 -- 172.21.15.42:0/2372116147 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908004d20 con 0x55989bd7a600 2024-04-25T00:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.465+0000 7f48f9600640 1 -- 172.21.15.42:0/2372116147 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f4908005020 con 0x55989bd7a600 2024-04-25T00:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.466+0000 7f4919600740 1 -- 172.21.15.42:0/2372116147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48fc004360 con 0x55989bd7a600 2024-04-25T00:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.467+0000 7f48f9600640 1 -- 172.21.15.42:0/2372116147 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4908005220 con 0x55989bd7a600 2024-04-25T00:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.467+0000 7f4919600740 10 client.26317 fetch_fsmap learned FSMap version 35 2024-04-25T00:32:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.467+0000 7f4919600740 1 -- 172.21.15.42:0/2372116147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55989bd7fa60 con 0x55989bd7a600 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.467+0000 7f48f9600640 1 -- 172.21.15.42:0/2372116147 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc004360 con 0x55989bd7a600 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.467+0000 7f48f9600640 1 -- 172.21.15.42:0/2372116147 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f490801eb50 con 0x55989bd7a600 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.468+0000 7f4919600740 10 client.26317 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.468+0000 7f4919600740 10 client.26317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.468+0000 7f4919600740 20 client.26317 populate_metadata read hostname 'smithi042' 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.468+0000 7f4919600740 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55989bd811f0 0x55989bd83690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.468+0000 7f4919600740 4 client.26317 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.468+0000 7f4919600740 1 -- 172.21.15.42:0/2372116147 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55989bcdaff0 con 0x55989bd811f0 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.468+0000 7f490f400640 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55989bd811f0 0x55989bd83690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.469+0000 7f490f400640 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55989bd811f0 0x55989bd83690 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47151a9958b74abc server_cookie=63b61d43039f6ce4 in_seq=0 out_seq=0 2024-04-25T00:32:58.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.469+0000 7f48f9600640 10 client.26317 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.474+0000 7f48f9600640 1 -- 172.21.15.42:0/2372116147 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55989bcdaff0 con 0x55989bd811f0 2024-04-25T00:32:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.474+0000 7f48f9600640 10 client.26317 handle_command_reply: tid=0 2024-04-25T00:32:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.475+0000 7f4919600740 1 client.26317 shutdown 2024-04-25T00:32:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.475+0000 7f4919600740 1 -- 172.21.15.42:0/2372116147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd7a600 msgr2=0x55989bd7c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.475+0000 7f4919600740 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd7a600 0x55989bd7c9f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f49080077f0 tx=0x7f4908004ab0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.475+0000 7f4919600740 1 -- 172.21.15.42:0/2372116147 shutdown_connections 2024-04-25T00:32:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.475+0000 7f4919600740 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55989bd811f0 0x55989bd83690 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.475+0000 7f4919600740 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd7a600 0x55989bd7c9f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.475+0000 7f4919600740 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd6b690 0x55989bd755d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.475+0000 7f4919600740 1 --2- 172.21.15.42:0/2372116147 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd75090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.475+0000 7f4919600740 1 -- 172.21.15.42:0/2372116147 >> 172.21.15.42:0/2372116147 conn(0x55989bd66310 msgr2=0x55989bd65480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.475+0000 7f48f9600640 0 client.26317 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.476+0000 7f4919600740 1 -- 172.21.15.42:0/2372116147 shutdown_connections 2024-04-25T00:32:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.477+0000 7f4919600740 1 -- 172.21.15.42:0/2372116147 wait complete. 2024-04-25T00:32:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.477+0000 7f4919600740 20 client.26317 trim_cache size 0 max 16384 2024-04-25T00:32:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.514+0000 7f4919600740 1 Processor -- start 2024-04-25T00:32:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.514+0000 7f4919600740 1 -- start start 2024-04-25T00:32:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.514+0000 7f4919600740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6aca0 0x55989bd88fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.514+0000 7f4919600740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6b690 0x55989bd894e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.514+0000 7f4919600740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 0x55989bd6ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.514+0000 7f4919600740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49101ba100 con 0x55989bd6b690 2024-04-25T00:32:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.514+0000 7f4919600740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b7910 con 0x55989bd7a600 2024-04-25T00:32:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.514+0000 7f4919600740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b5760 con 0x55989bd6aca0 2024-04-25T00:32:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.515+0000 7f490fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 0x55989bd6ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.515+0000 7f490ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6b690 0x55989bd894e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.515+0000 7f490ea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6b690 0x55989bd894e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47292/0 (socket says 172.21.15.42:47292) 2024-04-25T00:32:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.515+0000 7f490ea00640 1 -- 172.21.15.42:0/2729794965 learned_addr learned my addr 172.21.15.42:0/2729794965 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.515+0000 7f490f400640 1 --2- 172.21.15.42:0/2729794965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6aca0 0x55989bd88fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.515+0000 7f490ea00640 1 -- 172.21.15.42:0/2729794965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6aca0 msgr2=0x55989bd88fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.515+0000 7f490ea00640 1 --2- 172.21.15.42:0/2729794965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6aca0 0x55989bd88fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.515+0000 7f490ea00640 1 -- 172.21.15.42:0/2729794965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 msgr2=0x55989bd6ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.515+0000 7f490ea00640 1 --2- 172.21.15.42:0/2729794965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 0x55989bd6ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.515+0000 7f490ea00640 1 -- 172.21.15.42:0/2729794965 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55989bd66fe0 con 0x55989bd6b690 2024-04-25T00:32:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.516+0000 7f490ea00640 1 --2- 172.21.15.42:0/2729794965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6b690 0x55989bd894e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f48fc006010 tx=0x7f48fc005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e421d609425235be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.516+0000 7f48fbe00640 1 -- 172.21.15.42:0/2729794965 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc04a070 con 0x55989bd6b690 2024-04-25T00:32:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.516+0000 7f48fbe00640 1 -- 172.21.15.42:0/2729794965 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc031b90 con 0x55989bd6b690 2024-04-25T00:32:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.516+0000 7f48fbe00640 1 -- 172.21.15.42:0/2729794965 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc03e040 con 0x55989bd6b690 2024-04-25T00:32:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.517+0000 7f4919600740 1 -- 172.21.15.42:0/2729794965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6b690 msgr2=0x55989bd894e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.517+0000 7f4919600740 1 --2- 172.21.15.42:0/2729794965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6b690 0x55989bd894e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f48fc006010 tx=0x7f48fc005850 comp rx=0 tx=0).stop 2024-04-25T00:32:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.518+0000 7f4919600740 1 -- 172.21.15.42:0/2729794965 shutdown_connections 2024-04-25T00:32:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.518+0000 7f4919600740 1 --2- 172.21.15.42:0/2729794965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 0x55989bd6ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.518+0000 7f4919600740 1 --2- 172.21.15.42:0/2729794965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6b690 0x55989bd894e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.518+0000 7f4919600740 1 --2- 172.21.15.42:0/2729794965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6aca0 0x55989bd88fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.518+0000 7f4919600740 1 -- 172.21.15.42:0/2729794965 >> 172.21.15.42:0/2729794965 conn(0x55989bd66310 msgr2=0x55989bd60fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.518+0000 7f4919600740 1 -- 172.21.15.42:0/2729794965 shutdown_connections 2024-04-25T00:32:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.518+0000 7f4919600740 1 -- 172.21.15.42:0/2729794965 wait complete. 2024-04-25T00:32:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.519+0000 7f4919600740 1 -- start start 2024-04-25T00:32:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.519+0000 7f4919600740 1 Processor -- start 2024-04-25T00:32:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.520+0000 7f4919600740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd924e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f4919600740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6b690 0x55989bd92a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f4919600740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 0x55989bd93a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f4919600740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b5dd0 con 0x55989bd6aca0 2024-04-25T00:32:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f4919600740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b7240 con 0x55989bd7a600 2024-04-25T00:32:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f4919600740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f49101b64a0 con 0x55989bd6b690 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f4919600740 10 client.0 _ll_register_callbacks cb 0x7f4908007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f490ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6b690 0x55989bd92a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f490ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6b690 0x55989bd92a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:35062/0 (socket says 172.21.15.42:35062) 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f490ea00640 1 -- 172.21.15.42:0/3792385682 learned_addr learned my addr 172.21.15.42:0/3792385682 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f490f400640 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd924e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.521+0000 7f490fe00640 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 0x55989bd93a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f490ea00640 1 -- 172.21.15.42:0/3792385682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 msgr2=0x55989bd93a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f490ea00640 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 0x55989bd93a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f490ea00640 1 -- 172.21.15.42:0/3792385682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 msgr2=0x55989bd924e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f490ea00640 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd924e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f490ea00640 1 -- 172.21.15.42:0/3792385682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55989bd94020 con 0x55989bd6b690 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f490fe00640 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 0x55989bd93a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f490ea00640 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6b690 0x55989bd92a20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f48fc02a9f0 tx=0x7f48fc031d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=570232a8c6907fb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f48f9600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:32:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f48f9600640 1 -- 172.21.15.42:0/3792385682 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc052020 con 0x55989bd6b690 2024-04-25T00:32:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f48f9600640 1 -- 172.21.15.42:0/3792385682 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f48fc003bd0 con 0x55989bd6b690 2024-04-25T00:32:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.522+0000 7f4919600740 1 -- 172.21.15.42:0/3792385682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48fc031b90 con 0x55989bd6b690 2024-04-25T00:32:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.523+0000 7f48f9600640 1 -- 172.21.15.42:0/3792385682 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f48fc03e030 con 0x55989bd6b690 2024-04-25T00:32:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.523+0000 7f4919600740 10 client.26323 fetch_fsmap learned FSMap version 35 2024-04-25T00:32:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.523+0000 7f4919600740 1 -- 172.21.15.42:0/3792385682 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55989bde75c0 con 0x55989bd6b690 2024-04-25T00:32:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.523+0000 7f48f9600640 1 -- 172.21.15.42:0/3792385682 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f48fc031b90 con 0x55989bd6b690 2024-04-25T00:32:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.523+0000 7f48f9600640 1 -- 172.21.15.42:0/3792385682 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f48fc04eb20 con 0x55989bd6b690 2024-04-25T00:32:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.524+0000 7f4919600740 10 client.26323 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T00:32:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.524+0000 7f4919600740 10 client.26323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:32:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.524+0000 7f4919600740 20 client.26323 populate_metadata read hostname 'smithi042' 2024-04-25T00:32:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.524+0000 7f4919600740 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55989bde7a70 0x55989bde9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:32:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.524+0000 7f4919600740 4 client.26323 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:32:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.524+0000 7f4919600740 1 -- 172.21.15.42:0/3792385682 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55989bcd5340 con 0x55989bde7a70 2024-04-25T00:32:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.524+0000 7f490f400640 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55989bde7a70 0x55989bde9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:32:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.524+0000 7f490f400640 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55989bde7a70 0x55989bde9e60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64573363208232c9 server_cookie=16d486025a025d4b in_seq=0 out_seq=0 2024-04-25T00:32:58.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.524+0000 7f48f9600640 10 client.26323 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f48f9600640 1 -- 172.21.15.42:0/3792385682 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55989bcd5340 con 0x55989bde7a70 2024-04-25T00:32:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f48f9600640 10 client.26323 handle_command_reply: tid=0 2024-04-25T00:32:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f4919600740 1 client.26323 shutdown 2024-04-25T00:32:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f4919600740 1 -- 172.21.15.42:0/3792385682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6b690 msgr2=0x55989bd92a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f4919600740 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6b690 0x55989bd92a20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f48fc02a9f0 tx=0x7f48fc031d40 comp rx=0 tx=0).stop 2024-04-25T00:32:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f4919600740 1 -- 172.21.15.42:0/3792385682 shutdown_connections 2024-04-25T00:32:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f4919600740 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55989bde7a70 0x55989bde9e60 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.737 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f4919600740 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55989bd7a600 0x55989bd93a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f4919600740 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55989bd6b690 0x55989bd92a20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f4919600740 1 --2- 172.21.15.42:0/3792385682 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55989bd6aca0 0x55989bd924e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f4919600740 1 -- 172.21.15.42:0/3792385682 >> 172.21.15.42:0/3792385682 conn(0x55989bd66310 msgr2=0x55989bd6ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.736+0000 7f48f9600640 0 client.26323 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:32:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.737+0000 7f4919600740 1 -- 172.21.15.42:0/3792385682 shutdown_connections 2024-04-25T00:32:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.737+0000 7f4919600740 1 -- 172.21.15.42:0/3792385682 wait complete. 2024-04-25T00:32:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.737+0000 7f4919600740 20 client.26323 trim_cache size 0 max 16384 2024-04-25T00:32:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:32:58.738 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:32:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:32:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:32:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 -- 172.21.15.42:0/245645041 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f48f40a2d30 msgr2=0x7f48f40a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f48f40a2d30 0x7f48f40a51f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4908007310 tx=0x7f4908004a40 comp rx=0 tx=0).stop 2024-04-25T00:32:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 -- 172.21.15.42:0/245645041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4910100010 msgr2=0x7f491006fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:32:58.739 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4910100010 0x7f491006fed0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f490000c920 tx=0x7f490000cde0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 -- 172.21.15.42:0/245645041 shutdown_connections 2024-04-25T00:32:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f48f40a2d30 0x7f48f40a51f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4910105330 0x7f49101ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4910102980 0x7f4910070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 --2- 172.21.15.42:0/245645041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4910100010 0x7f491006fed0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:32:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 -- 172.21.15.42:0/245645041 >> 172.21.15.42:0/245645041 conn(0x7f49100fbc10 msgr2=0x7f4910100910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:32:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 -- 172.21.15.42:0/245645041 shutdown_connections 2024-04-25T00:32:58.740 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:32:58.738+0000 7f4916800640 1 -- 172.21.15.42:0/245645041 wait complete. 2024-04-25T00:32:58.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 26699 33.68 MB/sec warmup 48 sec latency 138.445 ms 2024-04-25T00:32:59.738 INFO:tasks.workunit.client.0.smithi042.stdout: 10 27353 33.60 MB/sec warmup 49 sec latency 62.231 ms 2024-04-25T00:32:59.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:32:59 smithi136 ceph-mon[32923]: pgmap v815: 97 pgs: 97 active+clean; 409 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 579 op/s 2024-04-25T00:32:59.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:32:59 smithi106 ceph-mon[32510]: pgmap v815: 97 pgs: 97 active+clean; 409 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 579 op/s 2024-04-25T00:32:59.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:32:59 smithi042 ceph-mon[28130]: pgmap v815: 97 pgs: 97 active+clean; 409 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 11 MiB/s wr, 579 op/s 2024-04-25T00:33:00.738 INFO:tasks.workunit.client.0.smithi042.stdout: 10 28263 33.72 MB/sec warmup 50 sec latency 117.414 ms 2024-04-25T00:33:01.739 INFO:tasks.workunit.client.0.smithi042.stdout: 10 28807 33.72 MB/sec warmup 51 sec latency 365.423 ms 2024-04-25T00:33:01.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:01 smithi136 ceph-mon[32923]: pgmap v816: 97 pgs: 97 active+clean; 410 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.7 MiB/s wr, 632 op/s 2024-04-25T00:33:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:01 smithi106 ceph-mon[32510]: pgmap v816: 97 pgs: 97 active+clean; 410 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.7 MiB/s wr, 632 op/s 2024-04-25T00:33:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:01 smithi042 ceph-mon[28130]: pgmap v816: 97 pgs: 97 active+clean; 410 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.7 MiB/s wr, 632 op/s 2024-04-25T00:33:02.739 INFO:tasks.workunit.client.0.smithi042.stdout: 10 29768 34.29 MB/sec warmup 52 sec latency 43.297 ms 2024-04-25T00:33:03.739 INFO:tasks.workunit.client.0.smithi042.stdout: 10 30350 34.26 MB/sec warmup 53 sec latency 48.085 ms 2024-04-25T00:33:03.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:03 smithi136 ceph-mon[32923]: pgmap v817: 97 pgs: 97 active+clean; 398 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.9 MiB/s wr, 750 op/s 2024-04-25T00:33:03.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:03 smithi106 ceph-mon[32510]: pgmap v817: 97 pgs: 97 active+clean; 398 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.9 MiB/s wr, 750 op/s 2024-04-25T00:33:03.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:03 smithi042 ceph-mon[28130]: pgmap v817: 97 pgs: 97 active+clean; 398 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.9 MiB/s wr, 750 op/s 2024-04-25T00:33:04.739 INFO:tasks.workunit.client.0.smithi042.stdout: 10 31010 34.14 MB/sec warmup 54 sec latency 52.531 ms 2024-04-25T00:33:05.739 INFO:tasks.workunit.client.0.smithi042.stdout: 10 31859 34.15 MB/sec warmup 55 sec latency 122.747 ms 2024-04-25T00:33:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:05 smithi136 ceph-mon[32923]: pgmap v818: 97 pgs: 97 active+clean; 400 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 635 op/s 2024-04-25T00:33:05.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:05 smithi106 ceph-mon[32510]: pgmap v818: 97 pgs: 97 active+clean; 400 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 635 op/s 2024-04-25T00:33:05.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:05 smithi042 ceph-mon[28130]: pgmap v818: 97 pgs: 97 active+clean; 400 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.5 MiB/s wr, 635 op/s 2024-04-25T00:33:06.739 INFO:tasks.workunit.client.0.smithi042.stdout: 10 32761 34.60 MB/sec warmup 56 sec latency 131.692 ms 2024-04-25T00:33:07.739 INFO:tasks.workunit.client.0.smithi042.stdout: 10 33545 34.93 MB/sec warmup 57 sec latency 33.888 ms 2024-04-25T00:33:07.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:07 smithi136 ceph-mon[32923]: pgmap v819: 97 pgs: 97 active+clean; 413 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 745 op/s 2024-04-25T00:33:07.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:07 smithi106 ceph-mon[32510]: pgmap v819: 97 pgs: 97 active+clean; 413 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 745 op/s 2024-04-25T00:33:07.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:07 smithi042 ceph-mon[28130]: pgmap v819: 97 pgs: 97 active+clean; 413 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.1 MiB/s wr, 745 op/s 2024-04-25T00:33:08.739 INFO:tasks.workunit.client.0.smithi042.stdout: 10 34060 34.69 MB/sec warmup 58 sec latency 44.682 ms 2024-04-25T00:33:09.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 34850 34.63 MB/sec warmup 59 sec latency 55.652 ms 2024-04-25T00:33:09.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:09 smithi136 ceph-mon[32923]: pgmap v820: 97 pgs: 97 active+clean; 422 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.8 MiB/s wr, 658 op/s 2024-04-25T00:33:09.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:09 smithi106 ceph-mon[32510]: pgmap v820: 97 pgs: 97 active+clean; 422 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.8 MiB/s wr, 658 op/s 2024-04-25T00:33:09.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:09 smithi042 ceph-mon[28130]: pgmap v820: 97 pgs: 97 active+clean; 422 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 6.8 MiB/s wr, 658 op/s 2024-04-25T00:33:10.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 35784 34.97 MB/sec warmup 60 sec latency 73.411 ms 2024-04-25T00:33:11.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 36529 35.11 MB/sec warmup 61 sec latency 177.396 ms 2024-04-25T00:33:11.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:11 smithi136 ceph-mon[32923]: pgmap v821: 97 pgs: 97 active+clean; 428 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.6 MiB/s wr, 700 op/s 2024-04-25T00:33:11.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:11 smithi106 ceph-mon[32510]: pgmap v821: 97 pgs: 97 active+clean; 428 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.6 MiB/s wr, 700 op/s 2024-04-25T00:33:11.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:11 smithi042 ceph-mon[28130]: pgmap v821: 97 pgs: 97 active+clean; 428 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 7.6 MiB/s wr, 700 op/s 2024-04-25T00:33:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.924+0000 7f80a0200640 1 -- 172.21.15.42:0/1925090128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80980ff460 msgr2=0x7f80980ff8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.924+0000 7f80a0200640 1 --2- 172.21.15.42:0/1925090128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80980ff460 0x7f80980ff8e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f8084009900 tx=0x7f808402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.924+0000 7f80a0200640 1 -- 172.21.15.42:0/1925090128 shutdown_connections 2024-04-25T00:33:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.924+0000 7f80a0200640 1 --2- 172.21.15.42:0/1925090128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80980ffe20 0x7f8098101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.924+0000 7f80a0200640 1 --2- 172.21.15.42:0/1925090128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80980ff460 0x7f80980ff8e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.924+0000 7f80a0200640 1 --2- 172.21.15.42:0/1925090128 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8098100990 0x7f8098100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.924+0000 7f80a0200640 1 -- 172.21.15.42:0/1925090128 >> 172.21.15.42:0/1925090128 conn(0x7f80980faf60 msgr2=0x7f80980fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.924+0000 7f80a0200640 1 -- 172.21.15.42:0/1925090128 shutdown_connections 2024-04-25T00:33:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.924+0000 7f80a0200640 1 -- 172.21.15.42:0/1925090128 wait complete. 2024-04-25T00:33:11.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.925+0000 7f80a0200640 1 Processor -- start 2024-04-25T00:33:11.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.926+0000 7f80a0200640 1 -- start start 2024-04-25T00:33:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.926+0000 7f80a0200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80980ff460 0x7f809819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.926+0000 7f80a0200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80980ffe20 0x7f809819e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.926+0000 7f80a0200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8098100990 0x7f80981a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.926+0000 7f80a0200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f809810f650 con 0x7f8098100990 2024-04-25T00:33:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.926+0000 7f80a0200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f809810f4d0 con 0x7f80980ffe20 2024-04-25T00:33:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.926+0000 7f80a0200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f809810f7d0 con 0x7f80980ff460 2024-04-25T00:33:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80980ff460 0x7f809819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80980ff460 0x7f809819dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:53136/0 (socket says 172.21.15.42:53136) 2024-04-25T00:33:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809d800640 1 -- 172.21.15.42:0/1729278207 learned_addr learned my addr 172.21.15.42:0/1729278207 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:11.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809ce00640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80980ffe20 0x7f809819e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809d800640 1 -- 172.21.15.42:0/1729278207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80980ffe20 msgr2=0x7f809819e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809d800640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80980ffe20 0x7f809819e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809d800640 1 -- 172.21.15.42:0/1729278207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8098100990 msgr2=0x7f80981a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809d800640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8098100990 0x7f80981a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809d800640 1 -- 172.21.15.42:0/1729278207 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80981a5c00 con 0x7f80980ff460 2024-04-25T00:33:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809ce00640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80980ffe20 0x7f809819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.927+0000 7f809d800640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80980ff460 0x7f809819dfe0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f808c00e9f0 tx=0x7f808c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.928+0000 7f8096000640 1 -- 172.21.15.42:0/1729278207 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c00cd80 con 0x7f80980ff460 2024-04-25T00:33:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.928+0000 7f8096000640 1 -- 172.21.15.42:0/1729278207 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f808c010910 con 0x7f80980ff460 2024-04-25T00:33:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.928+0000 7f80a0200640 1 -- 172.21.15.42:0/1729278207 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80981a5ef0 con 0x7f80980ff460 2024-04-25T00:33:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.928+0000 7f8096000640 1 -- 172.21.15.42:0/1729278207 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808c018990 con 0x7f80980ff460 2024-04-25T00:33:11.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.928+0000 7f80a0200640 1 -- 172.21.15.42:0/1729278207 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80981a63b0 con 0x7f80980ff460 2024-04-25T00:33:11.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.931+0000 7f80a319d740 1 Processor -- start 2024-04-25T00:33:11.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.934+0000 7f8096000640 1 -- 172.21.15.42:0/1729278207 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f808c010ac0 con 0x7f80980ff460 2024-04-25T00:33:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.934+0000 7f80a319d740 1 -- start start 2024-04-25T00:33:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.934+0000 7f80a319d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da72a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.934+0000 7f80a319d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72a690 0x55f1da72f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.934+0000 7f80a319d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72f800 0x55f1da731c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.934+0000 7f80a319d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f1da68cab0 con 0x55f1da72a690 2024-04-25T00:33:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.934+0000 7f80a319d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f1da635640 con 0x55f1da729ca0 2024-04-25T00:33:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.934+0000 7f80a319d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f1da688a70 con 0x55f1da72f800 2024-04-25T00:33:11.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.935+0000 7f809e200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da72a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.935+0000 7f809e200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da72a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35698/0 (socket says 172.21.15.42:35698) 2024-04-25T00:33:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.935+0000 7f809e200640 1 -- 172.21.15.42:0/1725291690 learned_addr learned my addr 172.21.15.42:0/1725291690 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.935+0000 7f809d800640 1 --2- 172.21.15.42:0/1725291690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72a690 0x55f1da72f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.936+0000 7f809e200640 1 -- 172.21.15.42:0/1725291690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72f800 msgr2=0x55f1da731c90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:33:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.936+0000 7f809e200640 1 --2- 172.21.15.42:0/1725291690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72f800 0x55f1da731c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.936+0000 7f809e200640 1 -- 172.21.15.42:0/1725291690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72a690 msgr2=0x55f1da72f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.936+0000 7f809e200640 1 --2- 172.21.15.42:0/1725291690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72a690 0x55f1da72f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.936+0000 7f809e200640 1 -- 172.21.15.42:0/1725291690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1da3a1210 con 0x55f1da729ca0 2024-04-25T00:33:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.936+0000 7f809d800640 1 --2- 172.21.15.42:0/1725291690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72a690 0x55f1da72f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.936+0000 7f809e200640 1 --2- 172.21.15.42:0/1725291690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da72a0c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8090009ea0 tx=0x7f8090009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daf0b87777fdea1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.936+0000 7f808aa00640 1 -- 172.21.15.42:0/1725291690 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8090004d20 con 0x55f1da729ca0 2024-04-25T00:33:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.936+0000 7f808aa00640 1 -- 172.21.15.42:0/1725291690 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f809000ee60 con 0x55f1da729ca0 2024-04-25T00:33:11.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.936+0000 7f808aa00640 1 -- 172.21.15.42:0/1725291690 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80900108f0 con 0x55f1da729ca0 2024-04-25T00:33:11.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.937+0000 7f80a319d740 1 -- 172.21.15.42:0/1725291690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 msgr2=0x55f1da72a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.938+0000 7f80a319d740 1 --2- 172.21.15.42:0/1725291690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da72a0c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8090009ea0 tx=0x7f8090009f90 comp rx=0 tx=0).stop 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.938+0000 7f80a319d740 1 -- 172.21.15.42:0/1725291690 shutdown_connections 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.938+0000 7f80a319d740 1 --2- 172.21.15.42:0/1725291690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72f800 0x55f1da731c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.938+0000 7f80a319d740 1 --2- 172.21.15.42:0/1725291690 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72a690 0x55f1da72f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.938+0000 7f80a319d740 1 --2- 172.21.15.42:0/1725291690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da72a0c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.938+0000 7f80a319d740 1 -- 172.21.15.42:0/1725291690 >> 172.21.15.42:0/1725291690 conn(0x55f1da725310 msgr2=0x55f1da725710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.938+0000 7f80a319d740 1 -- 172.21.15.42:0/1725291690 shutdown_connections 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.938+0000 7f80a319d740 1 -- 172.21.15.42:0/1725291690 wait complete. 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.938+0000 7f8096000640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80800a2bc0 0x7f80800a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.938+0000 7f8096000640 1 -- 172.21.15.42:0/1729278207 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f808c0cf420 con 0x7f80980ff460 2024-04-25T00:33:11.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.939+0000 7f80a319d740 1 -- start start 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.939+0000 7f80a319d740 1 Processor -- start 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.939+0000 7f809ce00640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80800a2bc0 0x7f80800a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f80a319d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da744f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f80a319d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da745470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f80a319d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 0x55f1da74c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f80a319d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f1da7437c0 con 0x55f1da72f800 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f80a319d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f1da648990 con 0x55f1da729ca0 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f80a319d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f1da743940 con 0x55f1da72a690 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f80a319d740 10 client.0 _ll_register_callbacks cb 0x55f1da64a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f809ce00640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80800a2bc0 0x7f80800a5080 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f808402a910 tx=0x7f8084005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f809ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da745470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f809ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da745470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:53156/0 (socket says 172.21.15.42:53156) 2024-04-25T00:33:11.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f809ce00640 1 -- 172.21.15.42:0/1651911968 learned_addr learned my addr 172.21.15.42:0/1651911968 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.940+0000 7f809d800640 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da744f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.941+0000 7f809e200640 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 0x55f1da74c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.941+0000 7f809ce00640 1 -- 172.21.15.42:0/1651911968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 msgr2=0x55f1da744f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.941+0000 7f809ce00640 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da744f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.941+0000 7f809ce00640 1 -- 172.21.15.42:0/1651911968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 msgr2=0x55f1da74c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:11.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.941+0000 7f809ce00640 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 0x55f1da74c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.941+0000 7f809ce00640 1 -- 172.21.15.42:0/1651911968 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1da46d620 con 0x55f1da72a690 2024-04-25T00:33:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.941+0000 7f809d800640 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da744f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.941+0000 7f809e200640 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 0x55f1da74c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.941+0000 7f809ce00640 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da745470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8084033000 tx=0x7f8084031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12b5043b097dbee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.941+0000 7f807f400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:33:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.942+0000 7f807f400640 1 -- 172.21.15.42:0/1651911968 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808404c020 con 0x55f1da72a690 2024-04-25T00:33:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.942+0000 7f807f400640 1 -- 172.21.15.42:0/1651911968 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f80840328c0 con 0x55f1da72a690 2024-04-25T00:33:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.942+0000 7f80a319d740 1 -- 172.21.15.42:0/1651911968 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f809000ee60 con 0x55f1da72a690 2024-04-25T00:33:11.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.942+0000 7f807f400640 1 -- 172.21.15.42:0/1651911968 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8084035050 con 0x55f1da72a690 2024-04-25T00:33:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.942+0000 7f80a319d740 10 client.26332 fetch_fsmap learned FSMap version 35 2024-04-25T00:33:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.942+0000 7f80a319d740 1 -- 172.21.15.42:0/1651911968 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f1da74f400 con 0x55f1da72a690 2024-04-25T00:33:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.943+0000 7f807f400640 1 -- 172.21.15.42:0/1651911968 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f809000ee60 con 0x55f1da72a690 2024-04-25T00:33:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.943+0000 7f807f400640 1 -- 172.21.15.42:0/1651911968 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f8084059020 con 0x55f1da72a690 2024-04-25T00:33:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.943+0000 7f80a319d740 10 client.26332 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T00:33:11.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.943+0000 7f80a319d740 10 client.26332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:33:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.943+0000 7f80a319d740 20 client.26332 populate_metadata read hostname 'smithi042' 2024-04-25T00:33:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.943+0000 7f80a319d740 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f1da750b40 0x55f1da752fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.943+0000 7f80a319d740 4 client.26332 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:33:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.943+0000 7f80a319d740 1 -- 172.21.15.42:0/1651911968 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1da699ff0 con 0x55f1da750b40 2024-04-25T00:33:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.943+0000 7f809d800640 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f1da750b40 0x55f1da752fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.944+0000 7f807f400640 10 client.26332 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:11.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.944+0000 7f809d800640 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f1da750b40 0x55f1da752fc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d775ae8e805ace30 server_cookie=a194c3b4552860f6 in_seq=0 out_seq=0 2024-04-25T00:33:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f807f400640 1 -- 172.21.15.42:0/1651911968 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f1da699ff0 con 0x55f1da750b40 2024-04-25T00:33:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f807f400640 10 client.26332 handle_command_reply: tid=0 2024-04-25T00:33:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 client.26332 shutdown 2024-04-25T00:33:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 -- 172.21.15.42:0/1651911968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 msgr2=0x55f1da745470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da745470 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8084033000 tx=0x7f8084031400 comp rx=0 tx=0).stop 2024-04-25T00:33:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 -- 172.21.15.42:0/1651911968 shutdown_connections 2024-04-25T00:33:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f1da750b40 0x55f1da752fc0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 0x55f1da74c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f807f400640 0 client.26332 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da745470 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 --2- 172.21.15.42:0/1651911968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da744f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 -- 172.21.15.42:0/1651911968 >> 172.21.15.42:0/1651911968 conn(0x55f1da725310 msgr2=0x55f1da724480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 -- 172.21.15.42:0/1651911968 shutdown_connections 2024-04-25T00:33:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.945+0000 7f80a319d740 1 -- 172.21.15.42:0/1651911968 wait complete. 2024-04-25T00:33:11.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.946+0000 7f80a319d740 20 client.26332 trim_cache size 0 max 16384 2024-04-25T00:33:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f80a319d740 1 Processor -- start 2024-04-25T00:33:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f80a319d740 1 -- start start 2024-04-25T00:33:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f80a319d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da729ca0 0x55f1da71ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f80a319d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da72a690 0x55f1da74c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.968 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f80a319d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72f800 0x55f1da727ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f80a319d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80981a9420 con 0x55f1da729ca0 2024-04-25T00:33:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f80a319d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80981a6c30 con 0x55f1da72a690 2024-04-25T00:33:11.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f80a319d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8098071d20 con 0x55f1da72f800 2024-04-25T00:33:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f809ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da72a690 0x55f1da74c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f809e200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72f800 0x55f1da727ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f809ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da72a690 0x55f1da74c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35708/0 (socket says 172.21.15.42:35708) 2024-04-25T00:33:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f809ce00640 1 -- 172.21.15.42:0/659837496 learned_addr learned my addr 172.21.15.42:0/659837496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.968+0000 7f809d800640 1 --2- 172.21.15.42:0/659837496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da729ca0 0x55f1da71ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.969+0000 7f809ce00640 1 -- 172.21.15.42:0/659837496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72f800 msgr2=0x55f1da727ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.969+0000 7f809ce00640 1 --2- 172.21.15.42:0/659837496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72f800 0x55f1da727ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.969+0000 7f809ce00640 1 -- 172.21.15.42:0/659837496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da729ca0 msgr2=0x55f1da71ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:11.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.969+0000 7f809ce00640 1 --2- 172.21.15.42:0/659837496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da729ca0 0x55f1da71ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.969+0000 7f809ce00640 1 -- 172.21.15.42:0/659837496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1da7298e0 con 0x55f1da72a690 2024-04-25T00:33:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.969+0000 7f809d800640 1 --2- 172.21.15.42:0/659837496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da729ca0 0x55f1da71ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.969+0000 7f809e200640 1 --2- 172.21.15.42:0/659837496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72f800 0x55f1da727ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.969+0000 7f809ce00640 1 --2- 172.21.15.42:0/659837496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da72a690 0x55f1da74c400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8084003830 tx=0x7f8084005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=628392dbce30733e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.969+0000 7f808a000640 1 -- 172.21.15.42:0/659837496 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808404c030 con 0x55f1da72a690 2024-04-25T00:33:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.970+0000 7f808a000640 1 -- 172.21.15.42:0/659837496 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8084041030 con 0x55f1da72a690 2024-04-25T00:33:11.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.970+0000 7f808a000640 1 -- 172.21.15.42:0/659837496 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084004920 con 0x55f1da72a690 2024-04-25T00:33:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.970+0000 7f80a319d740 1 -- 172.21.15.42:0/659837496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da72a690 msgr2=0x55f1da74c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.970+0000 7f80a319d740 1 --2- 172.21.15.42:0/659837496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da72a690 0x55f1da74c400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8084003830 tx=0x7f8084005770 comp rx=0 tx=0).stop 2024-04-25T00:33:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.970+0000 7f80a319d740 1 -- 172.21.15.42:0/659837496 shutdown_connections 2024-04-25T00:33:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.970+0000 7f80a319d740 1 --2- 172.21.15.42:0/659837496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72f800 0x55f1da727ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.970+0000 7f80a319d740 1 --2- 172.21.15.42:0/659837496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da72a690 0x55f1da74c400 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.970+0000 7f80a319d740 1 --2- 172.21.15.42:0/659837496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da729ca0 0x55f1da71ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.970+0000 7f80a319d740 1 -- 172.21.15.42:0/659837496 >> 172.21.15.42:0/659837496 conn(0x55f1da725310 msgr2=0x55f1da7256f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.971+0000 7f80a319d740 1 -- 172.21.15.42:0/659837496 shutdown_connections 2024-04-25T00:33:11.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.971+0000 7f80a319d740 1 -- 172.21.15.42:0/659837496 wait complete. 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.971+0000 7f80a319d740 1 -- start start 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.971+0000 7f80a319d740 1 Processor -- start 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.972+0000 7f80a319d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da749d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.972+0000 7f80a319d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da74a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.972+0000 7f80a319d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 0x55f1da74b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.972+0000 7f80a319d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8098072390 con 0x55f1da72f800 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.972+0000 7f80a319d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80981a66d0 con 0x55f1da729ca0 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.972+0000 7f80a319d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8098072a60 con 0x55f1da72a690 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.972+0000 7f80a319d740 10 client.0 _ll_register_callbacks cb 0x7f808c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.972+0000 7f809ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da74a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.972+0000 7f809ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da74a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:53174/0 (socket says 172.21.15.42:53174) 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.972+0000 7f809ce00640 1 -- 172.21.15.42:0/193226554 learned_addr learned my addr 172.21.15.42:0/193226554 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:11.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.973+0000 7f809e200640 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 0x55f1da74b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.973+0000 7f809ce00640 1 -- 172.21.15.42:0/193226554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 msgr2=0x55f1da749d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.973+0000 7f809ce00640 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da749d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.973+0000 7f809ce00640 1 -- 172.21.15.42:0/193226554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 msgr2=0x55f1da74b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.973+0000 7f809ce00640 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 0x55f1da74b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:11.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.973+0000 7f809ce00640 1 -- 172.21.15.42:0/193226554 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1da744010 con 0x55f1da72a690 2024-04-25T00:33:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.974+0000 7f809ce00640 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da74a2d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f80840036c0 tx=0x7f8084032c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d62c34008766b5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.974+0000 7f807f400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:33:11.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.974+0000 7f807f400640 1 -- 172.21.15.42:0/193226554 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808404c040 con 0x55f1da72a690 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.974+0000 7f807f400640 1 -- 172.21.15.42:0/193226554 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f8084003cf0 con 0x55f1da72a690 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.974+0000 7f80a319d740 1 -- 172.21.15.42:0/193226554 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8084041030 con 0x55f1da72a690 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f807f400640 1 -- 172.21.15.42:0/193226554 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f80840046c0 con 0x55f1da72a690 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f80a319d740 10 client.26338 fetch_fsmap learned FSMap version 35 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f80a319d740 1 -- 172.21.15.42:0/193226554 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55f1da725140 con 0x55f1da72a690 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f807f400640 1 -- 172.21.15.42:0/193226554 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8084041030 con 0x55f1da72a690 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f807f400640 1 -- 172.21.15.42:0/193226554 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7f8084059070 con 0x55f1da72a690 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f80a319d740 10 client.26338 fetch_fsmap finished waiting for FSMap version 35 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f80a319d740 10 client.26338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f80a319d740 20 client.26338 populate_metadata read hostname 'smithi042' 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f80a319d740 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f1da7b7760 0x55f1da7b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f80a319d740 4 client.26338 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.975+0000 7f80a319d740 1 -- 172.21.15.42:0/193226554 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1da694340 con 0x55f1da7b7760 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.976+0000 7f809d800640 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f1da7b7760 0x55f1da7b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:11.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.976+0000 7f807f400640 10 client.26338 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:11.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:11.976+0000 7f809d800640 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f1da7b7760 0x55f1da7b9b80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=324a0528ea0a2fbc server_cookie=d38f6a3f8ad869f in_seq=0 out_seq=0 2024-04-25T00:33:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.244+0000 7f807f400640 1 -- 172.21.15.42:0/193226554 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f1da694340 con 0x55f1da7b7760 2024-04-25T00:33:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.244+0000 7f807f400640 10 client.26338 handle_command_reply: tid=0 2024-04-25T00:33:12.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.244+0000 7f80a319d740 1 client.26338 shutdown 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 1 -- 172.21.15.42:0/193226554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 msgr2=0x55f1da74a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da74a2d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f80840036c0 tx=0x7f8084032c70 comp rx=0 tx=0).stop 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 1 -- 172.21.15.42:0/193226554 shutdown_connections 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f1da7b7760 0x55f1da7b9b80 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f1da72f800 0x55f1da74b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f807f400640 0 client.26338 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f1da72a690 0x55f1da74a2d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 1 --2- 172.21.15.42:0/193226554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f1da729ca0 0x55f1da749d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 1 -- 172.21.15.42:0/193226554 >> 172.21.15.42:0/193226554 conn(0x55f1da725310 msgr2=0x55f1da71ffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 1 -- 172.21.15.42:0/193226554 shutdown_connections 2024-04-25T00:33:12.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 1 -- 172.21.15.42:0/193226554 wait complete. 2024-04-25T00:33:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.245+0000 7f80a319d740 20 client.26338 trim_cache size 0 max 16384 2024-04-25T00:33:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:33:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:33:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:33:12.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:33:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.246+0000 7f80a0200640 1 -- 172.21.15.42:0/1729278207 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80800a2bc0 msgr2=0x7f80800a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.246+0000 7f80a0200640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80800a2bc0 0x7f80800a5080 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f808402a910 tx=0x7f8084005fd0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.247+0000 7f80a0200640 1 -- 172.21.15.42:0/1729278207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80980ff460 msgr2=0x7f809819dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.247+0000 7f80a0200640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80980ff460 0x7f809819dfe0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f808c00e9f0 tx=0x7f808c00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.247+0000 7f80a0200640 1 -- 172.21.15.42:0/1729278207 shutdown_connections 2024-04-25T00:33:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.247+0000 7f80a0200640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f80800a2bc0 0x7f80800a5080 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.247+0000 7f80a0200640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8098100990 0x7f80981a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.247+0000 7f80a0200640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80980ffe20 0x7f809819e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.247+0000 7f80a0200640 1 --2- 172.21.15.42:0/1729278207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80980ff460 0x7f809819dfe0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:12.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.247+0000 7f80a0200640 1 -- 172.21.15.42:0/1729278207 >> 172.21.15.42:0/1729278207 conn(0x7f80980faf60 msgr2=0x7f80980fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.247+0000 7f80a0200640 1 -- 172.21.15.42:0/1729278207 shutdown_connections 2024-04-25T00:33:12.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:12.247+0000 7f80a0200640 1 -- 172.21.15.42:0/1729278207 wait complete. 2024-04-25T00:33:12.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 37204 35.24 MB/sec warmup 62 sec latency 134.311 ms 2024-04-25T00:33:13.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 37864 35.15 MB/sec warmup 63 sec latency 91.506 ms 2024-04-25T00:33:13.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:13 smithi136 ceph-mon[32923]: pgmap v822: 97 pgs: 97 active+clean; 435 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 731 op/s 2024-04-25T00:33:13.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:13 smithi106 ceph-mon[32510]: pgmap v822: 97 pgs: 97 active+clean; 435 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 731 op/s 2024-04-25T00:33:13.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:13 smithi042 ceph-mon[28130]: pgmap v822: 97 pgs: 97 active+clean; 435 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.8 MiB/s wr, 731 op/s 2024-04-25T00:33:14.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 38326 34.98 MB/sec warmup 64 sec latency 142.252 ms 2024-04-25T00:33:15.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 39121 35.06 MB/sec warmup 65 sec latency 109.459 ms 2024-04-25T00:33:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:15 smithi106 ceph-mon[32510]: pgmap v823: 97 pgs: 97 active+clean; 435 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.4 MiB/s wr, 532 op/s 2024-04-25T00:33:15.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:15 smithi042 ceph-mon[28130]: pgmap v823: 97 pgs: 97 active+clean; 435 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.4 MiB/s wr, 532 op/s 2024-04-25T00:33:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:15 smithi136 ceph-mon[32923]: pgmap v823: 97 pgs: 97 active+clean; 435 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 7.4 MiB/s wr, 532 op/s 2024-04-25T00:33:16.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 39831 35.02 MB/sec warmup 66 sec latency 188.934 ms 2024-04-25T00:33:17.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 40735 35.46 MB/sec warmup 67 sec latency 34.238 ms 2024-04-25T00:33:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:17 smithi106 ceph-mon[32510]: pgmap v824: 97 pgs: 97 active+clean; 438 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-04-25T00:33:17.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:17 smithi042 ceph-mon[28130]: pgmap v824: 97 pgs: 97 active+clean; 438 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-04-25T00:33:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:17 smithi136 ceph-mon[32923]: pgmap v824: 97 pgs: 97 active+clean; 438 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 819 op/s 2024-04-25T00:33:18.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 41348 35.36 MB/sec warmup 68 sec latency 70.714 ms 2024-04-25T00:33:19.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 42012 35.36 MB/sec warmup 69 sec latency 73.722 ms 2024-04-25T00:33:19.815 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:19 smithi136 ceph-mon[32923]: pgmap v825: 97 pgs: 97 active+clean; 430 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.0 MiB/s wr, 761 op/s 2024-04-25T00:33:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:19 smithi106 ceph-mon[32510]: pgmap v825: 97 pgs: 97 active+clean; 430 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.0 MiB/s wr, 761 op/s 2024-04-25T00:33:19.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:19 smithi042 ceph-mon[28130]: pgmap v825: 97 pgs: 97 active+clean; 430 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 9.0 MiB/s wr, 761 op/s 2024-04-25T00:33:20.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 42870 35.34 MB/sec warmup 70 sec latency 52.595 ms 2024-04-25T00:33:21.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 43505 35.30 MB/sec warmup 71 sec latency 176.721 ms 2024-04-25T00:33:21.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:21 smithi106 ceph-mon[32510]: pgmap v826: 97 pgs: 97 active+clean; 438 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 782 op/s 2024-04-25T00:33:21.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:21 smithi042 ceph-mon[28130]: pgmap v826: 97 pgs: 97 active+clean; 438 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 782 op/s 2024-04-25T00:33:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:21 smithi136 ceph-mon[32923]: pgmap v826: 97 pgs: 97 active+clean; 438 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.6 MiB/s wr, 782 op/s 2024-04-25T00:33:22.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 44371 35.73 MB/sec warmup 72 sec latency 70.493 ms 2024-04-25T00:33:22.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:22 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:33:22.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:22 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:33:23.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:22 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:33:23.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 44973 35.70 MB/sec warmup 73 sec latency 69.015 ms 2024-04-25T00:33:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:23 smithi106 ceph-mon[32510]: pgmap v827: 97 pgs: 97 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 823 op/s 2024-04-25T00:33:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:33:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:33:23.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:33:23.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:23 smithi042 ceph-mon[28130]: pgmap v827: 97 pgs: 97 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 823 op/s 2024-04-25T00:33:23.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:33:23.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:33:23.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:33:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:23 smithi136 ceph-mon[32923]: pgmap v827: 97 pgs: 97 active+clean; 443 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 823 op/s 2024-04-25T00:33:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:33:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:33:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:33:24.740 INFO:tasks.workunit.client.0.smithi042.stdout: 10 45505 35.52 MB/sec warmup 74 sec latency 65.886 ms 2024-04-25T00:33:24.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:24 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:33:24.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:24 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:33:25.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:24 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:33:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.428+0000 7f2249e00640 1 -- 172.21.15.42:0/3619513277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22441015d0 msgr2=0x7f22441019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.428+0000 7f2249e00640 1 --2- 172.21.15.42:0/3619513277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22441015d0 0x7f22441019f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2238009900 tx=0x7f223802a410 comp rx=0 tx=0).stop 2024-04-25T00:33:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.428+0000 7f2249e00640 1 -- 172.21.15.42:0/3619513277 shutdown_connections 2024-04-25T00:33:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.428+0000 7f2249e00640 1 --2- 172.21.15.42:0/3619513277 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2244103180 0x7f2244109a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.428+0000 7f2249e00640 1 --2- 172.21.15.42:0/3619513277 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22441027c0 0x7f2244102c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.428+0000 7f2249e00640 1 --2- 172.21.15.42:0/3619513277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22441015d0 0x7f22441019f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.428+0000 7f2249e00640 1 -- 172.21.15.42:0/3619513277 >> 172.21.15.42:0/3619513277 conn(0x7f22440fcb40 msgr2=0x7f22440fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.428+0000 7f2249e00640 1 -- 172.21.15.42:0/3619513277 shutdown_connections 2024-04-25T00:33:25.430 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.428+0000 7f2249e00640 1 -- 172.21.15.42:0/3619513277 wait complete. 2024-04-25T00:33:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.430+0000 7f2249e00640 1 Processor -- start 2024-04-25T00:33:25.431 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.430+0000 7f2249e00640 1 -- start start 2024-04-25T00:33:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.430+0000 7f2249e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22441015d0 0x7f2244112190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.430+0000 7f2249e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22441027c0 0x7f22441126d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.430+0000 7f2249e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2244103180 0x7f22441107d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.432 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.430+0000 7f2249e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f224410b070 con 0x7f2244103180 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.430+0000 7f2249e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f224410aef0 con 0x7f22441027c0 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.430+0000 7f2249e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f224410b1f0 con 0x7f22441015d0 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.431+0000 7f2243e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22441027c0 0x7f22441126d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.431+0000 7f2243e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22441027c0 0x7f22441126d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56396/0 (socket says 172.21.15.42:56396) 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.431+0000 7f2243e00640 1 -- 172.21.15.42:0/3318134992 learned_addr learned my addr 172.21.15.42:0/3318134992 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.431+0000 7f2248a00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22441015d0 0x7f2244112190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.431+0000 7f2249400640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2244103180 0x7f22441107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.431+0000 7f2243e00640 1 -- 172.21.15.42:0/3318134992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22441015d0 msgr2=0x7f2244112190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.431+0000 7f2243e00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22441015d0 0x7f2244112190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.431+0000 7f2243e00640 1 -- 172.21.15.42:0/3318134992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2244103180 msgr2=0x7f22441107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.431+0000 7f2243e00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2244103180 0x7f22441107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.431+0000 7f2243e00640 1 -- 172.21.15.42:0/3318134992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2244110f50 con 0x7f22441027c0 2024-04-25T00:33:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.432+0000 7f2243e00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22441027c0 0x7f22441126d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f223000e9f0 tx=0x7f223000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.432+0000 7f2241600640 1 -- 172.21.15.42:0/3318134992 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223000cd80 con 0x7f22441027c0 2024-04-25T00:33:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.432+0000 7f2241600640 1 -- 172.21.15.42:0/3318134992 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2230005500 con 0x7f22441027c0 2024-04-25T00:33:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.432+0000 7f2241600640 1 -- 172.21.15.42:0/3318134992 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230010690 con 0x7f22441027c0 2024-04-25T00:33:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.432+0000 7f2249e00640 1 -- 172.21.15.42:0/3318134992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2244111240 con 0x7f22441027c0 2024-04-25T00:33:25.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.434+0000 7f2249e00640 1 -- 172.21.15.42:0/3318134992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22441aec10 con 0x7f22441027c0 2024-04-25T00:33:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.439+0000 7f224e1e0740 1 Processor -- start 2024-04-25T00:33:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.439+0000 7f224e1e0740 1 -- start start 2024-04-25T00:33:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.440+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678d4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.440+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d4840 0x55f3678d9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.440+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d99b0 0x55f3678dbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.440+0000 7f2249400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678d4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.440+0000 7f2249400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678d4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56404/0 (socket says 172.21.15.42:56404) 2024-04-25T00:33:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.440+0000 7f2249400640 1 -- 172.21.15.42:0/2801930985 learned_addr learned my addr 172.21.15.42:0/2801930985 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.440+0000 7f2248a00640 1 --2- 172.21.15.42:0/2801930985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d4840 0x55f3678d9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.440+0000 7f224e1e0740 1 -- 172.21.15.42:0/2801930985 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f367836ab0 con 0x55f3678d99b0 2024-04-25T00:33:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.440+0000 7f224e1e0740 1 -- 172.21.15.42:0/2801930985 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f3677df640 con 0x55f3678d3e50 2024-04-25T00:33:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.440+0000 7f224e1e0740 1 -- 172.21.15.42:0/2801930985 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f367832a70 con 0x55f3678d4840 2024-04-25T00:33:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.441+0000 7f2249400640 1 -- 172.21.15.42:0/2801930985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d4840 msgr2=0x55f3678d9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.441+0000 7f2241600640 1 -- 172.21.15.42:0/3318134992 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f22300046c0 con 0x7f22441027c0 2024-04-25T00:33:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.441+0000 7f2243e00640 1 --2- 172.21.15.42:0/2801930985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d99b0 0x55f3678dbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.441+0000 7f2249400640 1 --2- 172.21.15.42:0/2801930985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d4840 0x55f3678d9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.441+0000 7f2249400640 1 -- 172.21.15.42:0/2801930985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d99b0 msgr2=0x55f3678dbe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.441+0000 7f2249400640 1 --2- 172.21.15.42:0/2801930985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d99b0 0x55f3678dbe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.441+0000 7f2249400640 1 -- 172.21.15.42:0/2801930985 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f36754b210 con 0x55f3678d3e50 2024-04-25T00:33:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.441+0000 7f2243e00640 1 --2- 172.21.15.42:0/2801930985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d99b0 0x55f3678dbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.442+0000 7f2249400640 1 --2- 172.21.15.42:0/2801930985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678d4270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f223c009fd0 tx=0x7f223c010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4e4864d5acb93b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.443+0000 7f2235c00640 1 -- 172.21.15.42:0/2801930985 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c011700 con 0x55f3678d3e50 2024-04-25T00:33:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.443+0000 7f2235c00640 1 -- 172.21.15.42:0/2801930985 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f223c011d80 con 0x55f3678d3e50 2024-04-25T00:33:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.443+0000 7f2235c00640 1 -- 172.21.15.42:0/2801930985 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223c01a380 con 0x55f3678d3e50 2024-04-25T00:33:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.443+0000 7f2241600640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f222c0a2c00 0x7f222c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.443+0000 7f2241600640 1 -- 172.21.15.42:0/3318134992 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f22300d3060 con 0x7f22441027c0 2024-04-25T00:33:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.444+0000 7f224e1e0740 1 -- 172.21.15.42:0/2801930985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 msgr2=0x55f3678d4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:25.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.444+0000 7f224e1e0740 1 --2- 172.21.15.42:0/2801930985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678d4270 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f223c009fd0 tx=0x7f223c010860 comp rx=0 tx=0).stop 2024-04-25T00:33:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.444+0000 7f224e1e0740 1 -- 172.21.15.42:0/2801930985 shutdown_connections 2024-04-25T00:33:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.444+0000 7f224e1e0740 1 --2- 172.21.15.42:0/2801930985 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d99b0 0x55f3678dbe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.444+0000 7f224e1e0740 1 --2- 172.21.15.42:0/2801930985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d4840 0x55f3678d9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.444+0000 7f224e1e0740 1 --2- 172.21.15.42:0/2801930985 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678d4270 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.444+0000 7f224e1e0740 1 -- 172.21.15.42:0/2801930985 >> 172.21.15.42:0/2801930985 conn(0x55f3678cf4c0 msgr2=0x55f3678cf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.444+0000 7f2248a00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f222c0a2c00 0x7f222c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.444+0000 7f224e1e0740 1 -- 172.21.15.42:0/2801930985 shutdown_connections 2024-04-25T00:33:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.444+0000 7f224e1e0740 1 -- 172.21.15.42:0/2801930985 wait complete. 2024-04-25T00:33:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.445+0000 7f224e1e0740 1 -- start start 2024-04-25T00:33:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.445+0000 7f224e1e0740 1 Processor -- start 2024-04-25T00:33:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.445+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678de090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.445+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f3678de5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.445+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 0x55f3678e5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.445+0000 7f224e1e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f3678dc960 con 0x55f3678d4840 2024-04-25T00:33:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.445+0000 7f224e1e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f3677f2990 con 0x55f3678d3e50 2024-04-25T00:33:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.445+0000 7f224e1e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f3678dcae0 con 0x55f3678d99b0 2024-04-25T00:33:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.445+0000 7f224e1e0740 10 client.0 _ll_register_callbacks cb 0x55f3677f4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.446+0000 7f2248a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678de090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.446+0000 7f2248a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678de090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56410/0 (socket says 172.21.15.42:56410) 2024-04-25T00:33:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.446+0000 7f2248a00640 1 -- 172.21.15.42:0/3087154946 learned_addr learned my addr 172.21.15.42:0/3087154946 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.446+0000 7f2249400640 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 0x55f3678e5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.446+0000 7f2243e00640 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f3678de5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.446+0000 7f2248a00640 1 -- 172.21.15.42:0/3087154946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 msgr2=0x55f3678e5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.446+0000 7f2248a00640 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 0x55f3678e5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.446+0000 7f2248a00640 1 -- 172.21.15.42:0/3087154946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 msgr2=0x55f3678de5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.446+0000 7f2248a00640 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f3678de5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.446+0000 7f2248a00640 1 -- 172.21.15.42:0/3087154946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f367617620 con 0x55f3678d3e50 2024-04-25T00:33:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.447+0000 7f2248a00640 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678de090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2238006010 tx=0x7f22380052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27317868a4eaf335 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.447+0000 7f222a000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:33:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.447+0000 7f222a000640 1 -- 172.21.15.42:0/3087154946 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238035040 con 0x55f3678d3e50 2024-04-25T00:33:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.447+0000 7f222a000640 1 -- 172.21.15.42:0/3087154946 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f2238004970 con 0x55f3678d3e50 2024-04-25T00:33:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.448+0000 7f2248a00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f222c0a2c00 0x7f222c0a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2238005ab0 tx=0x7f2238042000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:25.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.448+0000 7f224e1e0740 1 -- 172.21.15.42:0/3087154946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f223c011d80 con 0x55f3678d3e50 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.448+0000 7f222a000640 1 -- 172.21.15.42:0/3087154946 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2238004e10 con 0x55f3678d3e50 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.448+0000 7f224e1e0740 10 client.26342 fetch_fsmap learned FSMap version 36 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.448+0000 7f224e1e0740 1 -- 172.21.15.42:0/3087154946 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f3678e85d0 con 0x55f3678d3e50 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.449+0000 7f222a000640 1 -- 172.21.15.42:0/3087154946 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f223c011d80 con 0x55f3678d3e50 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.449+0000 7f222a000640 1 -- 172.21.15.42:0/3087154946 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f223803c740 con 0x55f3678d3e50 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.449+0000 7f224e1e0740 10 client.26342 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.449+0000 7f224e1e0740 10 client.26342 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.449+0000 7f224e1e0740 20 client.26342 populate_metadata read hostname 'smithi042' 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.449+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3678e9cb0 0x55f3678ec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.449+0000 7f224e1e0740 4 client.26342 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.449+0000 7f224e1e0740 1 -- 172.21.15.42:0/3087154946 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f367843ff0 con 0x55f3678e9cb0 2024-04-25T00:33:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.450+0000 7f2243e00640 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3678e9cb0 0x55f3678ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.450+0000 7f2243e00640 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3678e9cb0 0x55f3678ec110 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cccdb1ad75648738 server_cookie=4494c9b4f63b3a44 in_seq=0 out_seq=0 2024-04-25T00:33:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.450+0000 7f222a000640 10 client.26342 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.452+0000 7f222a000640 1 -- 172.21.15.42:0/3087154946 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f367843ff0 con 0x55f3678e9cb0 2024-04-25T00:33:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.452+0000 7f222a000640 10 client.26342 handle_command_reply: tid=0 2024-04-25T00:33:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.452+0000 7f224e1e0740 1 client.26342 shutdown 2024-04-25T00:33:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 1 -- 172.21.15.42:0/3087154946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 msgr2=0x55f3678de090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678de090 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f2238006010 tx=0x7f22380052f0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 1 -- 172.21.15.42:0/3087154946 shutdown_connections 2024-04-25T00:33:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3678e9cb0 0x55f3678ec110 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 0x55f3678e5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f3678de5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3087154946 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678de090 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f222a000640 0 client.26342 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 1 -- 172.21.15.42:0/3087154946 >> 172.21.15.42:0/3087154946 conn(0x55f3678cf4c0 msgr2=0x55f3678ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 1 -- 172.21.15.42:0/3087154946 shutdown_connections 2024-04-25T00:33:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 1 -- 172.21.15.42:0/3087154946 wait complete. 2024-04-25T00:33:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.453+0000 7f224e1e0740 20 client.26342 trim_cache size 0 max 16384 2024-04-25T00:33:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.475+0000 7f224e1e0740 1 Processor -- start 2024-04-25T00:33:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.475+0000 7f224e1e0740 1 -- start start 2024-04-25T00:33:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.475+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678ca110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.475+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f3678ddca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.475+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 0x55f3678de1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.475+0000 7f224e1e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22441ba2d0 con 0x55f3678d4840 2024-04-25T00:33:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.476+0000 7f224e1e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f22441b7ab0 con 0x55f3678d3e50 2024-04-25T00:33:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.476+0000 7f224e1e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f22441b5950 con 0x55f3678d99b0 2024-04-25T00:33:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.476+0000 7f2249400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 0x55f3678de1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.476+0000 7f2249400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 0x55f3678de1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:58750/0 (socket says 172.21.15.42:58750) 2024-04-25T00:33:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.476+0000 7f2249400640 1 -- 172.21.15.42:0/2897134003 learned_addr learned my addr 172.21.15.42:0/2897134003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.476+0000 7f2243e00640 1 --2- 172.21.15.42:0/2897134003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f3678ddca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.476+0000 7f2248a00640 1 --2- 172.21.15.42:0/2897134003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.477+0000 7f2248a00640 1 -- 172.21.15.42:0/2897134003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 msgr2=0x55f3678de1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.477+0000 7f2248a00640 1 --2- 172.21.15.42:0/2897134003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 0x55f3678de1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.477+0000 7f2248a00640 1 -- 172.21.15.42:0/2897134003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 msgr2=0x55f3678ddca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.477+0000 7f2248a00640 1 --2- 172.21.15.42:0/2897134003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f3678ddca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.478+0000 7f2248a00640 1 -- 172.21.15.42:0/2897134003 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3678d4c20 con 0x55f3678d3e50 2024-04-25T00:33:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.478+0000 7f2243e00640 1 --2- 172.21.15.42:0/2897134003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f3678ddca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:33:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.478+0000 7f2248a00640 1 --2- 172.21.15.42:0/2897134003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678ca110 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f223802a930 tx=0x7f2238004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e43dd50b1de7f26f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.478+0000 7f2235200640 1 -- 172.21.15.42:0/2897134003 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238009400 con 0x55f3678d3e50 2024-04-25T00:33:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.478+0000 7f2235200640 1 -- 172.21.15.42:0/2897134003 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2238041070 con 0x55f3678d3e50 2024-04-25T00:33:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.478+0000 7f2235200640 1 -- 172.21.15.42:0/2897134003 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22380318a0 con 0x55f3678d3e50 2024-04-25T00:33:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.480+0000 7f224e1e0740 1 -- 172.21.15.42:0/2897134003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 msgr2=0x55f3678ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.480+0000 7f224e1e0740 1 --2- 172.21.15.42:0/2897134003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678ca110 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f223802a930 tx=0x7f2238004610 comp rx=0 tx=0).stop 2024-04-25T00:33:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.480+0000 7f224e1e0740 1 -- 172.21.15.42:0/2897134003 shutdown_connections 2024-04-25T00:33:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.480+0000 7f224e1e0740 1 --2- 172.21.15.42:0/2897134003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d99b0 0x55f3678de1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.480+0000 7f224e1e0740 1 --2- 172.21.15.42:0/2897134003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f3678ddca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.480+0000 7f224e1e0740 1 --2- 172.21.15.42:0/2897134003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d3e50 0x55f3678ca110 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.480+0000 7f224e1e0740 1 -- 172.21.15.42:0/2897134003 >> 172.21.15.42:0/2897134003 conn(0x55f3678cf4c0 msgr2=0x55f3678d90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.480+0000 7f224e1e0740 1 -- 172.21.15.42:0/2897134003 shutdown_connections 2024-04-25T00:33:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.480+0000 7f224e1e0740 1 -- 172.21.15.42:0/2897134003 wait complete. 2024-04-25T00:33:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.481+0000 7f224e1e0740 1 -- start start 2024-04-25T00:33:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.481+0000 7f224e1e0740 1 Processor -- start 2024-04-25T00:33:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.482+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d3e50 0x55f36794d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.482+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f36794d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.482+0000 7f224e1e0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d99b0 0x55f3678e38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.482+0000 7f224e1e0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f22441b5fc0 con 0x55f3678d4840 2024-04-25T00:33:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.482+0000 7f224e1e0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f22441b73e0 con 0x55f3678d99b0 2024-04-25T00:33:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.482+0000 7f224e1e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f22441b6640 con 0x55f3678d3e50 2024-04-25T00:33:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.482+0000 7f224e1e0740 10 client.0 _ll_register_callbacks cb 0x7f2238009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.483+0000 7f2248a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d3e50 0x55f36794d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.483+0000 7f2248a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d3e50 0x55f36794d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:58766/0 (socket says 172.21.15.42:58766) 2024-04-25T00:33:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.483+0000 7f2248a00640 1 -- 172.21.15.42:0/3247393419 learned_addr learned my addr 172.21.15.42:0/3247393419 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.483+0000 7f2243e00640 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f36794d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.483+0000 7f2249400640 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d99b0 0x55f3678e38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.483+0000 7f2248a00640 1 -- 172.21.15.42:0/3247393419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d99b0 msgr2=0x55f3678e38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.483+0000 7f2248a00640 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d99b0 0x55f3678e38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.483+0000 7f2248a00640 1 -- 172.21.15.42:0/3247393419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 msgr2=0x55f36794d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.483+0000 7f2248a00640 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f36794d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.483+0000 7f2248a00640 1 -- 172.21.15.42:0/3247393419 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3678e3eb0 con 0x55f3678d3e50 2024-04-25T00:33:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.484+0000 7f2248a00640 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d3e50 0x55f36794d0b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f223803cc00 tx=0x7f2238009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=320305d180da99a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.485+0000 7f222a000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:33:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.485+0000 7f222a000640 1 -- 172.21.15.42:0/3247393419 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2238065070 con 0x55f3678d3e50 2024-04-25T00:33:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.485+0000 7f222a000640 1 -- 172.21.15.42:0/3247393419 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f2238031850 con 0x55f3678d3e50 2024-04-25T00:33:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.486+0000 7f224e1e0740 1 -- 172.21.15.42:0/3247393419 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2238041070 con 0x55f3678d3e50 2024-04-25T00:33:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f222a000640 1 -- 172.21.15.42:0/3247393419 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f223803c920 con 0x55f3678d3e50 2024-04-25T00:33:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f224e1e0740 10 client.26356 fetch_fsmap learned FSMap version 36 2024-04-25T00:33:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f224e1e0740 1 -- 172.21.15.42:0/3247393419 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f3679613c0 con 0x55f3678d3e50 2024-04-25T00:33:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f222a000640 1 -- 172.21.15.42:0/3247393419 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2238041070 con 0x55f3678d3e50 2024-04-25T00:33:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f222a000640 1 -- 172.21.15.42:0/3247393419 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f2238068030 con 0x55f3678d3e50 2024-04-25T00:33:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f224e1e0740 10 client.26356 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T00:33:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f224e1e0740 10 client.26356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:33:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f224e1e0740 20 client.26356 populate_metadata read hostname 'smithi042' 2024-04-25T00:33:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f367961d60 0x55f367964180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f224e1e0740 4 client.26356 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:33:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.488+0000 7f224e1e0740 1 -- 172.21.15.42:0/3247393419 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f36783e340 con 0x55f367961d60 2024-04-25T00:33:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.489+0000 7f2243e00640 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f367961d60 0x55f367964180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.490+0000 7f2243e00640 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f367961d60 0x55f367964180 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba05637f877bcd5b server_cookie=9f48f0b85e466dc3 in_seq=0 out_seq=0 2024-04-25T00:33:25.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.490+0000 7f222a000640 10 client.26356 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:25.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 46198 35.42 MB/sec warmup 75 sec latency 146.743 ms 2024-04-25T00:33:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f222a000640 1 -- 172.21.15.42:0/3247393419 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f36783e340 con 0x55f367961d60 2024-04-25T00:33:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f222a000640 10 client.26356 handle_command_reply: tid=0 2024-04-25T00:33:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f224e1e0740 1 client.26356 shutdown 2024-04-25T00:33:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f224e1e0740 1 -- 172.21.15.42:0/3247393419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d3e50 msgr2=0x55f36794d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d3e50 0x55f36794d0b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f223803cc00 tx=0x7f2238009630 comp rx=0 tx=0).stop 2024-04-25T00:33:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f224e1e0740 1 -- 172.21.15.42:0/3247393419 shutdown_connections 2024-04-25T00:33:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f367961d60 0x55f367964180 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3678d99b0 0x55f3678e38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3678d4840 0x55f36794d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f224e1e0740 1 --2- 172.21.15.42:0/3247393419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3678d3e50 0x55f36794d0b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.792+0000 7f224e1e0740 1 -- 172.21.15.42:0/3247393419 >> 172.21.15.42:0/3247393419 conn(0x55f3678cf4c0 msgr2=0x55f3678d8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.793+0000 7f222a000640 0 client.26356 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.793+0000 7f224e1e0740 1 -- 172.21.15.42:0/3247393419 shutdown_connections 2024-04-25T00:33:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.793+0000 7f224e1e0740 1 -- 172.21.15.42:0/3247393419 wait complete. 2024-04-25T00:33:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.793+0000 7f224e1e0740 20 client.26356 trim_cache size 0 max 16384 2024-04-25T00:33:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:33:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:33:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:33:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:33:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.794+0000 7f2249e00640 1 -- 172.21.15.42:0/3318134992 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f222c0a2c00 msgr2=0x7f222c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.794+0000 7f2249e00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f222c0a2c00 0x7f222c0a50c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2238005ab0 tx=0x7f2238042000 comp rx=0 tx=0).stop 2024-04-25T00:33:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.794+0000 7f2249e00640 1 -- 172.21.15.42:0/3318134992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22441027c0 msgr2=0x7f22441126d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.794+0000 7f2249e00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22441027c0 0x7f22441126d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f223000e9f0 tx=0x7f223000edf0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.795+0000 7f2249e00640 1 -- 172.21.15.42:0/3318134992 shutdown_connections 2024-04-25T00:33:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.795+0000 7f2249e00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f222c0a2c00 0x7f222c0a50c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.795+0000 7f2249e00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2244103180 0x7f22441107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.795+0000 7f2249e00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22441027c0 0x7f22441126d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.795+0000 7f2249e00640 1 --2- 172.21.15.42:0/3318134992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f22441015d0 0x7f2244112190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.795+0000 7f2249e00640 1 -- 172.21.15.42:0/3318134992 >> 172.21.15.42:0/3318134992 conn(0x7f22440fcb40 msgr2=0x7f2244105b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.795+0000 7f2249e00640 1 -- 172.21.15.42:0/3318134992 shutdown_connections 2024-04-25T00:33:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:25.795+0000 7f2249e00640 1 -- 172.21.15.42:0/3318134992 wait complete. 2024-04-25T00:33:25.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:25 smithi106 ceph-mon[32510]: pgmap v828: 97 pgs: 97 active+clean; 443 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 690 op/s 2024-04-25T00:33:25.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:25 smithi042 ceph-mon[28130]: pgmap v828: 97 pgs: 97 active+clean; 443 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 690 op/s 2024-04-25T00:33:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:25 smithi136 ceph-mon[32923]: pgmap v828: 97 pgs: 97 active+clean; 443 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.0 MiB/s wr, 690 op/s 2024-04-25T00:33:26.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 46853 35.36 MB/sec warmup 76 sec latency 184.697 ms 2024-04-25T00:33:27.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 47877 35.77 MB/sec warmup 77 sec latency 55.281 ms 2024-04-25T00:33:27.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:27 smithi106 ceph-mon[32510]: pgmap v829: 97 pgs: 97 active+clean; 449 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 931 op/s 2024-04-25T00:33:27.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:27 smithi042 ceph-mon[28130]: pgmap v829: 97 pgs: 97 active+clean; 449 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 931 op/s 2024-04-25T00:33:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:27 smithi136 ceph-mon[32923]: pgmap v829: 97 pgs: 97 active+clean; 449 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 11 MiB/s wr, 931 op/s 2024-04-25T00:33:28.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 48589 35.97 MB/sec warmup 78 sec latency 43.346 ms 2024-04-25T00:33:29.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 49064 35.69 MB/sec warmup 79 sec latency 42.131 ms 2024-04-25T00:33:29.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:29 smithi106 ceph-mon[32510]: pgmap v830: 97 pgs: 97 active+clean; 450 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 702 op/s 2024-04-25T00:33:29.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:29 smithi042 ceph-mon[28130]: pgmap v830: 97 pgs: 97 active+clean; 450 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 702 op/s 2024-04-25T00:33:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:29 smithi136 ceph-mon[32923]: pgmap v830: 97 pgs: 97 active+clean; 450 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.3 MiB/s wr, 702 op/s 2024-04-25T00:33:30.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 49834 35.64 MB/sec warmup 80 sec latency 63.742 ms 2024-04-25T00:33:31.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 50900 35.91 MB/sec warmup 81 sec latency 76.276 ms 2024-04-25T00:33:31.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:31 smithi106 ceph-mon[32510]: pgmap v831: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 787 op/s 2024-04-25T00:33:31.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:31 smithi042 ceph-mon[28130]: pgmap v831: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 787 op/s 2024-04-25T00:33:32.018 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:31 smithi136 ceph-mon[32923]: pgmap v831: 97 pgs: 97 active+clean; 452 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 787 op/s 2024-04-25T00:33:32.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 51526 36.06 MB/sec warmup 82 sec latency 238.200 ms 2024-04-25T00:33:33.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 52103 36.02 MB/sec warmup 83 sec latency 76.298 ms 2024-04-25T00:33:33.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:33 smithi106 ceph-mon[32510]: pgmap v832: 97 pgs: 97 active+clean; 457 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 814 op/s 2024-04-25T00:33:33.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:33 smithi042 ceph-mon[28130]: pgmap v832: 97 pgs: 97 active+clean; 457 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 814 op/s 2024-04-25T00:33:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:33 smithi136 ceph-mon[32923]: pgmap v832: 97 pgs: 97 active+clean; 457 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 10 MiB/s wr, 814 op/s 2024-04-25T00:33:34.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 52715 35.83 MB/sec warmup 84 sec latency 44.213 ms 2024-04-25T00:33:35.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 53670 35.96 MB/sec warmup 85 sec latency 62.999 ms 2024-04-25T00:33:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:35 smithi106 ceph-mon[32510]: pgmap v833: 97 pgs: 97 active+clean; 457 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 688 op/s 2024-04-25T00:33:35.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:35 smithi042 ceph-mon[28130]: pgmap v833: 97 pgs: 97 active+clean; 457 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 688 op/s 2024-04-25T00:33:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:35 smithi136 ceph-mon[32923]: pgmap v833: 97 pgs: 97 active+clean; 457 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 688 op/s 2024-04-25T00:33:36.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 54396 36.08 MB/sec warmup 86 sec latency 294.170 ms 2024-04-25T00:33:37.741 INFO:tasks.workunit.client.0.smithi042.stdout: 10 54905 36.10 MB/sec warmup 87 sec latency 228.195 ms 2024-04-25T00:33:37.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:37 smithi042 ceph-mon[28130]: pgmap v834: 97 pgs: 97 active+clean; 467 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 910 op/s 2024-04-25T00:33:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:37 smithi136 ceph-mon[32923]: pgmap v834: 97 pgs: 97 active+clean; 467 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 910 op/s 2024-04-25T00:33:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:37 smithi106 ceph-mon[32510]: pgmap v834: 97 pgs: 97 active+clean; 467 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 12 MiB/s wr, 910 op/s 2024-04-25T00:33:38.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 55486 36.06 MB/sec warmup 88 sec latency 60.790 ms 2024-04-25T00:33:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.988+0000 7fb55ae00640 1 -- 172.21.15.42:0/2136922310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb554102980 msgr2=0x7fb554104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.988+0000 7fb55ae00640 1 --2- 172.21.15.42:0/2136922310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb554102980 0x7fb554104df0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009ca0 tx=0x7fb54c02a8e0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.988+0000 7fb55ae00640 1 -- 172.21.15.42:0/2136922310 shutdown_connections 2024-04-25T00:33:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.988+0000 7fb55ae00640 1 --2- 172.21.15.42:0/2136922310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554105330 0x7fb5541057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.988+0000 7fb55ae00640 1 --2- 172.21.15.42:0/2136922310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb554102980 0x7fb554104df0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.988+0000 7fb55ae00640 1 --2- 172.21.15.42:0/2136922310 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb554100010 0x7fb554102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.988+0000 7fb55ae00640 1 -- 172.21.15.42:0/2136922310 >> 172.21.15.42:0/2136922310 conn(0x7fb5540fbc10 msgr2=0x7fb5540fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.989+0000 7fb55ae00640 1 -- 172.21.15.42:0/2136922310 shutdown_connections 2024-04-25T00:33:38.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.989+0000 7fb55ae00640 1 -- 172.21.15.42:0/2136922310 wait complete. 2024-04-25T00:33:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.990+0000 7fb55ae00640 1 Processor -- start 2024-04-25T00:33:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.990+0000 7fb55ae00640 1 -- start start 2024-04-25T00:33:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.990+0000 7fb55ae00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100010 0x7fb55419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.990+0000 7fb55ae00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb554102980 0x7fb55419e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.991+0000 7fb55ae00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb554105330 0x7fb5541a5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.991+0000 7fb55ae00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb554113a40 con 0x7fb554102980 2024-04-25T00:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.991+0000 7fb55ae00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541138c0 con 0x7fb554100010 2024-04-25T00:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.991+0000 7fb55ae00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb554113bc0 con 0x7fb554105330 2024-04-25T00:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.991+0000 7fb553e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100010 0x7fb55419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.991+0000 7fb553e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100010 0x7fb55419dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59276/0 (socket says 172.21.15.42:59276) 2024-04-25T00:33:38.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.991+0000 7fb553e00640 1 -- 172.21.15.42:0/2666414766 learned_addr learned my addr 172.21.15.42:0/2666414766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.991+0000 7fb553400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb554102980 0x7fb55419e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.991+0000 7fb558e00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb554105330 0x7fb5541a5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.992+0000 7fb553e00640 1 -- 172.21.15.42:0/2666414766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb554105330 msgr2=0x7fb5541a5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.992+0000 7fb553e00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb554105330 0x7fb5541a5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.992+0000 7fb553e00640 1 -- 172.21.15.42:0/2666414766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb554102980 msgr2=0x7fb55419e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:38.996 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.992+0000 7fb553e00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb554102980 0x7fb55419e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.992+0000 7fb553e00640 1 -- 172.21.15.42:0/2666414766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb554106760 con 0x7fb554100010 2024-04-25T00:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.992+0000 7fb553e00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100010 0x7fb55419dfe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb54000d950 tx=0x7fb54000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.992+0000 7fb550c00640 1 -- 172.21.15.42:0/2666414766 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540017070 con 0x7fb554100010 2024-04-25T00:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.992+0000 7fb550c00640 1 -- 172.21.15.42:0/2666414766 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb540009c30 con 0x7fb554100010 2024-04-25T00:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.992+0000 7fb550c00640 1 -- 172.21.15.42:0/2666414766 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54000f690 con 0x7fb554100010 2024-04-25T00:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.992+0000 7fb55ae00640 1 -- 172.21.15.42:0/2666414766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb554106a50 con 0x7fb554100010 2024-04-25T00:33:38.997 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.993+0000 7fb55ae00640 1 -- 172.21.15.42:0/2666414766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb554106f50 con 0x7fb554100010 2024-04-25T00:33:38.998 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.997+0000 7fb55dc57740 1 Processor -- start 2024-04-25T00:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.997+0000 7fb55dc57740 1 -- start start 2024-04-25T00:33:38.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.998+0000 7fb550c00640 1 -- 172.21.15.42:0/2666414766 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb5400109d0 con 0x7fb554100010 2024-04-25T00:33:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 0x55a1689b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b7690 0x55a1689bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 0x55a1689bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb55dc57740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a168919ab0 con 0x55a1689b6ca0 2024-04-25T00:33:39.001 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb55dc57740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a1688c2640 con 0x55a1689bc800 2024-04-25T00:33:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb55dc57740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a168915a70 con 0x55a1689b7690 2024-04-25T00:33:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb553e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b7690 0x55a1689bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb553400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 0x55a1689bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb553400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 0x55a1689bec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59290/0 (socket says 172.21.15.42:59290) 2024-04-25T00:33:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb553400640 1 -- 172.21.15.42:0/492368880 learned_addr learned my addr 172.21.15.42:0/492368880 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb558e00640 1 --2- 172.21.15.42:0/492368880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 0x55a1689b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb553400640 1 -- 172.21.15.42:0/492368880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b7690 msgr2=0x55a1689bc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb553400640 1 --2- 172.21.15.42:0/492368880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b7690 0x55a1689bc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:38.999+0000 7fb553400640 1 -- 172.21.15.42:0/492368880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 msgr2=0x55a1689b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.000+0000 7fb553400640 1 --2- 172.21.15.42:0/492368880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 0x55a1689b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.000+0000 7fb553400640 1 -- 172.21.15.42:0/492368880 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a16862e210 con 0x55a1689bc800 2024-04-25T00:33:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.000+0000 7fb558e00640 1 --2- 172.21.15.42:0/492368880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 0x55a1689b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.001+0000 7fb550c00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb5380a2cb0 0x7fb5380a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.001+0000 7fb558e00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb5380a2cb0 0x7fb5380a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.001+0000 7fb553400640 1 --2- 172.21.15.42:0/492368880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 0x55a1689bec90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009ca0 tx=0x7fb54c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e34b7e8155924f92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.002+0000 7fb558e00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb5380a2cb0 0x7fb5380a5170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x55a1689b7ef0 tx=0x7fb544005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.002+0000 7fb549600640 1 -- 172.21.15.42:0/492368880 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c031ac0 con 0x55a1689bc800 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.002+0000 7fb549600640 1 -- 172.21.15.42:0/492368880 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb54c036040 con 0x55a1689bc800 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.002+0000 7fb549600640 1 -- 172.21.15.42:0/492368880 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0046e0 con 0x55a1689bc800 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.002+0000 7fb550c00640 1 -- 172.21.15.42:0/2666414766 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fb5400cebd0 con 0x7fb554100010 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.003+0000 7fb55dc57740 1 -- 172.21.15.42:0/492368880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 msgr2=0x55a1689bec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.003+0000 7fb55dc57740 1 --2- 172.21.15.42:0/492368880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 0x55a1689bec90 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009ca0 tx=0x7fb54c009a20 comp rx=0 tx=0).stop 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.003+0000 7fb55dc57740 1 -- 172.21.15.42:0/492368880 shutdown_connections 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.003+0000 7fb55dc57740 1 --2- 172.21.15.42:0/492368880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 0x55a1689bec90 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.003+0000 7fb55dc57740 1 --2- 172.21.15.42:0/492368880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b7690 0x55a1689bc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.003+0000 7fb55dc57740 1 --2- 172.21.15.42:0/492368880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 0x55a1689b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.003+0000 7fb55dc57740 1 -- 172.21.15.42:0/492368880 >> 172.21.15.42:0/492368880 conn(0x55a1689b2310 msgr2=0x55a1689b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.003+0000 7fb55dc57740 1 -- 172.21.15.42:0/492368880 shutdown_connections 2024-04-25T00:33:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.003+0000 7fb55dc57740 1 -- 172.21.15.42:0/492368880 wait complete. 2024-04-25T00:33:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.003+0000 7fb55dc57740 1 -- start start 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb55dc57740 1 Processor -- start 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 0x55a1689c0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689bc800 0x55a1689c8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb55dc57740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a1689bf750 con 0x55a1689b6ca0 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb55dc57740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a1688d5990 con 0x55a1689b7690 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb55dc57740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a1689bf8d0 con 0x55a1689bc800 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb55dc57740 10 client.0 _ll_register_callbacks cb 0x55a1688d7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb553400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb558e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689bc800 0x55a1689c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb553400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59304/0 (socket says 172.21.15.42:59304) 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.004+0000 7fb553400640 1 -- 172.21.15.42:0/690389528 learned_addr learned my addr 172.21.15.42:0/690389528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.005+0000 7fb553e00640 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 0x55a1689c0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.005+0000 7fb553400640 1 -- 172.21.15.42:0/690389528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689bc800 msgr2=0x55a1689c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.005+0000 7fb553400640 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689bc800 0x55a1689c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.005+0000 7fb553400640 1 -- 172.21.15.42:0/690389528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 msgr2=0x55a1689c0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.005+0000 7fb553400640 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 0x55a1689c0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.005+0000 7fb553400640 1 -- 172.21.15.42:0/690389528 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1686fa620 con 0x55a1689b7690 2024-04-25T00:33:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.005+0000 7fb553e00640 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 0x55a1689c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:33:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.006+0000 7fb553400640 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c13c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb54c006010 tx=0x7fb54c0317e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ba511c7b5d242e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.006+0000 7fb53e000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:33:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.006+0000 7fb53e000640 1 -- 172.21.15.42:0/690389528 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c055060 con 0x55a1689b7690 2024-04-25T00:33:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.006+0000 7fb55dc57740 1 -- 172.21.15.42:0/690389528 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb54c036040 con 0x55a1689b7690 2024-04-25T00:33:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.006+0000 7fb53e000640 1 -- 172.21.15.42:0/690389528 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fb54c0045f0 con 0x55a1689b7690 2024-04-25T00:33:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb53e000640 1 -- 172.21.15.42:0/690389528 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb54c0047f0 con 0x55a1689b7690 2024-04-25T00:33:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb55dc57740 10 client.26360 fetch_fsmap learned FSMap version 36 2024-04-25T00:33:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb55dc57740 1 -- 172.21.15.42:0/690389528 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a1689cb350 con 0x55a1689b7690 2024-04-25T00:33:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb53e000640 1 -- 172.21.15.42:0/690389528 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb54c036040 con 0x55a1689b7690 2024-04-25T00:33:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb53e000640 1 -- 172.21.15.42:0/690389528 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb54c030ad0 con 0x55a1689b7690 2024-04-25T00:33:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb55dc57740 10 client.26360 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T00:33:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb55dc57740 10 client.26360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:33:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb55dc57740 20 client.26360 populate_metadata read hostname 'smithi042' 2024-04-25T00:33:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb55dc57740 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1689cc6a0 0x55a1689cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb55dc57740 4 client.26360 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:33:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.008+0000 7fb55dc57740 1 -- 172.21.15.42:0/690389528 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a168926ff0 con 0x55a1689cc6a0 2024-04-25T00:33:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.009+0000 7fb553e00640 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1689cc6a0 0x55a1689cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.010+0000 7fb553e00640 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1689cc6a0 0x55a1689cea90 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b60502f8adfb0ee0 server_cookie=1551a549712ebb44 in_seq=0 out_seq=0 2024-04-25T00:33:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.010+0000 7fb53e000640 10 client.26360 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.012+0000 7fb53e000640 1 -- 172.21.15.42:0/690389528 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a168926ff0 con 0x55a1689cc6a0 2024-04-25T00:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.012+0000 7fb53e000640 10 client.26360 handle_command_reply: tid=0 2024-04-25T00:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.012+0000 7fb55dc57740 1 client.26360 shutdown 2024-04-25T00:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 1 -- 172.21.15.42:0/690389528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 msgr2=0x55a1689c13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c13c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb54c006010 tx=0x7fb54c0317e0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 1 -- 172.21.15.42:0/690389528 shutdown_connections 2024-04-25T00:33:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a1689cc6a0 0x55a1689cea90 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689bc800 0x55a1689c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c13c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 1 --2- 172.21.15.42:0/690389528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b6ca0 0x55a1689c0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 1 -- 172.21.15.42:0/690389528 >> 172.21.15.42:0/690389528 conn(0x55a1689b2310 msgr2=0x55a1689b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb53e000640 0 client.26360 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 1 -- 172.21.15.42:0/690389528 shutdown_connections 2024-04-25T00:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 1 -- 172.21.15.42:0/690389528 wait complete. 2024-04-25T00:33:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.013+0000 7fb55dc57740 20 client.26360 trim_cache size 0 max 16384 2024-04-25T00:33:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.038+0000 7fb55dc57740 1 Processor -- start 2024-04-25T00:33:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.038+0000 7fb55dc57740 1 -- start start 2024-04-25T00:33:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.039+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 0x55a1689b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.039+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b7690 0x55a1689b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.039+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 0x55a1689c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.039+0000 7fb55dc57740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541ba290 con 0x55a1689b7690 2024-04-25T00:33:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.039+0000 7fb55dc57740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541b7a70 con 0x55a1689bc800 2024-04-25T00:33:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.039+0000 7fb55dc57740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb55410dc20 con 0x55a1689b6ca0 2024-04-25T00:33:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.039+0000 7fb553e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 0x55a1689b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.039+0000 7fb553e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 0x55a1689b88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:42062/0 (socket says 172.21.15.42:42062) 2024-04-25T00:33:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.039+0000 7fb553e00640 1 -- 172.21.15.42:0/4126707068 learned_addr learned my addr 172.21.15.42:0/4126707068 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.039+0000 7fb558e00640 1 --2- 172.21.15.42:0/4126707068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 0x55a1689c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.040+0000 7fb553e00640 1 -- 172.21.15.42:0/4126707068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 msgr2=0x55a1689c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.040+0000 7fb553e00640 1 --2- 172.21.15.42:0/4126707068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 0x55a1689c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.040+0000 7fb553e00640 1 -- 172.21.15.42:0/4126707068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b7690 msgr2=0x55a1689b8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.040+0000 7fb553400640 1 --2- 172.21.15.42:0/4126707068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b7690 0x55a1689b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.040+0000 7fb553e00640 1 --2- 172.21.15.42:0/4126707068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b7690 0x55a1689b8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.040+0000 7fb553e00640 1 -- 172.21.15.42:0/4126707068 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1689b2140 con 0x55a1689b6ca0 2024-04-25T00:33:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.040+0000 7fb553400640 1 --2- 172.21.15.42:0/4126707068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b7690 0x55a1689b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.041+0000 7fb553e00640 1 --2- 172.21.15.42:0/4126707068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 0x55a1689b88b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb54009b1b0 tx=0x7fb5400cc570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de3e9fa1c9cd7f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.042+0000 7fb548c00640 1 -- 172.21.15.42:0/4126707068 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb540020040 con 0x55a1689b6ca0 2024-04-25T00:33:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.042+0000 7fb548c00640 1 -- 172.21.15.42:0/4126707068 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb540009de0 con 0x55a1689b6ca0 2024-04-25T00:33:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.042+0000 7fb548c00640 1 -- 172.21.15.42:0/4126707068 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5400cc7b0 con 0x55a1689b6ca0 2024-04-25T00:33:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.043+0000 7fb55dc57740 1 -- 172.21.15.42:0/4126707068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 msgr2=0x55a1689b88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.043+0000 7fb55dc57740 1 --2- 172.21.15.42:0/4126707068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 0x55a1689b88b0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb54009b1b0 tx=0x7fb5400cc570 comp rx=0 tx=0).stop 2024-04-25T00:33:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.043+0000 7fb55dc57740 1 -- 172.21.15.42:0/4126707068 shutdown_connections 2024-04-25T00:33:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.044+0000 7fb55dc57740 1 --2- 172.21.15.42:0/4126707068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689bc800 0x55a1689c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.044+0000 7fb55dc57740 1 --2- 172.21.15.42:0/4126707068 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689b7690 0x55a1689b8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.044+0000 7fb55dc57740 1 --2- 172.21.15.42:0/4126707068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 0x55a1689b88b0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.044+0000 7fb55dc57740 1 -- 172.21.15.42:0/4126707068 >> 172.21.15.42:0/4126707068 conn(0x55a1689b2310 msgr2=0x55a1689c0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.044+0000 7fb55dc57740 1 -- 172.21.15.42:0/4126707068 shutdown_connections 2024-04-25T00:33:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.044+0000 7fb55dc57740 1 -- 172.21.15.42:0/4126707068 wait complete. 2024-04-25T00:33:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.045+0000 7fb55dc57740 1 -- start start 2024-04-25T00:33:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.046+0000 7fb55dc57740 1 Processor -- start 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.046+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 0x55a1689c56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.046+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.046+0000 7fb55dc57740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689bc800 0x55a1689c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.046+0000 7fb55dc57740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb55410e290 con 0x55a1689bc800 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.046+0000 7fb55dc57740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb5541b73a0 con 0x55a1689b7690 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.046+0000 7fb55dc57740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb55410e960 con 0x55a1689b6ca0 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.046+0000 7fb55dc57740 10 client.0 _ll_register_callbacks cb 0x7fb5400d24e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.047+0000 7fb553400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.047+0000 7fb553e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 0x55a1689c56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.047+0000 7fb553400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59320/0 (socket says 172.21.15.42:59320) 2024-04-25T00:33:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.047+0000 7fb553400640 1 -- 172.21.15.42:0/3122108212 learned_addr learned my addr 172.21.15.42:0/3122108212 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.047+0000 7fb558e00640 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689bc800 0x55a1689c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.047+0000 7fb553400640 1 -- 172.21.15.42:0/3122108212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 msgr2=0x55a1689c56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.047+0000 7fb553400640 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 0x55a1689c56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.047+0000 7fb553400640 1 -- 172.21.15.42:0/3122108212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689bc800 msgr2=0x55a1689c6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.048+0000 7fb553400640 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689bc800 0x55a1689c6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.048+0000 7fb553400640 1 -- 172.21.15.42:0/3122108212 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1689ad620 con 0x55a1689b7690 2024-04-25T00:33:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.048+0000 7fb558e00640 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689bc800 0x55a1689c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:33:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.048+0000 7fb553400640 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c5bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb54c031040 tx=0x7fb54c009b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6c584f689b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.049+0000 7fb53e000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:33:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.049+0000 7fb53e000640 1 -- 172.21.15.42:0/3122108212 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c060070 con 0x55a1689b7690 2024-04-25T00:33:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.049+0000 7fb53e000640 1 -- 172.21.15.42:0/3122108212 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7fb54c0045f0 con 0x55a1689b7690 2024-04-25T00:33:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.049+0000 7fb55dc57740 1 -- 172.21.15.42:0/3122108212 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb540009de0 con 0x55a1689b7690 2024-04-25T00:33:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.049+0000 7fb53e000640 1 -- 172.21.15.42:0/3122108212 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb54c055030 con 0x55a1689b7690 2024-04-25T00:33:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.049+0000 7fb55dc57740 10 client.26366 fetch_fsmap learned FSMap version 36 2024-04-25T00:33:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.049+0000 7fb55dc57740 1 -- 172.21.15.42:0/3122108212 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55a1689c8280 con 0x55a1689b7690 2024-04-25T00:33:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.050+0000 7fb53e000640 1 -- 172.21.15.42:0/3122108212 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb540009de0 con 0x55a1689b7690 2024-04-25T00:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.050+0000 7fb53e000640 1 -- 172.21.15.42:0/3122108212 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb54c003970 con 0x55a1689b7690 2024-04-25T00:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.050+0000 7fb55dc57740 10 client.26366 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T00:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.050+0000 7fb55dc57740 10 client.26366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.050+0000 7fb55dc57740 20 client.26366 populate_metadata read hostname 'smithi042' 2024-04-25T00:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.050+0000 7fb55dc57740 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a168a32ee0 0x55a168a35300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.050+0000 7fb55dc57740 4 client.26366 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.050+0000 7fb55dc57740 1 -- 172.21.15.42:0/3122108212 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a168921340 con 0x55a168a32ee0 2024-04-25T00:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.050+0000 7fb553e00640 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a168a32ee0 0x55a168a35300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.051+0000 7fb553e00640 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a168a32ee0 0x55a168a35300 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27748a310a67b0ef server_cookie=a544a730e768dc3b in_seq=0 out_seq=0 2024-04-25T00:33:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.051+0000 7fb53e000640 10 client.26366 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.306+0000 7fb53e000640 1 -- 172.21.15.42:0/3122108212 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a168921340 con 0x55a168a32ee0 2024-04-25T00:33:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.307+0000 7fb53e000640 10 client.26366 handle_command_reply: tid=0 2024-04-25T00:33:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.307+0000 7fb55dc57740 1 client.26366 shutdown 2024-04-25T00:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 1 -- 172.21.15.42:0/3122108212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 msgr2=0x55a1689c5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c5bf0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb54c031040 tx=0x7fb54c009b40 comp rx=0 tx=0).stop 2024-04-25T00:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 1 -- 172.21.15.42:0/3122108212 shutdown_connections 2024-04-25T00:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a168a32ee0 0x55a168a35300 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a1689bc800 0x55a1689c6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a1689b7690 0x55a1689c5bf0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 1 --2- 172.21.15.42:0/3122108212 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a1689b6ca0 0x55a1689c56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb53e000640 0 client.26366 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 1 -- 172.21.15.42:0/3122108212 >> 172.21.15.42:0/3122108212 conn(0x55a1689b2310 msgr2=0x55a1689c0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 1 -- 172.21.15.42:0/3122108212 shutdown_connections 2024-04-25T00:33:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 1 -- 172.21.15.42:0/3122108212 wait complete. 2024-04-25T00:33:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.311+0000 7fb55dc57740 20 client.26366 trim_cache size 0 max 16384 2024-04-25T00:33:39.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:33:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:33:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:33:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:33:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.312+0000 7fb55ae00640 1 -- 172.21.15.42:0/2666414766 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb5380a2cb0 msgr2=0x7fb5380a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.312+0000 7fb55ae00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb5380a2cb0 0x7fb5380a5170 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x55a1689b7ef0 tx=0x7fb544005f90 comp rx=0 tx=0).stop 2024-04-25T00:33:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.312+0000 7fb55ae00640 1 -- 172.21.15.42:0/2666414766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100010 msgr2=0x7fb55419dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:39.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.312+0000 7fb55ae00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100010 0x7fb55419dfe0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb54000d950 tx=0x7fb54000de10 comp rx=0 tx=0).stop 2024-04-25T00:33:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.313+0000 7fb55ae00640 1 -- 172.21.15.42:0/2666414766 shutdown_connections 2024-04-25T00:33:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.313+0000 7fb55ae00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb5380a2cb0 0x7fb5380a5170 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.313+0000 7fb55ae00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb554105330 0x7fb5541a5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.313+0000 7fb55ae00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb554102980 0x7fb55419e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.313+0000 7fb55ae00640 1 --2- 172.21.15.42:0/2666414766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb554100010 0x7fb55419dfe0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.313+0000 7fb55ae00640 1 -- 172.21.15.42:0/2666414766 >> 172.21.15.42:0/2666414766 conn(0x7fb5540fbc10 msgr2=0x7fb554100950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.313+0000 7fb55ae00640 1 -- 172.21.15.42:0/2666414766 shutdown_connections 2024-04-25T00:33:39.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:39.313+0000 7fb55ae00640 1 -- 172.21.15.42:0/2666414766 wait complete. 2024-04-25T00:33:39.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 55913 35.83 MB/sec warmup 89 sec latency 185.417 ms 2024-04-25T00:33:39.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:39 smithi042 ceph-mon[28130]: pgmap v835: 97 pgs: 97 active+clean; 472 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 711 op/s 2024-04-25T00:33:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:39 smithi136 ceph-mon[32923]: pgmap v835: 97 pgs: 97 active+clean; 472 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 711 op/s 2024-04-25T00:33:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:39 smithi106 ceph-mon[32510]: pgmap v835: 97 pgs: 97 active+clean; 472 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 9.5 MiB/s wr, 711 op/s 2024-04-25T00:33:40.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 56608 35.72 MB/sec warmup 90 sec latency 218.959 ms 2024-04-25T00:33:41.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 57292 35.78 MB/sec warmup 91 sec latency 340.260 ms 2024-04-25T00:33:41.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:41 smithi042 ceph-mon[28130]: pgmap v836: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-04-25T00:33:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:41 smithi136 ceph-mon[32923]: pgmap v836: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-04-25T00:33:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:41 smithi106 ceph-mon[32510]: pgmap v836: 97 pgs: 97 active+clean; 465 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 10 MiB/s wr, 762 op/s 2024-04-25T00:33:42.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 58003 35.86 MB/sec warmup 92 sec latency 246.224 ms 2024-04-25T00:33:43.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 58501 35.83 MB/sec warmup 93 sec latency 358.015 ms 2024-04-25T00:33:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:43 smithi136 ceph-mon[32923]: pgmap v837: 97 pgs: 97 active+clean; 465 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 861 op/s 2024-04-25T00:33:44.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:43 smithi106 ceph-mon[32510]: pgmap v837: 97 pgs: 97 active+clean; 465 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 861 op/s 2024-04-25T00:33:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:43 smithi042 ceph-mon[28130]: pgmap v837: 97 pgs: 97 active+clean; 465 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 861 op/s 2024-04-25T00:33:44.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 59007 35.77 MB/sec warmup 94 sec latency 88.284 ms 2024-04-25T00:33:45.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 59584 35.64 MB/sec warmup 95 sec latency 75.259 ms 2024-04-25T00:33:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:45 smithi136 ceph-mon[32923]: pgmap v838: 97 pgs: 97 active+clean; 465 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.1 MiB/s wr, 723 op/s 2024-04-25T00:33:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:45 smithi106 ceph-mon[32510]: pgmap v838: 97 pgs: 97 active+clean; 465 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.1 MiB/s wr, 723 op/s 2024-04-25T00:33:46.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:45 smithi042 ceph-mon[28130]: pgmap v838: 97 pgs: 97 active+clean; 465 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 9.1 MiB/s wr, 723 op/s 2024-04-25T00:33:46.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 60351 35.65 MB/sec warmup 96 sec latency 101.629 ms 2024-04-25T00:33:47.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 61244 35.76 MB/sec warmup 97 sec latency 63.010 ms 2024-04-25T00:33:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:47 smithi136 ceph-mon[32923]: pgmap v839: 97 pgs: 97 active+clean; 471 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 982 op/s 2024-04-25T00:33:48.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:47 smithi106 ceph-mon[32510]: pgmap v839: 97 pgs: 97 active+clean; 471 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 982 op/s 2024-04-25T00:33:48.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:47 smithi042 ceph-mon[28130]: pgmap v839: 97 pgs: 97 active+clean; 471 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 982 op/s 2024-04-25T00:33:48.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 61633 35.59 MB/sec warmup 98 sec latency 489.444 ms 2024-04-25T00:33:49.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 61775 35.32 MB/sec warmup 99 sec latency 746.331 ms 2024-04-25T00:33:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:49 smithi136 ceph-mon[32923]: pgmap v840: 97 pgs: 97 active+clean; 469 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 799 op/s 2024-04-25T00:33:50.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:49 smithi106 ceph-mon[32510]: pgmap v840: 97 pgs: 97 active+clean; 469 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 799 op/s 2024-04-25T00:33:50.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:49 smithi042 ceph-mon[28130]: pgmap v840: 97 pgs: 97 active+clean; 469 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 12 MiB/s wr, 799 op/s 2024-04-25T00:33:50.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 62194 35.28 MB/sec warmup 100 sec latency 123.297 ms 2024-04-25T00:33:51.742 INFO:tasks.workunit.client.0.smithi042.stdout: 10 62844 35.30 MB/sec warmup 101 sec latency 56.749 ms 2024-04-25T00:33:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:51 smithi136 ceph-mon[32923]: pgmap v841: 97 pgs: 97 active+clean; 495 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 886 op/s 2024-04-25T00:33:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:51 smithi106 ceph-mon[32510]: pgmap v841: 97 pgs: 97 active+clean; 495 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 886 op/s 2024-04-25T00:33:52.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:51 smithi042 ceph-mon[28130]: pgmap v841: 97 pgs: 97 active+clean; 495 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 16 MiB/s wr, 886 op/s 2024-04-25T00:33:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.509+0000 7f921ec00640 1 -- 172.21.15.42:0/3368220798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9218102b20 msgr2=0x7f9218102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.511+0000 7f921ec00640 1 --2- 172.21.15.42:0/3368220798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9218102b20 0x7f9218102fa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9208009900 tx=0x7f920802a400 comp rx=0 tx=0).stop 2024-04-25T00:33:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.512+0000 7f921ec00640 1 -- 172.21.15.42:0/3368220798 shutdown_connections 2024-04-25T00:33:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.512+0000 7f921ec00640 1 --2- 172.21.15.42:0/3368220798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92181034e0 0x7f9218109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.512+0000 7f921ec00640 1 --2- 172.21.15.42:0/3368220798 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9218102b20 0x7f9218102fa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.512+0000 7f921ec00640 1 --2- 172.21.15.42:0/3368220798 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f92180ffab0 0x7f92180ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.512+0000 7f921ec00640 1 -- 172.21.15.42:0/3368220798 >> 172.21.15.42:0/3368220798 conn(0x7f92180fb3c0 msgr2=0x7f92180fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.512+0000 7f921ec00640 1 -- 172.21.15.42:0/3368220798 shutdown_connections 2024-04-25T00:33:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.513+0000 7f921ec00640 1 -- 172.21.15.42:0/3368220798 wait complete. 2024-04-25T00:33:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.514+0000 7f921ec00640 1 Processor -- start 2024-04-25T00:33:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.515+0000 7f921ec00640 1 -- start start 2024-04-25T00:33:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.515+0000 7f921ec00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92180ffab0 0x7f921819e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.515+0000 7f921ec00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9218102b20 0x7f921819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.515+0000 7f921ec00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92181034e0 0x7f921806d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.515+0000 7f921ec00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f921810f790 con 0x7f9218102b20 2024-04-25T00:33:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.515+0000 7f921ec00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f921810f610 con 0x7f92180ffab0 2024-04-25T00:33:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.515+0000 7f921ec00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f921810f910 con 0x7f92181034e0 2024-04-25T00:33:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.516+0000 7f9217e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92180ffab0 0x7f921819e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.516+0000 7f9217e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92180ffab0 0x7f921819e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:36402/0 (socket says 172.21.15.42:36402) 2024-04-25T00:33:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.516+0000 7f9217e00640 1 -- 172.21.15.42:0/4214913610 learned_addr learned my addr 172.21.15.42:0/4214913610 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.516+0000 7f921cc00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92181034e0 0x7f921806d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.516+0000 7f9217400640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9218102b20 0x7f921819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.517+0000 7f9217400640 1 -- 172.21.15.42:0/4214913610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92181034e0 msgr2=0x7f921806d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.517+0000 7f9217400640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92181034e0 0x7f921806d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.517+0000 7f9217400640 1 -- 172.21.15.42:0/4214913610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92180ffab0 msgr2=0x7f921819e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.517+0000 7f9217400640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92180ffab0 0x7f921819e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.517+0000 7f9217400640 1 -- 172.21.15.42:0/4214913610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f921806d7e0 con 0x7f9218102b20 2024-04-25T00:33:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.517+0000 7f9217e00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92180ffab0 0x7f921819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:33:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.518+0000 7f9217400640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9218102b20 0x7f921819e6e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f920802a920 tx=0x7f9208032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.519+0000 7f9214c00640 1 -- 172.21.15.42:0/4214913610 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f920803f070 con 0x7f9218102b20 2024-04-25T00:33:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.519+0000 7f9214c00640 1 -- 172.21.15.42:0/4214913610 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f920803b460 con 0x7f9218102b20 2024-04-25T00:33:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.519+0000 7f921ec00640 1 -- 172.21.15.42:0/4214913610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921806da70 con 0x7f9218102b20 2024-04-25T00:33:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.519+0000 7f9214c00640 1 -- 172.21.15.42:0/4214913610 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208035070 con 0x7f9218102b20 2024-04-25T00:33:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.519+0000 7f921ec00640 1 -- 172.21.15.42:0/4214913610 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921806e000 con 0x7f9218102b20 2024-04-25T00:33:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.524+0000 7f92219eb740 1 Processor -- start 2024-04-25T00:33:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.524+0000 7f92219eb740 1 -- start start 2024-04-25T00:33:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.525+0000 7f92219eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.525+0000 7f92219eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6ef690 0x55dc4c6f4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.525+0000 7f92219eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6f4800 0x55dc4c6f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.525+0000 7f92219eb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c651ab0 con 0x55dc4c6eeca0 2024-04-25T00:33:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.525+0000 7f92219eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c5fa640 con 0x55dc4c6ef690 2024-04-25T00:33:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.525+0000 7f92219eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c64da70 con 0x55dc4c6f4800 2024-04-25T00:33:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.525+0000 7f9217e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6ef690 0x55dc4c6f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.525+0000 7f9217e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6ef690 0x55dc4c6f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:36408/0 (socket says 172.21.15.42:36408) 2024-04-25T00:33:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.525+0000 7f9217e00640 1 -- 172.21.15.42:0/2256129257 learned_addr learned my addr 172.21.15.42:0/2256129257 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.526+0000 7f9217e00640 1 -- 172.21.15.42:0/2256129257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6f4800 msgr2=0x55dc4c6f6c90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:33:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.526+0000 7f9217e00640 1 --2- 172.21.15.42:0/2256129257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6f4800 0x55dc4c6f6c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.526+0000 7f9217e00640 1 -- 172.21.15.42:0/2256129257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 msgr2=0x55dc4c6ef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.526+0000 7f921cc00640 1 --2- 172.21.15.42:0/2256129257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.526+0000 7f9217e00640 1 --2- 172.21.15.42:0/2256129257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6ef0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.526+0000 7f9217e00640 1 -- 172.21.15.42:0/2256129257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc4c366210 con 0x55dc4c6ef690 2024-04-25T00:33:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.526+0000 7f921cc00640 1 --2- 172.21.15.42:0/2256129257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:33:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.527+0000 7f9217e00640 1 --2- 172.21.15.42:0/2256129257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6ef690 0x55dc4c6f4190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f921000c570 tx=0x7f9210009760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0616d59d7216c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.527+0000 7f9214c00640 1 -- 172.21.15.42:0/4214913610 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f920803b910 con 0x7f9218102b20 2024-04-25T00:33:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.529+0000 7f9214c00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91fc0a2cb0 0x7f91fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.529+0000 7f920d200640 1 -- 172.21.15.42:0/2256129257 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210018020 con 0x55dc4c6ef690 2024-04-25T00:33:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.529+0000 7f9214c00640 1 -- 172.21.15.42:0/4214913610 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f92080ef070 con 0x7f9218102b20 2024-04-25T00:33:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.529+0000 7f920d200640 1 -- 172.21.15.42:0/2256129257 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9210014d70 con 0x55dc4c6ef690 2024-04-25T00:33:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.529+0000 7f920d200640 1 -- 172.21.15.42:0/2256129257 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210007340 con 0x55dc4c6ef690 2024-04-25T00:33:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.530+0000 7f92219eb740 1 -- 172.21.15.42:0/2256129257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6ef690 msgr2=0x55dc4c6f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.530+0000 7f92219eb740 1 --2- 172.21.15.42:0/2256129257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6ef690 0x55dc4c6f4190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f921000c570 tx=0x7f9210009760 comp rx=0 tx=0).stop 2024-04-25T00:33:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.532+0000 7f921cc00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91fc0a2cb0 0x7f91fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.532+0000 7f92219eb740 1 -- 172.21.15.42:0/2256129257 shutdown_connections 2024-04-25T00:33:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.532+0000 7f92219eb740 1 --2- 172.21.15.42:0/2256129257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6f4800 0x55dc4c6f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.532+0000 7f92219eb740 1 --2- 172.21.15.42:0/2256129257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6ef690 0x55dc4c6f4190 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.532+0000 7f92219eb740 1 --2- 172.21.15.42:0/2256129257 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.532+0000 7f92219eb740 1 -- 172.21.15.42:0/2256129257 >> 172.21.15.42:0/2256129257 conn(0x55dc4c6ea310 msgr2=0x55dc4c6ea710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.533+0000 7f92219eb740 1 -- 172.21.15.42:0/2256129257 shutdown_connections 2024-04-25T00:33:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.534+0000 7f92219eb740 1 -- 172.21.15.42:0/2256129257 wait complete. 2024-04-25T00:33:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.534+0000 7f92219eb740 1 -- start start 2024-04-25T00:33:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.536+0000 7f921cc00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91fc0a2cb0 0x7f91fc0a5170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55dc4c6efef0 tx=0x7f9204008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:52.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.536+0000 7f92219eb740 1 Processor -- start 2024-04-25T00:33:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f92219eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6f9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f92219eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6ef690 0x55dc4c6f9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f92219eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6f4800 0x55dc4c700510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f92219eb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c6f77b0 con 0x55dc4c6ef690 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f92219eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c60d990 con 0x55dc4c6f4800 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f92219eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dc4c6f7930 con 0x55dc4c6eeca0 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f9217e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f9217400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6ef690 0x55dc4c6f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f9217e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6f9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33768/0 (socket says 172.21.15.42:33768) 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f9217e00640 1 -- 172.21.15.42:0/1321763285 learned_addr learned my addr 172.21.15.42:0/1321763285 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f92219eb740 10 client.0 _ll_register_callbacks cb 0x55dc4c60f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.537+0000 7f921cc00640 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6f4800 0x55dc4c700510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.538+0000 7f9217e00640 1 -- 172.21.15.42:0/1321763285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6f4800 msgr2=0x55dc4c700510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:52.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.538+0000 7f9217e00640 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6f4800 0x55dc4c700510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.538+0000 7f9217e00640 1 -- 172.21.15.42:0/1321763285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6ef690 msgr2=0x55dc4c6f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.538+0000 7f9217e00640 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6ef690 0x55dc4c6f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.538+0000 7f9217e00640 1 -- 172.21.15.42:0/1321763285 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4c432620 con 0x55dc4c6eeca0 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.539+0000 7f9217e00640 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6f9000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9210004b90 tx=0x7f9210013690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa17f3ae8ca28e9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.539+0000 7f9201600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.539+0000 7f9201600640 1 -- 172.21.15.42:0/1321763285 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210018040 con 0x55dc4c6eeca0 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.539+0000 7f9201600640 1 -- 172.21.15.42:0/1321763285 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f9210014890 con 0x55dc4c6eeca0 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.539+0000 7f92219eb740 1 -- 172.21.15.42:0/1321763285 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9210014d70 con 0x55dc4c6eeca0 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.540+0000 7f9201600640 1 -- 172.21.15.42:0/1321763285 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9210014a90 con 0x55dc4c6eeca0 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.540+0000 7f92219eb740 10 client.26380 fetch_fsmap learned FSMap version 36 2024-04-25T00:33:52.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.540+0000 7f92219eb740 1 -- 172.21.15.42:0/1321763285 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dc4c7034b0 con 0x55dc4c6eeca0 2024-04-25T00:33:52.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.540+0000 7f9201600640 1 -- 172.21.15.42:0/1321763285 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9210014d70 con 0x55dc4c6eeca0 2024-04-25T00:33:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.540+0000 7f9201600640 1 -- 172.21.15.42:0/1321763285 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f92100274c0 con 0x55dc4c6eeca0 2024-04-25T00:33:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.541+0000 7f92219eb740 10 client.26380 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T00:33:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.541+0000 7f92219eb740 10 client.26380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:33:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.541+0000 7f92219eb740 20 client.26380 populate_metadata read hostname 'smithi042' 2024-04-25T00:33:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.541+0000 7f92219eb740 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dc4c7049e0 0x55dc4c706df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.541+0000 7f92219eb740 4 client.26380 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:33:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.541+0000 7f92219eb740 1 -- 172.21.15.42:0/1321763285 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc4c65eff0 con 0x55dc4c7049e0 2024-04-25T00:33:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.541+0000 7f9217400640 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dc4c7049e0 0x55dc4c706df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.542+0000 7f9217400640 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dc4c7049e0 0x55dc4c706df0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0b313db36ef90be server_cookie=2af1263b5045ed66 in_seq=0 out_seq=0 2024-04-25T00:33:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.542+0000 7f9201600640 10 client.26380 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.545+0000 7f9201600640 1 -- 172.21.15.42:0/1321763285 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc4c65eff0 con 0x55dc4c7049e0 2024-04-25T00:33:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.545+0000 7f9201600640 10 client.26380 handle_command_reply: tid=0 2024-04-25T00:33:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 client.26380 shutdown 2024-04-25T00:33:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 -- 172.21.15.42:0/1321763285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6eeca0 msgr2=0x55dc4c6f9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6f9000 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9210004b90 tx=0x7f9210013690 comp rx=0 tx=0).stop 2024-04-25T00:33:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 -- 172.21.15.42:0/1321763285 shutdown_connections 2024-04-25T00:33:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dc4c7049e0 0x55dc4c706df0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6f4800 0x55dc4c700510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6ef690 0x55dc4c6f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 --2- 172.21.15.42:0/1321763285 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6f9000 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 -- 172.21.15.42:0/1321763285 >> 172.21.15.42:0/1321763285 conn(0x55dc4c6ea310 msgr2=0x55dc4c6e9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:52.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f9201600640 0 client.26380 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 -- 172.21.15.42:0/1321763285 shutdown_connections 2024-04-25T00:33:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.546+0000 7f92219eb740 1 -- 172.21.15.42:0/1321763285 wait complete. 2024-04-25T00:33:52.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.547+0000 7f92219eb740 20 client.26380 trim_cache size 0 max 16384 2024-04-25T00:33:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.573+0000 7f92219eb740 1 Processor -- start 2024-04-25T00:33:52.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f92219eb740 1 -- start start 2024-04-25T00:33:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f92219eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6eda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f92219eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6ef690 0x55dc4c7004d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f92219eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6f4800 0x55dc4c6f8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f92219eb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f92181ba430 con 0x55dc4c6eeca0 2024-04-25T00:33:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f92219eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b7c10 con 0x55dc4c6f4800 2024-04-25T00:33:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f92219eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b5a60 con 0x55dc4c6ef690 2024-04-25T00:33:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f9217400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6ef690 0x55dc4c7004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f9217e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6eda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f9217400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6ef690 0x55dc4c7004d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33784/0 (socket says 172.21.15.42:33784) 2024-04-25T00:33:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f9217400640 1 -- 172.21.15.42:0/2573646421 learned_addr learned my addr 172.21.15.42:0/2573646421 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.574+0000 7f921cc00640 1 --2- 172.21.15.42:0/2573646421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6f4800 0x55dc4c6f8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.575+0000 7f9217400640 1 -- 172.21.15.42:0/2573646421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6f4800 msgr2=0x55dc4c6f8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.575+0000 7f9217400640 1 --2- 172.21.15.42:0/2573646421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6f4800 0x55dc4c6f8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.575+0000 7f9217400640 1 -- 172.21.15.42:0/2573646421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 msgr2=0x55dc4c6eda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.575+0000 7f9217400640 1 --2- 172.21.15.42:0/2573646421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6eda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.575+0000 7f9217400640 1 -- 172.21.15.42:0/2573646421 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc4c6e5620 con 0x55dc4c6ef690 2024-04-25T00:33:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.575+0000 7f9217400640 1 --2- 172.21.15.42:0/2573646421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6ef690 0x55dc4c7004d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f92080b7e10 tx=0x7f920803bcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cbe0ef6709ad2aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.576+0000 7f9203e00640 1 -- 172.21.15.42:0/2573646421 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9208048040 con 0x55dc4c6ef690 2024-04-25T00:33:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.576+0000 7f9203e00640 1 -- 172.21.15.42:0/2573646421 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92080c4570 con 0x55dc4c6ef690 2024-04-25T00:33:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.576+0000 7f9203e00640 1 -- 172.21.15.42:0/2573646421 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92080bbe70 con 0x55dc4c6ef690 2024-04-25T00:33:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.577+0000 7f92219eb740 1 -- 172.21.15.42:0/2573646421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6ef690 msgr2=0x55dc4c7004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.577+0000 7f92219eb740 1 --2- 172.21.15.42:0/2573646421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6ef690 0x55dc4c7004d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f92080b7e10 tx=0x7f920803bcb0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.577+0000 7f92219eb740 1 -- 172.21.15.42:0/2573646421 shutdown_connections 2024-04-25T00:33:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.577+0000 7f92219eb740 1 --2- 172.21.15.42:0/2573646421 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6f4800 0x55dc4c6f8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.577+0000 7f92219eb740 1 --2- 172.21.15.42:0/2573646421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6ef690 0x55dc4c7004d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.577+0000 7f92219eb740 1 --2- 172.21.15.42:0/2573646421 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6eda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.577+0000 7f92219eb740 1 -- 172.21.15.42:0/2573646421 >> 172.21.15.42:0/2573646421 conn(0x55dc4c6ea310 msgr2=0x55dc4c6ed300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.577+0000 7f92219eb740 1 -- 172.21.15.42:0/2573646421 shutdown_connections 2024-04-25T00:33:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.577+0000 7f92219eb740 1 -- 172.21.15.42:0/2573646421 wait complete. 2024-04-25T00:33:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.578+0000 7f92219eb740 1 -- start start 2024-04-25T00:33:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.578+0000 7f92219eb740 1 Processor -- start 2024-04-25T00:33:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.578+0000 7f92219eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6fdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.578+0000 7f92219eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6ef690 0x55dc4c6fe060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.578+0000 7f92219eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6f4800 0x55dc4c6ff070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.578+0000 7f92219eb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b60d0 con 0x55dc4c6ef690 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.578+0000 7f92219eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b7540 con 0x55dc4c6eeca0 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.578+0000 7f92219eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f92181b67a0 con 0x55dc4c6f4800 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f92219eb740 10 client.0 _ll_register_callbacks cb 0x7f921000c470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f9217e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6fdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f9217e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6fdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:36426/0 (socket says 172.21.15.42:36426) 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f9217e00640 1 -- 172.21.15.42:0/1812778111 learned_addr learned my addr 172.21.15.42:0/1812778111 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f921cc00640 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6f4800 0x55dc4c6ff070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f9217400640 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6ef690 0x55dc4c6fe060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f9217e00640 1 -- 172.21.15.42:0/1812778111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6f4800 msgr2=0x55dc4c6ff070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f9217e00640 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6f4800 0x55dc4c6ff070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f9217e00640 1 -- 172.21.15.42:0/1812778111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6ef690 msgr2=0x55dc4c6fe060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f9217e00640 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6ef690 0x55dc4c6fe060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f9217e00640 1 -- 172.21.15.42:0/1812778111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc4c6ff660 con 0x55dc4c6eeca0 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.579+0000 7f9217e00640 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6fdb20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9210002230 tx=0x7f9210013770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c4ce46b471ed152 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.580+0000 7f9201600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.580+0000 7f9201600640 1 -- 172.21.15.42:0/1812778111 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210018040 con 0x55dc4c6eeca0 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.580+0000 7f9201600640 1 -- 172.21.15.42:0/1812778111 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8182+0+0 (secure 0 0 0) 0x7f9210014690 con 0x55dc4c6eeca0 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.580+0000 7f92219eb740 1 -- 172.21.15.42:0/1812778111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92080c4570 con 0x55dc4c6eeca0 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.580+0000 7f9201600640 1 -- 172.21.15.42:0/1812778111 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9210027d70 con 0x55dc4c6eeca0 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.580+0000 7f92219eb740 10 client.26384 fetch_fsmap learned FSMap version 36 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.580+0000 7f92219eb740 1 -- 172.21.15.42:0/1812778111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dc4c76af70 con 0x55dc4c6eeca0 2024-04-25T00:33:52.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.580+0000 7f9201600640 1 -- 172.21.15.42:0/1812778111 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92080c4570 con 0x55dc4c6eeca0 2024-04-25T00:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.581+0000 7f9201600640 1 -- 172.21.15.42:0/1812778111 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f9210045430 con 0x55dc4c6eeca0 2024-04-25T00:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.582+0000 7f92219eb740 10 client.26384 fetch_fsmap finished waiting for FSMap version 36 2024-04-25T00:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.582+0000 7f92219eb740 10 client.26384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.582+0000 7f92219eb740 20 client.26384 populate_metadata read hostname 'smithi042' 2024-04-25T00:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.582+0000 7f92219eb740 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dc4c76b7e0 0x55dc4c76dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.582+0000 7f92219eb740 4 client.26384 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.582+0000 7f92219eb740 1 -- 172.21.15.42:0/1812778111 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dc4c659340 con 0x55dc4c76b7e0 2024-04-25T00:33:52.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.582+0000 7f9217400640 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dc4c76b7e0 0x55dc4c76dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:33:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.583+0000 7f9217400640 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dc4c76b7e0 0x55dc4c76dc00 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e392aecfd7e1029 server_cookie=2170344995315f58 in_seq=0 out_seq=0 2024-04-25T00:33:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.583+0000 7f9201600640 10 client.26384 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:52.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 63378 35.25 MB/sec warmup 102 sec latency 130.623 ms 2024-04-25T00:33:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.813+0000 7f9201600640 1 -- 172.21.15.42:0/1812778111 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dc4c659340 con 0x55dc4c76b7e0 2024-04-25T00:33:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.813+0000 7f9201600640 10 client.26384 handle_command_reply: tid=0 2024-04-25T00:33:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.813+0000 7f92219eb740 1 client.26384 shutdown 2024-04-25T00:33:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 1 -- 172.21.15.42:0/1812778111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6eeca0 msgr2=0x55dc4c6fdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6fdb20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f9210002230 tx=0x7f9210013770 comp rx=0 tx=0).stop 2024-04-25T00:33:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 1 -- 172.21.15.42:0/1812778111 shutdown_connections 2024-04-25T00:33:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55dc4c76b7e0 0x55dc4c76dc00 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dc4c6f4800 0x55dc4c6ff070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55dc4c6ef690 0x55dc4c6fe060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 1 --2- 172.21.15.42:0/1812778111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55dc4c6eeca0 0x55dc4c6fdb20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 1 -- 172.21.15.42:0/1812778111 >> 172.21.15.42:0/1812778111 conn(0x55dc4c6ea310 msgr2=0x55dc4c6e8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f9201600640 0 client.26384 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:33:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 1 -- 172.21.15.42:0/1812778111 shutdown_connections 2024-04-25T00:33:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 1 -- 172.21.15.42:0/1812778111 wait complete. 2024-04-25T00:33:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.814+0000 7f92219eb740 20 client.26384 trim_cache size 0 max 16384 2024-04-25T00:33:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:33:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:33:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:33:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:33:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.815+0000 7f921ec00640 1 -- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91fc0a2cb0 msgr2=0x7f91fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.815+0000 7f921ec00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91fc0a2cb0 0x7f91fc0a5170 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55dc4c6efef0 tx=0x7f9204008040 comp rx=0 tx=0).stop 2024-04-25T00:33:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.815+0000 7f921ec00640 1 -- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9218102b20 msgr2=0x7f921819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:33:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.815+0000 7f921ec00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9218102b20 0x7f921819e6e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f920802a920 tx=0x7f9208032da0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.815+0000 7f921ec00640 1 -- 172.21.15.42:0/4214913610 shutdown_connections 2024-04-25T00:33:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.815+0000 7f921ec00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f91fc0a2cb0 0x7f91fc0a5170 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.815+0000 7f921ec00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f92181034e0 0x7f921806d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.815+0000 7f921ec00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9218102b20 0x7f921819e6e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.815+0000 7f921ec00640 1 --2- 172.21.15.42:0/4214913610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92180ffab0 0x7f921819e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:33:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.815+0000 7f921ec00640 1 -- 172.21.15.42:0/4214913610 >> 172.21.15.42:0/4214913610 conn(0x7f92180fb3c0 msgr2=0x7f9218105dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:33:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.816+0000 7f921ec00640 1 -- 172.21.15.42:0/4214913610 shutdown_connections 2024-04-25T00:33:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:33:52.816+0000 7f921ec00640 1 -- 172.21.15.42:0/4214913610 wait complete. 2024-04-25T00:33:53.263 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:33:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: ignoring --setuser ceph since I am not root 2024-04-25T00:33:53.263 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:33:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: ignoring --setgroup ceph since I am not root 2024-04-25T00:33:53.263 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:33:53 smithi136 ceph-mds[63607]: -- 172.21.15.136:0/1684942214 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557f06f57a40 con 0x557f06f2a800 2024-04-25T00:33:53.264 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:33:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: starting mds.i at 2024-04-25T00:33:53.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 64015 35.13 MB/sec warmup 103 sec latency 176.912 ms 2024-04-25T00:33:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:53 smithi136 ceph-mon[32923]: pgmap v842: 97 pgs: 97 active+clean; 492 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 22 MiB/s wr, 1.11k op/s 2024-04-25T00:33:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:53 smithi136 ceph-mon[32923]: daemon mds.i restarted 2024-04-25T00:33:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:53 smithi106 ceph-mon[32510]: pgmap v842: 97 pgs: 97 active+clean; 492 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 22 MiB/s wr, 1.11k op/s 2024-04-25T00:33:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:53 smithi106 ceph-mon[32510]: daemon mds.i restarted 2024-04-25T00:33:54.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:53 smithi042 ceph-mon[28130]: pgmap v842: 97 pgs: 97 active+clean; 492 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 22 MiB/s wr, 1.11k op/s 2024-04-25T00:33:54.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:53 smithi042 ceph-mon[28130]: daemon mds.i restarted 2024-04-25T00:33:54.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 64830 35.27 MB/sec warmup 104 sec latency 66.112 ms 2024-04-25T00:33:55.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:54 smithi136 ceph-mon[32923]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:33:55.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:54 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6838/4103858691,v1:172.21.15.136:6839/4103858691] up:boot 2024-04-25T00:33:55.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:54 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:33:55.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:33:55.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:54 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:33:55.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:54 smithi106 ceph-mon[32510]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:33:55.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:54 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6838/4103858691,v1:172.21.15.136:6839/4103858691] up:boot 2024-04-25T00:33:55.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:54 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:33:55.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:33:55.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:54 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:33:55.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:54 smithi042 ceph-mon[28130]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T00:33:55.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:54 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6838/4103858691,v1:172.21.15.136:6839/4103858691] up:boot 2024-04-25T00:33:55.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:54 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:33:55.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:33:55.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:54 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:33:55.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 65654 35.33 MB/sec warmup 105 sec latency 37.297 ms 2024-04-25T00:33:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:55 smithi136 ceph-mon[32923]: pgmap v844: 97 pgs: 97 active+clean; 484 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 24 MiB/s wr, 1.06k op/s 2024-04-25T00:33:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:55 smithi106 ceph-mon[32510]: pgmap v844: 97 pgs: 97 active+clean; 484 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 24 MiB/s wr, 1.06k op/s 2024-04-25T00:33:56.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:55 smithi042 ceph-mon[28130]: pgmap v844: 97 pgs: 97 active+clean; 484 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 24 MiB/s wr, 1.06k op/s 2024-04-25T00:33:56.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 66377 35.42 MB/sec warmup 106 sec latency 65.023 ms 2024-04-25T00:33:57.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 66982 35.42 MB/sec warmup 107 sec latency 89.579 ms 2024-04-25T00:33:58.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:57 smithi136 ceph-mon[32923]: pgmap v845: 97 pgs: 97 active+clean; 465 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 19 MiB/s wr, 1.00k op/s 2024-04-25T00:33:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:57 smithi106 ceph-mon[32510]: pgmap v845: 97 pgs: 97 active+clean; 465 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 19 MiB/s wr, 1.00k op/s 2024-04-25T00:33:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:57 smithi042 ceph-mon[28130]: pgmap v845: 97 pgs: 97 active+clean; 465 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 19 MiB/s wr, 1.00k op/s 2024-04-25T00:33:58.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 67654 35.34 MB/sec warmup 108 sec latency 40.455 ms 2024-04-25T00:33:59.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 68471 35.50 MB/sec warmup 109 sec latency 78.025 ms 2024-04-25T00:34:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:33:59 smithi136 ceph-mon[32923]: pgmap v846: 97 pgs: 97 active+clean; 463 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-04-25T00:34:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:33:59 smithi106 ceph-mon[32510]: pgmap v846: 97 pgs: 97 active+clean; 463 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-04-25T00:34:00.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:33:59 smithi042 ceph-mon[28130]: pgmap v846: 97 pgs: 97 active+clean; 463 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 18 MiB/s wr, 1.04k op/s 2024-04-25T00:34:00.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 69329 35.47 MB/sec warmup 110 sec latency 48.409 ms 2024-04-25T00:34:01.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 70097 35.64 MB/sec warmup 111 sec latency 90.267 ms 2024-04-25T00:34:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:01 smithi136 ceph-mon[32923]: pgmap v847: 97 pgs: 97 active+clean; 468 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 933 op/s 2024-04-25T00:34:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:01 smithi106 ceph-mon[32510]: pgmap v847: 97 pgs: 97 active+clean; 468 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 933 op/s 2024-04-25T00:34:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:01 smithi042 ceph-mon[28130]: pgmap v847: 97 pgs: 97 active+clean; 468 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 933 op/s 2024-04-25T00:34:02.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 70573 35.56 MB/sec warmup 112 sec latency 341.643 ms 2024-04-25T00:34:03.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 71246 35.50 MB/sec warmup 113 sec latency 43.470 ms 2024-04-25T00:34:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:03 smithi136 ceph-mon[32923]: pgmap v848: 97 pgs: 97 active+clean; 480 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 677 op/s 2024-04-25T00:34:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:03 smithi106 ceph-mon[32510]: pgmap v848: 97 pgs: 97 active+clean; 480 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 677 op/s 2024-04-25T00:34:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:03 smithi042 ceph-mon[28130]: pgmap v848: 97 pgs: 97 active+clean; 480 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 677 op/s 2024-04-25T00:34:04.743 INFO:tasks.workunit.client.0.smithi042.stdout: 10 72119 35.61 MB/sec warmup 114 sec latency 57.670 ms 2024-04-25T00:34:05.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 72991 35.74 MB/sec warmup 115 sec latency 47.282 ms 2024-04-25T00:34:05.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.960+0000 7f3672200640 1 -- 172.21.15.42:0/1940892311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c10b370 msgr2=0x7f366c10d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.960+0000 7f3672200640 1 --2- 172.21.15.42:0/1940892311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c10b370 0x7f366c10d7c0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f365800af00 tx=0x7f3658032a00 comp rx=0 tx=0).stop 2024-04-25T00:34:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.961+0000 7f3672200640 1 -- 172.21.15.42:0/1940892311 shutdown_connections 2024-04-25T00:34:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.961+0000 7f3672200640 1 --2- 172.21.15.42:0/1940892311 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c10de50 0x7f366c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.961+0000 7f3672200640 1 --2- 172.21.15.42:0/1940892311 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c10b370 0x7f366c10d7c0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.961+0000 7f3672200640 1 --2- 172.21.15.42:0/1940892311 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f366c1089b0 0x7f366c10ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.961+0000 7f3672200640 1 -- 172.21.15.42:0/1940892311 >> 172.21.15.42:0/1940892311 conn(0x7f366c06e970 msgr2=0x7f366c070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.961+0000 7f3672200640 1 -- 172.21.15.42:0/1940892311 shutdown_connections 2024-04-25T00:34:05.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.962+0000 7f3672200640 1 -- 172.21.15.42:0/1940892311 wait complete. 2024-04-25T00:34:05.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.963+0000 7f3672200640 1 Processor -- start 2024-04-25T00:34:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.964+0000 7f3672200640 1 -- start start 2024-04-25T00:34:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.964+0000 7f3672200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f366c1089b0 0x7f366c19e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.964+0000 7f3672200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c10b370 0x7f366c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.964+0000 7f3672200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c10de50 0x7f366c1a55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.964+0000 7f3672200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f366c113a80 con 0x7f366c1089b0 2024-04-25T00:34:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.964+0000 7f3672200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f366c113900 con 0x7f366c10de50 2024-04-25T00:34:05.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.964+0000 7f3672200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f366c113c00 con 0x7f366c10b370 2024-04-25T00:34:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.965+0000 7f366be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c10de50 0x7f366c1a55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.965+0000 7f366be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c10de50 0x7f366c1a55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60510/0 (socket says 172.21.15.42:60510) 2024-04-25T00:34:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.965+0000 7f366be00640 1 -- 172.21.15.42:0/951258843 learned_addr learned my addr 172.21.15.42:0/951258843 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.965+0000 7f366aa00640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c10b370 0x7f366c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.965+0000 7f366be00640 1 -- 172.21.15.42:0/951258843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c10b370 msgr2=0x7f366c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.965+0000 7f366be00640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c10b370 0x7f366c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.965+0000 7f366be00640 1 -- 172.21.15.42:0/951258843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f366c1089b0 msgr2=0x7f366c19e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.965+0000 7f366be00640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f366c1089b0 0x7f366c19e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.965+0000 7f366be00640 1 -- 172.21.15.42:0/951258843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c1a5c60 con 0x7f366c10de50 2024-04-25T00:34:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.965+0000 7f366aa00640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c10b370 0x7f366c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.966+0000 7f366be00640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c10de50 0x7f366c1a55f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f366400db00 tx=0x7f366400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.966+0000 7f3663e00640 1 -- 172.21.15.42:0/951258843 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3664015070 con 0x7f366c10de50 2024-04-25T00:34:05.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.966+0000 7f3672200640 1 -- 172.21.15.42:0/951258843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c1a5f50 con 0x7f366c10de50 2024-04-25T00:34:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.966+0000 7f3663e00640 1 -- 172.21.15.42:0/951258843 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3664005020 con 0x7f366c10de50 2024-04-25T00:34:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.966+0000 7f3663e00640 1 -- 172.21.15.42:0/951258843 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f366400f620 con 0x7f366c10de50 2024-04-25T00:34:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.966+0000 7f3672200640 1 -- 172.21.15.42:0/951258843 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1a6460 con 0x7f366c10de50 2024-04-25T00:34:05.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.971+0000 7f3674f67740 1 Processor -- start 2024-04-25T00:34:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.974+0000 7f3663e00640 1 -- 172.21.15.42:0/951258843 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f366400f880 con 0x7f366c10de50 2024-04-25T00:34:05.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.974+0000 7f3674f67740 1 -- start start 2024-04-25T00:34:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.975+0000 7f3674f67740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead56ca0 0x55beead570c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.975+0000 7f3674f67740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead57690 0x55beead5c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.975+0000 7f3674f67740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 0x55beead5ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.975+0000 7f3674f67740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55beeacb9ab0 con 0x55beead56ca0 2024-04-25T00:34:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.975+0000 7f3674f67740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55beeac62640 con 0x55beead5c800 2024-04-25T00:34:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.975+0000 7f3674f67740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55beeacb5a70 con 0x55beead57690 2024-04-25T00:34:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.975+0000 7f366aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 0x55beead5ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.976+0000 7f366aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 0x55beead5ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60526/0 (socket says 172.21.15.42:60526) 2024-04-25T00:34:05.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.976+0000 7f366aa00640 1 -- 172.21.15.42:0/425352060 learned_addr learned my addr 172.21.15.42:0/425352060 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.976+0000 7f3663e00640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36500a2cb0 0x7f36500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.976+0000 7f366aa00640 1 --2- 172.21.15.42:0/425352060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead56ca0 0x55beead570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:05.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.976+0000 7f3663e00640 1 -- 172.21.15.42:0/951258843 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f36640cf080 con 0x7f366c10de50 2024-04-25T00:34:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.976+0000 7f366aa00640 1 -- 172.21.15.42:0/425352060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead57690 msgr2=0x55beead5c190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:34:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.976+0000 7f366aa00640 1 --2- 172.21.15.42:0/425352060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead57690 0x55beead5c190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.976+0000 7f366aa00640 1 -- 172.21.15.42:0/425352060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead56ca0 msgr2=0x55beead570c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.976+0000 7f366aa00640 1 --2- 172.21.15.42:0/425352060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead56ca0 0x55beead570c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.976+0000 7f366aa00640 1 -- 172.21.15.42:0/425352060 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beea9ce210 con 0x55beead5c800 2024-04-25T00:34:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.977+0000 7f366aa00640 1 --2- 172.21.15.42:0/425352060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 0x55beead5ec90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3658008330 tx=0x7f3658000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a658e027906a08a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:05.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.978+0000 7f3657e00640 1 -- 172.21.15.42:0/425352060 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658004230 con 0x55beead5c800 2024-04-25T00:34:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.978+0000 7f366be00640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36500a2cb0 0x7f36500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.978+0000 7f3657e00640 1 -- 172.21.15.42:0/425352060 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36580043d0 con 0x55beead5c800 2024-04-25T00:34:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.978+0000 7f3657e00640 1 -- 172.21.15.42:0/425352060 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36580050b0 con 0x55beead5c800 2024-04-25T00:34:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.979+0000 7f3674f67740 1 -- 172.21.15.42:0/425352060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 msgr2=0x55beead5ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:05.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.979+0000 7f3674f67740 1 --2- 172.21.15.42:0/425352060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 0x55beead5ec90 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3658008330 tx=0x7f3658000bc0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.979+0000 7f3674f67740 1 -- 172.21.15.42:0/425352060 shutdown_connections 2024-04-25T00:34:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.979+0000 7f3674f67740 1 --2- 172.21.15.42:0/425352060 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 0x55beead5ec90 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.979+0000 7f3674f67740 1 --2- 172.21.15.42:0/425352060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead57690 0x55beead5c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.979+0000 7f3674f67740 1 --2- 172.21.15.42:0/425352060 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead56ca0 0x55beead570c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.979+0000 7f3674f67740 1 -- 172.21.15.42:0/425352060 >> 172.21.15.42:0/425352060 conn(0x55beead52310 msgr2=0x55beead52710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.979+0000 7f366be00640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36500a2cb0 0x7f36500a5170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f36640ccbb0 tx=0x7f3664005b40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:05.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.979+0000 7f3674f67740 1 -- 172.21.15.42:0/425352060 shutdown_connections 2024-04-25T00:34:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.980+0000 7f3674f67740 1 -- 172.21.15.42:0/425352060 wait complete. 2024-04-25T00:34:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.980+0000 7f3674f67740 1 -- start start 2024-04-25T00:34:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.981+0000 7f3674f67740 1 Processor -- start 2024-04-25T00:34:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.981+0000 7f3674f67740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead56ca0 0x55beead60ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.981+0000 7f3674f67740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 0x55beead61420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.981+0000 7f3674f67740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead5c800 0x55beead68460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.981+0000 7f3674f67740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55beead5f750 con 0x55beead57690 2024-04-25T00:34:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.981+0000 7f3674f67740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55beeac75990 con 0x55beead56ca0 2024-04-25T00:34:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.981+0000 7f3674f67740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55beead5f8d0 con 0x55beead5c800 2024-04-25T00:34:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.981+0000 7f3674f67740 10 client.0 _ll_register_callbacks cb 0x55beeac77950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.982+0000 7f366b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 0x55beead61420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.982+0000 7f366aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead56ca0 0x55beead60ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.982+0000 7f366aa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead56ca0 0x55beead60ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60528/0 (socket says 172.21.15.42:60528) 2024-04-25T00:34:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.982+0000 7f366aa00640 1 -- 172.21.15.42:0/2036447830 learned_addr learned my addr 172.21.15.42:0/2036447830 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:05.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.982+0000 7f366aa00640 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead5c800 0x55beead68460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.982+0000 7f366b400640 1 -- 172.21.15.42:0/2036447830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead5c800 msgr2=0x55beead68460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:05.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.982+0000 7f366b400640 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead5c800 0x55beead68460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.982+0000 7f366b400640 1 -- 172.21.15.42:0/2036447830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead56ca0 msgr2=0x55beead60ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.982+0000 7f366b400640 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead56ca0 0x55beead60ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.982+0000 7f366b400640 1 -- 172.21.15.42:0/2036447830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beeaa9a620 con 0x55beead57690 2024-04-25T00:34:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.983+0000 7f366b400640 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 0x55beead61420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55beead57a90 tx=0x7f365c00ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130345c12ba441f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.983+0000 7f3654c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:34:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.984+0000 7f3654c00640 1 -- 172.21.15.42:0/2036447830 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c004220 con 0x55beead57690 2024-04-25T00:34:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.984+0000 7f3674f67740 1 -- 172.21.15.42:0/2036447830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36580043d0 con 0x55beead57690 2024-04-25T00:34:05.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.984+0000 7f3654c00640 1 -- 172.21.15.42:0/2036447830 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f365c0043c0 con 0x55beead57690 2024-04-25T00:34:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.985+0000 7f3654c00640 1 -- 172.21.15.42:0/2036447830 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f365c01e070 con 0x55beead57690 2024-04-25T00:34:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.985+0000 7f3674f67740 10 client.16242 fetch_fsmap learned FSMap version 38 2024-04-25T00:34:05.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.985+0000 7f3674f67740 1 -- 172.21.15.42:0/2036447830 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55beead6b560 con 0x55beead57690 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.985+0000 7f3654c00640 1 -- 172.21.15.42:0/2036447830 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36580043d0 con 0x55beead57690 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.986+0000 7f3654c00640 1 -- 172.21.15.42:0/2036447830 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f365c010640 con 0x55beead57690 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.986+0000 7f3674f67740 10 client.16242 fetch_fsmap finished waiting for FSMap version 38 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.986+0000 7f3674f67740 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.986+0000 7f3674f67740 20 client.16242 populate_metadata read hostname 'smithi042' 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.986+0000 7f3674f67740 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55beead6cc50 0x55beead6f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.986+0000 7f3674f67740 4 client.16242 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.986+0000 7f3674f67740 1 -- 172.21.15.42:0/2036447830 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beeacc6ff0 con 0x55beead6cc50 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.986+0000 7f366aa00640 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55beead6cc50 0x55beead6f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.987+0000 7f366aa00640 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55beead6cc50 0x55beead6f0b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83dccffa1c435f11 server_cookie=25c403efb8ad5bc4 in_seq=0 out_seq=0 2024-04-25T00:34:05.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.987+0000 7f3654c00640 10 client.16242 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.988+0000 7f3654c00640 1 -- 172.21.15.42:0/2036447830 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55beeacc6ff0 con 0x55beead6cc50 2024-04-25T00:34:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.988+0000 7f3654c00640 10 client.16242 handle_command_reply: tid=0 2024-04-25T00:34:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.988+0000 7f3674f67740 1 client.16242 shutdown 2024-04-25T00:34:05.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.988+0000 7f3674f67740 1 -- 172.21.15.42:0/2036447830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 msgr2=0x55beead61420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:05.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.988+0000 7f3674f67740 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 0x55beead61420 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x55beead57a90 tx=0x7f365c00ea60 comp rx=0 tx=0).stop 2024-04-25T00:34:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.989+0000 7f3674f67740 1 -- 172.21.15.42:0/2036447830 shutdown_connections 2024-04-25T00:34:05.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.989+0000 7f3674f67740 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55beead6cc50 0x55beead6f0b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.989+0000 7f3674f67740 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead5c800 0x55beead68460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.990+0000 7f3674f67740 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 0x55beead61420 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.990+0000 7f3674f67740 1 --2- 172.21.15.42:0/2036447830 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead56ca0 0x55beead60ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.990+0000 7f3674f67740 1 -- 172.21.15.42:0/2036447830 >> 172.21.15.42:0/2036447830 conn(0x55beead52310 msgr2=0x55beead51480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.989+0000 7f3654c00640 0 client.16242 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.990+0000 7f3674f67740 1 -- 172.21.15.42:0/2036447830 shutdown_connections 2024-04-25T00:34:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.990+0000 7f3674f67740 1 -- 172.21.15.42:0/2036447830 wait complete. 2024-04-25T00:34:05.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:05.990+0000 7f3674f67740 20 client.16242 trim_cache size 0 max 16384 2024-04-25T00:34:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:05 smithi136 ceph-mon[32923]: pgmap v849: 97 pgs: 97 active+clean; 481 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 622 op/s 2024-04-25T00:34:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.017+0000 7f3674f67740 1 Processor -- start 2024-04-25T00:34:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.018+0000 7f3674f67740 1 -- start start 2024-04-25T00:34:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.018+0000 7f3674f67740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 0x55beead57d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.018+0000 7f3674f67740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead57690 0x55beead68420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.018+0000 7f3674f67740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead5c800 0x55beead58910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.018+0000 7f3674f67740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1b1b90 con 0x55beead5c800 2024-04-25T00:34:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.018+0000 7f3674f67740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1af370 con 0x55beead57690 2024-04-25T00:34:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.019+0000 7f3674f67740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1ad1c0 con 0x55beead56ca0 2024-04-25T00:34:06.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.019+0000 7f366b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead57690 0x55beead68420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.018+0000 7f366aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 0x55beead57d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.019+0000 7f366b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead57690 0x55beead68420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60534/0 (socket says 172.21.15.42:60534) 2024-04-25T00:34:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.019+0000 7f366b400640 1 -- 172.21.15.42:0/878036728 learned_addr learned my addr 172.21.15.42:0/878036728 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.019+0000 7f366aa00640 1 --2- 172.21.15.42:0/878036728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead5c800 0x55beead58910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.019+0000 7f366b400640 1 -- 172.21.15.42:0/878036728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 msgr2=0x55beead57d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.019+0000 7f366b400640 1 --2- 172.21.15.42:0/878036728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 0x55beead57d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.019+0000 7f366b400640 1 -- 172.21.15.42:0/878036728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead5c800 msgr2=0x55beead58910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.020+0000 7f366b400640 1 --2- 172.21.15.42:0/878036728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead5c800 0x55beead58910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.020+0000 7f366b400640 1 -- 172.21.15.42:0/878036728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beead52140 con 0x55beead57690 2024-04-25T00:34:06.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.020+0000 7f366aa00640 1 --2- 172.21.15.42:0/878036728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead5c800 0x55beead58910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:34:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.021+0000 7f366b400640 1 --2- 172.21.15.42:0/878036728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead57690 0x55beead68420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f365c000c00 tx=0x7f365c011fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a8021331a14d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.021+0000 7f3657400640 1 -- 172.21.15.42:0/878036728 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c010840 con 0x55beead57690 2024-04-25T00:34:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.021+0000 7f3657400640 1 -- 172.21.15.42:0/878036728 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365c004c80 con 0x55beead57690 2024-04-25T00:34:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.021+0000 7f3657400640 1 -- 172.21.15.42:0/878036728 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c003850 con 0x55beead57690 2024-04-25T00:34:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.022+0000 7f3674f67740 1 -- 172.21.15.42:0/878036728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead57690 msgr2=0x55beead68420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:06.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.022+0000 7f3674f67740 1 --2- 172.21.15.42:0/878036728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead57690 0x55beead68420 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f365c000c00 tx=0x7f365c011fe0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.022+0000 7f3674f67740 1 -- 172.21.15.42:0/878036728 shutdown_connections 2024-04-25T00:34:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.022+0000 7f3674f67740 1 --2- 172.21.15.42:0/878036728 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead5c800 0x55beead58910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.022+0000 7f3674f67740 1 --2- 172.21.15.42:0/878036728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead57690 0x55beead68420 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.022+0000 7f3674f67740 1 --2- 172.21.15.42:0/878036728 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 0x55beead57d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.022+0000 7f3674f67740 1 -- 172.21.15.42:0/878036728 >> 172.21.15.42:0/878036728 conn(0x55beead52310 msgr2=0x55beead5bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.022+0000 7f3674f67740 1 -- 172.21.15.42:0/878036728 shutdown_connections 2024-04-25T00:34:06.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.022+0000 7f3674f67740 1 -- 172.21.15.42:0/878036728 wait complete. 2024-04-25T00:34:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.023+0000 7f3674f67740 1 -- start start 2024-04-25T00:34:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.023+0000 7f3674f67740 1 Processor -- start 2024-04-25T00:34:06.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.024+0000 7f3674f67740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 0x55beead635b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.024+0000 7f3674f67740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 0x55beead63af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.024+0000 7f3674f67740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 0x55beead66ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.024+0000 7f3674f67740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1ad830 con 0x55beead57690 2024-04-25T00:34:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.024+0000 7f3674f67740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1aeca0 con 0x55beead5c800 2024-04-25T00:34:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.024+0000 7f3674f67740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1adf00 con 0x55beead56ca0 2024-04-25T00:34:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.025+0000 7f366aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 0x55beead635b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:06.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.025+0000 7f366aa00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 0x55beead635b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:54654/0 (socket says 172.21.15.42:54654) 2024-04-25T00:34:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.025+0000 7f366aa00640 1 -- 172.21.15.42:0/3109243240 learned_addr learned my addr 172.21.15.42:0/3109243240 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.025+0000 7f3674f67740 10 client.0 _ll_register_callbacks cb 0x7f365c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:06.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.025+0000 7f366aa00640 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 0x55beead66ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.025+0000 7f366aa00640 1 -- 172.21.15.42:0/3109243240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 msgr2=0x55beead66ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.025+0000 7f366aa00640 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 0x55beead66ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.025+0000 7f366aa00640 1 -- 172.21.15.42:0/3109243240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 msgr2=0x55beead63af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.025+0000 7f366b400640 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 0x55beead63af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.026+0000 7f366aa00640 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 0x55beead63af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.026+0000 7f366aa00640 1 -- 172.21.15.42:0/3109243240 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beead670b0 con 0x55beead56ca0 2024-04-25T00:34:06.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.026+0000 7f366b400640 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 0x55beead63af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.026+0000 7f366aa00640 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 0x55beead635b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f365800af00 tx=0x7f3658044040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe74d14254dd5bde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.026+0000 7f3654c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:34:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.026+0000 7f3654c00640 1 -- 172.21.15.42:0/3109243240 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365803b040 con 0x55beead56ca0 2024-04-25T00:34:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.026+0000 7f3654c00640 1 -- 172.21.15.42:0/3109243240 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f365805d540 con 0x55beead56ca0 2024-04-25T00:34:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.026+0000 7f3674f67740 1 -- 172.21.15.42:0/3109243240 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f365c004c80 con 0x55beead56ca0 2024-04-25T00:34:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.027+0000 7f3654c00640 1 -- 172.21.15.42:0/3109243240 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f365805d740 con 0x55beead56ca0 2024-04-25T00:34:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.027+0000 7f3674f67740 10 client.26404 fetch_fsmap learned FSMap version 38 2024-04-25T00:34:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.027+0000 7f3674f67740 1 -- 172.21.15.42:0/3109243240 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55beead683a0 con 0x55beead56ca0 2024-04-25T00:34:06.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.027+0000 7f3654c00640 1 -- 172.21.15.42:0/3109243240 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f365c004c80 con 0x55beead56ca0 2024-04-25T00:34:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.027+0000 7f3654c00640 1 -- 172.21.15.42:0/3109243240 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f3658005170 con 0x55beead56ca0 2024-04-25T00:34:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.028+0000 7f3674f67740 10 client.26404 fetch_fsmap finished waiting for FSMap version 38 2024-04-25T00:34:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.028+0000 7f3674f67740 10 client.26404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:34:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.028+0000 7f3674f67740 20 client.26404 populate_metadata read hostname 'smithi042' 2024-04-25T00:34:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.028+0000 7f3674f67740 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55beeade4290 0x55beeade66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.028+0000 7f3674f67740 4 client.26404 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:34:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.028+0000 7f3674f67740 1 -- 172.21.15.42:0/3109243240 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55beeacc1340 con 0x55beeade4290 2024-04-25T00:34:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.028+0000 7f366b400640 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55beeade4290 0x55beeade66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.030+0000 7f366b400640 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55beeade4290 0x55beeade66b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8409a37919a68da server_cookie=5341a10ba662cbd2 in_seq=0 out_seq=0 2024-04-25T00:34:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.030+0000 7f3654c00640 10 client.26404 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:05 smithi106 ceph-mon[32510]: pgmap v849: 97 pgs: 97 active+clean; 481 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 622 op/s 2024-04-25T00:34:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:05 smithi042 ceph-mon[28130]: pgmap v849: 97 pgs: 97 active+clean; 481 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.1 MiB/s wr, 622 op/s 2024-04-25T00:34:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3654c00640 1 -- 172.21.15.42:0/3109243240 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55beeacc1340 con 0x55beeade4290 2024-04-25T00:34:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3654c00640 10 client.26404 handle_command_reply: tid=0 2024-04-25T00:34:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3674f67740 1 client.26404 shutdown 2024-04-25T00:34:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3674f67740 1 -- 172.21.15.42:0/3109243240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 msgr2=0x55beead635b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3674f67740 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 0x55beead635b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f365800af00 tx=0x7f3658044040 comp rx=0 tx=0).stop 2024-04-25T00:34:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3674f67740 1 -- 172.21.15.42:0/3109243240 shutdown_connections 2024-04-25T00:34:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3674f67740 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55beeade4290 0x55beeade66b0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3674f67740 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55beead5c800 0x55beead66ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3674f67740 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55beead57690 0x55beead63af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3674f67740 1 --2- 172.21.15.42:0/3109243240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55beead56ca0 0x55beead635b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3674f67740 1 -- 172.21.15.42:0/3109243240 >> 172.21.15.42:0/3109243240 conn(0x55beead52310 msgr2=0x55beead58910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.238+0000 7f3654c00640 0 client.26404 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.239+0000 7f3674f67740 1 -- 172.21.15.42:0/3109243240 shutdown_connections 2024-04-25T00:34:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.239+0000 7f3674f67740 1 -- 172.21.15.42:0/3109243240 wait complete. 2024-04-25T00:34:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.239+0000 7f3674f67740 20 client.26404 trim_cache size 0 max 16384 2024-04-25T00:34:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:34:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:34:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:34:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:34:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.239+0000 7f3672200640 1 -- 172.21.15.42:0/951258843 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36500a2cb0 msgr2=0x7f36500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.239+0000 7f3672200640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36500a2cb0 0x7f36500a5170 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f36640ccbb0 tx=0x7f3664005b40 comp rx=0 tx=0).stop 2024-04-25T00:34:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.239+0000 7f3672200640 1 -- 172.21.15.42:0/951258843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c10de50 msgr2=0x7f366c1a55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.239+0000 7f3672200640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c10de50 0x7f366c1a55f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f366400db00 tx=0x7f366400b260 comp rx=0 tx=0).stop 2024-04-25T00:34:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.240+0000 7f3672200640 1 -- 172.21.15.42:0/951258843 shutdown_connections 2024-04-25T00:34:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.240+0000 7f3672200640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36500a2cb0 0x7f36500a5170 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.240+0000 7f3672200640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f366c10de50 0x7f366c1a55f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.240+0000 7f3672200640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c10b370 0x7f366c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.240+0000 7f3672200640 1 --2- 172.21.15.42:0/951258843 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f366c1089b0 0x7f366c19e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.240+0000 7f3672200640 1 -- 172.21.15.42:0/951258843 >> 172.21.15.42:0/951258843 conn(0x7f366c06e970 msgr2=0x7f366c109310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.240+0000 7f3672200640 1 -- 172.21.15.42:0/951258843 shutdown_connections 2024-04-25T00:34:06.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:06.240+0000 7f3672200640 1 -- 172.21.15.42:0/951258843 wait complete. 2024-04-25T00:34:06.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 73556 35.66 MB/sec warmup 116 sec latency 134.751 ms 2024-04-25T00:34:07.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 74303 35.73 MB/sec warmup 117 sec latency 47.105 ms 2024-04-25T00:34:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:07 smithi136 ceph-mon[32923]: pgmap v850: 97 pgs: 97 active+clean; 483 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 618 op/s 2024-04-25T00:34:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:07 smithi106 ceph-mon[32510]: pgmap v850: 97 pgs: 97 active+clean; 483 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 618 op/s 2024-04-25T00:34:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:07 smithi042 ceph-mon[28130]: pgmap v850: 97 pgs: 97 active+clean; 483 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.3 MiB/s wr, 618 op/s 2024-04-25T00:34:08.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 74958 35.69 MB/sec warmup 118 sec latency 41.631 ms 2024-04-25T00:34:09.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 75623 35.71 MB/sec warmup 119 sec latency 165.168 ms 2024-04-25T00:34:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:09 smithi136 ceph-mon[32923]: pgmap v851: 97 pgs: 97 active+clean; 489 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.1 MiB/s wr, 530 op/s 2024-04-25T00:34:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:09 smithi106 ceph-mon[32510]: pgmap v851: 97 pgs: 97 active+clean; 489 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.1 MiB/s wr, 530 op/s 2024-04-25T00:34:10.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:09 smithi042 ceph-mon[28130]: pgmap v851: 97 pgs: 97 active+clean; 489 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.1 MiB/s wr, 530 op/s 2024-04-25T00:34:11.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 77182 46.92 MB/sec execute 1 sec latency 116.036 ms 2024-04-25T00:34:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:11 smithi136 ceph-mon[32923]: pgmap v852: 97 pgs: 97 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 575 op/s 2024-04-25T00:34:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:11 smithi106 ceph-mon[32510]: pgmap v852: 97 pgs: 97 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 575 op/s 2024-04-25T00:34:12.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:11 smithi042 ceph-mon[28130]: pgmap v852: 97 pgs: 97 active+clean; 493 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.0 MiB/s wr, 575 op/s 2024-04-25T00:34:12.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 77889 42.81 MB/sec execute 2 sec latency 45.472 ms 2024-04-25T00:34:13.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 78550 38.30 MB/sec execute 3 sec latency 61.977 ms 2024-04-25T00:34:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:13 smithi106 ceph-mon[32510]: pgmap v853: 97 pgs: 97 active+clean; 501 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 691 op/s 2024-04-25T00:34:14.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:13 smithi042 ceph-mon[28130]: pgmap v853: 97 pgs: 97 active+clean; 501 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 691 op/s 2024-04-25T00:34:14.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:13 smithi136 ceph-mon[32923]: pgmap v853: 97 pgs: 97 active+clean; 501 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.7 MiB/s wr, 691 op/s 2024-04-25T00:34:14.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 78901 34.31 MB/sec execute 4 sec latency 581.776 ms 2024-04-25T00:34:15.744 INFO:tasks.workunit.client.0.smithi042.stdout: 10 79199 32.35 MB/sec execute 5 sec latency 747.806 ms 2024-04-25T00:34:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:15 smithi106 ceph-mon[32510]: pgmap v854: 97 pgs: 97 active+clean; 501 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.5 MiB/s wr, 547 op/s 2024-04-25T00:34:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:15 smithi042 ceph-mon[28130]: pgmap v854: 97 pgs: 97 active+clean; 501 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.5 MiB/s wr, 547 op/s 2024-04-25T00:34:16.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:15 smithi136 ceph-mon[32923]: pgmap v854: 97 pgs: 97 active+clean; 501 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.5 MiB/s wr, 547 op/s 2024-04-25T00:34:16.745 INFO:tasks.workunit.client.0.smithi042.stdout: 10 79624 29.24 MB/sec execute 6 sec latency 131.563 ms 2024-04-25T00:34:17.745 INFO:tasks.workunit.client.0.smithi042.stdout: 10 80281 29.30 MB/sec execute 7 sec latency 90.537 ms 2024-04-25T00:34:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:17 smithi106 ceph-mon[32510]: pgmap v855: 97 pgs: 97 active+clean; 586 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 864 op/s 2024-04-25T00:34:18.130 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:17 smithi042 ceph-mon[28130]: pgmap v855: 97 pgs: 97 active+clean; 586 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 864 op/s 2024-04-25T00:34:18.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:17 smithi136 ceph-mon[32923]: pgmap v855: 97 pgs: 97 active+clean; 586 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 864 op/s 2024-04-25T00:34:18.745 INFO:tasks.workunit.client.0.smithi042.stdout: 10 81087 31.82 MB/sec execute 8 sec latency 68.006 ms 2024-04-25T00:34:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.422+0000 7f36e3000640 1 -- 172.21.15.42:0/749767812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36dc075720 msgr2=0x7f36dc073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.422+0000 7f36e3000640 1 --2- 172.21.15.42:0/749767812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36dc075720 0x7f36dc073b70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009900 tx=0x7f36c402a480 comp rx=0 tx=0).stop 2024-04-25T00:34:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.422+0000 7f36e3000640 1 -- 172.21.15.42:0/749767812 shutdown_connections 2024-04-25T00:34:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.422+0000 7f36e3000640 1 --2- 172.21.15.42:0/749767812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36dc109630 0x7f36dc10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.422+0000 7f36e3000640 1 --2- 172.21.15.42:0/749767812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36dc074140 0x7f36dc0745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.422+0000 7f36e3000640 1 --2- 172.21.15.42:0/749767812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36dc075720 0x7f36dc073b70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.422+0000 7f36e3000640 1 -- 172.21.15.42:0/749767812 >> 172.21.15.42:0/749767812 conn(0x7f36dc0fcd30 msgr2=0x7f36dc0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.423+0000 7f36e3000640 1 -- 172.21.15.42:0/749767812 shutdown_connections 2024-04-25T00:34:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.423+0000 7f36e3000640 1 -- 172.21.15.42:0/749767812 wait complete. 2024-04-25T00:34:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.424+0000 7f36e3000640 1 Processor -- start 2024-04-25T00:34:19.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.424+0000 7f36e3000640 1 -- start start 2024-04-25T00:34:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.424+0000 7f36e3000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36dc074140 0x7f36dc19dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.424+0000 7f36e3000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36dc075720 0x7f36dc19e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.425+0000 7f36e3000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36dc109630 0x7f36dc1a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.425+0000 7f36e3000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc10f640 con 0x7f36dc074140 2024-04-25T00:34:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.425+0000 7f36e3000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc10f4c0 con 0x7f36dc075720 2024-04-25T00:34:19.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.425+0000 7f36e3000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc10f7c0 con 0x7f36dc109630 2024-04-25T00:34:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.425+0000 7f36db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36dc075720 0x7f36dc19e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.425+0000 7f36e1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36dc109630 0x7f36dc1a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.425+0000 7f36e1000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36dc109630 0x7f36dc1a5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:55362/0 (socket says 172.21.15.42:55362) 2024-04-25T00:34:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.425+0000 7f36e1000640 1 -- 172.21.15.42:0/3439564174 learned_addr learned my addr 172.21.15.42:0/3439564174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.425+0000 7f36dbe00640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36dc074140 0x7f36dc19dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.426+0000 7f36e1000640 1 -- 172.21.15.42:0/3439564174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36dc075720 msgr2=0x7f36dc19e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.426+0000 7f36e1000640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36dc075720 0x7f36dc19e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.426+0000 7f36e1000640 1 -- 172.21.15.42:0/3439564174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36dc074140 msgr2=0x7f36dc19dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.426+0000 7f36e1000640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36dc074140 0x7f36dc19dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.426 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.426+0000 7f36e1000640 1 -- 172.21.15.42:0/3439564174 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc1a5bd0 con 0x7f36dc109630 2024-04-25T00:34:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.426+0000 7f36db400640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36dc075720 0x7f36dc19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.426+0000 7f36e1000640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36dc109630 0x7f36dc1a5560 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f36d400eb10 tx=0x7f36d400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.426+0000 7f36d8c00640 1 -- 172.21.15.42:0/3439564174 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4004b20 con 0x7f36dc109630 2024-04-25T00:34:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.427+0000 7f36e3000640 1 -- 172.21.15.42:0/3439564174 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36dc1a5ec0 con 0x7f36dc109630 2024-04-25T00:34:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.427+0000 7f36e3000640 1 -- 172.21.15.42:0/3439564174 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36dc1a63d0 con 0x7f36dc109630 2024-04-25T00:34:19.427 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.427+0000 7f36d8c00640 1 -- 172.21.15.42:0/3439564174 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36d4004cc0 con 0x7f36dc109630 2024-04-25T00:34:19.428 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.427+0000 7f36d8c00640 1 -- 172.21.15.42:0/3439564174 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4010760 con 0x7f36dc109630 2024-04-25T00:34:19.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.434+0000 7f36e5fb2740 1 Processor -- start 2024-04-25T00:34:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.436+0000 7f36d8c00640 1 -- 172.21.15.42:0/3439564174 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36d4010900 con 0x7f36dc109630 2024-04-25T00:34:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.436+0000 7f36e5fb2740 1 -- start start 2024-04-25T00:34:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.436+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x5611596680c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.436+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561159668690 0x56115966d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.436+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56115966d800 0x56115966fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.436+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5611595caab0 con 0x561159668690 2024-04-25T00:34:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.436+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561159573640 con 0x56115966d800 2024-04-25T00:34:19.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.436+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5611595c6a70 con 0x561159667ca0 2024-04-25T00:34:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.437+0000 7f36dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561159668690 0x56115966d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.438+0000 7f36dbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561159668690 0x56115966d190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57250/0 (socket says 172.21.15.42:57250) 2024-04-25T00:34:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.438+0000 7f36dbe00640 1 -- 172.21.15.42:0/624408968 learned_addr learned my addr 172.21.15.42:0/624408968 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:19.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.438+0000 7f36db400640 1 --2- 172.21.15.42:0/624408968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56115966d800 0x56115966fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.438+0000 7f36db400640 1 --2- 172.21.15.42:0/624408968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x5611596680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.439+0000 7f36db400640 1 -- 172.21.15.42:0/624408968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56115966d800 msgr2=0x56115966fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.439+0000 7f36db400640 1 --2- 172.21.15.42:0/624408968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56115966d800 0x56115966fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.439+0000 7f36db400640 1 -- 172.21.15.42:0/624408968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561159668690 msgr2=0x56115966d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.439+0000 7f36db400640 1 --2- 172.21.15.42:0/624408968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561159668690 0x56115966d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.439+0000 7f36db400640 1 -- 172.21.15.42:0/624408968 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611592df210 con 0x561159667ca0 2024-04-25T00:34:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.440+0000 7f36db400640 1 --2- 172.21.15.42:0/624408968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x5611596680c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00b620 tx=0x7f36cc00b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7a394952a53eb37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.440+0000 7f36d1600640 1 -- 172.21.15.42:0/624408968 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc004020 con 0x561159667ca0 2024-04-25T00:34:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.440+0000 7f36d1600640 1 -- 172.21.15.42:0/624408968 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc00fd90 con 0x561159667ca0 2024-04-25T00:34:19.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.440+0000 7f36d1600640 1 -- 172.21.15.42:0/624408968 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0054e0 con 0x561159667ca0 2024-04-25T00:34:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.440+0000 7f36d8c00640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36c00a2cb0 0x7f36c00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.441+0000 7f36d8c00640 1 -- 172.21.15.42:0/3439564174 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f36d40cf620 con 0x7f36dc109630 2024-04-25T00:34:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.441+0000 7f36dbe00640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36c00a2cb0 0x7f36c00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.441+0000 7f36e5fb2740 1 -- 172.21.15.42:0/624408968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 msgr2=0x5611596680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.441+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/624408968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x5611596680c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00b620 tx=0x7f36cc00b390 comp rx=0 tx=0).stop 2024-04-25T00:34:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.441+0000 7f36e5fb2740 1 -- 172.21.15.42:0/624408968 shutdown_connections 2024-04-25T00:34:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.441+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/624408968 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56115966d800 0x56115966fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.441+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/624408968 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561159668690 0x56115966d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.441+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/624408968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x5611596680c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.441+0000 7f36e5fb2740 1 -- 172.21.15.42:0/624408968 >> 172.21.15.42:0/624408968 conn(0x561159663310 msgr2=0x561159663710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.442+0000 7f36dbe00640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36c00a2cb0 0x7f36c00a5170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x561159669100 tx=0x7f36c4005a20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.442+0000 7f36e5fb2740 1 -- 172.21.15.42:0/624408968 shutdown_connections 2024-04-25T00:34:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.442+0000 7f36e5fb2740 1 -- 172.21.15.42:0/624408968 wait complete. 2024-04-25T00:34:19.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.443+0000 7f36e5fb2740 1 -- start start 2024-04-25T00:34:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.443+0000 7f36e5fb2740 1 Processor -- start 2024-04-25T00:34:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.443+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159667ca0 0x561159671f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.443+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159668690 0x5611596724b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.443+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x5611596794f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5611596707b0 con 0x56115966d800 2024-04-25T00:34:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561159586990 con 0x561159667ca0 2024-04-25T00:34:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561159670930 con 0x561159668690 2024-04-25T00:34:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36db400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159668690 0x5611596724b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36e5fb2740 10 client.0 _ll_register_callbacks cb 0x561159588950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36db400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159668690 0x5611596724b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:55376/0 (socket says 172.21.15.42:55376) 2024-04-25T00:34:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36db400640 1 -- 172.21.15.42:0/387174607 learned_addr learned my addr 172.21.15.42:0/387174607 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36dbe00640 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159667ca0 0x561159671f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36db400640 1 -- 172.21.15.42:0/387174607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159667ca0 msgr2=0x561159671f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36db400640 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159667ca0 0x561159671f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36db400640 1 -- 172.21.15.42:0/387174607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 msgr2=0x5611596794f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36db400640 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x5611596794f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.444+0000 7f36db400640 1 -- 172.21.15.42:0/387174607 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611593ab620 con 0x561159668690 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.445+0000 7f36db400640 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159668690 0x5611596724b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00b5f0 tx=0x7f36cc00fb40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=683af961ceaaadd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.445+0000 7f36ca000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.445+0000 7f36ca000640 1 -- 172.21.15.42:0/387174607 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc02c060 con 0x561159668690 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.445+0000 7f36ca000640 1 -- 172.21.15.42:0/387174607 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f36cc0046c0 con 0x561159668690 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.445+0000 7f36e5fb2740 1 -- 172.21.15.42:0/387174607 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36cc00fd90 con 0x561159668690 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.445+0000 7f36ca000640 1 -- 172.21.15.42:0/387174607 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f36cc0048c0 con 0x561159668690 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.445+0000 7f36e5fb2740 10 client.26416 fetch_fsmap learned FSMap version 38 2024-04-25T00:34:19.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.445+0000 7f36e5fb2740 1 -- 172.21.15.42:0/387174607 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56115967c4d0 con 0x561159668690 2024-04-25T00:34:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.446+0000 7f36ca000640 1 -- 172.21.15.42:0/387174607 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc00fd90 con 0x561159668690 2024-04-25T00:34:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.447+0000 7f36ca000640 1 -- 172.21.15.42:0/387174607 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f36cc01f8f0 con 0x561159668690 2024-04-25T00:34:19.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.447+0000 7f36e5fb2740 10 client.26416 fetch_fsmap finished waiting for FSMap version 38 2024-04-25T00:34:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.447+0000 7f36e5fb2740 10 client.26416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:34:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.447+0000 7f36e5fb2740 20 client.26416 populate_metadata read hostname 'smithi042' 2024-04-25T00:34:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.447+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56115967da00 0x56115967fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.447+0000 7f36e5fb2740 4 client.26416 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:34:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.447+0000 7f36e5fb2740 1 -- 172.21.15.42:0/387174607 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611595d7ff0 con 0x56115967da00 2024-04-25T00:34:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.447+0000 7f36dbe00640 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56115967da00 0x56115967fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.448+0000 7f36dbe00640 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56115967da00 0x56115967fdf0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f82e694d7d5aa84 server_cookie=1a65c77cb680fddd in_seq=0 out_seq=0 2024-04-25T00:34:19.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.448+0000 7f36ca000640 10 client.26416 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.449+0000 7f36ca000640 1 -- 172.21.15.42:0/387174607 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5611595d7ff0 con 0x56115967da00 2024-04-25T00:34:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.449+0000 7f36ca000640 10 client.26416 handle_command_reply: tid=0 2024-04-25T00:34:19.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.449+0000 7f36e5fb2740 1 client.26416 shutdown 2024-04-25T00:34:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 1 -- 172.21.15.42:0/387174607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159668690 msgr2=0x5611596724b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159668690 0x5611596724b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00b5f0 tx=0x7f36cc00fb40 comp rx=0 tx=0).stop 2024-04-25T00:34:19.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 1 -- 172.21.15.42:0/387174607 shutdown_connections 2024-04-25T00:34:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56115967da00 0x56115967fdf0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x5611596794f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159668690 0x5611596724b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/387174607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159667ca0 0x561159671f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36ca000640 0 client.26416 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 1 -- 172.21.15.42:0/387174607 >> 172.21.15.42:0/387174607 conn(0x561159663310 msgr2=0x561159662480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 1 -- 172.21.15.42:0/387174607 shutdown_connections 2024-04-25T00:34:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 1 -- 172.21.15.42:0/387174607 wait complete. 2024-04-25T00:34:19.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.450+0000 7f36e5fb2740 20 client.26416 trim_cache size 0 max 16384 2024-04-25T00:34:19.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.476+0000 7f36e5fb2740 1 Processor -- start 2024-04-25T00:34:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.476+0000 7f36e5fb2740 1 -- start start 2024-04-25T00:34:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.477+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x561159669910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.477+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 0x561159669e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.477+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x561159670d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.477+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8010d60 con 0x56115966d800 2024-04-25T00:34:19.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.477+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f36b800e540 con 0x561159668690 2024-04-25T00:34:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.477+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f36b800c390 con 0x561159667ca0 2024-04-25T00:34:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 0x561159669e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x561159669910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36e1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x561159670d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36dbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x561159669910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:55384/0 (socket says 172.21.15.42:55384) 2024-04-25T00:34:19.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36dbe00640 1 -- 172.21.15.42:0/1959289024 learned_addr learned my addr 172.21.15.42:0/1959289024 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36dbe00640 1 -- 172.21.15.42:0/1959289024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 msgr2=0x561159669e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36dbe00640 1 --2- 172.21.15.42:0/1959289024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 0x561159669e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36dbe00640 1 -- 172.21.15.42:0/1959289024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 msgr2=0x561159670d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:19.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36dbe00640 1 --2- 172.21.15.42:0/1959289024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x561159670d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36dbe00640 1 -- 172.21.15.42:0/1959289024 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561159680160 con 0x561159667ca0 2024-04-25T00:34:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.478+0000 7f36e1000640 1 --2- 172.21.15.42:0/1959289024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x561159670d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:34:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.479+0000 7f36dbe00640 1 --2- 172.21.15.42:0/1959289024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x561159669910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f36c4005c40 tx=0x7f36c403e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f40bea4aa459a580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.479+0000 7f36d0c00640 1 -- 172.21.15.42:0/1959289024 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4035040 con 0x561159667ca0 2024-04-25T00:34:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.479+0000 7f36d0c00640 1 -- 172.21.15.42:0/1959289024 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36c4005560 con 0x561159667ca0 2024-04-25T00:34:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.479+0000 7f36d0c00640 1 -- 172.21.15.42:0/1959289024 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c4032e40 con 0x561159667ca0 2024-04-25T00:34:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.480+0000 7f36e5fb2740 1 -- 172.21.15.42:0/1959289024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 msgr2=0x561159669910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:19.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.480+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/1959289024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x561159669910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f36c4005c40 tx=0x7f36c403e000 comp rx=0 tx=0).stop 2024-04-25T00:34:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.480+0000 7f36e5fb2740 1 -- 172.21.15.42:0/1959289024 shutdown_connections 2024-04-25T00:34:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.480+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/1959289024 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x561159670d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.480+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/1959289024 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 0x561159669e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.480+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/1959289024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x561159669910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.480+0000 7f36e5fb2740 1 -- 172.21.15.42:0/1959289024 >> 172.21.15.42:0/1959289024 conn(0x561159663310 msgr2=0x56115965dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.480+0000 7f36e5fb2740 1 -- 172.21.15.42:0/1959289024 shutdown_connections 2024-04-25T00:34:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.480+0000 7f36e5fb2740 1 -- 172.21.15.42:0/1959289024 wait complete. 2024-04-25T00:34:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.481+0000 7f36e5fb2740 1 -- start start 2024-04-25T00:34:19.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.481+0000 7f36e5fb2740 1 Processor -- start 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.481+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x561159675e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 0x561159676370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36e5fb2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x561159677330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f36b800ca00 con 0x56115966d800 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f36b800de70 con 0x561159668690 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 0x561159676370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36e5fb2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f36b800d0d0 con 0x561159667ca0 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36db400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 0x561159676370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:36836/0 (socket says 172.21.15.42:36836) 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36e1000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x561159677330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36db400640 1 -- 172.21.15.42:0/3643547397 learned_addr learned my addr 172.21.15.42:0/3643547397 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36e5fb2740 10 client.0 _ll_register_callbacks cb 0x7f36d40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.482+0000 7f36dbe00640 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x561159675e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.483+0000 7f36db400640 1 -- 172.21.15.42:0/3643547397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 msgr2=0x561159675e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.483+0000 7f36db400640 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x561159675e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.483+0000 7f36db400640 1 -- 172.21.15.42:0/3643547397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 msgr2=0x561159677330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.483+0000 7f36db400640 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x561159677330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.483+0000 7f36db400640 1 -- 172.21.15.42:0/3643547397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561159677920 con 0x561159668690 2024-04-25T00:34:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.483+0000 7f36db400640 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 0x561159676370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f36cc012770 tx=0x7f36cc0128a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3f24725fe74cf59 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.483+0000 7f36ca000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:34:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.483+0000 7f36ca000640 1 -- 172.21.15.42:0/3643547397 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc037070 con 0x561159668690 2024-04-25T00:34:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.483+0000 7f36ca000640 1 -- 172.21.15.42:0/3643547397 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8182+0+0 (secure 0 0 0) 0x7f36cc005670 con 0x561159668690 2024-04-25T00:34:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.484+0000 7f36e5fb2740 1 -- 172.21.15.42:0/3643547397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c4005560 con 0x561159668690 2024-04-25T00:34:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.484+0000 7f36ca000640 1 -- 172.21.15.42:0/3643547397 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f36cc02c040 con 0x561159668690 2024-04-25T00:34:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.484+0000 7f36e5fb2740 10 client.26420 fetch_fsmap learned FSMap version 38 2024-04-25T00:34:19.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.484+0000 7f36e5fb2740 1 -- 172.21.15.42:0/3643547397 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x561159678c30 con 0x561159668690 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.484+0000 7f36ca000640 1 -- 172.21.15.42:0/3643547397 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36cc042070 con 0x561159668690 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.485+0000 7f36ca000640 1 -- 172.21.15.42:0/3643547397 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 38) ==== 8358+0+0 (secure 0 0 0) 0x7f36cc004a60 con 0x561159668690 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.485+0000 7f36e5fb2740 10 client.26420 fetch_fsmap finished waiting for FSMap version 38 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.485+0000 7f36e5fb2740 10 client.26420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.485+0000 7f36e5fb2740 20 client.26420 populate_metadata read hostname 'smithi042' 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.485+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5611596f5670 0x5611596f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.485+0000 7f36e5fb2740 4 client.26420 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.485+0000 7f36e5fb2740 1 -- 172.21.15.42:0/3643547397 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611595d2340 con 0x5611596f5670 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.485+0000 7f36dbe00640 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5611596f5670 0x5611596f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.486+0000 7f36dbe00640 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5611596f5670 0x5611596f7a90 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71e801d4a638e2d server_cookie=f722283dea9a34 in_seq=0 out_seq=0 2024-04-25T00:34:19.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.486+0000 7f36ca000640 10 client.26420 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36ca000640 1 -- 172.21.15.42:0/3643547397 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5611595d2340 con 0x5611596f5670 2024-04-25T00:34:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36ca000640 10 client.26420 handle_command_reply: tid=0 2024-04-25T00:34:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36e5fb2740 1 client.26420 shutdown 2024-04-25T00:34:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36e5fb2740 1 -- 172.21.15.42:0/3643547397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 msgr2=0x561159676370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 0x561159676370 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f36cc012770 tx=0x7f36cc0128a0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36e5fb2740 1 -- 172.21.15.42:0/3643547397 shutdown_connections 2024-04-25T00:34:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5611596f5670 0x5611596f7a90 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56115966d800 0x561159677330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561159668690 0x561159676370 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36e5fb2740 1 --2- 172.21.15.42:0/3643547397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561159667ca0 0x561159675e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36ca000640 0 client.26420 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.718+0000 7f36e5fb2740 1 -- 172.21.15.42:0/3643547397 >> 172.21.15.42:0/3643547397 conn(0x561159663310 msgr2=0x561159671f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.719+0000 7f36e5fb2740 1 -- 172.21.15.42:0/3643547397 shutdown_connections 2024-04-25T00:34:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.719+0000 7f36e5fb2740 1 -- 172.21.15.42:0/3643547397 wait complete. 2024-04-25T00:34:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.719+0000 7f36e5fb2740 20 client.26420 trim_cache size 0 max 16384 2024-04-25T00:34:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:34:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:34:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:34:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:34:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.720+0000 7f36e3000640 1 -- 172.21.15.42:0/3439564174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36c00a2cb0 msgr2=0x7f36c00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.720+0000 7f36e3000640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36c00a2cb0 0x7f36c00a5170 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x561159669100 tx=0x7f36c4005a20 comp rx=0 tx=0).stop 2024-04-25T00:34:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.720+0000 7f36e3000640 1 -- 172.21.15.42:0/3439564174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36dc109630 msgr2=0x7f36dc1a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.720+0000 7f36e3000640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36dc109630 0x7f36dc1a5560 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f36d400eb10 tx=0x7f36d400c2d0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.721+0000 7f36e3000640 1 -- 172.21.15.42:0/3439564174 shutdown_connections 2024-04-25T00:34:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.721+0000 7f36e3000640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f36c00a2cb0 0x7f36c00a5170 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.721+0000 7f36e3000640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f36dc109630 0x7f36dc1a5560 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.721+0000 7f36e3000640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f36dc075720 0x7f36dc19e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.721+0000 7f36e3000640 1 --2- 172.21.15.42:0/3439564174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f36dc074140 0x7f36dc19dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.721+0000 7f36e3000640 1 -- 172.21.15.42:0/3439564174 >> 172.21.15.42:0/3439564174 conn(0x7f36dc0fcd30 msgr2=0x7f36dc1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.721+0000 7f36e3000640 1 -- 172.21.15.42:0/3439564174 shutdown_connections 2024-04-25T00:34:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:19.721+0000 7f36e3000640 1 -- 172.21.15.42:0/3439564174 wait complete. 2024-04-25T00:34:19.745 INFO:tasks.workunit.client.0.smithi042.stdout: 10 81772 32.06 MB/sec execute 9 sec latency 152.902 ms 2024-04-25T00:34:20.065 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:19 smithi136 ceph-mon[32923]: pgmap v856: 97 pgs: 97 active+clean; 585 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 16 MiB/s wr, 869 op/s 2024-04-25T00:34:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:19 smithi106 ceph-mon[32510]: pgmap v856: 97 pgs: 97 active+clean; 585 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 16 MiB/s wr, 869 op/s 2024-04-25T00:34:20.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:19 smithi042 ceph-mon[28130]: pgmap v856: 97 pgs: 97 active+clean; 585 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 16 MiB/s wr, 869 op/s 2024-04-25T00:34:20.513 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:34:20 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: ignoring --setuser ceph since I am not root 2024-04-25T00:34:20.513 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:34:20 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: ignoring --setgroup ceph since I am not root 2024-04-25T00:34:20.514 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:34:20 smithi136 ceph-mds[63607]: -- 172.21.15.136:0/3801125102 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55657d877a40 con 0x55657d84ac00 2024-04-25T00:34:20.514 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:34:20 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: starting mds.i at 2024-04-25T00:34:20.745 INFO:tasks.workunit.client.0.smithi042.stdout: 10 82494 33.79 MB/sec execute 10 sec latency 42.593 ms 2024-04-25T00:34:21.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:20 smithi106 ceph-mon[32510]: daemon mds.i restarted 2024-04-25T00:34:21.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:20 smithi042 ceph-mon[28130]: daemon mds.i restarted 2024-04-25T00:34:21.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:20 smithi136 ceph-mon[32923]: daemon mds.i restarted 2024-04-25T00:34:21.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 83127 33.19 MB/sec execute 11 sec latency 62.270 ms 2024-04-25T00:34:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:21 smithi106 ceph-mon[32510]: pgmap v857: 97 pgs: 97 active+clean; 532 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 871 op/s 2024-04-25T00:34:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:21 smithi106 ceph-mon[32510]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:34:22.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:21 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6838/2386071853,v1:172.21.15.136:6839/2386071853] up:boot 2024-04-25T00:34:22.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:21 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:34:22.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:21 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:34:22.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:21 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:34:22.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:21 smithi042 ceph-mon[28130]: pgmap v857: 97 pgs: 97 active+clean; 532 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 871 op/s 2024-04-25T00:34:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:21 smithi042 ceph-mon[28130]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:34:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:21 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6838/2386071853,v1:172.21.15.136:6839/2386071853] up:boot 2024-04-25T00:34:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:21 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:34:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:21 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:34:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:21 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:34:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:21 smithi136 ceph-mon[32923]: pgmap v857: 97 pgs: 97 active+clean; 532 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 16 MiB/s wr, 871 op/s 2024-04-25T00:34:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:21 smithi136 ceph-mon[32923]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T00:34:22.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:21 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6838/2386071853,v1:172.21.15.136:6839/2386071853] up:boot 2024-04-25T00:34:22.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:21 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T00:34:22.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:21 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:34:22.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:21 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:34:22.745 INFO:tasks.workunit.client.0.smithi042.stdout: 10 83936 33.47 MB/sec execute 12 sec latency 67.323 ms 2024-04-25T00:34:23.745 INFO:tasks.workunit.client.0.smithi042.stdout: 10 84806 34.93 MB/sec execute 13 sec latency 44.148 ms 2024-04-25T00:34:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:23 smithi106 ceph-mon[32510]: pgmap v859: 97 pgs: 97 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 958 op/s 2024-04-25T00:34:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:23 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:34:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:23 smithi042 ceph-mon[28130]: pgmap v859: 97 pgs: 97 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 958 op/s 2024-04-25T00:34:24.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:23 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:34:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:23 smithi136 ceph-mon[32923]: pgmap v859: 97 pgs: 97 active+clean; 497 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 958 op/s 2024-04-25T00:34:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:23 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:34:24.745 INFO:tasks.workunit.client.0.smithi042.stdout: 10 85485 35.28 MB/sec execute 14 sec latency 53.398 ms 2024-04-25T00:34:25.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 86146 35.32 MB/sec execute 15 sec latency 86.737 ms 2024-04-25T00:34:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:25 smithi106 ceph-mon[32510]: pgmap v860: 97 pgs: 97 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 998 op/s 2024-04-25T00:34:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:34:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:34:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:34:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:25 smithi042 ceph-mon[28130]: pgmap v860: 97 pgs: 97 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 998 op/s 2024-04-25T00:34:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:34:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:34:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:34:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:25 smithi136 ceph-mon[32923]: pgmap v860: 97 pgs: 97 active+clean; 500 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 998 op/s 2024-04-25T00:34:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:34:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:34:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:34:26.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 86876 35.31 MB/sec execute 16 sec latency 50.034 ms 2024-04-25T00:34:27.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 87671 35.59 MB/sec execute 17 sec latency 87.601 ms 2024-04-25T00:34:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:27 smithi106 ceph-mon[32510]: pgmap v861: 97 pgs: 97 active+clean; 513 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 807 op/s 2024-04-25T00:34:28.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:27 smithi042 ceph-mon[28130]: pgmap v861: 97 pgs: 97 active+clean; 513 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 807 op/s 2024-04-25T00:34:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:27 smithi136 ceph-mon[32923]: pgmap v861: 97 pgs: 97 active+clean; 513 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 6.5 MiB/s wr, 807 op/s 2024-04-25T00:34:28.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 88512 36.50 MB/sec execute 18 sec latency 47.200 ms 2024-04-25T00:34:29.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 89182 36.79 MB/sec execute 19 sec latency 54.626 ms 2024-04-25T00:34:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:29 smithi106 ceph-mon[32510]: pgmap v862: 97 pgs: 97 active+clean; 517 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 749 op/s 2024-04-25T00:34:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:29 smithi042 ceph-mon[28130]: pgmap v862: 97 pgs: 97 active+clean; 517 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 749 op/s 2024-04-25T00:34:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:29 smithi136 ceph-mon[32923]: pgmap v862: 97 pgs: 97 active+clean; 517 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 749 op/s 2024-04-25T00:34:30.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 89587 35.79 MB/sec execute 20 sec latency 241.088 ms 2024-04-25T00:34:31.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 90065 35.03 MB/sec execute 21 sec latency 102.709 ms 2024-04-25T00:34:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:31 smithi106 ceph-mon[32510]: pgmap v863: 97 pgs: 97 active+clean; 523 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 624 op/s 2024-04-25T00:34:32.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:31 smithi042 ceph-mon[28130]: pgmap v863: 97 pgs: 97 active+clean; 523 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 624 op/s 2024-04-25T00:34:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:31 smithi136 ceph-mon[32923]: pgmap v863: 97 pgs: 97 active+clean; 523 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.7 MiB/s wr, 624 op/s 2024-04-25T00:34:32.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 90746 34.98 MB/sec execute 22 sec latency 55.802 ms 2024-04-25T00:34:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.917+0000 7f605a400640 1 -- 172.21.15.42:0/1656431162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60541049f0 msgr2=0x7f6054104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.917+0000 7f605a400640 1 --2- 172.21.15.42:0/1656431162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60541049f0 0x7f6054104e70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6040009900 tx=0x7f604002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.918+0000 7f605a400640 1 -- 172.21.15.42:0/1656431162 shutdown_connections 2024-04-25T00:34:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.918+0000 7f605a400640 1 --2- 172.21.15.42:0/1656431162 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60541053b0 0x7f605410bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.918+0000 7f605a400640 1 --2- 172.21.15.42:0/1656431162 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60541049f0 0x7f6054104e70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.918+0000 7f605a400640 1 --2- 172.21.15.42:0/1656431162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6054103800 0x7f6054103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.918+0000 7f605a400640 1 -- 172.21.15.42:0/1656431162 >> 172.21.15.42:0/1656431162 conn(0x7f60540fed70 msgr2=0x7f60541011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.918+0000 7f605a400640 1 -- 172.21.15.42:0/1656431162 shutdown_connections 2024-04-25T00:34:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.918+0000 7f605a400640 1 -- 172.21.15.42:0/1656431162 wait complete. 2024-04-25T00:34:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.919+0000 7f605a400640 1 Processor -- start 2024-04-25T00:34:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.920+0000 7f605a400640 1 -- start start 2024-04-25T00:34:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.920+0000 7f605a400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6054103800 0x7f6054071130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.920+0000 7f605a400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60541049f0 0x7f6054071670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.920+0000 7f605a400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60541053b0 0x7f60540726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.920+0000 7f605a400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f605406b8c0 con 0x7f60541053b0 2024-04-25T00:34:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.920+0000 7f605a400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f605406b740 con 0x7f6054103800 2024-04-25T00:34:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.920+0000 7f605a400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f605406ba40 con 0x7f60541049f0 2024-04-25T00:34:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.921+0000 7f6053e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60541049f0 0x7f6054071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.921+0000 7f6059000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6054103800 0x7f6054071130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.921+0000 7f6053e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60541049f0 0x7f6054071670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60518/0 (socket says 172.21.15.42:60518) 2024-04-25T00:34:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.921+0000 7f6053e00640 1 -- 172.21.15.42:0/2220571314 learned_addr learned my addr 172.21.15.42:0/2220571314 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.921+0000 7f6059000640 1 -- 172.21.15.42:0/2220571314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60541049f0 msgr2=0x7f6054071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.921+0000 7f6059000640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60541049f0 0x7f6054071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.921+0000 7f6059000640 1 -- 172.21.15.42:0/2220571314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60541053b0 msgr2=0x7f60540726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.921+0000 7f6059000640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60541053b0 0x7f60540726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.921+0000 7f6059000640 1 -- 172.21.15.42:0/2220571314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6054079cd0 con 0x7f6054103800 2024-04-25T00:34:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.922+0000 7f6059000640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6054103800 0x7f6054071130 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6048005fd0 tx=0x7f604800ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.922+0000 7f6051600640 1 -- 172.21.15.42:0/2220571314 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048003990 con 0x7f6054103800 2024-04-25T00:34:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.922+0000 7f6051600640 1 -- 172.21.15.42:0/2220571314 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6048003b30 con 0x7f6054103800 2024-04-25T00:34:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.922+0000 7f6051600640 1 -- 172.21.15.42:0/2220571314 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6048019880 con 0x7f6054103800 2024-04-25T00:34:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.922+0000 7f605a400640 1 -- 172.21.15.42:0/2220571314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6054072c50 con 0x7f6054103800 2024-04-25T00:34:32.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.922+0000 7f605a400640 1 -- 172.21.15.42:0/2220571314 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6054073120 con 0x7f6054103800 2024-04-25T00:34:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.928+0000 7f6051600640 1 -- 172.21.15.42:0/2220571314 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6048019a20 con 0x7f6054103800 2024-04-25T00:34:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.930+0000 7f605e836740 1 Processor -- start 2024-04-25T00:34:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.931+0000 7f6051600640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f603c0a2c80 0x7f603c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.931+0000 7f605e836740 1 -- start start 2024-04-25T00:34:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.931+0000 7f6059a00640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f603c0a2c80 0x7f603c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.931+0000 7f605e836740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2847270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.931+0000 7f605e836740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c2847840 0x55f3c284c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.931+0000 7f605e836740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c284c9b0 0x55f3c284ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.931+0000 7f605e836740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f3c27a9ab0 con 0x55f3c284c9b0 2024-04-25T00:34:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.931+0000 7f605e836740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2752640 con 0x55f3c2846e50 2024-04-25T00:34:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.931+0000 7f605e836740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f3c27a5a70 con 0x55f3c2847840 2024-04-25T00:34:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.931+0000 7f6051600640 1 -- 172.21.15.42:0/2220571314 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f60480cf820 con 0x7f6054103800 2024-04-25T00:34:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.932+0000 7f6059000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2847270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.932+0000 7f6059000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2847270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45790/0 (socket says 172.21.15.42:45790) 2024-04-25T00:34:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.932+0000 7f6059000640 1 -- 172.21.15.42:0/2537798116 learned_addr learned my addr 172.21.15.42:0/2537798116 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.932+0000 7f6053e00640 1 --2- 172.21.15.42:0/2537798116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c2847840 0x55f3c284c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.932+0000 7f6059a00640 1 --2- 172.21.15.42:0/2537798116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c284c9b0 0x55f3c284ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.933+0000 7f6059000640 1 -- 172.21.15.42:0/2537798116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c2847840 msgr2=0x55f3c284c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.933+0000 7f6059000640 1 --2- 172.21.15.42:0/2537798116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c2847840 0x55f3c284c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.933+0000 7f6059000640 1 -- 172.21.15.42:0/2537798116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c284c9b0 msgr2=0x55f3c284ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.933+0000 7f6059000640 1 --2- 172.21.15.42:0/2537798116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c284c9b0 0x55f3c284ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.933+0000 7f6059000640 1 -- 172.21.15.42:0/2537798116 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c24be210 con 0x55f3c2846e50 2024-04-25T00:34:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.933+0000 7f6059000640 1 --2- 172.21.15.42:0/2537798116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2847270 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f60480a4790 tx=0x7f604809b9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=644c2e84f9d53092 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.933+0000 7f6046000640 1 -- 172.21.15.42:0/2537798116 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604809c750 con 0x55f3c2846e50 2024-04-25T00:34:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.934+0000 7f6046000640 1 -- 172.21.15.42:0/2537798116 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60480bc110 con 0x55f3c2846e50 2024-04-25T00:34:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.934+0000 7f605e836740 1 -- 172.21.15.42:0/2537798116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 msgr2=0x55f3c2847270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.934+0000 7f605e836740 1 --2- 172.21.15.42:0/2537798116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2847270 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f60480a4790 tx=0x7f604809b9b0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.935+0000 7f6059a00640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f603c0a2c80 0x7f603c0a5140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f604c005fd0 tx=0x7f604c005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.935+0000 7f605e836740 1 -- 172.21.15.42:0/2537798116 shutdown_connections 2024-04-25T00:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.935+0000 7f605e836740 1 --2- 172.21.15.42:0/2537798116 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c284c9b0 0x55f3c284ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.935+0000 7f605e836740 1 --2- 172.21.15.42:0/2537798116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c2847840 0x55f3c284c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.935+0000 7f605e836740 1 --2- 172.21.15.42:0/2537798116 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2847270 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.935+0000 7f605e836740 1 -- 172.21.15.42:0/2537798116 >> 172.21.15.42:0/2537798116 conn(0x55f3c28424c0 msgr2=0x55f3c28428c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.935+0000 7f605e836740 1 -- 172.21.15.42:0/2537798116 shutdown_connections 2024-04-25T00:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.936+0000 7f605e836740 1 -- 172.21.15.42:0/2537798116 wait complete. 2024-04-25T00:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.936+0000 7f605e836740 1 -- start start 2024-04-25T00:34:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.936+0000 7f605e836740 1 Processor -- start 2024-04-25T00:34:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.937+0000 7f605e836740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2846e50 0x55f3c2850f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.937+0000 7f605e836740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2847840 0x55f3c28514b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.937+0000 7f605e836740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 0x55f3c28584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.937+0000 7f605e836740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f3c284f960 con 0x55f3c2846e50 2024-04-25T00:34:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.937+0000 7f605e836740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f3c2765990 con 0x55f3c2847840 2024-04-25T00:34:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.937+0000 7f605e836740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f3c284fae0 con 0x55f3c284c9b0 2024-04-25T00:34:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f605e836740 10 client.0 _ll_register_callbacks cb 0x55f3c2767950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f6059a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 0x55f3c28584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f6059000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2846e50 0x55f3c2850f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f6059a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 0x55f3c28584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60532/0 (socket says 172.21.15.42:60532) 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f6059a00640 1 -- 172.21.15.42:0/3949893192 learned_addr learned my addr 172.21.15.42:0/3949893192 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f6053e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2847840 0x55f3c28514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f6053e00640 1 -- 172.21.15.42:0/3949893192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 msgr2=0x55f3c28584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f6053e00640 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 0x55f3c28584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f6053e00640 1 -- 172.21.15.42:0/3949893192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2846e50 msgr2=0x55f3c2850f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f6053e00640 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2846e50 0x55f3c2850f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.938+0000 7f6053e00640 1 -- 172.21.15.42:0/3949893192 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c258a620 con 0x55f3c2847840 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.939+0000 7f6053e00640 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2847840 0x55f3c28514b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6040006010 tx=0x7f6040004580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ced9a60caae5c9c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.939+0000 7f603aa00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.939+0000 7f603aa00640 1 -- 172.21.15.42:0/3949893192 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040035040 con 0x55f3c2847840 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.939+0000 7f603aa00640 1 -- 172.21.15.42:0/3949893192 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f6040005220 con 0x55f3c2847840 2024-04-25T00:34:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.939+0000 7f605e836740 1 -- 172.21.15.42:0/3949893192 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60480bc110 con 0x55f3c2847840 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.940+0000 7f603aa00640 1 -- 172.21.15.42:0/3949893192 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f604004c070 con 0x55f3c2847840 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.940+0000 7f605e836740 10 client.26435 fetch_fsmap learned FSMap version 41 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.940+0000 7f605e836740 1 -- 172.21.15.42:0/3949893192 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55f3c285b590 con 0x55f3c2847840 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.940+0000 7f603aa00640 1 -- 172.21.15.42:0/3949893192 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60480bc110 con 0x55f3c2847840 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.940+0000 7f603aa00640 1 -- 172.21.15.42:0/3949893192 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f6040004b10 con 0x55f3c2847840 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.941+0000 7f605e836740 10 client.26435 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.941+0000 7f605e836740 10 client.26435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.941+0000 7f605e836740 20 client.26435 populate_metadata read hostname 'smithi042' 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.941+0000 7f605e836740 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3c285cc00 0x55f3c285f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.941+0000 7f605e836740 4 client.26435 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.941+0000 7f605e836740 1 -- 172.21.15.42:0/3949893192 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3c27b6ff0 con 0x55f3c285cc00 2024-04-25T00:34:32.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.941+0000 7f6059000640 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3c285cc00 0x55f3c285f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.941+0000 7f603aa00640 10 client.26435 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.942+0000 7f6059000640 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3c285cc00 0x55f3c285f090 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=527c0a19d10794b9 server_cookie=732e763091eab9e9 in_seq=0 out_seq=0 2024-04-25T00:34:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.942+0000 7f603aa00640 1 -- 172.21.15.42:0/3949893192 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3c27b6ff0 con 0x55f3c285cc00 2024-04-25T00:34:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.942+0000 7f603aa00640 10 client.26435 handle_command_reply: tid=0 2024-04-25T00:34:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.943+0000 7f605e836740 1 client.26435 shutdown 2024-04-25T00:34:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 1 -- 172.21.15.42:0/3949893192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2847840 msgr2=0x55f3c28514b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2847840 0x55f3c28514b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f6040006010 tx=0x7f6040004580 comp rx=0 tx=0).stop 2024-04-25T00:34:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 1 -- 172.21.15.42:0/3949893192 shutdown_connections 2024-04-25T00:34:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3c285cc00 0x55f3c285f090 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 0x55f3c28584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2847840 0x55f3c28514b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 1 --2- 172.21.15.42:0/3949893192 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2846e50 0x55f3c2850f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f603aa00640 0 client.26435 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 1 -- 172.21.15.42:0/3949893192 >> 172.21.15.42:0/3949893192 conn(0x55f3c28424c0 msgr2=0x55f3c2841620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 1 -- 172.21.15.42:0/3949893192 shutdown_connections 2024-04-25T00:34:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 1 -- 172.21.15.42:0/3949893192 wait complete. 2024-04-25T00:34:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.944+0000 7f605e836740 20 client.26435 trim_cache size 0 max 16384 2024-04-25T00:34:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.981+0000 7f605e836740 1 Processor -- start 2024-04-25T00:34:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.982+0000 7f605e836740 1 -- start start 2024-04-25T00:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f605e836740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c283d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f605e836740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2847840 0x55f3c28584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f605e836740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 0x55f3c28458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f605e836740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60541ba010 con 0x55f3c2847840 2024-04-25T00:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f605e836740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60541b7820 con 0x55f3c2846e50 2024-04-25T00:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f605e836740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60541b5670 con 0x55f3c284c9b0 2024-04-25T00:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f6059000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c283d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f6059000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c283d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45796/0 (socket says 172.21.15.42:45796) 2024-04-25T00:34:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f6059000640 1 -- 172.21.15.42:0/4053253172 learned_addr learned my addr 172.21.15.42:0/4053253172 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f6059a00640 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 0x55f3c28458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f6059000640 1 -- 172.21.15.42:0/4053253172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 msgr2=0x55f3c28458b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f6059000640 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 0x55f3c28458b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f6059000640 1 -- 172.21.15.42:0/4053253172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2847840 msgr2=0x55f3c28584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.983+0000 7f6053e00640 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2847840 0x55f3c28584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.984+0000 7f6059000640 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2847840 0x55f3c28584b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.984+0000 7f6059000640 1 -- 172.21.15.42:0/4053253172 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3c28422f0 con 0x55f3c2846e50 2024-04-25T00:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.984+0000 7f6059a00640 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 0x55f3c28458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.984+0000 7f6053e00640 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2847840 0x55f3c28584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.984+0000 7f6059000640 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c283d110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f604809bd50 tx=0x7f604809bae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a807d82c95f623d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.984+0000 7f6045600640 1 -- 172.21.15.42:0/4053253172 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604809c750 con 0x55f3c2846e50 2024-04-25T00:34:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.984+0000 7f6045600640 1 -- 172.21.15.42:0/4053253172 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60480bc110 con 0x55f3c2846e50 2024-04-25T00:34:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.984+0000 7f6045600640 1 -- 172.21.15.42:0/4053253172 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60480a4a70 con 0x55f3c2846e50 2024-04-25T00:34:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.986+0000 7f605e836740 1 -- 172.21.15.42:0/4053253172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 msgr2=0x55f3c283d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.986+0000 7f605e836740 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c283d110 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f604809bd50 tx=0x7f604809bae0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.986+0000 7f605e836740 1 -- 172.21.15.42:0/4053253172 shutdown_connections 2024-04-25T00:34:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.986+0000 7f605e836740 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c284c9b0 0x55f3c28458b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.986+0000 7f605e836740 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c2847840 0x55f3c28584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.986+0000 7f605e836740 1 --2- 172.21.15.42:0/4053253172 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c283d110 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.986+0000 7f605e836740 1 -- 172.21.15.42:0/4053253172 >> 172.21.15.42:0/4053253172 conn(0x55f3c28424c0 msgr2=0x55f3c28451a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.986+0000 7f605e836740 1 -- 172.21.15.42:0/4053253172 shutdown_connections 2024-04-25T00:34:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.986+0000 7f605e836740 1 -- 172.21.15.42:0/4053253172 wait complete. 2024-04-25T00:34:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.987+0000 7f605e836740 1 -- start start 2024-04-25T00:34:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.987+0000 7f605e836740 1 Processor -- start 2024-04-25T00:34:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.987+0000 7f605e836740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2857f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.987+0000 7f605e836740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c2847840 0x55f3c2858470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f605e836740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c284c9b0 0x55f3c2856560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f605e836740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f60541b5ce0 con 0x55f3c284c9b0 2024-04-25T00:34:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f605e836740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f60541b7150 con 0x55f3c2846e50 2024-04-25T00:34:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f605e836740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f60541b63b0 con 0x55f3c2847840 2024-04-25T00:34:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f6059000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2857f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f6059000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2857f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45806/0 (socket says 172.21.15.42:45806) 2024-04-25T00:34:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f6059000640 1 -- 172.21.15.42:0/1932384313 learned_addr learned my addr 172.21.15.42:0/1932384313 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f6059000640 1 -- 172.21.15.42:0/1932384313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c2847840 msgr2=0x55f3c2858470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:34:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f6059000640 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c2847840 0x55f3c2858470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f6059000640 1 -- 172.21.15.42:0/1932384313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c284c9b0 msgr2=0x55f3c2856560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f6059000640 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c284c9b0 0x55f3c2856560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f6059000640 1 -- 172.21.15.42:0/1932384313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3c2856d90 con 0x55f3c2846e50 2024-04-25T00:34:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f605e836740 10 client.0 _ll_register_callbacks cb 0x7f6040004210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.988+0000 7f605e836740 1 -- 172.21.15.42:0/1932384313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60480bc110 con 0x55f3c2846e50 2024-04-25T00:34:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.989+0000 7f6059000640 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2857f30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f604809a300 tx=0x7f6048003dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a0bea6500a1897 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.989+0000 7f603aa00640 10 client.26441 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.989+0000 7f603aa00640 1 -- 172.21.15.42:0/1932384313 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60480a4a30 con 0x55f3c2846e50 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.989+0000 7f603aa00640 1 -- 172.21.15.42:0/1932384313 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f6048096bc0 con 0x55f3c2846e50 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.989+0000 7f603aa00640 1 -- 172.21.15.42:0/1932384313 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f6048096dc0 con 0x55f3c2846e50 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.989+0000 7f605e836740 10 client.26441 fetch_fsmap learned FSMap version 41 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.989+0000 7f605e836740 1 -- 172.21.15.42:0/1932384313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=41}) -- 0x55f3c28c3b70 con 0x55f3c2846e50 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.990+0000 7f603aa00640 1 -- 172.21.15.42:0/1932384313 <== mon.2 v2:172.21.15.106:3300/0 4 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f60480b7070 con 0x55f3c2846e50 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.990+0000 7f605e836740 10 client.26441 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.990+0000 7f605e836740 10 client.26441 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.990+0000 7f605e836740 20 client.26441 populate_metadata read hostname 'smithi042' 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.990+0000 7f605e836740 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3c28c3e50 0x55f3c28c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.990+0000 7f605e836740 4 client.26441 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:34:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.990+0000 7f605e836740 1 -- 172.21.15.42:0/1932384313 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3c27b1340 con 0x55f3c28c3e50 2024-04-25T00:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.990+0000 7f6053e00640 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3c28c3e50 0x55f3c28c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.991+0000 7f6053e00640 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3c28c3e50 0x55f3c28c6310 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74e015def3f92f97 server_cookie=f4995d812f329c5 in_seq=0 out_seq=0 2024-04-25T00:34:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:32.991+0000 7f603aa00640 10 client.26441 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f603aa00640 1 -- 172.21.15.42:0/1932384313 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3c27b1340 con 0x55f3c28c3e50 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f603aa00640 10 client.26441 handle_command_reply: tid=0 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f605e836740 1 client.26441 shutdown 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f605e836740 1 -- 172.21.15.42:0/1932384313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 msgr2=0x55f3c2857f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f605e836740 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2857f30 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f604809a300 tx=0x7f6048003dd0 comp rx=0 tx=0).stop 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f605e836740 1 -- 172.21.15.42:0/1932384313 shutdown_connections 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f605e836740 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f3c28c3e50 0x55f3c28c6310 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f605e836740 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f3c284c9b0 0x55f3c2856560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f605e836740 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f3c2847840 0x55f3c2858470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f605e836740 1 --2- 172.21.15.42:0/1932384313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f3c2846e50 0x55f3c2857f30 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f605e836740 1 -- 172.21.15.42:0/1932384313 >> 172.21.15.42:0/1932384313 conn(0x55f3c28424c0 msgr2=0x55f3c2840d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f603aa00640 0 client.26441 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.251+0000 7f605e836740 1 -- 172.21.15.42:0/1932384313 shutdown_connections 2024-04-25T00:34:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.252+0000 7f605e836740 1 -- 172.21.15.42:0/1932384313 wait complete. 2024-04-25T00:34:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.252+0000 7f605e836740 20 client.26441 trim_cache size 0 max 16384 2024-04-25T00:34:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:34:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:34:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:34:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:34:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.253+0000 7f605a400640 1 -- 172.21.15.42:0/2220571314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f603c0a2c80 msgr2=0x7f603c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.253+0000 7f605a400640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f603c0a2c80 0x7f603c0a5140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f604c005fd0 tx=0x7f604c005dc0 comp rx=0 tx=0).stop 2024-04-25T00:34:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.253+0000 7f605a400640 1 -- 172.21.15.42:0/2220571314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6054103800 msgr2=0x7f6054071130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.253+0000 7f605a400640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6054103800 0x7f6054071130 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f6048005fd0 tx=0x7f604800ef90 comp rx=0 tx=0).stop 2024-04-25T00:34:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.253+0000 7f605a400640 1 -- 172.21.15.42:0/2220571314 shutdown_connections 2024-04-25T00:34:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.253+0000 7f605a400640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f603c0a2c80 0x7f603c0a5140 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.253+0000 7f605a400640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f60541053b0 0x7f60540726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.253+0000 7f605a400640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f60541049f0 0x7f6054071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.253+0000 7f605a400640 1 --2- 172.21.15.42:0/2220571314 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6054103800 0x7f6054071130 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.253+0000 7f605a400640 1 -- 172.21.15.42:0/2220571314 >> 172.21.15.42:0/2220571314 conn(0x7f60540fed70 msgr2=0x7f6054107c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:33.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.254+0000 7f605a400640 1 -- 172.21.15.42:0/2220571314 shutdown_connections 2024-04-25T00:34:33.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:33.256+0000 7f605a400640 1 -- 172.21.15.42:0/2220571314 wait complete. 2024-04-25T00:34:33.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 91346 34.80 MB/sec execute 23 sec latency 160.856 ms 2024-04-25T00:34:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:33 smithi106 ceph-mon[32510]: pgmap v864: 97 pgs: 97 active+clean; 544 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 725 op/s 2024-04-25T00:34:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:33 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:34:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:33 smithi042 ceph-mon[28130]: pgmap v864: 97 pgs: 97 active+clean; 544 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 725 op/s 2024-04-25T00:34:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:33 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:34:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:33 smithi136 ceph-mon[32923]: pgmap v864: 97 pgs: 97 active+clean; 544 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 7.4 MiB/s wr, 725 op/s 2024-04-25T00:34:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:33 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:34:34.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 92129 35.50 MB/sec execute 24 sec latency 41.549 ms 2024-04-25T00:34:35.746 INFO:tasks.workunit.client.0.smithi042.stdout: 10 92389 34.74 MB/sec execute 25 sec latency 580.517 ms 2024-04-25T00:34:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:35 smithi042 ceph-mon[28130]: pgmap v865: 97 pgs: 97 active+clean; 540 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.2 MiB/s wr, 503 op/s 2024-04-25T00:34:36.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:35 smithi136 ceph-mon[32923]: pgmap v865: 97 pgs: 97 active+clean; 540 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.2 MiB/s wr, 503 op/s 2024-04-25T00:34:36.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:35 smithi106 ceph-mon[32510]: pgmap v865: 97 pgs: 97 active+clean; 540 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.2 MiB/s wr, 503 op/s 2024-04-25T00:34:36.747 INFO:tasks.workunit.client.0.smithi042.stdout: 10 92769 34.24 MB/sec execute 26 sec latency 832.678 ms 2024-04-25T00:34:37.747 INFO:tasks.workunit.client.0.smithi042.stdout: 10 93099 33.64 MB/sec execute 27 sec latency 142.926 ms 2024-04-25T00:34:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:37 smithi042 ceph-mon[28130]: pgmap v866: 97 pgs: 97 active+clean; 563 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 721 op/s 2024-04-25T00:34:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:37 smithi136 ceph-mon[32923]: pgmap v866: 97 pgs: 97 active+clean; 563 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 721 op/s 2024-04-25T00:34:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:37 smithi106 ceph-mon[32510]: pgmap v866: 97 pgs: 97 active+clean; 563 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 721 op/s 2024-04-25T00:34:38.747 INFO:tasks.workunit.client.0.smithi042.stdout: 10 93710 33.25 MB/sec execute 28 sec latency 58.408 ms 2024-04-25T00:34:39.747 INFO:tasks.workunit.client.0.smithi042.stdout: 10 94546 33.65 MB/sec execute 29 sec latency 49.620 ms 2024-04-25T00:34:40.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:39 smithi042 ceph-mon[28130]: pgmap v867: 97 pgs: 97 active+clean; 579 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 708 op/s 2024-04-25T00:34:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:39 smithi136 ceph-mon[32923]: pgmap v867: 97 pgs: 97 active+clean; 579 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 708 op/s 2024-04-25T00:34:40.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:39 smithi106 ceph-mon[32510]: pgmap v867: 97 pgs: 97 active+clean; 579 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 708 op/s 2024-04-25T00:34:40.747 INFO:tasks.workunit.client.0.smithi042.stdout: 10 95469 34.27 MB/sec execute 30 sec latency 42.550 ms 2024-04-25T00:34:41.747 INFO:tasks.workunit.client.0.smithi042.stdout: 10 96301 34.90 MB/sec execute 31 sec latency 34.180 ms 2024-04-25T00:34:42.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:41 smithi042 ceph-mon[28130]: pgmap v868: 97 pgs: 97 active+clean; 569 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 745 op/s 2024-04-25T00:34:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:41 smithi136 ceph-mon[32923]: pgmap v868: 97 pgs: 97 active+clean; 569 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 745 op/s 2024-04-25T00:34:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:41 smithi106 ceph-mon[32510]: pgmap v868: 97 pgs: 97 active+clean; 569 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 13 MiB/s wr, 745 op/s 2024-04-25T00:34:42.747 INFO:tasks.workunit.client.0.smithi042.stdout: 10 96792 34.58 MB/sec execute 32 sec latency 80.355 ms 2024-04-25T00:34:43.747 INFO:tasks.workunit.client.0.smithi042.stdout: 10 97398 34.26 MB/sec execute 33 sec latency 90.609 ms 2024-04-25T00:34:44.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:43 smithi042 ceph-mon[28130]: pgmap v869: 97 pgs: 97 active+clean; 522 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 948 op/s 2024-04-25T00:34:44.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:43 smithi136 ceph-mon[32923]: pgmap v869: 97 pgs: 97 active+clean; 522 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 948 op/s 2024-04-25T00:34:44.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:43 smithi106 ceph-mon[32510]: pgmap v869: 97 pgs: 97 active+clean; 522 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 948 op/s 2024-04-25T00:34:44.747 INFO:tasks.workunit.client.0.smithi042.stdout: 10 98285 34.60 MB/sec execute 34 sec latency 38.976 ms 2024-04-25T00:34:45.747 INFO:tasks.workunit.client.0.smithi042.stdout: 10 99289 35.45 MB/sec execute 35 sec latency 35.920 ms 2024-04-25T00:34:46.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:45 smithi042 ceph-mon[28130]: pgmap v870: 97 pgs: 97 active+clean; 522 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-04-25T00:34:46.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:45 smithi136 ceph-mon[32923]: pgmap v870: 97 pgs: 97 active+clean; 522 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-04-25T00:34:46.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:45 smithi106 ceph-mon[32510]: pgmap v870: 97 pgs: 97 active+clean; 522 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 11 MiB/s wr, 731 op/s 2024-04-25T00:34:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.434+0000 7f468b000640 1 -- 172.21.15.42:0/4225517749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684077340 msgr2=0x7f46840777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.434+0000 7f468b000640 1 --2- 172.21.15.42:0/4225517749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684077340 0x7f46840777a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4674009900 tx=0x7f467402a420 comp rx=0 tx=0).stop 2024-04-25T00:34:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.435+0000 7f468b000640 1 -- 172.21.15.42:0/4225517749 shutdown_connections 2024-04-25T00:34:46.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.435+0000 7f468b000640 1 --2- 172.21.15.42:0/4225517749 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4684077ce0 0x7f4684078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.435+0000 7f468b000640 1 --2- 172.21.15.42:0/4225517749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684077340 0x7f46840777a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.435+0000 7f468b000640 1 --2- 172.21.15.42:0/4225517749 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4684076100 0x7f4684076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.435+0000 7f468b000640 1 -- 172.21.15.42:0/4225517749 >> 172.21.15.42:0/4225517749 conn(0x7f46840fef60 msgr2=0x7f4684101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.435+0000 7f468b000640 1 -- 172.21.15.42:0/4225517749 shutdown_connections 2024-04-25T00:34:46.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.435+0000 7f468b000640 1 -- 172.21.15.42:0/4225517749 wait complete. 2024-04-25T00:34:46.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.437+0000 7f468b000640 1 Processor -- start 2024-04-25T00:34:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.437+0000 7f468b000640 1 -- start start 2024-04-25T00:34:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.437+0000 7f468b000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4684076100 0x7f468419e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.437+0000 7f468b000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684077340 0x7f468419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.437+0000 7f468b000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4684077ce0 0x7f46841a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.437+0000 7f468b000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4684113b80 con 0x7f4684076100 2024-04-25T00:34:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f468b000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4684113a00 con 0x7f4684077ce0 2024-04-25T00:34:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f468b000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4684113d00 con 0x7f4684077340 2024-04-25T00:34:46.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f4683400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684077340 0x7f468419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f4683400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684077340 0x7f468419e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60428/0 (socket says 172.21.15.42:60428) 2024-04-25T00:34:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f4683400640 1 -- 172.21.15.42:0/1673553999 learned_addr learned my addr 172.21.15.42:0/1673553999 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:46.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f4689000640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4684077ce0 0x7f46841a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f4683400640 1 -- 172.21.15.42:0/1673553999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4684077ce0 msgr2=0x7f46841a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f4683400640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4684077ce0 0x7f46841a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f4683400640 1 -- 172.21.15.42:0/1673553999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4684076100 msgr2=0x7f468419e160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:34:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f4683400640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4684076100 0x7f468419e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.438+0000 7f4683400640 1 -- 172.21.15.42:0/1673553999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4684103530 con 0x7f4684077340 2024-04-25T00:34:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.439+0000 7f4683400640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684077340 0x7f468419e6a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f467402aa10 tx=0x7f4674032da0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.439+0000 7f4680c00640 1 -- 172.21.15.42:0/1673553999 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467403f030 con 0x7f4684077340 2024-04-25T00:34:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.439+0000 7f4680c00640 1 -- 172.21.15.42:0/1673553999 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4674003ce0 con 0x7f4684077340 2024-04-25T00:34:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.439+0000 7f4680c00640 1 -- 172.21.15.42:0/1673553999 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4674035070 con 0x7f4684077340 2024-04-25T00:34:46.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.439+0000 7f468b000640 1 -- 172.21.15.42:0/1673553999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46841037c0 con 0x7f4684077340 2024-04-25T00:34:46.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.440+0000 7f468b000640 1 -- 172.21.15.42:0/1673553999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4684103ce0 con 0x7f4684077340 2024-04-25T00:34:46.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.445+0000 7f468ddc9740 1 Processor -- start 2024-04-25T00:34:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.446+0000 7f468ddc9740 1 -- start start 2024-04-25T00:34:46.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.446+0000 7f4680c00640 1 -- 172.21.15.42:0/1673553999 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4674042030 con 0x7f4684077340 2024-04-25T00:34:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.446+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a09ca0 0x55c626a0a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.446+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0a690 0x55c626a0f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.447+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a11c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.447+0000 7f4689000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a09ca0 0x55c626a0a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.447+0000 7f4689000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a09ca0 0x55c626a0a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46790/0 (socket says 172.21.15.42:46790) 2024-04-25T00:34:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.448+0000 7f468ddc9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c62696cab0 con 0x55c626a0f800 2024-04-25T00:34:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.448+0000 7f468ddc9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c626915640 con 0x55c626a09ca0 2024-04-25T00:34:46.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.448+0000 7f468ddc9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c626968a70 con 0x55c626a0a690 2024-04-25T00:34:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.448+0000 7f4680c00640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46680a2da0 0x7f46680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.448+0000 7f4689000640 1 -- 172.21.15.42:0/3915987814 learned_addr learned my addr 172.21.15.42:0/3915987814 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.448+0000 7f4680c00640 1 -- 172.21.15.42:0/1673553999 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f46740ef6c0 con 0x7f4684077340 2024-04-25T00:34:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.448+0000 7f4683400640 1 --2- 172.21.15.42:0/3915987814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a11c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.449+0000 7f4689000640 1 -- 172.21.15.42:0/3915987814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0a690 msgr2=0x55c626a0f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.447+0000 7f4683e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0a690 0x55c626a0f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.450+0000 7f4689000640 1 --2- 172.21.15.42:0/3915987814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0a690 0x55c626a0f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.450+0000 7f4689000640 1 -- 172.21.15.42:0/3915987814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 msgr2=0x55c626a11c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:46.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.450+0000 7f4689000640 1 --2- 172.21.15.42:0/3915987814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a11c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.450+0000 7f4689000640 1 -- 172.21.15.42:0/3915987814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c626681210 con 0x55c626a09ca0 2024-04-25T00:34:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.450+0000 7f4689000640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46680a2da0 0x7f46680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.450+0000 7f4683400640 1 --2- 172.21.15.42:0/3915987814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a11c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:34:46.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.450+0000 7f4683e00640 1 --2- 172.21.15.42:0/3915987814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0a690 0x55c626a0f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:34:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.452+0000 7f4689000640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46680a2da0 0x7f46680a5260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4670017000 tx=0x7f4670019000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.452+0000 7f4689000640 1 --2- 172.21.15.42:0/3915987814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a09ca0 0x55c626a0a0c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4670009b00 tx=0x7f46700101f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be01e774e60096e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.452+0000 7f4679600640 1 -- 172.21.15.42:0/3915987814 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467001c400 con 0x55c626a09ca0 2024-04-25T00:34:46.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.452+0000 7f4679600640 1 -- 172.21.15.42:0/3915987814 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467001ca80 con 0x55c626a09ca0 2024-04-25T00:34:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.453+0000 7f468ddc9740 1 -- 172.21.15.42:0/3915987814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a09ca0 msgr2=0x55c626a0a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.453+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3915987814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a09ca0 0x55c626a0a0c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4670009b00 tx=0x7f46700101f0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.453+0000 7f468ddc9740 1 -- 172.21.15.42:0/3915987814 shutdown_connections 2024-04-25T00:34:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.453+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3915987814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a11c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.453+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3915987814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0a690 0x55c626a0f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.453+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3915987814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a09ca0 0x55c626a0a0c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.453+0000 7f468ddc9740 1 -- 172.21.15.42:0/3915987814 >> 172.21.15.42:0/3915987814 conn(0x55c626a05310 msgr2=0x55c626a05710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.454+0000 7f468ddc9740 1 -- 172.21.15.42:0/3915987814 shutdown_connections 2024-04-25T00:34:46.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.454+0000 7f468ddc9740 1 -- 172.21.15.42:0/3915987814 wait complete. 2024-04-25T00:34:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.454+0000 7f468ddc9740 1 -- start start 2024-04-25T00:34:46.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.455+0000 7f468ddc9740 1 Processor -- start 2024-04-25T00:34:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.455+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a13ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.455+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a14420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.456+0000 7f4683e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a13ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.456+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a1b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.456+0000 7f4683e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a13ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60442/0 (socket says 172.21.15.42:60442) 2024-04-25T00:34:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.456+0000 7f4683400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a14420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.456+0000 7f4683400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a14420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46800/0 (socket says 172.21.15.42:46800) 2024-04-25T00:34:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.456+0000 7f468ddc9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c626a12850 con 0x55c626a0f800 2024-04-25T00:34:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.456+0000 7f468ddc9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c626928990 con 0x55c626a0a690 2024-04-25T00:34:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.456+0000 7f468ddc9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c626a129d0 con 0x55c626a09ca0 2024-04-25T00:34:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.457+0000 7f4683e00640 1 -- 172.21.15.42:0/446453563 learned_addr learned my addr 172.21.15.42:0/446453563 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.457+0000 7f468ddc9740 10 client.0 _ll_register_callbacks cb 0x55c62692a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:46.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.457+0000 7f4689000640 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a1b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.457+0000 7f4683e00640 1 -- 172.21.15.42:0/446453563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 msgr2=0x55c626a14420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.457+0000 7f4683e00640 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a14420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.457+0000 7f4683e00640 1 -- 172.21.15.42:0/446453563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 msgr2=0x55c626a1b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.457+0000 7f4683e00640 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a1b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.457+0000 7f4683e00640 1 -- 172.21.15.42:0/446453563 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c62674d620 con 0x55c626a09ca0 2024-04-25T00:34:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.459+0000 7f4683e00640 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a13ee0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c380 tx=0x7f467c00c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a049ef3baacd4b4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.459+0000 7f466e000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:34:46.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.459+0000 7f466e000640 1 -- 172.21.15.42:0/446453563 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c014070 con 0x55c626a09ca0 2024-04-25T00:34:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.459+0000 7f466e000640 1 -- 172.21.15.42:0/446453563 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f467c010870 con 0x55c626a09ca0 2024-04-25T00:34:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.459+0000 7f468ddc9740 1 -- 172.21.15.42:0/446453563 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f467001ca80 con 0x55c626a09ca0 2024-04-25T00:34:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.460+0000 7f466e000640 1 -- 172.21.15.42:0/446453563 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f467c020050 con 0x55c626a09ca0 2024-04-25T00:34:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.460+0000 7f468ddc9740 10 client.26449 fetch_fsmap learned FSMap version 41 2024-04-25T00:34:46.462 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.460+0000 7f468ddc9740 1 -- 172.21.15.42:0/446453563 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c626a1e3f0 con 0x55c626a09ca0 2024-04-25T00:34:46.463 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.460+0000 7f466e000640 1 -- 172.21.15.42:0/446453563 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467001ca80 con 0x55c626a09ca0 2024-04-25T00:34:46.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.460+0000 7f466e000640 1 -- 172.21.15.42:0/446453563 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f467c00f3e0 con 0x55c626a09ca0 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.461+0000 7f468ddc9740 10 client.26449 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.461+0000 7f468ddc9740 10 client.26449 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.461+0000 7f468ddc9740 20 client.26449 populate_metadata read hostname 'smithi042' 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.462+0000 7f468ddc9740 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c626a1fae0 0x55c626a21f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.462+0000 7f468ddc9740 4 client.26449 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.462+0000 7f468ddc9740 1 -- 172.21.15.42:0/446453563 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c626979ff0 con 0x55c626a1fae0 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.462+0000 7f4683400640 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c626a1fae0 0x55c626a21f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.463+0000 7f4683400640 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c626a1fae0 0x55c626a21f30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d941f95d8fc49f2 server_cookie=bf5e1aba78e3fae5 in_seq=0 out_seq=0 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.463+0000 7f466e000640 10 client.26449 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.464+0000 7f466e000640 1 -- 172.21.15.42:0/446453563 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c626979ff0 con 0x55c626a1fae0 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.464+0000 7f466e000640 10 client.26449 handle_command_reply: tid=0 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.464+0000 7f468ddc9740 1 client.26449 shutdown 2024-04-25T00:34:46.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.464+0000 7f468ddc9740 1 -- 172.21.15.42:0/446453563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 msgr2=0x55c626a13ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.464+0000 7f468ddc9740 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a13ee0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f467c00c380 tx=0x7f467c00c840 comp rx=0 tx=0).stop 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.465+0000 7f468ddc9740 1 -- 172.21.15.42:0/446453563 shutdown_connections 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.465+0000 7f468ddc9740 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c626a1fae0 0x55c626a21f30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.465+0000 7f468ddc9740 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a1b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.465+0000 7f468ddc9740 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a14420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.465+0000 7f468ddc9740 1 --2- 172.21.15.42:0/446453563 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a13ee0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.465+0000 7f468ddc9740 1 -- 172.21.15.42:0/446453563 >> 172.21.15.42:0/446453563 conn(0x55c626a05310 msgr2=0x55c626a04480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.465+0000 7f466e000640 0 client.26449 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.465+0000 7f468ddc9740 1 -- 172.21.15.42:0/446453563 shutdown_connections 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.465+0000 7f468ddc9740 1 -- 172.21.15.42:0/446453563 wait complete. 2024-04-25T00:34:46.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.466+0000 7f468ddc9740 20 client.26449 trim_cache size 0 max 16384 2024-04-25T00:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.495+0000 7f468ddc9740 1 Processor -- start 2024-04-25T00:34:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f468ddc9740 1 -- start start 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a04300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a04840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a1b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f468ddc9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f46841ba330 con 0x55c626a0f800 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f468ddc9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f46841b7b40 con 0x55c626a0a690 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f468ddc9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f468410a940 con 0x55c626a09ca0 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f4683e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a04300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f4683400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a04840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f4683e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a04300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60454/0 (socket says 172.21.15.42:60454) 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f4683e00640 1 -- 172.21.15.42:0/3302204839 learned_addr learned my addr 172.21.15.42:0/3302204839 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:46.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.496+0000 7f4683400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a04840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46812/0 (socket says 172.21.15.42:46812) 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.497+0000 7f4689000640 1 --2- 172.21.15.42:0/3302204839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a1b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.497+0000 7f4683e00640 1 -- 172.21.15.42:0/3302204839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 msgr2=0x55c626a04840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.497+0000 7f4683e00640 1 --2- 172.21.15.42:0/3302204839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a04840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.497+0000 7f4683e00640 1 -- 172.21.15.42:0/3302204839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 msgr2=0x55c626a1b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.497+0000 7f4683e00640 1 --2- 172.21.15.42:0/3302204839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a1b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.497+0000 7f4683e00640 1 -- 172.21.15.42:0/3302204839 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c626a05fe0 con 0x55c626a09ca0 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.497+0000 7f4683400640 1 --2- 172.21.15.42:0/3302204839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a04840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.497+0000 7f4683e00640 1 --2- 172.21.15.42:0/3302204839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a04300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f467c00d010 tx=0x7f467c00ce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8629a02a1538cba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.497+0000 7f4678c00640 1 -- 172.21.15.42:0/3302204839 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c02a030 con 0x55c626a09ca0 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.498+0000 7f4678c00640 1 -- 172.21.15.42:0/3302204839 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467c0077a0 con 0x55c626a09ca0 2024-04-25T00:34:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.498+0000 7f4678c00640 1 -- 172.21.15.42:0/3302204839 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467c014040 con 0x55c626a09ca0 2024-04-25T00:34:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.498+0000 7f468ddc9740 1 -- 172.21.15.42:0/3302204839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 msgr2=0x55c626a04300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.498+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3302204839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a04300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f467c00d010 tx=0x7f467c00ce20 comp rx=0 tx=0).stop 2024-04-25T00:34:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.499+0000 7f468ddc9740 1 -- 172.21.15.42:0/3302204839 shutdown_connections 2024-04-25T00:34:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.499+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3302204839 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a0f800 0x55c626a1b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.499+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3302204839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a04840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.499+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3302204839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a09ca0 0x55c626a04300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.499+0000 7f468ddc9740 1 -- 172.21.15.42:0/3302204839 >> 172.21.15.42:0/3302204839 conn(0x55c626a05310 msgr2=0x55c626a03bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.499+0000 7f468ddc9740 1 -- 172.21.15.42:0/3302204839 shutdown_connections 2024-04-25T00:34:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.499+0000 7f468ddc9740 1 -- 172.21.15.42:0/3302204839 wait complete. 2024-04-25T00:34:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.499+0000 7f468ddc9740 1 -- start start 2024-04-25T00:34:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f468ddc9740 1 Processor -- start 2024-04-25T00:34:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a09ca0 0x55c626a19080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a195c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f468ddc9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0f800 0x55c626a1a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f468ddc9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f468410afb0 con 0x55c626a09ca0 2024-04-25T00:34:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f468ddc9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f46841b7470 con 0x55c626a0a690 2024-04-25T00:34:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f468ddc9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f468410b680 con 0x55c626a0f800 2024-04-25T00:34:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f468ddc9740 10 client.0 _ll_register_callbacks cb 0x7f467c007510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:34:46.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f4683e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a09ca0 0x55c626a19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f4689000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0f800 0x55c626a1a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.500+0000 7f4689000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0f800 0x55c626a1a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:60470/0 (socket says 172.21.15.42:60470) 2024-04-25T00:34:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f4689000640 1 -- 172.21.15.42:0/3850354201 learned_addr learned my addr 172.21.15.42:0/3850354201 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:34:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f4683400640 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a195c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f4683400640 1 -- 172.21.15.42:0/3850354201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0f800 msgr2=0x55c626a1a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f4683400640 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0f800 0x55c626a1a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f4683400640 1 -- 172.21.15.42:0/3850354201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a09ca0 msgr2=0x55c626a19080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:34:46.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f4683400640 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a09ca0 0x55c626a19080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f4683400640 1 -- 172.21.15.42:0/3850354201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c626a1abc0 con 0x55c626a0a690 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f4689000640 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0f800 0x55c626a1a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f4683e00640 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a09ca0 0x55c626a19080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f4683400640 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a195c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4674005ec0 tx=0x7f467402a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6a7bbe741d0f06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f466e000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.501+0000 7f466e000640 1 -- 172.21.15.42:0/3850354201 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46740c4bc0 con 0x55c626a0a690 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.502+0000 7f466e000640 1 -- 172.21.15.42:0/3850354201 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f46740b85c0 con 0x55c626a0a690 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.502+0000 7f468ddc9740 1 -- 172.21.15.42:0/3850354201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f467c0077a0 con 0x55c626a0a690 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.502+0000 7f466e000640 1 -- 172.21.15.42:0/3850354201 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f467403b5f0 con 0x55c626a0a690 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.502+0000 7f468ddc9740 10 client.26456 fetch_fsmap learned FSMap version 41 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.502+0000 7f468ddc9740 1 -- 172.21.15.42:0/3850354201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c626a86700 con 0x55c626a0a690 2024-04-25T00:34:46.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.502+0000 7f466e000640 1 -- 172.21.15.42:0/3850354201 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f467c0077a0 con 0x55c626a0a690 2024-04-25T00:34:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.503+0000 7f466e000640 1 -- 172.21.15.42:0/3850354201 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f467404c060 con 0x55c626a0a690 2024-04-25T00:34:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.503+0000 7f468ddc9740 10 client.26456 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T00:34:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.503+0000 7f468ddc9740 10 client.26456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:34:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.503+0000 7f468ddc9740 20 client.26456 populate_metadata read hostname 'smithi042' 2024-04-25T00:34:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.503+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c626a87100 0x55c626a89520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:34:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.503+0000 7f468ddc9740 4 client.26456 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:34:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.503+0000 7f468ddc9740 1 -- 172.21.15.42:0/3850354201 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c626a0a450 con 0x55c626a87100 2024-04-25T00:34:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.503+0000 7f4683e00640 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c626a87100 0x55c626a89520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:34:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.504+0000 7f4683e00640 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c626a87100 0x55c626a89520 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20292df95e6e5943 server_cookie=4f83587d3240e34a in_seq=0 out_seq=0 2024-04-25T00:34:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.504+0000 7f466e000640 10 client.26456 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:46.749 INFO:tasks.workunit.client.0.smithi042.stdout: 10 99900 35.48 MB/sec execute 36 sec latency 189.460 ms 2024-04-25T00:34:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f466e000640 1 -- 172.21.15.42:0/3850354201 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c626a0a450 con 0x55c626a87100 2024-04-25T00:34:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f466e000640 10 client.26456 handle_command_reply: tid=0 2024-04-25T00:34:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f468ddc9740 1 client.26456 shutdown 2024-04-25T00:34:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f468ddc9740 1 -- 172.21.15.42:0/3850354201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 msgr2=0x55c626a195c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a195c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f4674005ec0 tx=0x7f467402a420 comp rx=0 tx=0).stop 2024-04-25T00:34:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f468ddc9740 1 -- 172.21.15.42:0/3850354201 shutdown_connections 2024-04-25T00:34:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c626a87100 0x55c626a89520 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c626a0f800 0x55c626a1a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c626a0a690 0x55c626a195c0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f468ddc9740 1 --2- 172.21.15.42:0/3850354201 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c626a09ca0 0x55c626a19080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f468ddc9740 1 -- 172.21.15.42:0/3850354201 >> 172.21.15.42:0/3850354201 conn(0x55c626a05310 msgr2=0x55c626a04480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.819+0000 7f466e000640 0 client.26456 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:34:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.820+0000 7f468ddc9740 1 -- 172.21.15.42:0/3850354201 shutdown_connections 2024-04-25T00:34:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.820+0000 7f468ddc9740 1 -- 172.21.15.42:0/3850354201 wait complete. 2024-04-25T00:34:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.820+0000 7f468ddc9740 20 client.26456 trim_cache size 0 max 16384 2024-04-25T00:34:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:34:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:34:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:34:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:34:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.821+0000 7f468b000640 1 -- 172.21.15.42:0/1673553999 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46680a2da0 msgr2=0x7f46680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.821+0000 7f468b000640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46680a2da0 0x7f46680a5260 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4670017000 tx=0x7f4670019000 comp rx=0 tx=0).stop 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.821+0000 7f468b000640 1 -- 172.21.15.42:0/1673553999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684077340 msgr2=0x7f468419e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.821+0000 7f468b000640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684077340 0x7f468419e6a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f467402aa10 tx=0x7f4674032da0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.822+0000 7f468b000640 1 -- 172.21.15.42:0/1673553999 shutdown_connections 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.822+0000 7f468b000640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f46680a2da0 0x7f46680a5260 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.822+0000 7f468b000640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4684077ce0 0x7f46841a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.822+0000 7f468b000640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4684077340 0x7f468419e6a0 secure :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f467402aa10 tx=0x7f4674032da0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.822+0000 7f468b000640 1 --2- 172.21.15.42:0/1673553999 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4684076100 0x7f468419e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.822+0000 7f468b000640 1 -- 172.21.15.42:0/1673553999 >> 172.21.15.42:0/1673553999 conn(0x7f46840fef60 msgr2=0x7f4684100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.822+0000 7f468b000640 1 -- 172.21.15.42:0/1673553999 shutdown_connections 2024-04-25T00:34:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:34:46.822+0000 7f468b000640 1 -- 172.21.15.42:0/1673553999 wait complete. 2024-04-25T00:34:47.749 INFO:tasks.workunit.client.0.smithi042.stdout: 10 100325 35.01 MB/sec execute 37 sec latency 203.909 ms 2024-04-25T00:34:48.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:47 smithi042 ceph-mon[28130]: pgmap v871: 97 pgs: 97 active+clean; 536 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 888 op/s 2024-04-25T00:34:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:47 smithi136 ceph-mon[32923]: pgmap v871: 97 pgs: 97 active+clean; 536 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 888 op/s 2024-04-25T00:34:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:47 smithi106 ceph-mon[32510]: pgmap v871: 97 pgs: 97 active+clean; 536 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 12 MiB/s wr, 888 op/s 2024-04-25T00:34:48.749 INFO:tasks.workunit.client.0.smithi042.stdout: 10 100945 34.86 MB/sec execute 38 sec latency 84.986 ms 2024-04-25T00:34:49.749 INFO:tasks.workunit.client.0.smithi042.stdout: 10 101706 34.93 MB/sec execute 39 sec latency 110.655 ms 2024-04-25T00:34:50.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:49 smithi042 ceph-mon[28130]: pgmap v872: 97 pgs: 97 active+clean; 532 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-04-25T00:34:50.139 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:49 smithi106 ceph-mon[32510]: pgmap v872: 97 pgs: 97 active+clean; 532 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-04-25T00:34:50.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:49 smithi136 ceph-mon[32923]: pgmap v872: 97 pgs: 97 active+clean; 532 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 7.8 MiB/s wr, 711 op/s 2024-04-25T00:34:50.749 INFO:tasks.workunit.client.0.smithi042.stdout: 10 102702 35.45 MB/sec execute 40 sec latency 50.417 ms 2024-04-25T00:34:51.749 INFO:tasks.workunit.client.0.smithi042.stdout: 10 103467 35.80 MB/sec execute 41 sec latency 34.524 ms 2024-04-25T00:34:52.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:51 smithi136 ceph-mon[32923]: pgmap v873: 97 pgs: 97 active+clean; 537 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 660 op/s 2024-04-25T00:34:52.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:51 smithi106 ceph-mon[32510]: pgmap v873: 97 pgs: 97 active+clean; 537 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 660 op/s 2024-04-25T00:34:52.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:51 smithi042 ceph-mon[28130]: pgmap v873: 97 pgs: 97 active+clean; 537 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.6 MiB/s wr, 660 op/s 2024-04-25T00:34:52.749 INFO:tasks.workunit.client.0.smithi042.stdout: 10 103997 35.56 MB/sec execute 42 sec latency 53.130 ms 2024-04-25T00:34:53.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 104590 35.31 MB/sec execute 43 sec latency 66.180 ms 2024-04-25T00:34:54.180 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:53 smithi042 ceph-mon[28130]: pgmap v874: 97 pgs: 97 active+clean; 546 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.1 MiB/s wr, 742 op/s 2024-04-25T00:34:54.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:53 smithi136 ceph-mon[32923]: pgmap v874: 97 pgs: 97 active+clean; 546 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.1 MiB/s wr, 742 op/s 2024-04-25T00:34:54.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:53 smithi106 ceph-mon[32510]: pgmap v874: 97 pgs: 97 active+clean; 546 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.1 MiB/s wr, 742 op/s 2024-04-25T00:34:54.749 INFO:tasks.workunit.client.0.smithi042.stdout: 10 105056 35.10 MB/sec execute 44 sec latency 448.129 ms 2024-04-25T00:34:55.749 INFO:tasks.workunit.client.0.smithi042.stdout: 10 106060 35.45 MB/sec execute 45 sec latency 49.996 ms 2024-04-25T00:34:56.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:55 smithi136 ceph-mon[32923]: pgmap v875: 97 pgs: 97 active+clean; 548 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.6 MiB/s wr, 516 op/s 2024-04-25T00:34:56.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:55 smithi106 ceph-mon[32510]: pgmap v875: 97 pgs: 97 active+clean; 548 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.6 MiB/s wr, 516 op/s 2024-04-25T00:34:56.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:55 smithi042 ceph-mon[28130]: pgmap v875: 97 pgs: 97 active+clean; 548 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 3.6 MiB/s wr, 516 op/s 2024-04-25T00:34:56.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 106747 35.69 MB/sec execute 46 sec latency 299.988 ms 2024-04-25T00:34:57.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 107336 35.62 MB/sec execute 47 sec latency 55.290 ms 2024-04-25T00:34:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:57 smithi136 ceph-mon[32923]: pgmap v876: 97 pgs: 97 active+clean; 562 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.1 MiB/s wr, 690 op/s 2024-04-25T00:34:58.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:57 smithi106 ceph-mon[32510]: pgmap v876: 97 pgs: 97 active+clean; 562 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.1 MiB/s wr, 690 op/s 2024-04-25T00:34:58.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:57 smithi042 ceph-mon[28130]: pgmap v876: 97 pgs: 97 active+clean; 562 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 5.1 MiB/s wr, 690 op/s 2024-04-25T00:34:58.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 107901 35.54 MB/sec execute 48 sec latency 47.928 ms 2024-04-25T00:34:59.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 108613 35.34 MB/sec execute 49 sec latency 45.127 ms 2024-04-25T00:35:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.014+0000 7f801c600640 1 -- 172.21.15.42:0/682161063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8014104b90 msgr2=0x7f8014105010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.014+0000 7f801c600640 1 --2- 172.21.15.42:0/682161063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8014104b90 0x7f8014105010 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8000009900 tx=0x7f800002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.014+0000 7f801c600640 1 -- 172.21.15.42:0/682161063 shutdown_connections 2024-04-25T00:35:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.014+0000 7f801c600640 1 --2- 172.21.15.42:0/682161063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8014105550 0x7f801410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.014+0000 7f801c600640 1 --2- 172.21.15.42:0/682161063 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8014104b90 0x7f8014105010 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.014+0000 7f801c600640 1 --2- 172.21.15.42:0/682161063 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f80141039a0 0x7f8014103dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.014+0000 7f801c600640 1 -- 172.21.15.42:0/682161063 >> 172.21.15.42:0/682161063 conn(0x7f80140fef30 msgr2=0x7f8014101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.014+0000 7f801c600640 1 -- 172.21.15.42:0/682161063 shutdown_connections 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.014+0000 7f801c600640 1 -- 172.21.15.42:0/682161063 wait complete. 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.015+0000 7f801c600640 1 Processor -- start 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f801c600640 1 -- start start 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f801c600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80141039a0 0x7f80141957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f801c600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8014104b90 0x7f8014195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f801c600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8014105550 0x7f801419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f801c600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f801410f670 con 0x7f8014105550 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f801c600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f801410f4f0 con 0x7f8014104b90 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f801c600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f801410f7f0 con 0x7f80141039a0 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f8019c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80141039a0 0x7f80141957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f8019c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80141039a0 0x7f80141957e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:32818/0 (socket says 172.21.15.42:32818) 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f8019c00640 1 -- 172.21.15.42:0/2236423913 learned_addr learned my addr 172.21.15.42:0/2236423913 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f8019200640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8014104b90 0x7f8014195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.016+0000 7f801a600640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8014105550 0x7f801419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f8019c00640 1 -- 172.21.15.42:0/2236423913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8014104b90 msgr2=0x7f8014195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f8019c00640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8014104b90 0x7f8014195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f8019c00640 1 -- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8014105550 msgr2=0x7f801419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f8019c00640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8014105550 0x7f801419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f8019c00640 1 -- 172.21.15.42:0/2236423913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80141ae570 con 0x7f80141039a0 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f801a600640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8014105550 0x7f801419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f8019200640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8014104b90 0x7f8014195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f8019c00640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80141039a0 0x7f80141957e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f800800e9c0 tx=0x7f800800ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f8012000640 1 -- 172.21.15.42:0/2236423913 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800800cde0 con 0x7f80141039a0 2024-04-25T00:35:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f801c600640 1 -- 172.21.15.42:0/2236423913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80141ae740 con 0x7f80141039a0 2024-04-25T00:35:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f8012000640 1 -- 172.21.15.42:0/2236423913 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8008010910 con 0x7f80141039a0 2024-04-25T00:35:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.017+0000 7f8012000640 1 -- 172.21.15.42:0/2236423913 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8008018a10 con 0x7f80141039a0 2024-04-25T00:35:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.018+0000 7f801c600640 1 -- 172.21.15.42:0/2236423913 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80141aec50 con 0x7f80141039a0 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.021+0000 7f801f413740 1 Processor -- start 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.021+0000 7f801f413740 1 -- start start 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.021+0000 7f8012000640 1 -- 172.21.15.42:0/2236423913 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f800801f020 con 0x7f80141039a0 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.021+0000 7f801f413740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17eca0 0x55c5ce17f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.021+0000 7f801f413740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17f690 0x55c5ce184190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.021+0000 7f801f413740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce186c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.021+0000 7f801f413740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce0e1ab0 con 0x55c5ce17f690 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.021+0000 7f801f413740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce08a640 con 0x55c5ce17eca0 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.021+0000 7f801f413740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce0dda70 con 0x55c5ce184800 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.022+0000 7f8019200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce186c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.022+0000 7f8019200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce186c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:32822/0 (socket says 172.21.15.42:32822) 2024-04-25T00:35:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.022+0000 7f8019200640 1 -- 172.21.15.42:0/2713116194 learned_addr learned my addr 172.21.15.42:0/2713116194 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.022+0000 7f801a600640 1 --2- 172.21.15.42:0/2713116194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17eca0 0x55c5ce17f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.022+0000 7f8019200640 1 -- 172.21.15.42:0/2713116194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17eca0 msgr2=0x55c5ce17f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.022+0000 7f8019200640 1 --2- 172.21.15.42:0/2713116194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17eca0 0x55c5ce17f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.022+0000 7f8019200640 1 -- 172.21.15.42:0/2713116194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17f690 msgr2=0x55c5ce184190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.022+0000 7f8019200640 1 --2- 172.21.15.42:0/2713116194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17f690 0x55c5ce184190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.022+0000 7f8019200640 1 -- 172.21.15.42:0/2713116194 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5cddf6210 con 0x55c5ce184800 2024-04-25T00:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.023+0000 7f8019200640 1 --2- 172.21.15.42:0/2713116194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce186c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8000009900 tx=0x7f80000043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e641b19c2d1b90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.023+0000 7f8006a00640 1 -- 172.21.15.42:0/2713116194 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000035040 con 0x55c5ce184800 2024-04-25T00:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.024+0000 7f8006a00640 1 -- 172.21.15.42:0/2713116194 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8000004770 con 0x55c5ce184800 2024-04-25T00:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.024+0000 7f8006a00640 1 -- 172.21.15.42:0/2713116194 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000031ab0 con 0x55c5ce184800 2024-04-25T00:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.024+0000 7f8012000640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7ffc0a2d30 0x7f7ffc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.024+0000 7f8012000640 1 -- 172.21.15.42:0/2236423913 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f80080cf2c0 con 0x7f80141039a0 2024-04-25T00:35:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.024+0000 7f801f413740 1 -- 172.21.15.42:0/2713116194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 msgr2=0x55c5ce186c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.024+0000 7f801f413740 1 --2- 172.21.15.42:0/2713116194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce186c90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8000009900 tx=0x7f80000043f0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.025+0000 7f801a600640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7ffc0a2d30 0x7f7ffc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.025+0000 7f801f413740 1 -- 172.21.15.42:0/2713116194 shutdown_connections 2024-04-25T00:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.025+0000 7f801f413740 1 --2- 172.21.15.42:0/2713116194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce186c90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.025+0000 7f801f413740 1 --2- 172.21.15.42:0/2713116194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17f690 0x55c5ce184190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.025+0000 7f801f413740 1 --2- 172.21.15.42:0/2713116194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17eca0 0x55c5ce17f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.025+0000 7f801f413740 1 -- 172.21.15.42:0/2713116194 >> 172.21.15.42:0/2713116194 conn(0x55c5ce17a310 msgr2=0x55c5ce17a710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.025+0000 7f801f413740 1 -- 172.21.15.42:0/2713116194 shutdown_connections 2024-04-25T00:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.025+0000 7f801f413740 1 -- 172.21.15.42:0/2713116194 wait complete. 2024-04-25T00:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.026+0000 7f801f413740 1 -- start start 2024-04-25T00:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.026+0000 7f801f413740 1 Processor -- start 2024-04-25T00:35:00.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.026+0000 7f801a600640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7ffc0a2d30 0x7f7ffc0a51f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55c5ce17fef0 tx=0x7f800c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.027+0000 7f801f413740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17eca0 0x55c5ce188ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.027+0000 7f801f413740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17f690 0x55c5ce189430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.027+0000 7f801f413740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce190470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.027+0000 7f801f413740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce187750 con 0x55c5ce17f690 2024-04-25T00:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.027+0000 7f801f413740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce09d990 con 0x55c5ce17eca0 2024-04-25T00:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.027+0000 7f801f413740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c5ce1878d0 con 0x55c5ce184800 2024-04-25T00:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.027+0000 7f801f413740 10 client.0 _ll_register_callbacks cb 0x55c5ce09f950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.028+0000 7f801a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce190470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.028+0000 7f801a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce190470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:32830/0 (socket says 172.21.15.42:32830) 2024-04-25T00:35:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.028+0000 7f801a600640 1 -- 172.21.15.42:0/1879240817 learned_addr learned my addr 172.21.15.42:0/1879240817 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.028+0000 7f8019c00640 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17eca0 0x55c5ce188ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.028+0000 7f8019200640 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17f690 0x55c5ce189430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.028+0000 7f801a600640 1 -- 172.21.15.42:0/1879240817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17eca0 msgr2=0x55c5ce188ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.028+0000 7f801a600640 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17eca0 0x55c5ce188ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.028+0000 7f801a600640 1 -- 172.21.15.42:0/1879240817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17f690 msgr2=0x55c5ce189430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.028+0000 7f801a600640 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17f690 0x55c5ce189430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.028+0000 7f801a600640 1 -- 172.21.15.42:0/1879240817 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5cdec2620 con 0x55c5ce184800 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.029+0000 7f801a600640 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce190470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f800c010a40 tx=0x7f800c004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2e9c44cb708565 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.029+0000 7f7ffb400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.029+0000 7f7ffb400640 1 -- 172.21.15.42:0/1879240817 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c01d8f0 con 0x55c5ce184800 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.029+0000 7f801f413740 1 -- 172.21.15.42:0/1879240817 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8000004770 con 0x55c5ce184800 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.029+0000 7f7ffb400640 1 -- 172.21.15.42:0/1879240817 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f800c01da90 con 0x55c5ce184800 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.029+0000 7f7ffb400640 1 -- 172.21.15.42:0/1879240817 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f800c01dc90 con 0x55c5ce184800 2024-04-25T00:35:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.029+0000 7f801f413740 10 client.26467 fetch_fsmap learned FSMap version 41 2024-04-25T00:35:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.030+0000 7f801f413740 1 -- 172.21.15.42:0/1879240817 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c5ce193520 con 0x55c5ce184800 2024-04-25T00:35:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.030+0000 7f7ffb400640 1 -- 172.21.15.42:0/1879240817 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8000004770 con 0x55c5ce184800 2024-04-25T00:35:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.031+0000 7f7ffb400640 1 -- 172.21.15.42:0/1879240817 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f800c033560 con 0x55c5ce184800 2024-04-25T00:35:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.031+0000 7f801f413740 10 client.26467 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T00:35:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.031+0000 7f801f413740 10 client.26467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:35:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.031+0000 7f801f413740 20 client.26467 populate_metadata read hostname 'smithi042' 2024-04-25T00:35:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.031+0000 7f801f413740 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5ce194b50 0x55c5ce196fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.031+0000 7f801f413740 4 client.26467 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.031+0000 7f801f413740 1 -- 172.21.15.42:0/1879240817 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5ce0eeff0 con 0x55c5ce194b50 2024-04-25T00:35:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.032+0000 7f8019c00640 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5ce194b50 0x55c5ce196fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.032+0000 7f8019c00640 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5ce194b50 0x55c5ce196fe0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bce88521977b8a7 server_cookie=44a8f32c4d471bbe in_seq=0 out_seq=0 2024-04-25T00:35:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.032+0000 7f7ffb400640 10 client.26467 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.034+0000 7f7ffb400640 1 -- 172.21.15.42:0/1879240817 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5ce0eeff0 con 0x55c5ce194b50 2024-04-25T00:35:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.034+0000 7f7ffb400640 10 client.26467 handle_command_reply: tid=0 2024-04-25T00:35:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.034+0000 7f801f413740 1 client.26467 shutdown 2024-04-25T00:35:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.034+0000 7f801f413740 1 -- 172.21.15.42:0/1879240817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 msgr2=0x55c5ce190470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.034+0000 7f801f413740 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce190470 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f800c010a40 tx=0x7f800c004a40 comp rx=0 tx=0).stop 2024-04-25T00:35:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.035+0000 7f801f413740 1 -- 172.21.15.42:0/1879240817 shutdown_connections 2024-04-25T00:35:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.035+0000 7f801f413740 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5ce194b50 0x55c5ce196fe0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.035+0000 7f801f413740 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce190470 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.035+0000 7f801f413740 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17f690 0x55c5ce189430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.035+0000 7f801f413740 1 --2- 172.21.15.42:0/1879240817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17eca0 0x55c5ce188ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.035+0000 7f801f413740 1 -- 172.21.15.42:0/1879240817 >> 172.21.15.42:0/1879240817 conn(0x55c5ce17a310 msgr2=0x55c5ce179480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.035+0000 7f7ffb400640 0 client.26467 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.036+0000 7f801f413740 1 -- 172.21.15.42:0/1879240817 shutdown_connections 2024-04-25T00:35:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.036+0000 7f801f413740 1 -- 172.21.15.42:0/1879240817 wait complete. 2024-04-25T00:35:00.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.036+0000 7f801f413740 20 client.26467 trim_cache size 0 max 16384 2024-04-25T00:35:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.059+0000 7f801f413740 1 Processor -- start 2024-04-25T00:35:00.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f801f413740 1 -- start start 2024-04-25T00:35:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f801f413740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 0x55c5ce180950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f801f413740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17f690 0x55c5ce180e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f801f413740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce1969a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f801f413740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80141ba350 con 0x55c5ce17eca0 2024-04-25T00:35:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f801f413740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80141b7b60 con 0x55c5ce17f690 2024-04-25T00:35:00.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f801f413740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80141b5950 con 0x55c5ce184800 2024-04-25T00:35:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f801a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce1969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f801a600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce1969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:32836/0 (socket says 172.21.15.42:32836) 2024-04-25T00:35:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f801a600640 1 -- 172.21.15.42:0/1074024942 learned_addr learned my addr 172.21.15.42:0/1074024942 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:00.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.060+0000 7f8019c00640 1 --2- 172.21.15.42:0/1074024942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 0x55c5ce180950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.061+0000 7f8019200640 1 --2- 172.21.15.42:0/1074024942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17f690 0x55c5ce180e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.061+0000 7f801a600640 1 -- 172.21.15.42:0/1074024942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17f690 msgr2=0x55c5ce180e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.061+0000 7f801a600640 1 --2- 172.21.15.42:0/1074024942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17f690 0x55c5ce180e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.061+0000 7f801a600640 1 -- 172.21.15.42:0/1074024942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 msgr2=0x55c5ce180950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.061+0000 7f801a600640 1 --2- 172.21.15.42:0/1074024942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 0x55c5ce180950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.061+0000 7f801a600640 1 -- 172.21.15.42:0/1074024942 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ce17a140 con 0x55c5ce184800 2024-04-25T00:35:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.061+0000 7f801a600640 1 --2- 172.21.15.42:0/1074024942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce1969a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f800c012eb0 tx=0x7f800c009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d5595acc6552ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.061+0000 7f8006000640 1 -- 172.21.15.42:0/1074024942 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c01d6f0 con 0x55c5ce184800 2024-04-25T00:35:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.061+0000 7f8006000640 1 -- 172.21.15.42:0/1074024942 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800c01d890 con 0x55c5ce184800 2024-04-25T00:35:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.061+0000 7f8006000640 1 -- 172.21.15.42:0/1074024942 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c005030 con 0x55c5ce184800 2024-04-25T00:35:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.062+0000 7f801f413740 1 -- 172.21.15.42:0/1074024942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 msgr2=0x55c5ce1969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.062+0000 7f801f413740 1 --2- 172.21.15.42:0/1074024942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce1969a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f800c012eb0 tx=0x7f800c009e70 comp rx=0 tx=0).stop 2024-04-25T00:35:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.062+0000 7f801f413740 1 -- 172.21.15.42:0/1074024942 shutdown_connections 2024-04-25T00:35:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.062+0000 7f801f413740 1 --2- 172.21.15.42:0/1074024942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce184800 0x55c5ce1969a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.062+0000 7f801f413740 1 --2- 172.21.15.42:0/1074024942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce17f690 0x55c5ce180e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.062+0000 7f801f413740 1 --2- 172.21.15.42:0/1074024942 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 0x55c5ce180950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.063+0000 7f801f413740 1 -- 172.21.15.42:0/1074024942 >> 172.21.15.42:0/1074024942 conn(0x55c5ce17a310 msgr2=0x55c5ce174fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.063+0000 7f801f413740 1 -- 172.21.15.42:0/1074024942 shutdown_connections 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.063+0000 7f801f413740 1 -- 172.21.15.42:0/1074024942 wait complete. 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.064+0000 7f801f413740 1 -- start start 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.064+0000 7f801f413740 1 Processor -- start 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.064+0000 7f801f413740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 0x55c5ce18d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.064+0000 7f801f413740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce17f690 0x55c5ce18d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.064+0000 7f801f413740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce184800 0x55c5ce18e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.064+0000 7f801f413740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f80141b6020 con 0x55c5ce17eca0 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.065+0000 7f801a600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce184800 0x55c5ce18e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.065+0000 7f801f413740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f80141b7490 con 0x55c5ce184800 2024-04-25T00:35:00.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.065+0000 7f801a600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce184800 0x55c5ce18e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:50378/0 (socket says 172.21.15.42:50378) 2024-04-25T00:35:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.065+0000 7f801a600640 1 -- 172.21.15.42:0/1646985457 learned_addr learned my addr 172.21.15.42:0/1646985457 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.065+0000 7f8019200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce17f690 0x55c5ce18d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.065+0000 7f8019c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 0x55c5ce18d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.065+0000 7f801f413740 1 -- 172.21.15.42:0/1646985457 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f80141b66f0 con 0x55c5ce17f690 2024-04-25T00:35:00.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.066+0000 7f801f413740 10 client.0 _ll_register_callbacks cb 0x7f800c009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.066+0000 7f801a600640 1 -- 172.21.15.42:0/1646985457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce17f690 msgr2=0x55c5ce18d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.066+0000 7f801a600640 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce17f690 0x55c5ce18d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.066+0000 7f801a600640 1 -- 172.21.15.42:0/1646985457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 msgr2=0x55c5ce18d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.066+0000 7f801a600640 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 0x55c5ce18d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.066+0000 7f801a600640 1 -- 172.21.15.42:0/1646985457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ce18ec70 con 0x55c5ce184800 2024-04-25T00:35:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.066+0000 7f8019c00640 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 0x55c5ce18d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:00.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.067+0000 7f801a600640 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce184800 0x55c5ce18e680 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f800c033a70 tx=0x7f800c005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e25e2f87a1a1c17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.067+0000 7f7ffb400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:35:00.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.067+0000 7f7ffb400640 1 -- 172.21.15.42:0/1646985457 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c040070 con 0x55c5ce184800 2024-04-25T00:35:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.067+0000 7f7ffb400640 1 -- 172.21.15.42:0/1646985457 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f800c004c00 con 0x55c5ce184800 2024-04-25T00:35:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.067+0000 7f801f413740 1 -- 172.21.15.42:0/1646985457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f800c01d890 con 0x55c5ce184800 2024-04-25T00:35:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.067+0000 7f7ffb400640 1 -- 172.21.15.42:0/1646985457 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f800c04c050 con 0x55c5ce184800 2024-04-25T00:35:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.067+0000 7f801f413740 10 client.26474 fetch_fsmap learned FSMap version 41 2024-04-25T00:35:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.067+0000 7f801f413740 1 -- 172.21.15.42:0/1646985457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55c5ce1fb210 con 0x55c5ce184800 2024-04-25T00:35:00.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.068+0000 7f7ffb400640 1 -- 172.21.15.42:0/1646985457 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f800c01d890 con 0x55c5ce184800 2024-04-25T00:35:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.069+0000 7f7ffb400640 1 -- 172.21.15.42:0/1646985457 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7f800c029480 con 0x55c5ce184800 2024-04-25T00:35:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.069+0000 7f801f413740 10 client.26474 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T00:35:00.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.069+0000 7f801f413740 10 client.26474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:35:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.069+0000 7f801f413740 20 client.26474 populate_metadata read hostname 'smithi042' 2024-04-25T00:35:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.069+0000 7f801f413740 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5ce1fba30 0x55c5ce1fde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.069+0000 7f801f413740 4 client.26474 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:35:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.069+0000 7f801f413740 1 -- 172.21.15.42:0/1646985457 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c5ce0e9340 con 0x55c5ce1fba30 2024-04-25T00:35:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.069+0000 7f8019c00640 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5ce1fba30 0x55c5ce1fde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.070+0000 7f7ffb400640 10 client.26474 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:00.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.070+0000 7f8019c00640 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5ce1fba30 0x55c5ce1fde50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcc4c2f400a6a979 server_cookie=805c469fd2d257d0 in_seq=0 out_seq=0 2024-04-25T00:35:00.269 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:34:59 smithi136 ceph-mon[32923]: pgmap v877: 97 pgs: 97 active+clean; 569 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.5 MiB/s wr, 580 op/s 2024-04-25T00:35:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.337+0000 7f7ffb400640 1 -- 172.21.15.42:0/1646985457 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c5ce0e9340 con 0x55c5ce1fba30 2024-04-25T00:35:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.337+0000 7f7ffb400640 10 client.26474 handle_command_reply: tid=0 2024-04-25T00:35:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.337+0000 7f801f413740 1 client.26474 shutdown 2024-04-25T00:35:00.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:34:59 smithi106 ceph-mon[32510]: pgmap v877: 97 pgs: 97 active+clean; 569 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.5 MiB/s wr, 580 op/s 2024-04-25T00:35:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.338+0000 7f801f413740 1 -- 172.21.15.42:0/1646985457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce184800 msgr2=0x55c5ce18e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.338+0000 7f801f413740 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce184800 0x55c5ce18e680 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f800c033a70 tx=0x7f800c005bf0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.339+0000 7f801f413740 1 -- 172.21.15.42:0/1646985457 shutdown_connections 2024-04-25T00:35:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.339+0000 7f801f413740 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c5ce1fba30 0x55c5ce1fde50 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.339+0000 7f801f413740 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c5ce184800 0x55c5ce18e680 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.339+0000 7f801f413740 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c5ce17f690 0x55c5ce18d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.339+0000 7f801f413740 1 --2- 172.21.15.42:0/1646985457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c5ce17eca0 0x55c5ce18d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.339+0000 7f7ffb400640 0 client.26474 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.339+0000 7f801f413740 1 -- 172.21.15.42:0/1646985457 >> 172.21.15.42:0/1646985457 conn(0x55c5ce17a310 msgr2=0x55c5ce188ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.339+0000 7f801f413740 1 -- 172.21.15.42:0/1646985457 shutdown_connections 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.339+0000 7f801f413740 1 -- 172.21.15.42:0/1646985457 wait complete. 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.340+0000 7f801f413740 20 client.26474 trim_cache size 0 max 16384 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:35:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:35:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.340+0000 7f801c600640 1 -- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7ffc0a2d30 msgr2=0x7f7ffc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.340+0000 7f801c600640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7ffc0a2d30 0x7f7ffc0a51f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x55c5ce17fef0 tx=0x7f800c009500 comp rx=0 tx=0).stop 2024-04-25T00:35:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.341+0000 7f801c600640 1 -- 172.21.15.42:0/2236423913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80141039a0 msgr2=0x7f80141957e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.341+0000 7f801c600640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80141039a0 0x7f80141957e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f800800e9c0 tx=0x7f800800ee80 comp rx=0 tx=0).stop 2024-04-25T00:35:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.341+0000 7f801c600640 1 -- 172.21.15.42:0/2236423913 shutdown_connections 2024-04-25T00:35:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.341+0000 7f801c600640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7ffc0a2d30 0x7f7ffc0a51f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.341+0000 7f801c600640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8014105550 0x7f801419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.341+0000 7f801c600640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8014104b90 0x7f8014195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.341+0000 7f801c600640 1 --2- 172.21.15.42:0/2236423913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f80141039a0 0x7f80141957e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.341+0000 7f801c600640 1 -- 172.21.15.42:0/2236423913 >> 172.21.15.42:0/2236423913 conn(0x7f80140fef30 msgr2=0x7f8014077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.341+0000 7f801c600640 1 -- 172.21.15.42:0/2236423913 shutdown_connections 2024-04-25T00:35:00.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:00.341+0000 7f801c600640 1 -- 172.21.15.42:0/2236423913 wait complete. 2024-04-25T00:35:00.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:34:59 smithi042 ceph-mon[28130]: pgmap v877: 97 pgs: 97 active+clean; 569 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 4.5 MiB/s wr, 580 op/s 2024-04-25T00:35:00.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 109457 35.54 MB/sec execute 50 sec latency 197.691 ms 2024-04-25T00:35:01.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 109899 35.46 MB/sec execute 51 sec latency 388.393 ms 2024-04-25T00:35:02.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:01 smithi136 ceph-mon[32923]: pgmap v878: 97 pgs: 97 active+clean; 571 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.1 MiB/s wr, 582 op/s 2024-04-25T00:35:02.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:01 smithi106 ceph-mon[32510]: pgmap v878: 97 pgs: 97 active+clean; 571 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.1 MiB/s wr, 582 op/s 2024-04-25T00:35:02.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:01 smithi042 ceph-mon[28130]: pgmap v878: 97 pgs: 97 active+clean; 571 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 5.1 MiB/s wr, 582 op/s 2024-04-25T00:35:02.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 110036 34.91 MB/sec execute 52 sec latency 789.561 ms 2024-04-25T00:35:03.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 110393 34.63 MB/sec execute 53 sec latency 122.391 ms 2024-04-25T00:35:04.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:03 smithi136 ceph-mon[32923]: pgmap v879: 97 pgs: 97 active+clean; 580 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.8 MiB/s wr, 741 op/s 2024-04-25T00:35:04.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:03 smithi106 ceph-mon[32510]: pgmap v879: 97 pgs: 97 active+clean; 580 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.8 MiB/s wr, 741 op/s 2024-04-25T00:35:04.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:03 smithi042 ceph-mon[28130]: pgmap v879: 97 pgs: 97 active+clean; 580 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 8.8 MiB/s wr, 741 op/s 2024-04-25T00:35:04.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 110890 34.54 MB/sec execute 54 sec latency 168.363 ms 2024-04-25T00:35:05.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 111543 34.50 MB/sec execute 55 sec latency 47.196 ms 2024-04-25T00:35:06.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:05 smithi136 ceph-mon[32923]: pgmap v880: 97 pgs: 97 active+clean; 578 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.2 MiB/s wr, 606 op/s 2024-04-25T00:35:06.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:05 smithi106 ceph-mon[32510]: pgmap v880: 97 pgs: 97 active+clean; 578 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.2 MiB/s wr, 606 op/s 2024-04-25T00:35:06.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:05 smithi042 ceph-mon[28130]: pgmap v880: 97 pgs: 97 active+clean; 578 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.2 MiB/s wr, 606 op/s 2024-04-25T00:35:06.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 112095 34.37 MB/sec execute 56 sec latency 52.294 ms 2024-04-25T00:35:07.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 112888 34.49 MB/sec execute 57 sec latency 83.679 ms 2024-04-25T00:35:08.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:07 smithi136 ceph-mon[32923]: pgmap v881: 97 pgs: 97 active+clean; 565 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 905 op/s 2024-04-25T00:35:08.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:07 smithi106 ceph-mon[32510]: pgmap v881: 97 pgs: 97 active+clean; 565 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 905 op/s 2024-04-25T00:35:08.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:07 smithi042 ceph-mon[28130]: pgmap v881: 97 pgs: 97 active+clean; 565 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 905 op/s 2024-04-25T00:35:08.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 113721 34.68 MB/sec execute 58 sec latency 47.305 ms 2024-04-25T00:35:09.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 114528 35.00 MB/sec execute 59 sec latency 60.599 ms 2024-04-25T00:35:10.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:09 smithi136 ceph-mon[32923]: pgmap v882: 97 pgs: 97 active+clean; 543 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 878 op/s 2024-04-25T00:35:10.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:09 smithi106 ceph-mon[32510]: pgmap v882: 97 pgs: 97 active+clean; 543 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 878 op/s 2024-04-25T00:35:10.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:09 smithi042 ceph-mon[28130]: pgmap v882: 97 pgs: 97 active+clean; 543 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 15 MiB/s wr, 878 op/s 2024-04-25T00:35:10.750 INFO:tasks.workunit.client.0.smithi042.stdout: 10 115177 34.90 MB/sec execute 60 sec latency 51.993 ms 2024-04-25T00:35:11.751 INFO:tasks.workunit.client.0.smithi042.stdout: 10 115875 34.92 MB/sec execute 61 sec latency 54.554 ms 2024-04-25T00:35:12.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:11 smithi136 ceph-mon[32923]: pgmap v883: 97 pgs: 97 active+clean; 553 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 942 op/s 2024-04-25T00:35:12.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:11 smithi106 ceph-mon[32510]: pgmap v883: 97 pgs: 97 active+clean; 553 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 942 op/s 2024-04-25T00:35:12.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:11 smithi042 ceph-mon[28130]: pgmap v883: 97 pgs: 97 active+clean; 553 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 15 MiB/s wr, 942 op/s 2024-04-25T00:35:12.751 INFO:tasks.workunit.client.0.smithi042.stdout: 10 116701 35.04 MB/sec execute 62 sec latency 89.345 ms 2024-04-25T00:35:13.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:12 smithi136 ceph-mon[32923]: pgmap v884: 97 pgs: 97 active+clean; 557 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 913 op/s 2024-04-25T00:35:13.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:12 smithi106 ceph-mon[32510]: pgmap v884: 97 pgs: 97 active+clean; 557 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 913 op/s 2024-04-25T00:35:13.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:12 smithi042 ceph-mon[28130]: pgmap v884: 97 pgs: 97 active+clean; 557 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 913 op/s 2024-04-25T00:35:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.536+0000 7ff84ce00640 1 -- 172.21.15.42:0/2891322375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8480741a0 msgr2=0x7ff848074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.536+0000 7ff84ce00640 1 --2- 172.21.15.42:0/2891322375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8480741a0 0x7ff848074620 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff834009930 tx=0x7ff83402a440 comp rx=0 tx=0).stop 2024-04-25T00:35:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.536+0000 7ff84ce00640 1 -- 172.21.15.42:0/2891322375 shutdown_connections 2024-04-25T00:35:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.536+0000 7ff84ce00640 1 --2- 172.21.15.42:0/2891322375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff84810dba0 0x7ff84810ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.536+0000 7ff84ce00640 1 --2- 172.21.15.42:0/2891322375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8480741a0 0x7ff848074620 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.536+0000 7ff84ce00640 1 --2- 172.21.15.42:0/2891322375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff848075780 0x7ff848073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.536+0000 7ff84ce00640 1 -- 172.21.15.42:0/2891322375 >> 172.21.15.42:0/2891322375 conn(0x7ff8480fcb40 msgr2=0x7ff8480fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.536+0000 7ff84ce00640 1 -- 172.21.15.42:0/2891322375 shutdown_connections 2024-04-25T00:35:13.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.536+0000 7ff84ce00640 1 -- 172.21.15.42:0/2891322375 wait complete. 2024-04-25T00:35:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff84ce00640 1 Processor -- start 2024-04-25T00:35:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff84ce00640 1 -- start start 2024-04-25T00:35:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff84ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8480741a0 0x7ff8480700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff84ce00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848075780 0x7ff848070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff84ce00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff84810dba0 0x7ff8481adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff84ce00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff848113890 con 0x7ff8480741a0 2024-04-25T00:35:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff84ce00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff848113710 con 0x7ff84810dba0 2024-04-25T00:35:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff84ce00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff848113a10 con 0x7ff848075780 2024-04-25T00:35:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff846a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848075780 0x7ff848070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff846a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848075780 0x7ff848070630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38578/0 (socket says 172.21.15.42:38578) 2024-04-25T00:35:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.538+0000 7ff846a00640 1 -- 172.21.15.42:0/2620731149 learned_addr learned my addr 172.21.15.42:0/2620731149 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.539+0000 7ff847400640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8480741a0 0x7ff8480700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.539+0000 7ff847e00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff84810dba0 0x7ff8481adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.539+0000 7ff846a00640 1 -- 172.21.15.42:0/2620731149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff84810dba0 msgr2=0x7ff8481adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.539+0000 7ff846a00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff84810dba0 0x7ff8481adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.539+0000 7ff846a00640 1 -- 172.21.15.42:0/2620731149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8480741a0 msgr2=0x7ff8480700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.539+0000 7ff846a00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8480741a0 0x7ff8480700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.539+0000 7ff846a00640 1 -- 172.21.15.42:0/2620731149 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8481ae4e0 con 0x7ff848075780 2024-04-25T00:35:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.540+0000 7ff846a00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848075780 0x7ff848070630 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff834031440 tx=0x7ff834032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.540+0000 7ff83fe00640 1 -- 172.21.15.42:0/2620731149 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83403f030 con 0x7ff848075780 2024-04-25T00:35:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.540+0000 7ff83fe00640 1 -- 172.21.15.42:0/2620731149 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff834003ce0 con 0x7ff848075780 2024-04-25T00:35:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.540+0000 7ff83fe00640 1 -- 172.21.15.42:0/2620731149 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834035070 con 0x7ff848075780 2024-04-25T00:35:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.540+0000 7ff84ce00640 1 -- 172.21.15.42:0/2620731149 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8481ae6b0 con 0x7ff848075780 2024-04-25T00:35:13.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.541+0000 7ff84ce00640 1 -- 172.21.15.42:0/2620731149 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8481aea60 con 0x7ff848075780 2024-04-25T00:35:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.549+0000 7ff851237740 1 Processor -- start 2024-04-25T00:35:13.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.550+0000 7ff851237740 1 -- start start 2024-04-25T00:35:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.551+0000 7ff83fe00640 1 -- 172.21.15.42:0/2620731149 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff834042030 con 0x7ff848075780 2024-04-25T00:35:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.551+0000 7ff851237740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.552+0000 7ff851237740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d2840 0x555b404d7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.552+0000 7ff851237740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 0x555b404d9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.552+0000 7ff851237740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555b40434ab0 con 0x555b404d1e50 2024-04-25T00:35:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.552+0000 7ff851237740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555b403dd640 con 0x555b404d79b0 2024-04-25T00:35:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.552+0000 7ff851237740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555b40430a70 con 0x555b404d2840 2024-04-25T00:35:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.553+0000 7ff847400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d2840 0x555b404d7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.553+0000 7ff847400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d2840 0x555b404d7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38586/0 (socket says 172.21.15.42:38586) 2024-04-25T00:35:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.553+0000 7ff847400640 1 -- 172.21.15.42:0/2094517773 learned_addr learned my addr 172.21.15.42:0/2094517773 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:13.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.554+0000 7ff846a00640 1 --2- 172.21.15.42:0/2094517773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 0x555b404d9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.554+0000 7ff846a00640 1 -- 172.21.15.42:0/2094517773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d2840 msgr2=0x555b404d7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.554+0000 7ff846a00640 1 --2- 172.21.15.42:0/2094517773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d2840 0x555b404d7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.554+0000 7ff846a00640 1 -- 172.21.15.42:0/2094517773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 msgr2=0x555b404d2270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:35:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.554+0000 7ff846a00640 1 --2- 172.21.15.42:0/2094517773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404d2270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.554+0000 7ff846a00640 1 -- 172.21.15.42:0/2094517773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b40149210 con 0x555b404d79b0 2024-04-25T00:35:13.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.555+0000 7ff846a00640 1 --2- 172.21.15.42:0/2094517773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 0x555b404d9e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff8340ecf80 tx=0x7ff8340f7620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2788bea767177ec5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.557+0000 7ff83fe00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff82c0a2c80 0x7ff82c0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.557+0000 7ff83cc00640 1 -- 172.21.15.42:0/2094517773 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8340b8060 con 0x555b404d79b0 2024-04-25T00:35:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.557+0000 7ff83cc00640 1 -- 172.21.15.42:0/2094517773 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8340fc030 con 0x555b404d79b0 2024-04-25T00:35:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.557+0000 7ff83cc00640 1 -- 172.21.15.42:0/2094517773 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8340bb210 con 0x555b404d79b0 2024-04-25T00:35:13.557 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.557+0000 7ff83fe00640 1 -- 172.21.15.42:0/2620731149 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ff8340ef4e0 con 0x7ff848075780 2024-04-25T00:35:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.558+0000 7ff847e00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff82c0a2c80 0x7ff82c0a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.558+0000 7ff851237740 1 -- 172.21.15.42:0/2094517773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 msgr2=0x555b404d9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.558+0000 7ff851237740 1 --2- 172.21.15.42:0/2094517773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 0x555b404d9e40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff8340ecf80 tx=0x7ff8340f7620 comp rx=0 tx=0).stop 2024-04-25T00:35:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.558+0000 7ff851237740 1 -- 172.21.15.42:0/2094517773 shutdown_connections 2024-04-25T00:35:13.558 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.558+0000 7ff851237740 1 --2- 172.21.15.42:0/2094517773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 0x555b404d9e40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.558+0000 7ff851237740 1 --2- 172.21.15.42:0/2094517773 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d2840 0x555b404d7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.558+0000 7ff851237740 1 --2- 172.21.15.42:0/2094517773 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.558+0000 7ff851237740 1 -- 172.21.15.42:0/2094517773 >> 172.21.15.42:0/2094517773 conn(0x555b404cd4c0 msgr2=0x555b404cd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.559+0000 7ff851237740 1 -- 172.21.15.42:0/2094517773 shutdown_connections 2024-04-25T00:35:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.559+0000 7ff851237740 1 -- 172.21.15.42:0/2094517773 wait complete. 2024-04-25T00:35:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.559+0000 7ff851237740 1 -- start start 2024-04-25T00:35:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.559+0000 7ff851237740 1 Processor -- start 2024-04-25T00:35:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.560+0000 7ff847e00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff82c0a2c80 0x7ff82c0a5140 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x555b404d30a0 tx=0x7ff840009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.561+0000 7ff851237740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404dbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.561+0000 7ff851237740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 0x555b404dc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.561+0000 7ff851237740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 0x555b404e3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.561+0000 7ff847400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404dbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.561+0000 7ff847400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404dbdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57470/0 (socket says 172.21.15.42:57470) 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.561+0000 7ff851237740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555b404da720 con 0x555b404d1e50 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.561+0000 7ff851237740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555b403f0990 con 0x555b404d2840 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.561+0000 7ff851237740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555b404da8a0 con 0x555b404d79b0 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.561+0000 7ff846a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 0x555b404dc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.562+0000 7ff846a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 0x555b404dc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:54660/0 (socket says 172.21.15.42:54660) 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.562+0000 7ff846a00640 1 -- 172.21.15.42:0/2576288508 learned_addr learned my addr 172.21.15.42:0/2576288508 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.562+0000 7ff851237740 10 client.0 _ll_register_callbacks cb 0x555b403f2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.562+0000 7ff847e00640 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 0x555b404e3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.562+0000 7ff846a00640 1 -- 172.21.15.42:0/2576288508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 msgr2=0x555b404e3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.562+0000 7ff846a00640 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 0x555b404e3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.562+0000 7ff846a00640 1 -- 172.21.15.42:0/2576288508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 msgr2=0x555b404dbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.562+0000 7ff846a00640 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404dbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.562+0000 7ff846a00640 1 -- 172.21.15.42:0/2576288508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b40215620 con 0x555b404d2840 2024-04-25T00:35:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.562+0000 7ff847400640 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404dbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.563+0000 7ff846a00640 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 0x555b404dc330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff8340b8790 tx=0x7ff83403b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3cb784e376409df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.563+0000 7ff831600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:35:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.563+0000 7ff831600640 1 -- 172.21.15.42:0/2576288508 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834049030 con 0x555b404d2840 2024-04-25T00:35:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.563+0000 7ff831600640 1 -- 172.21.15.42:0/2576288508 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ff8340ed0c0 con 0x555b404d2840 2024-04-25T00:35:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.563+0000 7ff851237740 1 -- 172.21.15.42:0/2576288508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff8340fc030 con 0x555b404d2840 2024-04-25T00:35:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.563+0000 7ff831600640 1 -- 172.21.15.42:0/2576288508 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff8340bb540 con 0x555b404d2840 2024-04-25T00:35:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.563+0000 7ff851237740 10 client.26486 fetch_fsmap learned FSMap version 41 2024-04-25T00:35:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.563+0000 7ff851237740 1 -- 172.21.15.42:0/2576288508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555b404da720 con 0x555b404d2840 2024-04-25T00:35:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.564+0000 7ff831600640 1 -- 172.21.15.42:0/2576288508 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff8340fc030 con 0x555b404d2840 2024-04-25T00:35:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.564+0000 7ff831600640 1 -- 172.21.15.42:0/2576288508 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7ff8340b8430 con 0x555b404d2840 2024-04-25T00:35:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.565+0000 7ff851237740 10 client.26486 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T00:35:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.565+0000 7ff851237740 10 client.26486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:35:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.565+0000 7ff851237740 20 client.26486 populate_metadata read hostname 'smithi042' 2024-04-25T00:35:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.565+0000 7ff851237740 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555b404e7780 0x555b404e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.565+0000 7ff851237740 4 client.26486 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.565+0000 7ff851237740 1 -- 172.21.15.42:0/2576288508 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b40441ff0 con 0x555b404e7780 2024-04-25T00:35:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.565+0000 7ff847400640 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555b404e7780 0x555b404e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.566+0000 7ff847400640 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555b404e7780 0x555b404e9bc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e044d9580f2a037 server_cookie=d2fca684464093d in_seq=0 out_seq=0 2024-04-25T00:35:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.566+0000 7ff831600640 10 client.26486 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.567+0000 7ff831600640 1 -- 172.21.15.42:0/2576288508 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b40441ff0 con 0x555b404e7780 2024-04-25T00:35:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.567+0000 7ff831600640 10 client.26486 handle_command_reply: tid=0 2024-04-25T00:35:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.567+0000 7ff851237740 1 client.26486 shutdown 2024-04-25T00:35:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.568+0000 7ff851237740 1 -- 172.21.15.42:0/2576288508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 msgr2=0x555b404dc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.568+0000 7ff851237740 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 0x555b404dc330 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff8340b8790 tx=0x7ff83403b800 comp rx=0 tx=0).stop 2024-04-25T00:35:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.568+0000 7ff851237740 1 -- 172.21.15.42:0/2576288508 shutdown_connections 2024-04-25T00:35:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.568+0000 7ff851237740 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555b404e7780 0x555b404e9bc0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.568+0000 7ff851237740 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 0x555b404e3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.568+0000 7ff851237740 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 0x555b404dc330 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.568+0000 7ff831600640 0 client.26486 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.568+0000 7ff851237740 1 --2- 172.21.15.42:0/2576288508 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404dbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.568+0000 7ff851237740 1 -- 172.21.15.42:0/2576288508 >> 172.21.15.42:0/2576288508 conn(0x555b404cd4c0 msgr2=0x555b404cc620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.569+0000 7ff851237740 1 -- 172.21.15.42:0/2576288508 shutdown_connections 2024-04-25T00:35:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.570+0000 7ff851237740 1 -- 172.21.15.42:0/2576288508 wait complete. 2024-04-25T00:35:13.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.571+0000 7ff851237740 20 client.26486 trim_cache size 0 max 16384 2024-04-25T00:35:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.595+0000 7ff851237740 1 Processor -- start 2024-04-25T00:35:13.596 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.595+0000 7ff851237740 1 -- start start 2024-04-25T00:35:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.595+0000 7ff851237740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.595+0000 7ff851237740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 0x555b404e3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.595+0000 7ff851237740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 0x555b404d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.595+0000 7ff851237740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481ba190 con 0x555b404d1e50 2024-04-25T00:35:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.595+0000 7ff851237740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b79a0 con 0x555b404d2840 2024-04-25T00:35:13.597 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.595+0000 7ff851237740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b57f0 con 0x555b404d79b0 2024-04-25T00:35:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.595+0000 7ff847e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 0x555b404d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.596+0000 7ff847e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 0x555b404d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38604/0 (socket says 172.21.15.42:38604) 2024-04-25T00:35:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.596+0000 7ff846a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 0x555b404e3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.596+0000 7ff847e00640 1 -- 172.21.15.42:0/1906719544 learned_addr learned my addr 172.21.15.42:0/1906719544 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:13.598 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.596+0000 7ff847400640 1 --2- 172.21.15.42:0/1906719544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.596+0000 7ff847e00640 1 -- 172.21.15.42:0/1906719544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 msgr2=0x555b404e3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.596+0000 7ff847e00640 1 --2- 172.21.15.42:0/1906719544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 0x555b404e3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.596+0000 7ff847e00640 1 -- 172.21.15.42:0/1906719544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 msgr2=0x555b404d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.604 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.596+0000 7ff847e00640 1 --2- 172.21.15.42:0/1906719544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.596+0000 7ff847e00640 1 -- 172.21.15.42:0/1906719544 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b404ea240 con 0x555b404d79b0 2024-04-25T00:35:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.597+0000 7ff847e00640 1 --2- 172.21.15.42:0/1906719544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 0x555b404d0910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff840003f90 tx=0x7ff840012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb8b0aa23ca5bc92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.598+0000 7ff833e00640 1 -- 172.21.15.42:0/1906719544 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84002a070 con 0x555b404d79b0 2024-04-25T00:35:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.598+0000 7ff833e00640 1 -- 172.21.15.42:0/1906719544 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff84001bc10 con 0x555b404d79b0 2024-04-25T00:35:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.598+0000 7ff833e00640 1 -- 172.21.15.42:0/1906719544 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff84001ad60 con 0x555b404d79b0 2024-04-25T00:35:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.599+0000 7ff851237740 1 -- 172.21.15.42:0/1906719544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 msgr2=0x555b404d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.599+0000 7ff851237740 1 --2- 172.21.15.42:0/1906719544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 0x555b404d0910 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff840003f90 tx=0x7ff840012000 comp rx=0 tx=0).stop 2024-04-25T00:35:13.605 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.599+0000 7ff851237740 1 -- 172.21.15.42:0/1906719544 shutdown_connections 2024-04-25T00:35:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.599+0000 7ff851237740 1 --2- 172.21.15.42:0/1906719544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d79b0 0x555b404d0910 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.599+0000 7ff851237740 1 --2- 172.21.15.42:0/1906719544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d2840 0x555b404e3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.599+0000 7ff851237740 1 --2- 172.21.15.42:0/1906719544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d1e50 0x555b404d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.599+0000 7ff851237740 1 -- 172.21.15.42:0/1906719544 >> 172.21.15.42:0/1906719544 conn(0x555b404cd4c0 msgr2=0x555b404d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.599+0000 7ff851237740 1 -- 172.21.15.42:0/1906719544 shutdown_connections 2024-04-25T00:35:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.599+0000 7ff851237740 1 -- 172.21.15.42:0/1906719544 wait complete. 2024-04-25T00:35:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.600+0000 7ff851237740 1 -- start start 2024-04-25T00:35:13.606 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.600+0000 7ff851237740 1 Processor -- start 2024-04-25T00:35:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.600+0000 7ff851237740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d1e50 0x555b404e0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.600+0000 7ff851237740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d2840 0x555b404e1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.600+0000 7ff851237740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 0x555b404e2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.600+0000 7ff851237740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b5e60 con 0x555b404d2840 2024-04-25T00:35:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.600+0000 7ff851237740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b72d0 con 0x555b404d79b0 2024-04-25T00:35:13.607 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.600+0000 7ff851237740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff8481b6530 con 0x555b404d1e50 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.600+0000 7ff851237740 10 client.0 _ll_register_callbacks cb 0x7ff83800b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff847400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d1e50 0x555b404e0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff847400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d1e50 0x555b404e0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:38612/0 (socket says 172.21.15.42:38612) 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff847e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 0x555b404e2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff847400640 1 -- 172.21.15.42:0/1720590365 learned_addr learned my addr 172.21.15.42:0/1720590365 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff846a00640 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d2840 0x555b404e1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff847400640 1 -- 172.21.15.42:0/1720590365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 msgr2=0x555b404e2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff847400640 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 0x555b404e2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff847400640 1 -- 172.21.15.42:0/1720590365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d2840 msgr2=0x555b404e1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff847400640 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d2840 0x555b404e1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff847400640 1 -- 172.21.15.42:0/1720590365 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b404e2940 con 0x555b404d1e50 2024-04-25T00:35:13.608 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff847400640 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d1e50 0x555b404e0ec0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff838011870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a97cc128d4ea4a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.601+0000 7ff831600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.602+0000 7ff831600640 1 -- 172.21.15.42:0/1720590365 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff838004220 con 0x555b404d1e50 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.602+0000 7ff831600640 1 -- 172.21.15.42:0/1720590365 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ff8380043c0 con 0x555b404d1e50 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.602+0000 7ff851237740 1 -- 172.21.15.42:0/1720590365 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff84001bc10 con 0x555b404d1e50 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.602+0000 7ff831600640 1 -- 172.21.15.42:0/1720590365 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7ff83802f050 con 0x555b404d1e50 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.602+0000 7ff851237740 10 client.26491 fetch_fsmap learned FSMap version 41 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.602+0000 7ff851237740 1 -- 172.21.15.42:0/1720590365 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x555b4054e1f0 con 0x555b404d1e50 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.603+0000 7ff831600640 1 -- 172.21.15.42:0/1720590365 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff84001bc10 con 0x555b404d1e50 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.603+0000 7ff831600640 1 -- 172.21.15.42:0/1720590365 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 41) ==== 8354+0+0 (secure 0 0 0) 0x7ff838010830 con 0x555b404d1e50 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.603+0000 7ff851237740 10 client.26491 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.603+0000 7ff851237740 10 client.26491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.603+0000 7ff851237740 20 client.26491 populate_metadata read hostname 'smithi042' 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.603+0000 7ff851237740 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555b4054ec80 0x555b405510a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:13.609 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.603+0000 7ff851237740 4 client.26491 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:35:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.603+0000 7ff851237740 1 -- 172.21.15.42:0/1720590365 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b4043c340 con 0x555b4054ec80 2024-04-25T00:35:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.603+0000 7ff846a00640 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555b4054ec80 0x555b405510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.604+0000 7ff846a00640 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555b4054ec80 0x555b405510a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c3dad58198a7ed5 server_cookie=ca420f37c4e52c7f in_seq=0 out_seq=0 2024-04-25T00:35:13.610 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.604+0000 7ff831600640 10 client.26491 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:13.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 117418 35.09 MB/sec execute 63 sec latency 156.116 ms 2024-04-25T00:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.886+0000 7ff831600640 1 -- 172.21.15.42:0/1720590365 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b4043c340 con 0x555b4054ec80 2024-04-25T00:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.886+0000 7ff831600640 10 client.26491 handle_command_reply: tid=0 2024-04-25T00:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.886+0000 7ff851237740 1 client.26491 shutdown 2024-04-25T00:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.886+0000 7ff851237740 1 -- 172.21.15.42:0/1720590365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d1e50 msgr2=0x555b404e0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.886+0000 7ff851237740 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d1e50 0x555b404e0ec0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff838011870 comp rx=0 tx=0).stop 2024-04-25T00:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.886+0000 7ff851237740 1 -- 172.21.15.42:0/1720590365 shutdown_connections 2024-04-25T00:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.887+0000 7ff851237740 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555b4054ec80 0x555b405510a0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.887+0000 7ff851237740 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555b404d79b0 0x555b404e2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.887+0000 7ff851237740 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555b404d2840 0x555b404e1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.887+0000 7ff851237740 1 --2- 172.21.15.42:0/1720590365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555b404d1e50 0x555b404e0ec0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.887+0000 7ff851237740 1 -- 172.21.15.42:0/1720590365 >> 172.21.15.42:0/1720590365 conn(0x555b404cd4c0 msgr2=0x555b404d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.887+0000 7ff831600640 0 client.26491 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.887+0000 7ff851237740 1 -- 172.21.15.42:0/1720590365 shutdown_connections 2024-04-25T00:35:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.887+0000 7ff851237740 1 -- 172.21.15.42:0/1720590365 wait complete. 2024-04-25T00:35:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.888+0000 7ff851237740 20 client.26491 trim_cache size 0 max 16384 2024-04-25T00:35:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:35:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:35:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:35:13.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:35:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 -- 172.21.15.42:0/2620731149 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff82c0a2c80 msgr2=0x7ff82c0a5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff82c0a2c80 0x7ff82c0a5140 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x555b404d30a0 tx=0x7ff840009500 comp rx=0 tx=0).stop 2024-04-25T00:35:13.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 -- 172.21.15.42:0/2620731149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848075780 msgr2=0x7ff848070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848075780 0x7ff848070630 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff834031440 tx=0x7ff834032ce0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 -- 172.21.15.42:0/2620731149 shutdown_connections 2024-04-25T00:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff82c0a2c80 0x7ff82c0a5140 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff84810dba0 0x7ff8481adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff848075780 0x7ff848070630 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 --2- 172.21.15.42:0/2620731149 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8480741a0 0x7ff8480700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 -- 172.21.15.42:0/2620731149 >> 172.21.15.42:0/2620731149 conn(0x7ff8480fcb40 msgr2=0x7ff848100f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 -- 172.21.15.42:0/2620731149 shutdown_connections 2024-04-25T00:35:13.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:13.891+0000 7ff84ce00640 1 -- 172.21.15.42:0/2620731149 wait complete. 2024-04-25T00:35:14.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 118177 35.37 MB/sec execute 64 sec latency 193.308 ms 2024-04-25T00:35:15.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 118851 35.23 MB/sec execute 65 sec latency 66.363 ms 2024-04-25T00:35:15.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:15 smithi136 ceph-mon[32923]: pgmap v885: 97 pgs: 97 active+clean; 557 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 659 op/s 2024-04-25T00:35:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:15 smithi106 ceph-mon[32510]: pgmap v885: 97 pgs: 97 active+clean; 557 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 659 op/s 2024-04-25T00:35:15.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:15 smithi042 ceph-mon[28130]: pgmap v885: 97 pgs: 97 active+clean; 557 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 659 op/s 2024-04-25T00:35:16.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 119624 35.41 MB/sec execute 66 sec latency 48.830 ms 2024-04-25T00:35:17.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 120486 35.36 MB/sec execute 67 sec latency 38.352 ms 2024-04-25T00:35:17.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:17 smithi136 ceph-mon[32923]: pgmap v886: 97 pgs: 97 active+clean; 569 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 900 op/s 2024-04-25T00:35:17.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:17 smithi106 ceph-mon[32510]: pgmap v886: 97 pgs: 97 active+clean; 569 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 900 op/s 2024-04-25T00:35:17.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:17 smithi042 ceph-mon[28130]: pgmap v886: 97 pgs: 97 active+clean; 569 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 900 op/s 2024-04-25T00:35:18.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 121321 35.75 MB/sec execute 68 sec latency 47.306 ms 2024-04-25T00:35:19.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 121999 35.69 MB/sec execute 69 sec latency 49.140 ms 2024-04-25T00:35:19.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:19 smithi136 ceph-mon[32923]: pgmap v887: 97 pgs: 97 active+clean; 563 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.8 MiB/s wr, 647 op/s 2024-04-25T00:35:19.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:19 smithi106 ceph-mon[32510]: pgmap v887: 97 pgs: 97 active+clean; 563 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.8 MiB/s wr, 647 op/s 2024-04-25T00:35:19.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:19 smithi042 ceph-mon[28130]: pgmap v887: 97 pgs: 97 active+clean; 563 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.8 MiB/s wr, 647 op/s 2024-04-25T00:35:20.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 122645 35.64 MB/sec execute 70 sec latency 64.909 ms 2024-04-25T00:35:21.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 123319 35.72 MB/sec execute 71 sec latency 83.163 ms 2024-04-25T00:35:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:21 smithi136 ceph-mon[32923]: pgmap v888: 97 pgs: 97 active+clean; 575 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 644 op/s 2024-04-25T00:35:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:21 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:21 smithi106 ceph-mon[32510]: pgmap v888: 97 pgs: 97 active+clean; 575 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 644 op/s 2024-04-25T00:35:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:21 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:22.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:21 smithi042 ceph-mon[28130]: pgmap v888: 97 pgs: 97 active+clean; 575 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 644 op/s 2024-04-25T00:35:22.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:21 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:22.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 124118 35.64 MB/sec execute 72 sec latency 142.967 ms 2024-04-25T00:35:23.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 124919 35.99 MB/sec execute 73 sec latency 62.827 ms 2024-04-25T00:35:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:23 smithi136 ceph-mon[32923]: pgmap v889: 97 pgs: 97 active+clean; 581 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 608 op/s 2024-04-25T00:35:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:23 smithi106 ceph-mon[32510]: pgmap v889: 97 pgs: 97 active+clean; 581 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 608 op/s 2024-04-25T00:35:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:23 smithi042 ceph-mon[28130]: pgmap v889: 97 pgs: 97 active+clean; 581 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 608 op/s 2024-04-25T00:35:24.752 INFO:tasks.workunit.client.0.smithi042.stdout: 10 125457 35.80 MB/sec execute 74 sec latency 275.039 ms 2024-04-25T00:35:25.753 INFO:tasks.workunit.client.0.smithi042.stdout: 10 126025 35.73 MB/sec execute 75 sec latency 294.274 ms 2024-04-25T00:35:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:25 smithi136 ceph-mon[32923]: pgmap v890: 97 pgs: 97 active+clean; 581 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.9 MiB/s wr, 560 op/s 2024-04-25T00:35:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:25 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:35:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:25 smithi106 ceph-mon[32510]: pgmap v890: 97 pgs: 97 active+clean; 581 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.9 MiB/s wr, 560 op/s 2024-04-25T00:35:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:25 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:35:26.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:25 smithi042 ceph-mon[28130]: pgmap v890: 97 pgs: 97 active+clean; 581 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.9 MiB/s wr, 560 op/s 2024-04-25T00:35:26.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:25 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:35:26.753 INFO:tasks.workunit.client.0.smithi042.stdout: 10 126734 35.78 MB/sec execute 76 sec latency 72.050 ms 2024-04-25T00:35:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:26 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:26 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:26 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:27.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:26 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.110+0000 7fd5f3600640 1 -- 172.21.15.42:0/3862063836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec104bc0 msgr2=0x7fd5ec105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.110+0000 7fd5f3600640 1 --2- 172.21.15.42:0/3862063836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec104bc0 0x7fd5ec105040 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009900 tx=0x7fd5d802a400 comp rx=0 tx=0).stop 2024-04-25T00:35:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.111+0000 7fd5f3600640 1 -- 172.21.15.42:0/3862063836 shutdown_connections 2024-04-25T00:35:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.111+0000 7fd5f3600640 1 --2- 172.21.15.42:0/3862063836 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5ec105580 0x7fd5ec10be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.111+0000 7fd5f3600640 1 --2- 172.21.15.42:0/3862063836 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec104bc0 0x7fd5ec105040 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.111+0000 7fd5f3600640 1 --2- 172.21.15.42:0/3862063836 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ec1039d0 0x7fd5ec103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.111+0000 7fd5f3600640 1 -- 172.21.15.42:0/3862063836 >> 172.21.15.42:0/3862063836 conn(0x7fd5ec0fef60 msgr2=0x7fd5ec101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.111+0000 7fd5f3600640 1 -- 172.21.15.42:0/3862063836 shutdown_connections 2024-04-25T00:35:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.111+0000 7fd5f3600640 1 -- 172.21.15.42:0/3862063836 wait complete. 2024-04-25T00:35:27.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.112+0000 7fd5f3600640 1 Processor -- start 2024-04-25T00:35:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.112+0000 7fd5f3600640 1 -- start start 2024-04-25T00:35:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.113+0000 7fd5f3600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5ec1039d0 0x7fd5ec199c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.113+0000 7fd5f3600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ec104bc0 0x7fd5ec19a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.113+0000 7fd5f3600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec105580 0x7fd5ec1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.113+0000 7fd5f3600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec10f680 con 0x7fd5ec104bc0 2024-04-25T00:35:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.113+0000 7fd5f3600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec10f500 con 0x7fd5ec105580 2024-04-25T00:35:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.113+0000 7fd5f3600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec10f800 con 0x7fd5ec1039d0 2024-04-25T00:35:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.113+0000 7fd5f1600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec105580 0x7fd5ec1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.113+0000 7fd5f1600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec105580 0x7fd5ec1a11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51184/0 (socket says 172.21.15.42:51184) 2024-04-25T00:35:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.113+0000 7fd5f1600640 1 -- 172.21.15.42:0/758119874 learned_addr learned my addr 172.21.15.42:0/758119874 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:27.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.113+0000 7fd5f0c00640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5ec1039d0 0x7fd5ec199c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.114+0000 7fd5ebe00640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ec104bc0 0x7fd5ec19a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.114+0000 7fd5f1600640 1 -- 172.21.15.42:0/758119874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5ec1039d0 msgr2=0x7fd5ec199c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.114+0000 7fd5f1600640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5ec1039d0 0x7fd5ec199c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.114+0000 7fd5f1600640 1 -- 172.21.15.42:0/758119874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ec104bc0 msgr2=0x7fd5ec19a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.114+0000 7fd5f1600640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ec104bc0 0x7fd5ec19a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.114+0000 7fd5f1600640 1 -- 172.21.15.42:0/758119874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ec1a1860 con 0x7fd5ec105580 2024-04-25T00:35:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.114+0000 7fd5f1600640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec105580 0x7fd5ec1a11c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400eb10 tx=0x7fd5e400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:27.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.115+0000 7fd5e9600640 1 -- 172.21.15.42:0/758119874 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4004b20 con 0x7fd5ec105580 2024-04-25T00:35:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.115+0000 7fd5e9600640 1 -- 172.21.15.42:0/758119874 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5e4004cc0 con 0x7fd5ec105580 2024-04-25T00:35:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.115+0000 7fd5e9600640 1 -- 172.21.15.42:0/758119874 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4010760 con 0x7fd5ec105580 2024-04-25T00:35:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.115+0000 7fd5f3600640 1 -- 172.21.15.42:0/758119874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ec1a1b50 con 0x7fd5ec105580 2024-04-25T00:35:27.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.116+0000 7fd5f3600640 1 -- 172.21.15.42:0/758119874 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ec1a2020 con 0x7fd5ec105580 2024-04-25T00:35:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.120+0000 7fd5f65a9740 1 Processor -- start 2024-04-25T00:35:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.120+0000 7fd5e9600640 1 -- 172.21.15.42:0/758119874 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd5e4005000 con 0x7fd5ec105580 2024-04-25T00:35:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.121+0000 7fd5f65a9740 1 -- start start 2024-04-25T00:35:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.121+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc4824b0 0x557ffc4828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.121+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc4879a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.121+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 0x557ffc48a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.121+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557ffc3dcab0 con 0x557ffc482ea0 2024-04-25T00:35:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.121+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ffc385640 con 0x557ffc488010 2024-04-25T00:35:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.121+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557ffc3d8a70 con 0x557ffc4824b0 2024-04-25T00:35:27.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.121+0000 7fd5f0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc4879a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.122+0000 7fd5f0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc4879a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55190/0 (socket says 172.21.15.42:55190) 2024-04-25T00:35:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.122+0000 7fd5f0c00640 1 -- 172.21.15.42:0/3656885012 learned_addr learned my addr 172.21.15.42:0/3656885012 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.122+0000 7fd5f0c00640 1 -- 172.21.15.42:0/3656885012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc4824b0 msgr2=0x557ffc4828d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.122+0000 7fd5f0c00640 1 --2- 172.21.15.42:0/3656885012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc4824b0 0x557ffc4828d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.122+0000 7fd5f0c00640 1 -- 172.21.15.42:0/3656885012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 msgr2=0x557ffc48a4a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:35:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.122+0000 7fd5f0c00640 1 --2- 172.21.15.42:0/3656885012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 0x557ffc48a4a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.122+0000 7fd5f0c00640 1 -- 172.21.15.42:0/3656885012 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ffc0f1210 con 0x557ffc482ea0 2024-04-25T00:35:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.123+0000 7fd5f0c00640 1 --2- 172.21.15.42:0/3656885012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc4879a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009c60 tx=0x7fd5e0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf2b747f1f42857 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:27.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.123+0000 7fd5e9600640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5d40a2d00 0x7fd5d40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.124+0000 7fd5ddc00640 1 -- 172.21.15.42:0/3656885012 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0004220 con 0x557ffc482ea0 2024-04-25T00:35:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.124+0000 7fd5ddc00640 1 -- 172.21.15.42:0/3656885012 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5e00043c0 con 0x557ffc482ea0 2024-04-25T00:35:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.124+0000 7fd5ddc00640 1 -- 172.21.15.42:0/3656885012 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e0004df0 con 0x557ffc482ea0 2024-04-25T00:35:27.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.124+0000 7fd5ebe00640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5d40a2d00 0x7fd5d40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.126+0000 7fd5f65a9740 1 -- 172.21.15.42:0/3656885012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 msgr2=0x557ffc4879a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.126+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/3656885012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc4879a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd5e0009c60 tx=0x7fd5e0011810 comp rx=0 tx=0).stop 2024-04-25T00:35:27.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:26 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:27.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:26 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.127+0000 7fd5e9600640 1 -- 172.21.15.42:0/758119874 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fd5e40cf720 con 0x7fd5ec105580 2024-04-25T00:35:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.127+0000 7fd5ebe00640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5d40a2d00 0x7fd5d40a51c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80096c0 tx=0x7fd5d8031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.129+0000 7fd5f65a9740 1 -- 172.21.15.42:0/3656885012 shutdown_connections 2024-04-25T00:35:27.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.129+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/3656885012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 0x557ffc48a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.129+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/3656885012 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc4879a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.129+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/3656885012 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc4824b0 0x557ffc4828d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.129+0000 7fd5f65a9740 1 -- 172.21.15.42:0/3656885012 >> 172.21.15.42:0/3656885012 conn(0x557ffc47db20 msgr2=0x557ffc47df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.129+0000 7fd5f65a9740 1 -- 172.21.15.42:0/3656885012 shutdown_connections 2024-04-25T00:35:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.129+0000 7fd5f65a9740 1 -- 172.21.15.42:0/3656885012 wait complete. 2024-04-25T00:35:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.130+0000 7fd5f65a9740 1 -- start start 2024-04-25T00:35:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.130+0000 7fd5f65a9740 1 Processor -- start 2024-04-25T00:35:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.131+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc4824b0 0x557ffc48c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.131+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc48ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.131+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 0x557ffc493d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.131+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557ffc48afc0 con 0x557ffc4824b0 2024-04-25T00:35:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.131+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ffc398990 con 0x557ffc488010 2024-04-25T00:35:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.131+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557ffc48b140 con 0x557ffc482ea0 2024-04-25T00:35:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.131+0000 7fd5f65a9740 10 client.0 _ll_register_callbacks cb 0x557ffc39a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.132+0000 7fd5f1600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 0x557ffc493d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.132+0000 7fd5f1600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 0x557ffc493d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51194/0 (socket says 172.21.15.42:51194) 2024-04-25T00:35:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.132+0000 7fd5f1600640 1 -- 172.21.15.42:0/1406997119 learned_addr learned my addr 172.21.15.42:0/1406997119 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.132+0000 7fd5ebe00640 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc48ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.132+0000 7fd5f0c00640 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc4824b0 0x557ffc48c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.133+0000 7fd5ebe00640 1 -- 172.21.15.42:0/1406997119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 msgr2=0x557ffc493d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.133+0000 7fd5ebe00640 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 0x557ffc493d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.133+0000 7fd5ebe00640 1 -- 172.21.15.42:0/1406997119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc4824b0 msgr2=0x557ffc48c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.133+0000 7fd5ebe00640 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc4824b0 0x557ffc48c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.133+0000 7fd5ebe00640 1 -- 172.21.15.42:0/1406997119 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ffc1bd620 con 0x557ffc482ea0 2024-04-25T00:35:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.133+0000 7fd5f0c00640 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc4824b0 0x557ffc48c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.133+0000 7fd5f1600640 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 0x557ffc493d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.134+0000 7fd5ebe00640 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc48ccc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009230 tx=0x7fd5d802a400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a28a1ce3a8c8236 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.134+0000 7fd5d2000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:35:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.134+0000 7fd5d2000640 1 -- 172.21.15.42:0/1406997119 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8035050 con 0x557ffc482ea0 2024-04-25T00:35:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.134+0000 7fd5d2000640 1 -- 172.21.15.42:0/1406997119 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fd5d80046c0 con 0x557ffc482ea0 2024-04-25T00:35:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.134+0000 7fd5f65a9740 1 -- 172.21.15.42:0/1406997119 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5e00043c0 con 0x557ffc482ea0 2024-04-25T00:35:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.135+0000 7fd5d2000640 1 -- 172.21.15.42:0/1406997119 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd5d80048c0 con 0x557ffc482ea0 2024-04-25T00:35:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.135+0000 7fd5f65a9740 10 client.26500 fetch_fsmap learned FSMap version 43 2024-04-25T00:35:27.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.135+0000 7fd5f65a9740 1 -- 172.21.15.42:0/1406997119 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557ffc496d50 con 0x557ffc482ea0 2024-04-25T00:35:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.135+0000 7fd5d2000640 1 -- 172.21.15.42:0/1406997119 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5e00043c0 con 0x557ffc482ea0 2024-04-25T00:35:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.135+0000 7fd5d2000640 1 -- 172.21.15.42:0/1406997119 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fd5d80035f0 con 0x557ffc482ea0 2024-04-25T00:35:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.135+0000 7fd5f65a9740 10 client.26500 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T00:35:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.135+0000 7fd5f65a9740 10 client.26500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:35:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.136+0000 7fd5f65a9740 20 client.26500 populate_metadata read hostname 'smithi042' 2024-04-25T00:35:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.136+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ffc498470 0x557ffc49a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.136+0000 7fd5f65a9740 4 client.26500 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:27.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.136+0000 7fd5f65a9740 1 -- 172.21.15.42:0/1406997119 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ffc3e9ff0 con 0x557ffc498470 2024-04-25T00:35:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.136+0000 7fd5f0c00640 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ffc498470 0x557ffc49a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.137+0000 7fd5f0c00640 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ffc498470 0x557ffc49a860 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ced8a1a4a568f05 server_cookie=96c6238ef97ada89 in_seq=0 out_seq=0 2024-04-25T00:35:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.137+0000 7fd5d2000640 10 client.26500 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:27.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.137+0000 7fd5d2000640 1 -- 172.21.15.42:0/1406997119 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ffc3e9ff0 con 0x557ffc498470 2024-04-25T00:35:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.137+0000 7fd5d2000640 10 client.26500 handle_command_reply: tid=0 2024-04-25T00:35:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.137+0000 7fd5f65a9740 1 client.26500 shutdown 2024-04-25T00:35:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.138+0000 7fd5f65a9740 1 -- 172.21.15.42:0/1406997119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 msgr2=0x557ffc48ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.138+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc48ccc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fd5d8009230 tx=0x7fd5d802a400 comp rx=0 tx=0).stop 2024-04-25T00:35:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.138+0000 7fd5f65a9740 1 -- 172.21.15.42:0/1406997119 shutdown_connections 2024-04-25T00:35:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.138+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ffc498470 0x557ffc49a860 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.138+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc488010 0x557ffc493d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.138+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc48ccc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.138+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/1406997119 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc4824b0 0x557ffc48c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.138+0000 7fd5f65a9740 1 -- 172.21.15.42:0/1406997119 >> 172.21.15.42:0/1406997119 conn(0x557ffc47db20 msgr2=0x557ffc474480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.138+0000 7fd5d2000640 0 client.26500 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.139+0000 7fd5f65a9740 1 -- 172.21.15.42:0/1406997119 shutdown_connections 2024-04-25T00:35:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.139+0000 7fd5f65a9740 1 -- 172.21.15.42:0/1406997119 wait complete. 2024-04-25T00:35:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.139+0000 7fd5f65a9740 20 client.26500 trim_cache size 0 max 16384 2024-04-25T00:35:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.169+0000 7fd5f65a9740 1 Processor -- start 2024-04-25T00:35:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.169+0000 7fd5f65a9740 1 -- start start 2024-04-25T00:35:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.170+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc47db20 0x557ffc481140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.170+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc493cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.170+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc49f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.170+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec1a9270 con 0x557ffc482ea0 2024-04-25T00:35:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.170+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec1a6ae0 con 0x557ffc4824b0 2024-04-25T00:35:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.170+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec1a4930 con 0x557ffc47db20 2024-04-25T00:35:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.170+0000 7fd5ebe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc493cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.170+0000 7fd5ebe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc493cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51196/0 (socket says 172.21.15.42:51196) 2024-04-25T00:35:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.170+0000 7fd5f0c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc47db20 0x557ffc481140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.170+0000 7fd5ebe00640 1 -- 172.21.15.42:0/204867093 learned_addr learned my addr 172.21.15.42:0/204867093 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:27.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.171+0000 7fd5f1600640 1 --2- 172.21.15.42:0/204867093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc49f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.171+0000 7fd5ebe00640 1 -- 172.21.15.42:0/204867093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc47db20 msgr2=0x557ffc481140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.171+0000 7fd5ebe00640 1 --2- 172.21.15.42:0/204867093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc47db20 0x557ffc481140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.171+0000 7fd5ebe00640 1 -- 172.21.15.42:0/204867093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 msgr2=0x557ffc49f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.171+0000 7fd5ebe00640 1 --2- 172.21.15.42:0/204867093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc49f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.171+0000 7fd5ebe00640 1 -- 172.21.15.42:0/204867093 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ffc49abd0 con 0x557ffc4824b0 2024-04-25T00:35:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.171+0000 7fd5f1600640 1 --2- 172.21.15.42:0/204867093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc49f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.171+0000 7fd5ebe00640 1 --2- 172.21.15.42:0/204867093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc493cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80040f0 tx=0x7fd5d8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64a6668b3e9474aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.172+0000 7fd5dd200640 1 -- 172.21.15.42:0/204867093 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8035040 con 0x557ffc4824b0 2024-04-25T00:35:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.172+0000 7fd5dd200640 1 -- 172.21.15.42:0/204867093 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d8003cd0 con 0x557ffc4824b0 2024-04-25T00:35:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.172+0000 7fd5dd200640 1 -- 172.21.15.42:0/204867093 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8032a20 con 0x557ffc4824b0 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.172+0000 7fd5f65a9740 1 -- 172.21.15.42:0/204867093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 msgr2=0x557ffc493cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.172+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/204867093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc493cc0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80040f0 tx=0x7fd5d8000bc0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.173+0000 7fd5f65a9740 1 -- 172.21.15.42:0/204867093 shutdown_connections 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.173+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/204867093 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc482ea0 0x557ffc49f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.173+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/204867093 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc493cc0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.173+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/204867093 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc47db20 0x557ffc481140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.173+0000 7fd5f65a9740 1 -- 172.21.15.42:0/204867093 >> 172.21.15.42:0/204867093 conn(0x557ffc487ae0 msgr2=0x557ffc480a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.173+0000 7fd5f65a9740 1 -- 172.21.15.42:0/204867093 shutdown_connections 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.173+0000 7fd5f65a9740 1 -- 172.21.15.42:0/204867093 wait complete. 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.173+0000 7fd5f65a9740 1 -- start start 2024-04-25T00:35:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.174+0000 7fd5f65a9740 1 Processor -- start 2024-04-25T00:35:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.174+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc47db20 0x557ffc491710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.174+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc491c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.174+0000 7fd5f65a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc492c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.174+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec1a4fa0 con 0x557ffc47db20 2024-04-25T00:35:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.174+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec1a6410 con 0x557ffc4824b0 2024-04-25T00:35:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.174+0000 7fd5f65a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec1a5670 con 0x557ffc482ea0 2024-04-25T00:35:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.174+0000 7fd5f65a9740 10 client.0 _ll_register_callbacks cb 0x7fd5e0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.175+0000 7fd5f1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc492c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.175+0000 7fd5f0c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc47db20 0x557ffc491710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.175+0000 7fd5ebe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc491c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.175+0000 7fd5ebe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc491c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51202/0 (socket says 172.21.15.42:51202) 2024-04-25T00:35:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.175+0000 7fd5ebe00640 1 -- 172.21.15.42:0/124166463 learned_addr learned my addr 172.21.15.42:0/124166463 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.175+0000 7fd5f1600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc492c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:34496/0 (socket says 172.21.15.42:34496) 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.176+0000 7fd5f1600640 1 -- 172.21.15.42:0/124166463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 msgr2=0x557ffc491c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.176+0000 7fd5f1600640 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc491c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.176+0000 7fd5f1600640 1 -- 172.21.15.42:0/124166463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc47db20 msgr2=0x557ffc491710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.176+0000 7fd5f1600640 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc47db20 0x557ffc491710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.176+0000 7fd5f1600640 1 -- 172.21.15.42:0/124166463 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ffc493200 con 0x557ffc482ea0 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.176+0000 7fd5f1600640 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc492c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd5e409bee0 tx=0x7fd5e409bb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa0ecc023aaeba34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.176+0000 7fd5d2000640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.177+0000 7fd5d2000640 1 -- 172.21.15.42:0/124166463 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e409bf50 con 0x557ffc482ea0 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.177+0000 7fd5d2000640 1 -- 172.21.15.42:0/124166463 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fd5e40052f0 con 0x557ffc482ea0 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.177+0000 7fd5f65a9740 1 -- 172.21.15.42:0/124166463 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5d8003cd0 con 0x557ffc482ea0 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.177+0000 7fd5d2000640 1 -- 172.21.15.42:0/124166463 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd5e40a0300 con 0x557ffc482ea0 2024-04-25T00:35:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.178+0000 7fd5f65a9740 10 client.26506 fetch_fsmap learned FSMap version 43 2024-04-25T00:35:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.178+0000 7fd5f65a9740 1 -- 172.21.15.42:0/124166463 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557ffc475140 con 0x557ffc482ea0 2024-04-25T00:35:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.178+0000 7fd5d2000640 1 -- 172.21.15.42:0/124166463 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5d8003cd0 con 0x557ffc482ea0 2024-04-25T00:35:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.179+0000 7fd5d2000640 1 -- 172.21.15.42:0/124166463 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fd5e409f030 con 0x557ffc482ea0 2024-04-25T00:35:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.179+0000 7fd5f65a9740 10 client.26506 fetch_fsmap finished waiting for FSMap version 43 2024-04-25T00:35:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.179+0000 7fd5f65a9740 10 client.26506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:35:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.179+0000 7fd5f65a9740 20 client.26506 populate_metadata read hostname 'smithi042' 2024-04-25T00:35:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.179+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ffc488010 0x557ffc477b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.179+0000 7fd5f65a9740 4 client.26506 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:35:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.179+0000 7fd5f65a9740 1 -- 172.21.15.42:0/124166463 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ffc3e4340 con 0x557ffc488010 2024-04-25T00:35:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.180+0000 7fd5f0c00640 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ffc488010 0x557ffc477b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.180+0000 7fd5f0c00640 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ffc488010 0x557ffc477b80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14cf438d137b04c1 server_cookie=7d810aba4fcc5264 in_seq=0 out_seq=0 2024-04-25T00:35:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.181+0000 7fd5d2000640 10 client.26506 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.700+0000 7fd5d2000640 1 -- 172.21.15.42:0/124166463 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ffc3e4340 con 0x557ffc488010 2024-04-25T00:35:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.700+0000 7fd5d2000640 10 client.26506 handle_command_reply: tid=0 2024-04-25T00:35:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.700+0000 7fd5f65a9740 1 client.26506 shutdown 2024-04-25T00:35:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.700+0000 7fd5f65a9740 1 -- 172.21.15.42:0/124166463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 msgr2=0x557ffc492c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.700+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc492c10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd5e409bee0 tx=0x7fd5e409bb90 comp rx=0 tx=0).stop 2024-04-25T00:35:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.701+0000 7fd5f65a9740 1 -- 172.21.15.42:0/124166463 shutdown_connections 2024-04-25T00:35:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.701+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ffc488010 0x557ffc477b80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.701+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ffc482ea0 0x557ffc492c10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.701+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ffc4824b0 0x557ffc491c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.701+0000 7fd5f65a9740 1 --2- 172.21.15.42:0/124166463 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ffc47db20 0x557ffc491710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.701+0000 7fd5f65a9740 1 -- 172.21.15.42:0/124166463 >> 172.21.15.42:0/124166463 conn(0x557ffc487ae0 msgr2=0x557ffc480a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.701+0000 7fd5d2000640 0 client.26506 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.702+0000 7fd5f65a9740 1 -- 172.21.15.42:0/124166463 shutdown_connections 2024-04-25T00:35:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.702+0000 7fd5f65a9740 1 -- 172.21.15.42:0/124166463 wait complete. 2024-04-25T00:35:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.702+0000 7fd5f65a9740 20 client.26506 trim_cache size 0 max 16384 2024-04-25T00:35:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:35:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:35:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:35:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:35:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.704+0000 7fd5f3600640 1 -- 172.21.15.42:0/758119874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5d40a2d00 msgr2=0x7fd5d40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.704+0000 7fd5f3600640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5d40a2d00 0x7fd5d40a51c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd5d80096c0 tx=0x7fd5d8031400 comp rx=0 tx=0).stop 2024-04-25T00:35:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.704+0000 7fd5f3600640 1 -- 172.21.15.42:0/758119874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec105580 msgr2=0x7fd5ec1a11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.704+0000 7fd5f3600640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec105580 0x7fd5ec1a11c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400eb10 tx=0x7fd5e400c2d0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.705+0000 7fd5f3600640 1 -- 172.21.15.42:0/758119874 shutdown_connections 2024-04-25T00:35:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.705+0000 7fd5f3600640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fd5d40a2d00 0x7fd5d40a51c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.705+0000 7fd5f3600640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd5ec105580 0x7fd5ec1a11c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.705+0000 7fd5f3600640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fd5ec104bc0 0x7fd5ec19a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.705+0000 7fd5f3600640 1 --2- 172.21.15.42:0/758119874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5ec1039d0 0x7fd5ec199c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.705+0000 7fd5f3600640 1 -- 172.21.15.42:0/758119874 >> 172.21.15.42:0/758119874 conn(0x7fd5ec0fef60 msgr2=0x7fd5ec107e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.705+0000 7fd5f3600640 1 -- 172.21.15.42:0/758119874 shutdown_connections 2024-04-25T00:35:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:27.706+0000 7fd5f3600640 1 -- 172.21.15.42:0/758119874 wait complete. 2024-04-25T00:35:27.753 INFO:tasks.workunit.client.0.smithi042.stdout: 10 127029 35.46 MB/sec execute 77 sec latency 368.545 ms 2024-04-25T00:35:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:27 smithi106 ceph-mon[32510]: pgmap v891: 97 pgs: 97 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 689 op/s 2024-04-25T00:35:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:27 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-25T00:35:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:27 smithi106 ceph-mon[32510]: Unable to set osd_memory_target on smithi106 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T00:35:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:35:28.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:35:28.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:35:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:27 smithi042 ceph-mon[28130]: pgmap v891: 97 pgs: 97 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 689 op/s 2024-04-25T00:35:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:27 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-25T00:35:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:27 smithi042 ceph-mon[28130]: Unable to set osd_memory_target on smithi106 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T00:35:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:35:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:35:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:35:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:27 smithi136 ceph-mon[32923]: pgmap v891: 97 pgs: 97 active+clean; 592 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.0 MiB/s wr, 689 op/s 2024-04-25T00:35:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:27 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi106 to 176.2M 2024-04-25T00:35:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:27 smithi136 ceph-mon[32923]: Unable to set osd_memory_target on smithi106 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-04-25T00:35:28.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:35:28.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:35:28.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:35:28.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:35:28.754 INFO:tasks.workunit.client.0.smithi042.stdout: 10 127575 35.35 MB/sec execute 78 sec latency 172.740 ms 2024-04-25T00:35:29.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:28 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-25T00:35:29.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:28 smithi106 ceph-mon[32510]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T00:35:29.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:28 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-25T00:35:29.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:28 smithi042 ceph-mon[28130]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T00:35:29.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:28 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi042 to 177.5M 2024-04-25T00:35:29.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:28 smithi136 ceph-mon[32923]: Unable to set osd_memory_target on smithi042 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-25T00:35:29.753 INFO:tasks.workunit.client.0.smithi042.stdout: 10 128197 35.35 MB/sec execute 79 sec latency 322.080 ms 2024-04-25T00:35:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:29 smithi106 ceph-mon[32510]: pgmap v892: 97 pgs: 97 active+clean; 599 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 491 op/s 2024-04-25T00:35:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:29 smithi042 ceph-mon[28130]: pgmap v892: 97 pgs: 97 active+clean; 599 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 491 op/s 2024-04-25T00:35:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:29 smithi136 ceph-mon[32923]: pgmap v892: 97 pgs: 97 active+clean; 599 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 491 op/s 2024-04-25T00:35:30.754 INFO:tasks.workunit.client.0.smithi042.stdout: 10 129021 35.47 MB/sec execute 80 sec latency 334.210 ms 2024-04-25T00:35:31.753 INFO:tasks.workunit.client.0.smithi042.stdout: 10 129784 35.61 MB/sec execute 81 sec latency 43.363 ms 2024-04-25T00:35:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:31 smithi106 ceph-mon[32510]: pgmap v893: 97 pgs: 97 active+clean; 598 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 555 op/s 2024-04-25T00:35:32.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:31 smithi042 ceph-mon[28130]: pgmap v893: 97 pgs: 97 active+clean; 598 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 555 op/s 2024-04-25T00:35:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:31 smithi136 ceph-mon[32923]: pgmap v893: 97 pgs: 97 active+clean; 598 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.1 MiB/s wr, 555 op/s 2024-04-25T00:35:32.753 INFO:tasks.workunit.client.0.smithi042.stdout: 10 130415 35.64 MB/sec execute 82 sec latency 69.783 ms 2024-04-25T00:35:33.753 INFO:tasks.workunit.client.0.smithi042.stdout: 10 130566 35.26 MB/sec execute 83 sec latency 615.801 ms 2024-04-25T00:35:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:33 smithi106 ceph-mon[32510]: pgmap v894: 97 pgs: 97 active+clean; 587 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 639 op/s 2024-04-25T00:35:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:33 smithi042 ceph-mon[28130]: pgmap v894: 97 pgs: 97 active+clean; 587 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 639 op/s 2024-04-25T00:35:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:33 smithi136 ceph-mon[32923]: pgmap v894: 97 pgs: 97 active+clean; 587 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 7.3 MiB/s wr, 639 op/s 2024-04-25T00:35:34.754 INFO:tasks.workunit.client.0.smithi042.stdout: 10 130734 34.92 MB/sec execute 84 sec latency 865.462 ms 2024-04-25T00:35:35.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:34 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:35.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:34 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:35.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:34 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:35.754 INFO:tasks.workunit.client.0.smithi042.stdout: 10 131082 34.67 MB/sec execute 85 sec latency 219.323 ms 2024-04-25T00:35:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:35 smithi106 ceph-mon[32510]: pgmap v895: 97 pgs: 97 active+clean; 588 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-04-25T00:35:36.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:35 smithi042 ceph-mon[28130]: pgmap v895: 97 pgs: 97 active+clean; 588 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-04-25T00:35:36.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:35 smithi136 ceph-mon[32923]: pgmap v895: 97 pgs: 97 active+clean; 588 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 6.6 MiB/s wr, 575 op/s 2024-04-25T00:35:36.754 INFO:tasks.workunit.client.0.smithi042.stdout: 10 131897 34.75 MB/sec execute 86 sec latency 81.112 ms 2024-04-25T00:35:37.754 INFO:tasks.workunit.client.0.smithi042.stdout: 10 132886 35.06 MB/sec execute 87 sec latency 38.133 ms 2024-04-25T00:35:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:37 smithi106 ceph-mon[32510]: pgmap v896: 97 pgs: 97 active+clean; 613 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 935 op/s 2024-04-25T00:35:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:37 smithi042 ceph-mon[28130]: pgmap v896: 97 pgs: 97 active+clean; 613 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 935 op/s 2024-04-25T00:35:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:37 smithi136 ceph-mon[32923]: pgmap v896: 97 pgs: 97 active+clean; 613 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 935 op/s 2024-04-25T00:35:38.754 INFO:tasks.workunit.client.0.smithi042.stdout: 10 133617 35.19 MB/sec execute 88 sec latency 41.822 ms 2024-04-25T00:35:39.754 INFO:tasks.workunit.client.0.smithi042.stdout: 10 134149 35.05 MB/sec execute 89 sec latency 68.040 ms 2024-04-25T00:35:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:39 smithi106 ceph-mon[32510]: pgmap v897: 97 pgs: 97 active+clean; 607 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 960 op/s 2024-04-25T00:35:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:39 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:40.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:39 smithi042 ceph-mon[28130]: pgmap v897: 97 pgs: 97 active+clean; 607 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 960 op/s 2024-04-25T00:35:40.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:39 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:39 smithi136 ceph-mon[32923]: pgmap v897: 97 pgs: 97 active+clean; 607 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 18 MiB/s wr, 960 op/s 2024-04-25T00:35:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:39 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:35:40.755 INFO:tasks.workunit.client.0.smithi042.stdout: 10 134694 34.89 MB/sec execute 90 sec latency 71.538 ms 2024-04-25T00:35:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.898+0000 7fef84400640 1 -- 172.21.15.42:0/1769175209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7c074140 msgr2=0x7fef7c0745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.898+0000 7fef84400640 1 --2- 172.21.15.42:0/1769175209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7c074140 0x7fef7c0745a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fef68009900 tx=0x7fef6802a490 comp rx=0 tx=0).stop 2024-04-25T00:35:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.898+0000 7fef84400640 1 -- 172.21.15.42:0/1769175209 shutdown_connections 2024-04-25T00:35:40.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.898+0000 7fef84400640 1 --2- 172.21.15.42:0/1769175209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c109630 0x7fef7c10bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.898+0000 7fef84400640 1 --2- 172.21.15.42:0/1769175209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7c074140 0x7fef7c0745a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.898+0000 7fef84400640 1 --2- 172.21.15.42:0/1769175209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c075720 0x7fef7c073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.898+0000 7fef84400640 1 -- 172.21.15.42:0/1769175209 >> 172.21.15.42:0/1769175209 conn(0x7fef7c0fcd10 msgr2=0x7fef7c0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.898+0000 7fef84400640 1 -- 172.21.15.42:0/1769175209 shutdown_connections 2024-04-25T00:35:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.898+0000 7fef84400640 1 -- 172.21.15.42:0/1769175209 wait complete. 2024-04-25T00:35:40.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.899+0000 7fef84400640 1 Processor -- start 2024-04-25T00:35:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.900+0000 7fef84400640 1 -- start start 2024-04-25T00:35:40.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.900+0000 7fef84400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7c074140 0x7fef7c104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.900+0000 7fef84400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c075720 0x7fef7c105060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.900+0000 7fef84400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c109630 0x7fef7c101690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.900+0000 7fef84400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c10f6c0 con 0x7fef7c075720 2024-04-25T00:35:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.900+0000 7fef84400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c10f540 con 0x7fef7c074140 2024-04-25T00:35:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.900+0000 7fef84400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c10f840 con 0x7fef7c109630 2024-04-25T00:35:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.901+0000 7fef81a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7c074140 0x7fef7c104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.901+0000 7fef81a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7c074140 0x7fef7c104b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:55836/0 (socket says 172.21.15.42:55836) 2024-04-25T00:35:40.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.901+0000 7fef81a00640 1 -- 172.21.15.42:0/3230355947 learned_addr learned my addr 172.21.15.42:0/3230355947 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.901+0000 7fef81a00640 1 -- 172.21.15.42:0/3230355947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c109630 msgr2=0x7fef7c101690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.901+0000 7fef81000640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c075720 0x7fef7c105060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.901+0000 7fef81a00640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c109630 0x7fef7c101690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.901+0000 7fef81a00640 1 -- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c075720 msgr2=0x7fef7c105060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.901+0000 7fef81a00640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c075720 0x7fef7c105060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.901+0000 7fef81a00640 1 -- 172.21.15.42:0/3230355947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c101f50 con 0x7fef7c074140 2024-04-25T00:35:40.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.901+0000 7fef81000640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c075720 0x7fef7c105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:35:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.902+0000 7fef81a00640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7c074140 0x7fef7c104b20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fef7000b4a0 tx=0x7fef7000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.902+0000 7fef7a000640 1 -- 172.21.15.42:0/3230355947 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70014070 con 0x7fef7c074140 2024-04-25T00:35:40.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.902+0000 7fef7a000640 1 -- 172.21.15.42:0/3230355947 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef70010af0 con 0x7fef7c074140 2024-04-25T00:35:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.902+0000 7fef7a000640 1 -- 172.21.15.42:0/3230355947 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef70017d10 con 0x7fef7c074140 2024-04-25T00:35:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.902+0000 7fef84400640 1 -- 172.21.15.42:0/3230355947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c1a60a0 con 0x7fef7c074140 2024-04-25T00:35:40.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.902+0000 7fef84400640 1 -- 172.21.15.42:0/3230355947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c1a6530 con 0x7fef7c074140 2024-04-25T00:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.907+0000 7fef87251740 1 Processor -- start 2024-04-25T00:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.908+0000 7fef87251740 1 -- start start 2024-04-25T00:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.908+0000 7fef87251740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce23ca0 0x55577ce240c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.908+0000 7fef87251740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 0x55577ce29190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.908+0000 7fef87251740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce29800 0x55577ce2bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.908+0000 7fef87251740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55577cd86ab0 con 0x55577ce24690 2024-04-25T00:35:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.908+0000 7fef87251740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55577cd2f640 con 0x55577ce29800 2024-04-25T00:35:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.908+0000 7fef87251740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55577cd82a70 con 0x55577ce23ca0 2024-04-25T00:35:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.908+0000 7fef7a000640 1 -- 172.21.15.42:0/3230355947 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fef70017070 con 0x7fef7c074140 2024-04-25T00:35:40.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.909+0000 7fef81a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 0x55577ce29190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.909+0000 7fef81a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 0x55577ce29190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49800/0 (socket says 172.21.15.42:49800) 2024-04-25T00:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.909+0000 7fef81a00640 1 -- 172.21.15.42:0/1791083952 learned_addr learned my addr 172.21.15.42:0/1791083952 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.909+0000 7fef82400640 1 --2- 172.21.15.42:0/1791083952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce23ca0 0x55577ce240c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.909+0000 7fef81000640 1 --2- 172.21.15.42:0/1791083952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce29800 0x55577ce2bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.909+0000 7fef81a00640 1 -- 172.21.15.42:0/1791083952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce23ca0 msgr2=0x55577ce240c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.909+0000 7fef81a00640 1 --2- 172.21.15.42:0/1791083952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce23ca0 0x55577ce240c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.909+0000 7fef81a00640 1 -- 172.21.15.42:0/1791083952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce29800 msgr2=0x55577ce2bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.909+0000 7fef81a00640 1 --2- 172.21.15.42:0/1791083952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce29800 0x55577ce2bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.909+0000 7fef81a00640 1 -- 172.21.15.42:0/1791083952 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55577ca9b210 con 0x55577ce24690 2024-04-25T00:35:40.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.910+0000 7fef81a00640 1 --2- 172.21.15.42:0/1791083952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 0x55577ce29190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fef700cbf00 tx=0x7fef700d4b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1011d64845cf5c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.910+0000 7fef6ea00640 1 -- 172.21.15.42:0/1791083952 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef700e1070 con 0x55577ce24690 2024-04-25T00:35:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.910+0000 7fef6ea00640 1 -- 172.21.15.42:0/1791083952 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef700d0550 con 0x55577ce24690 2024-04-25T00:35:40.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.910+0000 7fef6ea00640 1 -- 172.21.15.42:0/1791083952 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef700d6b90 con 0x55577ce24690 2024-04-25T00:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.911+0000 7fef7a000640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fef640a2c30 0x7fef640a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.911+0000 7fef87251740 1 -- 172.21.15.42:0/1791083952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 msgr2=0x55577ce29190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.911+0000 7fef87251740 1 --2- 172.21.15.42:0/1791083952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 0x55577ce29190 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fef700cbf00 tx=0x7fef700d4b80 comp rx=0 tx=0).stop 2024-04-25T00:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.912+0000 7fef7a000640 1 -- 172.21.15.42:0/3230355947 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fef700ce490 con 0x7fef7c074140 2024-04-25T00:35:40.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.912+0000 7fef82400640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fef640a2c30 0x7fef640a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.913+0000 7fef87251740 1 -- 172.21.15.42:0/1791083952 shutdown_connections 2024-04-25T00:35:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.913+0000 7fef87251740 1 --2- 172.21.15.42:0/1791083952 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce29800 0x55577ce2bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.913+0000 7fef87251740 1 --2- 172.21.15.42:0/1791083952 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 0x55577ce29190 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.913+0000 7fef87251740 1 --2- 172.21.15.42:0/1791083952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce23ca0 0x55577ce240c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.913+0000 7fef87251740 1 -- 172.21.15.42:0/1791083952 >> 172.21.15.42:0/1791083952 conn(0x55577ce1f310 msgr2=0x55577ce1f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.913+0000 7fef82400640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fef640a2c30 0x7fef640a50f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fef74005c40 tx=0x7fef74005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.913+0000 7fef87251740 1 -- 172.21.15.42:0/1791083952 shutdown_connections 2024-04-25T00:35:40.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.913+0000 7fef87251740 1 -- 172.21.15.42:0/1791083952 wait complete. 2024-04-25T00:35:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.914+0000 7fef87251740 1 -- start start 2024-04-25T00:35:40.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.914+0000 7fef87251740 1 Processor -- start 2024-04-25T00:35:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.915+0000 7fef87251740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce23ca0 0x55577ce2df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.915+0000 7fef87251740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce24690 0x55577ce2e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.915+0000 7fef87251740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 0x55577ce354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.915+0000 7fef87251740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55577ce2c7b0 con 0x55577ce23ca0 2024-04-25T00:35:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.915+0000 7fef87251740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55577cd42990 con 0x55577ce24690 2024-04-25T00:35:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.915+0000 7fef87251740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55577ce2c930 con 0x55577ce29800 2024-04-25T00:35:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.915+0000 7fef87251740 10 client.0 _ll_register_callbacks cb 0x55577cd44950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:40.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef81000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce24690 0x55577ce2e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef82400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 0x55577ce354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef82400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 0x55577ce354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39648/0 (socket says 172.21.15.42:39648) 2024-04-25T00:35:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef81000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce24690 0x55577ce2e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:55850/0 (socket says 172.21.15.42:55850) 2024-04-25T00:35:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef82400640 1 -- 172.21.15.42:0/2877791671 learned_addr learned my addr 172.21.15.42:0/2877791671 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:40.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef81a00640 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce23ca0 0x55577ce2df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef82400640 1 -- 172.21.15.42:0/2877791671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce24690 msgr2=0x55577ce2e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef82400640 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce24690 0x55577ce2e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef82400640 1 -- 172.21.15.42:0/2877791671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce23ca0 msgr2=0x55577ce2df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef82400640 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce23ca0 0x55577ce2df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.916+0000 7fef82400640 1 -- 172.21.15.42:0/2877791671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55577cb67620 con 0x55577ce29800 2024-04-25T00:35:40.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.917+0000 7fef82400640 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 0x55577ce354f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fef74009a70 tx=0x7fef74004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88d7b755f12e58c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.917+0000 7fef63400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:35:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.917+0000 7fef63400640 1 -- 172.21.15.42:0/2877791671 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74022070 con 0x55577ce29800 2024-04-25T00:35:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.917+0000 7fef63400640 1 -- 172.21.15.42:0/2877791671 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fef74010920 con 0x55577ce29800 2024-04-25T00:35:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.918+0000 7fef87251740 1 -- 172.21.15.42:0/2877791671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef700d0550 con 0x55577ce29800 2024-04-25T00:35:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.919+0000 7fef63400640 1 -- 172.21.15.42:0/2877791671 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fef7402c050 con 0x55577ce29800 2024-04-25T00:35:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.919+0000 7fef87251740 10 client.26512 fetch_fsmap learned FSMap version 45 2024-04-25T00:35:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.919+0000 7fef87251740 1 -- 172.21.15.42:0/2877791671 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55577ce385a0 con 0x55577ce29800 2024-04-25T00:35:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.920+0000 7fef63400640 1 -- 172.21.15.42:0/2877791671 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef700d0550 con 0x55577ce29800 2024-04-25T00:35:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.920+0000 7fef63400640 1 -- 172.21.15.42:0/2877791671 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fef74027440 con 0x55577ce29800 2024-04-25T00:35:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.920+0000 7fef87251740 10 client.26512 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:35:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.920+0000 7fef87251740 10 client.26512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:35:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.920+0000 7fef87251740 20 client.26512 populate_metadata read hostname 'smithi042' 2024-04-25T00:35:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.920+0000 7fef87251740 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55577ce39ce0 0x55577ce3c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.921+0000 7fef87251740 4 client.26512 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.921+0000 7fef87251740 1 -- 172.21.15.42:0/2877791671 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55577cd93ff0 con 0x55577ce39ce0 2024-04-25T00:35:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.921+0000 7fef81a00640 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55577ce39ce0 0x55577ce3c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.922+0000 7fef81a00640 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55577ce39ce0 0x55577ce3c110 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83a45631259e4fbd server_cookie=3f3edc0172c1211b in_seq=0 out_seq=0 2024-04-25T00:35:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.923+0000 7fef63400640 10 client.26512 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.925+0000 7fef63400640 1 -- 172.21.15.42:0/2877791671 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55577cd93ff0 con 0x55577ce39ce0 2024-04-25T00:35:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.925+0000 7fef63400640 10 client.26512 handle_command_reply: tid=0 2024-04-25T00:35:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.925+0000 7fef87251740 1 client.26512 shutdown 2024-04-25T00:35:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.925+0000 7fef87251740 1 -- 172.21.15.42:0/2877791671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 msgr2=0x55577ce354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.925+0000 7fef87251740 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 0x55577ce354f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fef74009a70 tx=0x7fef74004a40 comp rx=0 tx=0).stop 2024-04-25T00:35:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.926+0000 7fef87251740 1 -- 172.21.15.42:0/2877791671 shutdown_connections 2024-04-25T00:35:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.926+0000 7fef87251740 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55577ce39ce0 0x55577ce3c110 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.926+0000 7fef87251740 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 0x55577ce354f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.926+0000 7fef87251740 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce24690 0x55577ce2e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.926+0000 7fef87251740 1 --2- 172.21.15.42:0/2877791671 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce23ca0 0x55577ce2df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.926+0000 7fef87251740 1 -- 172.21.15.42:0/2877791671 >> 172.21.15.42:0/2877791671 conn(0x55577ce1f310 msgr2=0x55577ce1e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.926+0000 7fef63400640 0 client.26512 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.927+0000 7fef87251740 1 -- 172.21.15.42:0/2877791671 shutdown_connections 2024-04-25T00:35:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.927+0000 7fef87251740 1 -- 172.21.15.42:0/2877791671 wait complete. 2024-04-25T00:35:40.929 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.927+0000 7fef87251740 20 client.26512 trim_cache size 0 max 16384 2024-04-25T00:35:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.960+0000 7fef87251740 1 Processor -- start 2024-04-25T00:35:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.961+0000 7fef87251740 1 -- start start 2024-04-25T00:35:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.961+0000 7fef87251740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce22700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.961+0000 7fef87251740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 0x55577ce354b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.961+0000 7fef87251740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 0x55577ce40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.961+0000 7fef87251740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1b1c00 con 0x55577ce24690 2024-04-25T00:35:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.961+0000 7fef87251740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1af4d0 con 0x55577ce23ca0 2024-04-25T00:35:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.961+0000 7fef87251740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1ad320 con 0x55577ce29800 2024-04-25T00:35:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.962+0000 7fef81a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce22700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.962+0000 7fef82400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 0x55577ce40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.962+0000 7fef81a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce22700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:55854/0 (socket says 172.21.15.42:55854) 2024-04-25T00:35:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.962+0000 7fef81a00640 1 -- 172.21.15.42:0/450578056 learned_addr learned my addr 172.21.15.42:0/450578056 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.962+0000 7fef81000640 1 --2- 172.21.15.42:0/450578056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 0x55577ce354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.962+0000 7fef81a00640 1 -- 172.21.15.42:0/450578056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 msgr2=0x55577ce40a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.962+0000 7fef81a00640 1 --2- 172.21.15.42:0/450578056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 0x55577ce40a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.962+0000 7fef81a00640 1 -- 172.21.15.42:0/450578056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 msgr2=0x55577ce354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.962+0000 7fef81a00640 1 --2- 172.21.15.42:0/450578056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 0x55577ce354b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.962+0000 7fef81a00640 1 -- 172.21.15.42:0/450578056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55577ce1edd0 con 0x55577ce23ca0 2024-04-25T00:35:40.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.963+0000 7fef81a00640 1 --2- 172.21.15.42:0/450578056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce22700 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fef70017fd0 tx=0x7fef700047d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29da0d942dfb5702 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.964+0000 7fef6e000640 1 -- 172.21.15.42:0/450578056 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7001e040 con 0x55577ce23ca0 2024-04-25T00:35:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.965+0000 7fef6e000640 1 -- 172.21.15.42:0/450578056 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef70009230 con 0x55577ce23ca0 2024-04-25T00:35:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.965+0000 7fef6e000640 1 -- 172.21.15.42:0/450578056 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7009f0d0 con 0x55577ce23ca0 2024-04-25T00:35:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.966+0000 7fef87251740 1 -- 172.21.15.42:0/450578056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 msgr2=0x55577ce22700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.966+0000 7fef87251740 1 --2- 172.21.15.42:0/450578056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce22700 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fef70017fd0 tx=0x7fef700047d0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.966+0000 7fef87251740 1 -- 172.21.15.42:0/450578056 shutdown_connections 2024-04-25T00:35:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.966+0000 7fef87251740 1 --2- 172.21.15.42:0/450578056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce29800 0x55577ce40a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.966+0000 7fef87251740 1 --2- 172.21.15.42:0/450578056 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce24690 0x55577ce354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.966+0000 7fef87251740 1 --2- 172.21.15.42:0/450578056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce22700 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.966+0000 7fef87251740 1 -- 172.21.15.42:0/450578056 >> 172.21.15.42:0/450578056 conn(0x55577ce1f310 msgr2=0x55577ce21ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.966+0000 7fef87251740 1 -- 172.21.15.42:0/450578056 shutdown_connections 2024-04-25T00:35:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.967+0000 7fef87251740 1 -- 172.21.15.42:0/450578056 wait complete. 2024-04-25T00:35:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.967+0000 7fef87251740 1 -- start start 2024-04-25T00:35:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.968+0000 7fef87251740 1 Processor -- start 2024-04-25T00:35:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.970+0000 7fef87251740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce32fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.970+0000 7fef87251740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce24690 0x55577ce334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.970+0000 7fef87251740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce29800 0x55577ce344f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.970+0000 7fef81a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce32fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.970+0000 7fef81a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce32fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:55860/0 (socket says 172.21.15.42:55860) 2024-04-25T00:35:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.971+0000 7fef81000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce24690 0x55577ce334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.971+0000 7fef82400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce29800 0x55577ce344f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.971+0000 7fef82400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce29800 0x55577ce344f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49818/0 (socket says 172.21.15.42:49818) 2024-04-25T00:35:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.971+0000 7fef81000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce24690 0x55577ce334e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:39674/0 (socket says 172.21.15.42:39674) 2024-04-25T00:35:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.971+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1ad990 con 0x55577ce29800 2024-04-25T00:35:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.971+0000 7fef81a00640 1 -- 172.21.15.42:0/4007705247 learned_addr learned my addr 172.21.15.42:0/4007705247 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.971+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1aee00 con 0x55577ce23ca0 2024-04-25T00:35:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.971+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1ae060 con 0x55577ce24690 2024-04-25T00:35:40.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.971+0000 7fef87251740 10 client.0 _ll_register_callbacks cb 0x7fef74015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.972+0000 7fef81000640 1 -- 172.21.15.42:0/4007705247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 msgr2=0x55577ce32fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.972+0000 7fef81000640 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce32fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.972+0000 7fef81000640 1 -- 172.21.15.42:0/4007705247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce29800 msgr2=0x55577ce344f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.972+0000 7fef81000640 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce29800 0x55577ce344f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.972+0000 7fef81000640 1 -- 172.21.15.42:0/4007705247 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55577ce34ae0 con 0x55577ce24690 2024-04-25T00:35:40.974 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.973+0000 7fef82400640 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce29800 0x55577ce344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:35:40.975 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.973+0000 7fef81000640 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce24690 0x55577ce334e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fef68006010 tx=0x7fef68005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5dc915e38255250f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.973+0000 7fef63400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:35:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.973+0000 7fef63400640 1 -- 172.21.15.42:0/4007705247 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef68035040 con 0x55577ce24690 2024-04-25T00:35:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.973+0000 7fef63400640 1 -- 172.21.15.42:0/4007705247 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fef680316b0 con 0x55577ce24690 2024-04-25T00:35:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.973+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef70009230 con 0x55577ce24690 2024-04-25T00:35:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.974+0000 7fef63400640 1 -- 172.21.15.42:0/4007705247 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fef680318b0 con 0x55577ce24690 2024-04-25T00:35:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.974+0000 7fef87251740 10 client.26518 fetch_fsmap learned FSMap version 45 2024-04-25T00:35:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.974+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55577cea0650 con 0x55577ce24690 2024-04-25T00:35:40.976 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.974+0000 7fef63400640 1 -- 172.21.15.42:0/4007705247 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef70009230 con 0x55577ce24690 2024-04-25T00:35:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.975+0000 7fef63400640 1 -- 172.21.15.42:0/4007705247 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fef680035f0 con 0x55577ce24690 2024-04-25T00:35:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.975+0000 7fef87251740 10 client.26518 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:35:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.975+0000 7fef87251740 10 client.26518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:35:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.975+0000 7fef87251740 20 client.26518 populate_metadata read hostname 'smithi042' 2024-04-25T00:35:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.976+0000 7fef87251740 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55577cea1080 0x55577cea34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.976+0000 7fef87251740 4 client.26518 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:35:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.976+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55577cd8e340 con 0x55577cea1080 2024-04-25T00:35:40.977 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.976+0000 7fef81a00640 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55577cea1080 0x55577cea34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.977+0000 7fef81a00640 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55577cea1080 0x55577cea34a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1861b5690c16db8 server_cookie=bdad926126170470 in_seq=0 out_seq=0 2024-04-25T00:35:40.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:40.977+0000 7fef63400640 10 client.26518 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.210+0000 7fef63400640 1 -- 172.21.15.42:0/4007705247 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55577cd8e340 con 0x55577cea1080 2024-04-25T00:35:41.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.210+0000 7fef63400640 10 client.26518 handle_command_reply: tid=0 2024-04-25T00:35:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.210+0000 7fef87251740 1 client.26518 shutdown 2024-04-25T00:35:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.211+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce24690 msgr2=0x55577ce334e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.211+0000 7fef87251740 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce24690 0x55577ce334e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fef68006010 tx=0x7fef68005250 comp rx=0 tx=0).stop 2024-04-25T00:35:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.211+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 shutdown_connections 2024-04-25T00:35:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.211+0000 7fef87251740 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55577cea1080 0x55577cea34a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.211+0000 7fef87251740 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55577ce29800 0x55577ce344f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.211+0000 7fef63400640 0 client.26518 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.211+0000 7fef87251740 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55577ce24690 0x55577ce334e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.211+0000 7fef87251740 1 --2- 172.21.15.42:0/4007705247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55577ce23ca0 0x55577ce32fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.211+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 >> 172.21.15.42:0/4007705247 conn(0x55577ce1f310 msgr2=0x55577ce22880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.212+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 shutdown_connections 2024-04-25T00:35:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.212+0000 7fef87251740 1 -- 172.21.15.42:0/4007705247 wait complete. 2024-04-25T00:35:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.212+0000 7fef87251740 20 client.26518 trim_cache size 0 max 16384 2024-04-25T00:35:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:35:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:35:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:35:41.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:35:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 -- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fef640a2c30 msgr2=0x7fef640a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fef640a2c30 0x7fef640a50f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fef74005c40 tx=0x7fef74005bd0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 -- 172.21.15.42:0/3230355947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7c074140 msgr2=0x7fef7c104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7c074140 0x7fef7c104b20 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fef7000b4a0 tx=0x7fef7000b960 comp rx=0 tx=0).stop 2024-04-25T00:35:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 -- 172.21.15.42:0/3230355947 shutdown_connections 2024-04-25T00:35:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fef640a2c30 0x7fef640a50f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c109630 0x7fef7c101690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fef7c075720 0x7fef7c105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 --2- 172.21.15.42:0/3230355947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef7c074140 0x7fef7c104b20 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 -- 172.21.15.42:0/3230355947 >> 172.21.15.42:0/3230355947 conn(0x7fef7c0fcd10 msgr2=0x7fef7c1055c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 -- 172.21.15.42:0/3230355947 shutdown_connections 2024-04-25T00:35:41.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:41.213+0000 7fef84400640 1 -- 172.21.15.42:0/3230355947 wait complete. 2024-04-25T00:35:41.755 INFO:tasks.workunit.client.0.smithi042.stdout: 10 135527 34.94 MB/sec execute 91 sec latency 157.154 ms 2024-04-25T00:35:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:41 smithi106 ceph-mon[32510]: pgmap v898: 97 pgs: 97 active+clean; 593 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 990 op/s 2024-04-25T00:35:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:41 smithi042 ceph-mon[28130]: pgmap v898: 97 pgs: 97 active+clean; 593 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 990 op/s 2024-04-25T00:35:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:41 smithi136 ceph-mon[32923]: pgmap v898: 97 pgs: 97 active+clean; 593 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 18 MiB/s wr, 990 op/s 2024-04-25T00:35:42.755 INFO:tasks.workunit.client.0.smithi042.stdout: 10 136559 35.33 MB/sec execute 92 sec latency 34.201 ms 2024-04-25T00:35:43.755 INFO:tasks.workunit.client.0.smithi042.stdout: 10 137239 35.43 MB/sec execute 93 sec latency 56.389 ms 2024-04-25T00:35:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:43 smithi106 ceph-mon[32510]: pgmap v899: 97 pgs: 97 active+clean; 593 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-04-25T00:35:44.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:43 smithi042 ceph-mon[28130]: pgmap v899: 97 pgs: 97 active+clean; 593 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-04-25T00:35:44.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:43 smithi136 ceph-mon[32923]: pgmap v899: 97 pgs: 97 active+clean; 593 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 18 MiB/s wr, 1.01k op/s 2024-04-25T00:35:44.755 INFO:tasks.workunit.client.0.smithi042.stdout: 10 137730 35.21 MB/sec execute 94 sec latency 54.733 ms 2024-04-25T00:35:45.755 INFO:tasks.workunit.client.0.smithi042.stdout: 10 138450 35.24 MB/sec execute 95 sec latency 106.985 ms 2024-04-25T00:35:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:45 smithi106 ceph-mon[32510]: pgmap v900: 97 pgs: 97 active+clean; 591 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 808 op/s 2024-04-25T00:35:46.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:45 smithi042 ceph-mon[28130]: pgmap v900: 97 pgs: 97 active+clean; 591 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 808 op/s 2024-04-25T00:35:46.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:45 smithi136 ceph-mon[32923]: pgmap v900: 97 pgs: 97 active+clean; 591 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 15 MiB/s wr, 808 op/s 2024-04-25T00:35:46.755 INFO:tasks.workunit.client.0.smithi042.stdout: 10 139425 35.28 MB/sec execute 96 sec latency 95.775 ms 2024-04-25T00:35:47.756 INFO:tasks.workunit.client.0.smithi042.stdout: 10 140402 35.69 MB/sec execute 97 sec latency 30.592 ms 2024-04-25T00:35:48.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:47 smithi106 ceph-mon[32510]: pgmap v901: 97 pgs: 97 active+clean; 594 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 988 op/s 2024-04-25T00:35:48.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:47 smithi042 ceph-mon[28130]: pgmap v901: 97 pgs: 97 active+clean; 594 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 988 op/s 2024-04-25T00:35:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:47 smithi136 ceph-mon[32923]: pgmap v901: 97 pgs: 97 active+clean; 594 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 988 op/s 2024-04-25T00:35:48.755 INFO:tasks.workunit.client.0.smithi042.stdout: 10 140936 35.65 MB/sec execute 98 sec latency 75.994 ms 2024-04-25T00:35:49.755 INFO:tasks.workunit.client.0.smithi042.stdout: 10 141433 35.43 MB/sec execute 99 sec latency 67.267 ms 2024-04-25T00:35:50.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:49 smithi042 ceph-mon[28130]: pgmap v902: 97 pgs: 97 active+clean; 599 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-04-25T00:35:50.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:49 smithi136 ceph-mon[32923]: pgmap v902: 97 pgs: 97 active+clean; 599 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-04-25T00:35:50.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:49 smithi106 ceph-mon[32510]: pgmap v902: 97 pgs: 97 active+clean; 599 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.4 MiB/s wr, 681 op/s 2024-04-25T00:35:50.758 INFO:tasks.workunit.client.0.smithi042.stdout: 10 142229 35.47 MB/sec execute 100 sec latency 77.307 ms 2024-04-25T00:35:51.757 INFO:tasks.workunit.client.0.smithi042.stdout: 10 143182 35.59 MB/sec execute 101 sec latency 56.136 ms 2024-04-25T00:35:52.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:51 smithi042 ceph-mon[28130]: pgmap v903: 97 pgs: 97 active+clean; 607 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 633 op/s 2024-04-25T00:35:52.139 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:51 smithi106 ceph-mon[32510]: pgmap v903: 97 pgs: 97 active+clean; 607 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 633 op/s 2024-04-25T00:35:52.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:51 smithi136 ceph-mon[32923]: pgmap v903: 97 pgs: 97 active+clean; 607 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 633 op/s 2024-04-25T00:35:52.757 INFO:tasks.workunit.client.0.smithi042.stdout: 10 144024 35.84 MB/sec execute 102 sec latency 48.277 ms 2024-04-25T00:35:53.757 INFO:tasks.workunit.client.0.smithi042.stdout: 10 144561 35.79 MB/sec execute 103 sec latency 92.091 ms 2024-04-25T00:35:54.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:53 smithi042 ceph-mon[28130]: pgmap v904: 97 pgs: 97 active+clean; 619 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 659 op/s 2024-04-25T00:35:54.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:53 smithi136 ceph-mon[32923]: pgmap v904: 97 pgs: 97 active+clean; 619 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 659 op/s 2024-04-25T00:35:54.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:53 smithi106 ceph-mon[32510]: pgmap v904: 97 pgs: 97 active+clean; 619 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.8 MiB/s wr, 659 op/s 2024-04-25T00:35:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.402+0000 7fbe4e000640 1 -- 172.21.15.42:0/3925312683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe48108740 msgr2=0x7fbe4810ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.402+0000 7fbe4e000640 1 --2- 172.21.15.42:0/3925312683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe48108740 0x7fbe4810ab70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbe38009860 tx=0x7fbe3802a340 comp rx=0 tx=0).stop 2024-04-25T00:35:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.402+0000 7fbe4e000640 1 -- 172.21.15.42:0/3925312683 shutdown_connections 2024-04-25T00:35:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.402+0000 7fbe4e000640 1 --2- 172.21.15.42:0/3925312683 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe4810da60 0x7fbe4810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.402+0000 7fbe4e000640 1 --2- 172.21.15.42:0/3925312683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe4810b0b0 0x7fbe4810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.402+0000 7fbe4e000640 1 --2- 172.21.15.42:0/3925312683 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe48108740 0x7fbe4810ab70 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.404 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.402+0000 7fbe4e000640 1 -- 172.21.15.42:0/3925312683 >> 172.21.15.42:0/3925312683 conn(0x7fbe4806e7b0 msgr2=0x7fbe48070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.402+0000 7fbe4e000640 1 -- 172.21.15.42:0/3925312683 shutdown_connections 2024-04-25T00:35:54.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.403+0000 7fbe4e000640 1 -- 172.21.15.42:0/3925312683 wait complete. 2024-04-25T00:35:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.403+0000 7fbe4e000640 1 Processor -- start 2024-04-25T00:35:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.404+0000 7fbe4e000640 1 -- start start 2024-04-25T00:35:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.404+0000 7fbe4e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe48108740 0x7fbe481a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.404+0000 7fbe4e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe4810b0b0 0x7fbe481a6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.404+0000 7fbe4e000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe4810da60 0x7fbe481addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe4810da60 0x7fbe481addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe46a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe4810b0b0 0x7fbe481a6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe4810da60 0x7fbe481addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41020/0 (socket says 172.21.15.42:41020) 2024-04-25T00:35:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe47e00640 1 -- 172.21.15.42:0/3290039528 learned_addr learned my addr 172.21.15.42:0/3290039528 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe47400640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe48108740 0x7fbe481a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4811c280 con 0x7fbe48108740 2024-04-25T00:35:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4811c100 con 0x7fbe4810b0b0 2024-04-25T00:35:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4811c400 con 0x7fbe4810da60 2024-04-25T00:35:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe47400640 1 -- 172.21.15.42:0/3290039528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe4810da60 msgr2=0x7fbe481addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe47400640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe4810da60 0x7fbe481addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe47400640 1 -- 172.21.15.42:0/3290039528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe4810b0b0 msgr2=0x7fbe481a6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe47400640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe4810b0b0 0x7fbe481a6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe47400640 1 -- 172.21.15.42:0/3290039528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe481ae420 con 0x7fbe48108740 2024-04-25T00:35:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.405+0000 7fbe46a00640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe4810b0b0 0x7fbe481a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:35:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.406+0000 7fbe47400640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe48108740 0x7fbe481a6850 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbe38035040 tx=0x7fbe38033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.406+0000 7fbe3fe00640 1 -- 172.21.15.42:0/3290039528 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38037040 con 0x7fbe48108740 2024-04-25T00:35:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.406+0000 7fbe3fe00640 1 -- 172.21.15.42:0/3290039528 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe38004050 con 0x7fbe48108740 2024-04-25T00:35:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.406+0000 7fbe3fe00640 1 -- 172.21.15.42:0/3290039528 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe38003a10 con 0x7fbe48108740 2024-04-25T00:35:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.407+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe481ae6b0 con 0x7fbe48108740 2024-04-25T00:35:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.407+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe481aeb50 con 0x7fbe48108740 2024-04-25T00:35:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.410+0000 7fbe50dd9740 1 Processor -- start 2024-04-25T00:35:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe50dd9740 1 -- start start 2024-04-25T00:35:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.412 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0f5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0f5800 0x55d9ce0f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d9ce052ab0 con 0x55d9ce0f5800 2024-04-25T00:35:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d9cdffb640 con 0x55d9ce0f0690 2024-04-25T00:35:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d9ce04ea70 con 0x55d9ce0efca0 2024-04-25T00:35:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe3fe00640 1 -- 172.21.15.42:0/3290039528 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbe38042030 con 0x7fbe48108740 2024-04-25T00:35:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe46a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0f5800 0x55d9ce0f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f00c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41028/0 (socket says 172.21.15.42:41028) 2024-04-25T00:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe47e00640 1 -- 172.21.15.42:0/1910288086 learned_addr learned my addr 172.21.15.42:0/1910288086 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.411+0000 7fbe47400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.412+0000 7fbe47e00640 1 -- 172.21.15.42:0/1910288086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 msgr2=0x55d9ce0f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.412+0000 7fbe47e00640 1 --2- 172.21.15.42:0/1910288086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.412+0000 7fbe47e00640 1 -- 172.21.15.42:0/1910288086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0f5800 msgr2=0x55d9ce0f7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.412+0000 7fbe47e00640 1 --2- 172.21.15.42:0/1910288086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0f5800 0x55d9ce0f7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.412+0000 7fbe47e00640 1 -- 172.21.15.42:0/1910288086 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9cdd67210 con 0x55d9ce0efca0 2024-04-25T00:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.412+0000 7fbe47e00640 1 --2- 172.21.15.42:0/1910288086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f00c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbe40011570 tx=0x7fbe40011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f299a822c198df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.412+0000 7fbe33e00640 1 -- 172.21.15.42:0/1910288086 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40004b20 con 0x55d9ce0efca0 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.412+0000 7fbe33e00640 1 -- 172.21.15.42:0/1910288086 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe40004cc0 con 0x55d9ce0efca0 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.412+0000 7fbe33e00640 1 -- 172.21.15.42:0/1910288086 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40005520 con 0x55d9ce0efca0 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.413+0000 7fbe3fe00640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fbe2c0a2e70 0x7fbe2c0a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.413+0000 7fbe3fe00640 1 -- 172.21.15.42:0/3290039528 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fbe380ee4f0 con 0x7fbe48108740 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.413+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1910288086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 msgr2=0x55d9ce0f00c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.413+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1910288086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f00c0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fbe40011570 tx=0x7fbe40011a30 comp rx=0 tx=0).stop 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.413+0000 7fbe47400640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fbe2c0a2e70 0x7fbe2c0a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.413+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1910288086 shutdown_connections 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.413+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1910288086 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0f5800 0x55d9ce0f7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.413+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1910288086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.413+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1910288086 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f00c0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.413+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1910288086 >> 172.21.15.42:0/1910288086 conn(0x55d9ce0eb310 msgr2=0x55d9ce0eb710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.414+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1910288086 shutdown_connections 2024-04-25T00:35:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.414+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1910288086 wait complete. 2024-04-25T00:35:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.414+0000 7fbe50dd9740 1 -- start start 2024-04-25T00:35:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.414+0000 7fbe47400640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fbe2c0a2e70 0x7fbe2c0a5330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55d9ce0f0a90 tx=0x7fbe38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.414+0000 7fbe50dd9740 1 Processor -- start 2024-04-25T00:35:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0f5800 0x55d9ce1013e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d9ce0f87b0 con 0x55d9ce0efca0 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d9ce00e990 con 0x55d9ce0f0690 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d9ce0f8930 con 0x55d9ce0f5800 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe50dd9740 10 client.0 _ll_register_callbacks cb 0x55d9ce010950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe46a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe46a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45816/0 (socket says 172.21.15.42:45816) 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47104/0 (socket says 172.21.15.42:47104) 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.415+0000 7fbe46a00640 1 -- 172.21.15.42:0/2520569014 learned_addr learned my addr 172.21.15.42:0/2520569014 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.416+0000 7fbe46a00640 1 -- 172.21.15.42:0/2520569014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0f5800 msgr2=0x55d9ce1013e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.416+0000 7fbe46a00640 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0f5800 0x55d9ce1013e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.416+0000 7fbe46a00640 1 -- 172.21.15.42:0/2520569014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 msgr2=0x55d9ce0f9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.416+0000 7fbe46a00640 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.416+0000 7fbe46a00640 1 -- 172.21.15.42:0/2520569014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9cde33620 con 0x55d9ce0f0690 2024-04-25T00:35:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.416+0000 7fbe46a00640 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fa3a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009d70 tx=0x7fbe340039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90d40974de23faec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.416+0000 7fbe30c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:35:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.416+0000 7fbe30c00640 1 -- 172.21.15.42:0/2520569014 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34013930 con 0x55d9ce0f0690 2024-04-25T00:35:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.417+0000 7fbe50dd9740 1 -- 172.21.15.42:0/2520569014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe40004cc0 con 0x55d9ce0f0690 2024-04-25T00:35:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.417+0000 7fbe30c00640 1 -- 172.21.15.42:0/2520569014 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fbe34013ad0 con 0x55d9ce0f0690 2024-04-25T00:35:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.417+0000 7fbe30c00640 1 -- 172.21.15.42:0/2520569014 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbe34013d10 con 0x55d9ce0f0690 2024-04-25T00:35:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.417+0000 7fbe50dd9740 10 client.26537 fetch_fsmap learned FSMap version 45 2024-04-25T00:35:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.417+0000 7fbe50dd9740 1 -- 172.21.15.42:0/2520569014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d9ce104480 con 0x55d9ce0f0690 2024-04-25T00:35:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.417+0000 7fbe30c00640 1 -- 172.21.15.42:0/2520569014 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe40004cc0 con 0x55d9ce0f0690 2024-04-25T00:35:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.417+0000 7fbe30c00640 1 -- 172.21.15.42:0/2520569014 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fbe3401a070 con 0x55d9ce0f0690 2024-04-25T00:35:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.418+0000 7fbe50dd9740 10 client.26537 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:35:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.418+0000 7fbe50dd9740 10 client.26537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:35:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.418+0000 7fbe50dd9740 20 client.26537 populate_metadata read hostname 'smithi042' 2024-04-25T00:35:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.418+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d9ce105d00 0x55d9ce108180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.418+0000 7fbe50dd9740 4 client.26537 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:35:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.418+0000 7fbe50dd9740 1 -- 172.21.15.42:0/2520569014 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9ce05fff0 con 0x55d9ce105d00 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.418+0000 7fbe47e00640 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d9ce105d00 0x55d9ce108180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.418+0000 7fbe47e00640 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d9ce105d00 0x55d9ce108180 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e902bbe1c0f8192 server_cookie=464648304742bc8e in_seq=0 out_seq=0 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.419+0000 7fbe30c00640 10 client.26537 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.419+0000 7fbe30c00640 1 -- 172.21.15.42:0/2520569014 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9ce05fff0 con 0x55d9ce105d00 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.419+0000 7fbe30c00640 10 client.26537 handle_command_reply: tid=0 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.419+0000 7fbe50dd9740 1 client.26537 shutdown 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 1 -- 172.21.15.42:0/2520569014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 msgr2=0x55d9ce0fa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fa3a0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009d70 tx=0x7fbe340039c0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 1 -- 172.21.15.42:0/2520569014 shutdown_connections 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d9ce105d00 0x55d9ce108180 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0f5800 0x55d9ce1013e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fa3a0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/2520569014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 0x55d9ce0f9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 1 -- 172.21.15.42:0/2520569014 >> 172.21.15.42:0/2520569014 conn(0x55d9ce0eb310 msgr2=0x55d9ce0ea480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe30c00640 0 client.26537 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 1 -- 172.21.15.42:0/2520569014 shutdown_connections 2024-04-25T00:35:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 1 -- 172.21.15.42:0/2520569014 wait complete. 2024-04-25T00:35:54.425 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.420+0000 7fbe50dd9740 20 client.26537 trim_cache size 0 max 16384 2024-04-25T00:35:54.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.441+0000 7fbe50dd9740 1 Processor -- start 2024-04-25T00:35:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.441+0000 7fbe50dd9740 1 -- start start 2024-04-25T00:35:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.442+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 0x55d9ce0ee1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.442+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce1013a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.442+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0f5800 0x55d9ce0ee730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.442+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481ba1c0 con 0x55d9ce0efca0 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.442+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481b7a90 con 0x55d9ce0f0690 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.442+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481b58e0 con 0x55d9ce0f5800 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.442+0000 7fbe46a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce1013a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.442+0000 7fbe46a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce1013a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45828/0 (socket says 172.21.15.42:45828) 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.442+0000 7fbe46a00640 1 -- 172.21.15.42:0/1393824180 learned_addr learned my addr 172.21.15.42:0/1393824180 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.442+0000 7fbe47e00640 1 --2- 172.21.15.42:0/1393824180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 0x55d9ce0ee1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.443+0000 7fbe46a00640 1 -- 172.21.15.42:0/1393824180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0f5800 msgr2=0x55d9ce0ee730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.443+0000 7fbe46a00640 1 --2- 172.21.15.42:0/1393824180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0f5800 0x55d9ce0ee730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.443+0000 7fbe46a00640 1 -- 172.21.15.42:0/1393824180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 msgr2=0x55d9ce0ee1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.443+0000 7fbe46a00640 1 --2- 172.21.15.42:0/1393824180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 0x55d9ce0ee1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.443+0000 7fbe46a00640 1 -- 172.21.15.42:0/1393824180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9ce0eb140 con 0x55d9ce0f0690 2024-04-25T00:35:54.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.443+0000 7fbe46a00640 1 --2- 172.21.15.42:0/1393824180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce1013a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbe340042c0 tx=0x7fbe34009730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cd2bf106cb2748d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.443+0000 7fbe33400640 1 -- 172.21.15.42:0/1393824180 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34000e60 con 0x55d9ce0f0690 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.444+0000 7fbe33400640 1 -- 172.21.15.42:0/1393824180 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe3402b070 con 0x55d9ce0f0690 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.444+0000 7fbe33400640 1 -- 172.21.15.42:0/1393824180 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3401c6a0 con 0x55d9ce0f0690 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.445+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1393824180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 msgr2=0x55d9ce1013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.445+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1393824180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce1013a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbe340042c0 tx=0x7fbe34009730 comp rx=0 tx=0).stop 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.445+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1393824180 shutdown_connections 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.445+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1393824180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0f5800 0x55d9ce0ee730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.445+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1393824180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce1013a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.445+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1393824180 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0efca0 0x55d9ce0ee1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.445+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1393824180 >> 172.21.15.42:0/1393824180 conn(0x55d9ce0eb310 msgr2=0x55d9ce0f9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.445+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1393824180 shutdown_connections 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.445+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1393824180 wait complete. 2024-04-25T00:35:54.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.446+0000 7fbe50dd9740 1 -- start start 2024-04-25T00:35:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.446+0000 7fbe50dd9740 1 Processor -- start 2024-04-25T00:35:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.446+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 0x55d9ce0fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.446+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.447+0000 7fbe50dd9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0f5800 0x55d9ce17b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.447+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481b5f50 con 0x55d9ce0f5800 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.447+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481b73c0 con 0x55d9ce0f0690 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.447+0000 7fbe50dd9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe481b6620 con 0x55d9ce0efca0 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.447+0000 7fbe50dd9740 10 client.0 _ll_register_callbacks cb 0x7fbe34009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.447+0000 7fbe46a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.447+0000 7fbe46a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45832/0 (socket says 172.21.15.42:45832) 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.447+0000 7fbe46a00640 1 -- 172.21.15.42:0/1839368423 learned_addr learned my addr 172.21.15.42:0/1839368423 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.447+0000 7fbe47e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 0x55d9ce0fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.447+0000 7fbe47e00640 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0f5800 0x55d9ce17b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.448+0000 7fbe46a00640 1 -- 172.21.15.42:0/1839368423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 msgr2=0x55d9ce0fc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.448+0000 7fbe46a00640 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 0x55d9ce0fc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.448+0000 7fbe46a00640 1 -- 172.21.15.42:0/1839368423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0f5800 msgr2=0x55d9ce17b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.448+0000 7fbe46a00640 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0f5800 0x55d9ce17b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.448+0000 7fbe46a00640 1 -- 172.21.15.42:0/1839368423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9ce17b800 con 0x55d9ce0f0690 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.448+0000 7fbe46a00640 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fc7e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbe340118c0 tx=0x7fbe34011f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c837a48bd94c50af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.449+0000 7fbe30c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.449+0000 7fbe30c00640 1 -- 172.21.15.42:0/1839368423 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe34037070 con 0x55d9ce0f0690 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.449+0000 7fbe30c00640 1 -- 172.21.15.42:0/1839368423 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fbe34013940 con 0x55d9ce0f0690 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.449+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1839368423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe3402b070 con 0x55d9ce0f0690 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.449+0000 7fbe30c00640 1 -- 172.21.15.42:0/1839368423 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fbe34043070 con 0x55d9ce0f0690 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.449+0000 7fbe50dd9740 10 client.26543 fetch_fsmap learned FSMap version 45 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.449+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1839368423 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d9ce17cb10 con 0x55d9ce0f0690 2024-04-25T00:35:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.449+0000 7fbe30c00640 1 -- 172.21.15.42:0/1839368423 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe3402b070 con 0x55d9ce0f0690 2024-04-25T00:35:54.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.450+0000 7fbe30c00640 1 -- 172.21.15.42:0/1839368423 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fbe3401a050 con 0x55d9ce0f0690 2024-04-25T00:35:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.450+0000 7fbe50dd9740 10 client.26543 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:35:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.450+0000 7fbe50dd9740 10 client.26543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:35:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.450+0000 7fbe50dd9740 20 client.26543 populate_metadata read hostname 'smithi042' 2024-04-25T00:35:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.450+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d9ce17d4e0 0x55d9ce17f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:35:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.450+0000 7fbe50dd9740 4 client.26543 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:35:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.450+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1839368423 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9ce05a340 con 0x55d9ce17d4e0 2024-04-25T00:35:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.450+0000 7fbe47e00640 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d9ce17d4e0 0x55d9ce17f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:35:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.451+0000 7fbe47e00640 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d9ce17d4e0 0x55d9ce17f900 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c255831b6ed20ea1 server_cookie=5aec57e5b94d7e59 in_seq=0 out_seq=0 2024-04-25T00:35:54.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.451+0000 7fbe30c00640 10 client.26543 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:54.757 INFO:tasks.workunit.client.0.smithi042.stdout: 10 144913 35.56 MB/sec execute 104 sec latency 196.342 ms 2024-04-25T00:35:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.796+0000 7fbe30c00640 1 -- 172.21.15.42:0/1839368423 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9ce05a340 con 0x55d9ce17d4e0 2024-04-25T00:35:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.796+0000 7fbe30c00640 10 client.26543 handle_command_reply: tid=0 2024-04-25T00:35:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.797+0000 7fbe50dd9740 1 client.26543 shutdown 2024-04-25T00:35:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.798+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1839368423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 msgr2=0x55d9ce0fc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.798+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fc7e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fbe340118c0 tx=0x7fbe34011f00 comp rx=0 tx=0).stop 2024-04-25T00:35:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.798+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1839368423 shutdown_connections 2024-04-25T00:35:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.798+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d9ce17d4e0 0x55d9ce17f900 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.798+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d9ce0f5800 0x55d9ce17b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.798+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d9ce0f0690 0x55d9ce0fc7e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.798+0000 7fbe50dd9740 1 --2- 172.21.15.42:0/1839368423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d9ce0efca0 0x55d9ce0fc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.798+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1839368423 >> 172.21.15.42:0/1839368423 conn(0x55d9ce0eb310 msgr2=0x55d9ce0ee1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.799+0000 7fbe30c00640 0 client.26543 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:35:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.799+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1839368423 shutdown_connections 2024-04-25T00:35:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.799+0000 7fbe50dd9740 1 -- 172.21.15.42:0/1839368423 wait complete. 2024-04-25T00:35:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.799+0000 7fbe50dd9740 20 client.26543 trim_cache size 0 max 16384 2024-04-25T00:35:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:35:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:35:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:35:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:35:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.802+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fbe2c0a2e70 msgr2=0x7fbe2c0a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.802+0000 7fbe4e000640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fbe2c0a2e70 0x7fbe2c0a5330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55d9ce0f0a90 tx=0x7fbe38005f90 comp rx=0 tx=0).stop 2024-04-25T00:35:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.802+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe48108740 msgr2=0x7fbe481a6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:35:54.803 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.802+0000 7fbe4e000640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe48108740 0x7fbe481a6850 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbe38035040 tx=0x7fbe38033d10 comp rx=0 tx=0).stop 2024-04-25T00:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.804+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 shutdown_connections 2024-04-25T00:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.804+0000 7fbe4e000640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fbe2c0a2e70 0x7fbe2c0a5330 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.804+0000 7fbe4e000640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe4810da60 0x7fbe481addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.804+0000 7fbe4e000640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbe4810b0b0 0x7fbe481a6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.804+0000 7fbe4e000640 1 --2- 172.21.15.42:0/3290039528 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fbe48108740 0x7fbe481a6850 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.804+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 >> 172.21.15.42:0/3290039528 conn(0x7fbe4806e7b0 msgr2=0x7fbe4810d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:35:54.805 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.804+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 shutdown_connections 2024-04-25T00:35:54.806 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:35:54.804+0000 7fbe4e000640 1 -- 172.21.15.42:0/3290039528 wait complete. 2024-04-25T00:35:55.757 INFO:tasks.workunit.client.0.smithi042.stdout: 10 145308 35.40 MB/sec execute 105 sec latency 439.175 ms 2024-04-25T00:35:56.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:55 smithi042 ceph-mon[28130]: pgmap v905: 97 pgs: 97 active+clean; 619 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-04-25T00:35:56.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:55 smithi136 ceph-mon[32923]: pgmap v905: 97 pgs: 97 active+clean; 619 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-04-25T00:35:56.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:55 smithi106 ceph-mon[32510]: pgmap v905: 97 pgs: 97 active+clean; 619 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 3.9 MiB/s wr, 522 op/s 2024-04-25T00:35:56.757 INFO:tasks.workunit.client.0.smithi042.stdout: 10 146017 35.34 MB/sec execute 106 sec latency 62.599 ms 2024-04-25T00:35:57.758 INFO:tasks.workunit.client.0.smithi042.stdout: 10 146819 35.46 MB/sec execute 107 sec latency 68.735 ms 2024-04-25T00:35:58.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:57 smithi042 ceph-mon[28130]: pgmap v906: 97 pgs: 97 active+clean; 621 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 633 op/s 2024-04-25T00:35:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:57 smithi136 ceph-mon[32923]: pgmap v906: 97 pgs: 97 active+clean; 621 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 633 op/s 2024-04-25T00:35:58.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:57 smithi106 ceph-mon[32510]: pgmap v906: 97 pgs: 97 active+clean; 621 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.5 MiB/s wr, 633 op/s 2024-04-25T00:35:58.758 INFO:tasks.workunit.client.0.smithi042.stdout: 10 147529 35.56 MB/sec execute 108 sec latency 52.147 ms 2024-04-25T00:35:59.758 INFO:tasks.workunit.client.0.smithi042.stdout: 10 147772 35.37 MB/sec execute 109 sec latency 354.896 ms 2024-04-25T00:36:00.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:35:59 smithi042 ceph-mon[28130]: pgmap v907: 97 pgs: 97 active+clean; 624 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.0 MiB/s wr, 556 op/s 2024-04-25T00:36:00.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:35:59 smithi136 ceph-mon[32923]: pgmap v907: 97 pgs: 97 active+clean; 624 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.0 MiB/s wr, 556 op/s 2024-04-25T00:36:00.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:35:59 smithi106 ceph-mon[32510]: pgmap v907: 97 pgs: 97 active+clean; 624 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.0 MiB/s wr, 556 op/s 2024-04-25T00:36:00.758 INFO:tasks.workunit.client.0.smithi042.stdout: 10 148003 35.14 MB/sec execute 110 sec latency 464.861 ms 2024-04-25T00:36:01.758 INFO:tasks.workunit.client.0.smithi042.stdout: 10 148586 35.10 MB/sec execute 111 sec latency 125.037 ms 2024-04-25T00:36:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:01 smithi042 ceph-mon[28130]: pgmap v908: 97 pgs: 97 active+clean; 635 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 586 op/s 2024-04-25T00:36:02.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:01 smithi136 ceph-mon[32923]: pgmap v908: 97 pgs: 97 active+clean; 635 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 586 op/s 2024-04-25T00:36:02.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:01 smithi106 ceph-mon[32510]: pgmap v908: 97 pgs: 97 active+clean; 635 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 5.0 MiB/s wr, 586 op/s 2024-04-25T00:36:02.758 INFO:tasks.workunit.client.0.smithi042.stdout: 10 149326 35.09 MB/sec execute 112 sec latency 65.871 ms 2024-04-25T00:36:03.758 INFO:tasks.workunit.client.0.smithi042.stdout: 10 150063 35.15 MB/sec execute 113 sec latency 43.489 ms 2024-04-25T00:36:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:03 smithi042 ceph-mon[28130]: pgmap v909: 97 pgs: 97 active+clean; 632 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.6 MiB/s wr, 678 op/s 2024-04-25T00:36:04.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:03 smithi136 ceph-mon[32923]: pgmap v909: 97 pgs: 97 active+clean; 632 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.6 MiB/s wr, 678 op/s 2024-04-25T00:36:04.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:03 smithi106 ceph-mon[32510]: pgmap v909: 97 pgs: 97 active+clean; 632 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 8.6 MiB/s wr, 678 op/s 2024-04-25T00:36:04.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 150841 35.24 MB/sec execute 114 sec latency 64.251 ms 2024-04-25T00:36:05.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 150877 34.96 MB/sec execute 115 sec latency 1008.285 ms 2024-04-25T00:36:06.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:05 smithi042 ceph-mon[28130]: pgmap v910: 97 pgs: 97 active+clean; 632 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.5 MiB/s wr, 547 op/s 2024-04-25T00:36:06.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:05 smithi136 ceph-mon[32923]: pgmap v910: 97 pgs: 97 active+clean; 632 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.5 MiB/s wr, 547 op/s 2024-04-25T00:36:06.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:05 smithi106 ceph-mon[32510]: pgmap v910: 97 pgs: 97 active+clean; 632 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 7.5 MiB/s wr, 547 op/s 2024-04-25T00:36:06.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 151126 34.77 MB/sec execute 116 sec latency 1026.679 ms 2024-04-25T00:36:07.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 151541 34.64 MB/sec execute 117 sec latency 152.850 ms 2024-04-25T00:36:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.000+0000 7fb89dc00640 1 -- 172.21.15.42:0/547656136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8980741a0 msgr2=0x7fb898074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.000+0000 7fb89dc00640 1 --2- 172.21.15.42:0/547656136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8980741a0 0x7fb898074620 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb884009930 tx=0x7fb88402a440 comp rx=0 tx=0).stop 2024-04-25T00:36:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.000+0000 7fb89dc00640 1 -- 172.21.15.42:0/547656136 shutdown_connections 2024-04-25T00:36:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.000+0000 7fb89dc00640 1 --2- 172.21.15.42:0/547656136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb89810dd70 0x7fb898110160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.002 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.000+0000 7fb89dc00640 1 --2- 172.21.15.42:0/547656136 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8980741a0 0x7fb898074620 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.000+0000 7fb89dc00640 1 --2- 172.21.15.42:0/547656136 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb898075780 0x7fb898073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.000+0000 7fb89dc00640 1 -- 172.21.15.42:0/547656136 >> 172.21.15.42:0/547656136 conn(0x7fb8980fcd30 msgr2=0x7fb8980ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.000+0000 7fb89dc00640 1 -- 172.21.15.42:0/547656136 shutdown_connections 2024-04-25T00:36:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.001+0000 7fb89dc00640 1 -- 172.21.15.42:0/547656136 wait complete. 2024-04-25T00:36:08.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.001+0000 7fb89dc00640 1 Processor -- start 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.002+0000 7fb89dc00640 1 -- start start 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.002+0000 7fb89dc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8980741a0 0x7fb89810a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.002+0000 7fb89dc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb898075780 0x7fb89810a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.002+0000 7fb89dc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb89810dd70 0x7fb8981ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.002+0000 7fb896000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb898075780 0x7fb89810a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.002+0000 7fb896000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb898075780 0x7fb89810a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:43180/0 (socket says 172.21.15.42:43180) 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.002+0000 7fb896a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8980741a0 0x7fb89810a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.002+0000 7fb896a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8980741a0 0x7fb89810a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45684/0 (socket says 172.21.15.42:45684) 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb896a00640 1 -- 172.21.15.42:0/3303503550 learned_addr learned my addr 172.21.15.42:0/3303503550 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb898113a60 con 0x7fb89810dd70 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981138e0 con 0x7fb8980741a0 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb898113be0 con 0x7fb898075780 2024-04-25T00:36:08.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb897400640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb89810dd70 0x7fb8981ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb896000640 1 -- 172.21.15.42:0/3303503550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8980741a0 msgr2=0x7fb89810a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb896000640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8980741a0 0x7fb89810a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb896000640 1 -- 172.21.15.42:0/3303503550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb89810dd70 msgr2=0x7fb8981ae170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb896000640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb89810dd70 0x7fb8981ae170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb896000640 1 -- 172.21.15.42:0/3303503550 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89810daa0 con 0x7fb898075780 2024-04-25T00:36:08.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.003+0000 7fb896000640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb898075780 0x7fb89810a920 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb884009850 tx=0x7fb884032bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.004+0000 7fb88f400640 1 -- 172.21.15.42:0/3303503550 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88403f030 con 0x7fb898075780 2024-04-25T00:36:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.004+0000 7fb88f400640 1 -- 172.21.15.42:0/3303503550 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb884003ad0 con 0x7fb898075780 2024-04-25T00:36:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.004+0000 7fb88f400640 1 -- 172.21.15.42:0/3303503550 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884035070 con 0x7fb898075780 2024-04-25T00:36:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.004+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8981ae6b0 con 0x7fb898075780 2024-04-25T00:36:08.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.004+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8981aeb00 con 0x7fb898075780 2024-04-25T00:36:08.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.008+0000 7fb88f400640 1 -- 172.21.15.42:0/3303503550 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb884042030 con 0x7fb898075780 2024-04-25T00:36:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.010+0000 7fb88f400640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb87c0a2d00 0x7fb87c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.010+0000 7fb88f400640 1 -- 172.21.15.42:0/3303503550 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fb8840ef2d0 con 0x7fb898075780 2024-04-25T00:36:08.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.010+0000 7fb8a0a11740 1 Processor -- start 2024-04-25T00:36:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.010+0000 7fb8a0a11740 1 -- start start 2024-04-25T00:36:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.011+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.011+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.012 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.011+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c7800 0x55bb7b0c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.011+0000 7fb8a0a11740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bb7b024ab0 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.011+0000 7fb8a0a11740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bb7afcd640 con 0x55bb7b0c2690 2024-04-25T00:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.011+0000 7fb8a0a11740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bb7b020a70 con 0x55bb7b0c7800 2024-04-25T00:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.011+0000 7fb896a00640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb87c0a2d00 0x7fb87c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.011+0000 7fb896000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.011+0000 7fb896000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45698/0 (socket says 172.21.15.42:45698) 2024-04-25T00:36:08.013 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.011+0000 7fb896000640 1 -- 172.21.15.42:0/1347049025 learned_addr learned my addr 172.21.15.42:0/1347049025 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.012+0000 7fb896a00640 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.012+0000 7fb896000640 1 -- 172.21.15.42:0/1347049025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c7800 msgr2=0x55bb7b0c9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.012+0000 7fb897400640 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c7800 0x55bb7b0c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.014 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.012+0000 7fb896000640 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c7800 0x55bb7b0c9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.012+0000 7fb896000640 1 -- 172.21.15.42:0/1347049025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 msgr2=0x55bb7b0c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.012+0000 7fb896000640 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.012+0000 7fb896000640 1 -- 172.21.15.42:0/1347049025 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7ad39210 con 0x55bb7b0c2690 2024-04-25T00:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.013+0000 7fb896a00640 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.013+0000 7fb897400640 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c7800 0x55bb7b0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.013+0000 7fb896a00640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb87c0a2d00 0x7fb87c0a51c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb88800ae50 tx=0x7fb888013830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.013+0000 7fb896000640 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0c7190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb88402aa30 tx=0x7fb8840bb510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=775592224885a820 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.013+0000 7fb883e00640 1 -- 172.21.15.42:0/1347049025 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8840d4070 con 0x55bb7b0c2690 2024-04-25T00:36:08.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.013+0000 7fb883e00640 1 -- 172.21.15.42:0/1347049025 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8840c4770 con 0x55bb7b0c2690 2024-04-25T00:36:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.014+0000 7fb883e00640 1 -- 172.21.15.42:0/1347049025 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884049040 con 0x55bb7b0c2690 2024-04-25T00:36:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.014+0000 7fb8a0a11740 1 -- 172.21.15.42:0/1347049025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 msgr2=0x55bb7b0c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.014+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0c7190 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb88402aa30 tx=0x7fb8840bb510 comp rx=0 tx=0).stop 2024-04-25T00:36:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.014+0000 7fb8a0a11740 1 -- 172.21.15.42:0/1347049025 shutdown_connections 2024-04-25T00:36:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.014+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c7800 0x55bb7b0c9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.014+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0c7190 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.014+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/1347049025 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.014+0000 7fb8a0a11740 1 -- 172.21.15.42:0/1347049025 >> 172.21.15.42:0/1347049025 conn(0x55bb7b0bd310 msgr2=0x55bb7b0bd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.014+0000 7fb8a0a11740 1 -- 172.21.15.42:0/1347049025 shutdown_connections 2024-04-25T00:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.014+0000 7fb8a0a11740 1 -- 172.21.15.42:0/1347049025 wait complete. 2024-04-25T00:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.015+0000 7fb8a0a11740 1 -- start start 2024-04-25T00:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.015+0000 7fb8a0a11740 1 Processor -- start 2024-04-25T00:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.016+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.016+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.016+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c7800 0x55bb7b0d3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.016+0000 7fb8a0a11740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55bb7b0ca570 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.016+0000 7fb8a0a11740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55bb7afe0990 con 0x55bb7b0c2690 2024-04-25T00:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.016+0000 7fb8a0a11740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bb7b0ca6f0 con 0x55bb7b0c7800 2024-04-25T00:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.016+0000 7fb8a0a11740 10 client.0 _ll_register_callbacks cb 0x55bb7afe2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.016+0000 7fb896a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.016+0000 7fb896a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37682/0 (socket says 172.21.15.42:37682) 2024-04-25T00:36:08.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.016+0000 7fb896a00640 1 -- 172.21.15.42:0/2702378647 learned_addr learned my addr 172.21.15.42:0/2702378647 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.017+0000 7fb896a00640 1 -- 172.21.15.42:0/2702378647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c7800 msgr2=0x55bb7b0d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.017+0000 7fb896000640 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.017+0000 7fb896a00640 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c7800 0x55bb7b0d3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.017+0000 7fb896a00640 1 -- 172.21.15.42:0/2702378647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 msgr2=0x55bb7b0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.017+0000 7fb896a00640 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.017+0000 7fb896a00640 1 -- 172.21.15.42:0/2702378647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7ae05620 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.017+0000 7fb896a00640 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0cbbf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb88800ec40 tx=0x7fb888004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f0e1b13922e34c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.017+0000 7fb880c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.018+0000 7fb880c00640 1 -- 172.21.15.42:0/2702378647 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb88801e640 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.019 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.018+0000 7fb880c00640 1 -- 172.21.15.42:0/2702378647 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fb88801e7e0 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.018+0000 7fb8a0a11740 1 -- 172.21.15.42:0/2702378647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8840c4770 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.018+0000 7fb880c00640 1 -- 172.21.15.42:0/2702378647 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb88801e9e0 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.018+0000 7fb8a0a11740 10 client.16353 fetch_fsmap learned FSMap version 45 2024-04-25T00:36:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.018+0000 7fb8a0a11740 1 -- 172.21.15.42:0/2702378647 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bb7b0d6130 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.020 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.019+0000 7fb880c00640 1 -- 172.21.15.42:0/2702378647 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb8840c4770 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.020+0000 7fb880c00640 1 -- 172.21.15.42:0/2702378647 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fb88801c040 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.020+0000 7fb8a0a11740 10 client.16353 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:36:08.021 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.020+0000 7fb8a0a11740 10 client.16353 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:36:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.020+0000 7fb8a0a11740 20 client.16353 populate_metadata read hostname 'smithi042' 2024-04-25T00:36:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.020+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7b0d7870 0x55bb7b0d9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.020+0000 7fb8a0a11740 4 client.16353 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:36:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.020+0000 7fb8a0a11740 1 -- 172.21.15.42:0/2702378647 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb7b031ff0 con 0x55bb7b0d7870 2024-04-25T00:36:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.020+0000 7fb896000640 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7b0d7870 0x55bb7b0d9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.021+0000 7fb896000640 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7b0d7870 0x55bb7b0d9cc0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18582874a62feb72 server_cookie=f8f21e3c5b79847b in_seq=0 out_seq=0 2024-04-25T00:36:08.022 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.021+0000 7fb880c00640 10 client.16353 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.021+0000 7fb880c00640 1 -- 172.21.15.42:0/2702378647 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bb7b031ff0 con 0x55bb7b0d7870 2024-04-25T00:36:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.021+0000 7fb880c00640 10 client.16353 handle_command_reply: tid=0 2024-04-25T00:36:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb8a0a11740 1 client.16353 shutdown 2024-04-25T00:36:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb8a0a11740 1 -- 172.21.15.42:0/2702378647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 msgr2=0x55bb7b0cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.023 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0cbbf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb88800ec40 tx=0x7fb888004730 comp rx=0 tx=0).stop 2024-04-25T00:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb8a0a11740 1 -- 172.21.15.42:0/2702378647 shutdown_connections 2024-04-25T00:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7b0d7870 0x55bb7b0d9cc0 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c7800 0x55bb7b0d3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/2702378647 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0cbbf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb8a0a11740 1 -- 172.21.15.42:0/2702378647 >> 172.21.15.42:0/2702378647 conn(0x55bb7b0bd310 msgr2=0x55bb7b0bc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb880c00640 0 client.16353 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:08.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.022+0000 7fb8a0a11740 1 -- 172.21.15.42:0/2702378647 shutdown_connections 2024-04-25T00:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.023+0000 7fb8a0a11740 1 -- 172.21.15.42:0/2702378647 wait complete. 2024-04-25T00:36:08.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.023+0000 7fb8a0a11740 20 client.16353 trim_cache size 0 max 16384 2024-04-25T00:36:08.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.051+0000 7fb8a0a11740 1 Processor -- start 2024-04-25T00:36:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.051+0000 7fb8a0a11740 1 -- start start 2024-04-25T00:36:08.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.052+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.052+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0d3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.052+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c7800 0x55bb7b0cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.052+0000 7fb8a0a11740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981ba140 con 0x55bb7b0c7800 2024-04-25T00:36:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.052+0000 7fb8a0a11740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b7a10 con 0x55bb7b0c2690 2024-04-25T00:36:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.052+0000 7fb8a0a11740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b5860 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.052+0000 7fb896000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.052+0000 7fb896000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0d3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:45714/0 (socket says 172.21.15.42:45714) 2024-04-25T00:36:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.052+0000 7fb896000640 1 -- 172.21.15.42:0/1987137766 learned_addr learned my addr 172.21.15.42:0/1987137766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.053+0000 7fb897400640 1 --2- 172.21.15.42:0/1987137766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c7800 0x55bb7b0cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.053+0000 7fb896a00640 1 --2- 172.21.15.42:0/1987137766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.053+0000 7fb897400640 1 -- 172.21.15.42:0/1987137766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c1ca0 msgr2=0x55bb7b0cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.053+0000 7fb897400640 1 --2- 172.21.15.42:0/1987137766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.053+0000 7fb897400640 1 -- 172.21.15.42:0/1987137766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 msgr2=0x55bb7b0d3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.053+0000 7fb897400640 1 --2- 172.21.15.42:0/1987137766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0d3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.053+0000 7fb897400640 1 -- 172.21.15.42:0/1987137766 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb7b0ca850 con 0x55bb7b0c7800 2024-04-25T00:36:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.054+0000 7fb897400640 1 --2- 172.21.15.42:0/1987137766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c7800 0x55bb7b0cbd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55bb7b0be1e0 tx=0x7fb8900129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a564b8fc5d927b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.054+0000 7fb883400640 1 -- 172.21.15.42:0/1987137766 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb890021070 con 0x55bb7b0c7800 2024-04-25T00:36:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.054+0000 7fb883400640 1 -- 172.21.15.42:0/1987137766 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb890010b70 con 0x55bb7b0c7800 2024-04-25T00:36:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.054+0000 7fb883400640 1 -- 172.21.15.42:0/1987137766 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89001cda0 con 0x55bb7b0c7800 2024-04-25T00:36:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.056+0000 7fb8a0a11740 1 -- 172.21.15.42:0/1987137766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c7800 msgr2=0x55bb7b0cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.056+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/1987137766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c7800 0x55bb7b0cbd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x55bb7b0be1e0 tx=0x7fb8900129e0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.056+0000 7fb8a0a11740 1 -- 172.21.15.42:0/1987137766 shutdown_connections 2024-04-25T00:36:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.056+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/1987137766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c7800 0x55bb7b0cbd40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.056+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/1987137766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c2690 0x55bb7b0d3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.056+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/1987137766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.056+0000 7fb8a0a11740 1 -- 172.21.15.42:0/1987137766 >> 172.21.15.42:0/1987137766 conn(0x55bb7b0bd310 msgr2=0x55bb7b0bd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.057+0000 7fb8a0a11740 1 -- 172.21.15.42:0/1987137766 shutdown_connections 2024-04-25T00:36:08.058 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.057+0000 7fb8a0a11740 1 -- 172.21.15.42:0/1987137766 wait complete. 2024-04-25T00:36:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.057+0000 7fb8a0a11740 1 -- start start 2024-04-25T00:36:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.058+0000 7fb8a0a11740 1 Processor -- start 2024-04-25T00:36:08.059 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.058+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0d0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c2690 0x55bb7b0d0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb8a0a11740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c7800 0x55bb7b0d1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb896a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0d0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb896a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0d0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37700/0 (socket says 172.21.15.42:37700) 2024-04-25T00:36:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb896a00640 1 -- 172.21.15.42:0/887206884 learned_addr learned my addr 172.21.15.42:0/887206884 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb8a0a11740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b5ed0 con 0x55bb7b0c1ca0 2024-04-25T00:36:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb8a0a11740 1 -- 172.21.15.42:0/887206884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b7340 con 0x55bb7b0c7800 2024-04-25T00:36:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb8a0a11740 1 -- 172.21.15.42:0/887206884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8981b65a0 con 0x55bb7b0c2690 2024-04-25T00:36:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb8a0a11740 10 client.0 _ll_register_callbacks cb 0x7fb88800e910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb897400640 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c7800 0x55bb7b0d1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.059+0000 7fb896000640 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c2690 0x55bb7b0d0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.060+0000 7fb896000640 1 -- 172.21.15.42:0/887206884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c7800 msgr2=0x55bb7b0d1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.060+0000 7fb896000640 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c7800 0x55bb7b0d1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.060+0000 7fb896000640 1 -- 172.21.15.42:0/887206884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 msgr2=0x55bb7b0d0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.060+0000 7fb896000640 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0d0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.060+0000 7fb896000640 1 -- 172.21.15.42:0/887206884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb7b0d20d0 con 0x55bb7b0c2690 2024-04-25T00:36:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.060+0000 7fb896a00640 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.061+0000 7fb896000640 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c2690 0x55bb7b0d0ad0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb8840b8010 tx=0x7fb8840c0030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d4cd56b296cc7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.061+0000 7fb880c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:36:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.061+0000 7fb880c00640 1 -- 172.21.15.42:0/887206884 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884049040 con 0x55bb7b0c2690 2024-04-25T00:36:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.061+0000 7fb880c00640 1 -- 172.21.15.42:0/887206884 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fb88403b460 con 0x55bb7b0c2690 2024-04-25T00:36:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.061+0000 7fb897400640 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c7800 0x55bb7b0d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.061+0000 7fb8a0a11740 1 -- 172.21.15.42:0/887206884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb890010b70 con 0x55bb7b0c2690 2024-04-25T00:36:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.062+0000 7fb880c00640 1 -- 172.21.15.42:0/887206884 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fb8840d4040 con 0x55bb7b0c2690 2024-04-25T00:36:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.062+0000 7fb8a0a11740 10 client.26542 fetch_fsmap learned FSMap version 45 2024-04-25T00:36:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.062+0000 7fb8a0a11740 1 -- 172.21.15.42:0/887206884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55bb7b0d33e0 con 0x55bb7b0c2690 2024-04-25T00:36:08.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.063+0000 7fb880c00640 1 -- 172.21.15.42:0/887206884 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb890010b70 con 0x55bb7b0c2690 2024-04-25T00:36:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.063+0000 7fb880c00640 1 -- 172.21.15.42:0/887206884 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fb88404c060 con 0x55bb7b0c2690 2024-04-25T00:36:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.065+0000 7fb8a0a11740 10 client.26542 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:36:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.065+0000 7fb8a0a11740 10 client.26542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:36:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.065+0000 7fb8a0a11740 20 client.26542 populate_metadata read hostname 'smithi042' 2024-04-25T00:36:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.065+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7b13e180 0x55bb7b140570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.065+0000 7fb8a0a11740 4 client.26542 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:36:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.065+0000 7fb8a0a11740 1 -- 172.21.15.42:0/887206884 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb7b02c340 con 0x55bb7b13e180 2024-04-25T00:36:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.066+0000 7fb896a00640 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7b13e180 0x55bb7b140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.067+0000 7fb896a00640 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7b13e180 0x55bb7b140570 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44b78c45c9d7ab6 server_cookie=ed8cef73c948947 in_seq=0 out_seq=0 2024-04-25T00:36:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.067+0000 7fb880c00640 10 client.26542 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:08.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:07 smithi136 ceph-mon[32923]: pgmap v911: 97 pgs: 97 active+clean; 682 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 753 op/s 2024-04-25T00:36:08.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:07 smithi106 ceph-mon[32510]: pgmap v911: 97 pgs: 97 active+clean; 682 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 753 op/s 2024-04-25T00:36:08.347 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.344+0000 7fb880c00640 1 -- 172.21.15.42:0/887206884 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb7b02c340 con 0x55bb7b13e180 2024-04-25T00:36:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.344+0000 7fb880c00640 10 client.26542 handle_command_reply: tid=0 2024-04-25T00:36:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.344+0000 7fb8a0a11740 1 client.26542 shutdown 2024-04-25T00:36:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.345+0000 7fb8a0a11740 1 -- 172.21.15.42:0/887206884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c2690 msgr2=0x55bb7b0d0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.345+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c2690 0x55bb7b0d0ad0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb8840b8010 tx=0x7fb8840c0030 comp rx=0 tx=0).stop 2024-04-25T00:36:08.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.345+0000 7fb8a0a11740 1 -- 172.21.15.42:0/887206884 shutdown_connections 2024-04-25T00:36:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.345+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55bb7b13e180 0x55bb7b140570 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.345+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55bb7b0c7800 0x55bb7b0d1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.345+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bb7b0c2690 0x55bb7b0d0ad0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.345+0000 7fb8a0a11740 1 --2- 172.21.15.42:0/887206884 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55bb7b0c1ca0 0x55bb7b0d0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.345+0000 7fb8a0a11740 1 -- 172.21.15.42:0/887206884 >> 172.21.15.42:0/887206884 conn(0x55bb7b0bd310 msgr2=0x55bb7b0cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.345+0000 7fb880c00640 0 client.26542 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.346+0000 7fb8a0a11740 1 -- 172.21.15.42:0/887206884 shutdown_connections 2024-04-25T00:36:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.346+0000 7fb8a0a11740 1 -- 172.21.15.42:0/887206884 wait complete. 2024-04-25T00:36:08.350 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.346+0000 7fb8a0a11740 20 client.26542 trim_cache size 0 max 16384 2024-04-25T00:36:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:36:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:36:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:36:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:36:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.349+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb87c0a2d00 msgr2=0x7fb87c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.349+0000 7fb89dc00640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb87c0a2d00 0x7fb87c0a51c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb88800ae50 tx=0x7fb888013830 comp rx=0 tx=0).stop 2024-04-25T00:36:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.349+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb898075780 msgr2=0x7fb89810a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.349+0000 7fb89dc00640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb898075780 0x7fb89810a920 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb884009850 tx=0x7fb884032bf0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.349+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 shutdown_connections 2024-04-25T00:36:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.349+0000 7fb89dc00640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fb87c0a2d00 0x7fb87c0a51c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.349+0000 7fb89dc00640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fb89810dd70 0x7fb8981ae170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.349+0000 7fb89dc00640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb898075780 0x7fb89810a920 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.349+0000 7fb89dc00640 1 --2- 172.21.15.42:0/3303503550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8980741a0 0x7fb89810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.349+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 >> 172.21.15.42:0/3303503550 conn(0x7fb8980fcd30 msgr2=0x7fb898101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.350+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 shutdown_connections 2024-04-25T00:36:08.355 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:08.350+0000 7fb89dc00640 1 -- 172.21.15.42:0/3303503550 wait complete. 2024-04-25T00:36:08.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:07 smithi042 ceph-mon[28130]: pgmap v911: 97 pgs: 97 active+clean; 682 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 14 MiB/s wr, 753 op/s 2024-04-25T00:36:08.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 152123 34.64 MB/sec execute 118 sec latency 129.014 ms 2024-04-25T00:36:09.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 152891 34.63 MB/sec execute 119 sec latency 51.037 ms 2024-04-25T00:36:10.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:09 smithi136 ceph-mon[32923]: pgmap v912: 97 pgs: 97 active+clean; 702 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 17 MiB/s wr, 777 op/s 2024-04-25T00:36:10.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:09 smithi106 ceph-mon[32510]: pgmap v912: 97 pgs: 97 active+clean; 702 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 17 MiB/s wr, 777 op/s 2024-04-25T00:36:10.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:09 smithi042 ceph-mon[28130]: pgmap v912: 97 pgs: 97 active+clean; 702 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 17 MiB/s wr, 777 op/s 2024-04-25T00:36:10.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 153678 34.76 MB/sec execute 120 sec latency 42.118 ms 2024-04-25T00:36:11.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 154457 34.79 MB/sec execute 121 sec latency 49.365 ms 2024-04-25T00:36:12.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:11 smithi136 ceph-mon[32923]: pgmap v913: 97 pgs: 97 active+clean; 678 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 765 op/s 2024-04-25T00:36:12.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:11 smithi106 ceph-mon[32510]: pgmap v913: 97 pgs: 97 active+clean; 678 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 765 op/s 2024-04-25T00:36:12.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:11 smithi042 ceph-mon[28130]: pgmap v913: 97 pgs: 97 active+clean; 678 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 18 MiB/s wr, 765 op/s 2024-04-25T00:36:12.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 155134 34.84 MB/sec execute 122 sec latency 52.428 ms 2024-04-25T00:36:13.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 155700 34.82 MB/sec execute 123 sec latency 135.364 ms 2024-04-25T00:36:14.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:13 smithi136 ceph-mon[32923]: pgmap v914: 97 pgs: 97 active+clean; 611 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 17 MiB/s wr, 956 op/s 2024-04-25T00:36:14.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:13 smithi106 ceph-mon[32510]: pgmap v914: 97 pgs: 97 active+clean; 611 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 17 MiB/s wr, 956 op/s 2024-04-25T00:36:14.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:13 smithi042 ceph-mon[28130]: pgmap v914: 97 pgs: 97 active+clean; 611 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 17 MiB/s wr, 956 op/s 2024-04-25T00:36:14.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 156471 34.74 MB/sec execute 124 sec latency 56.385 ms 2024-04-25T00:36:15.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 157311 34.97 MB/sec execute 125 sec latency 38.110 ms 2024-04-25T00:36:16.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:15 smithi136 ceph-mon[32923]: pgmap v915: 97 pgs: 97 active+clean; 611 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 758 op/s 2024-04-25T00:36:16.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:15 smithi106 ceph-mon[32510]: pgmap v915: 97 pgs: 97 active+clean; 611 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 758 op/s 2024-04-25T00:36:16.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:15 smithi042 ceph-mon[28130]: pgmap v915: 97 pgs: 97 active+clean; 611 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 13 MiB/s wr, 758 op/s 2024-04-25T00:36:16.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 158070 34.97 MB/sec execute 126 sec latency 46.990 ms 2024-04-25T00:36:17.759 INFO:tasks.workunit.client.0.smithi042.stdout: 10 158700 34.98 MB/sec execute 127 sec latency 80.141 ms 2024-04-25T00:36:18.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:17 smithi136 ceph-mon[32923]: pgmap v916: 97 pgs: 97 active+clean; 625 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 928 op/s 2024-04-25T00:36:18.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:17 smithi106 ceph-mon[32510]: pgmap v916: 97 pgs: 97 active+clean; 625 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 928 op/s 2024-04-25T00:36:18.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:17 smithi042 ceph-mon[28130]: pgmap v916: 97 pgs: 97 active+clean; 625 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 14 MiB/s wr, 928 op/s 2024-04-25T00:36:18.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 159323 34.98 MB/sec execute 128 sec latency 60.904 ms 2024-04-25T00:36:19.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 160130 34.95 MB/sec execute 129 sec latency 53.536 ms 2024-04-25T00:36:20.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:19 smithi136 ceph-mon[32923]: pgmap v917: 97 pgs: 97 active+clean; 630 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.5 MiB/s wr, 772 op/s 2024-04-25T00:36:20.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:19 smithi106 ceph-mon[32510]: pgmap v917: 97 pgs: 97 active+clean; 630 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.5 MiB/s wr, 772 op/s 2024-04-25T00:36:20.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:19 smithi042 ceph-mon[28130]: pgmap v917: 97 pgs: 97 active+clean; 630 MiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 8.5 MiB/s wr, 772 op/s 2024-04-25T00:36:20.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 160759 35.00 MB/sec execute 130 sec latency 268.335 ms 2024-04-25T00:36:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.526+0000 7fc608200640 1 -- 172.21.15.42:0/185164902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6001009f0 msgr2=0x7fc600100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.526+0000 7fc608200640 1 --2- 172.21.15.42:0/185164902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6001009f0 0x7fc600100e50 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec009900 tx=0x7fc5ec02a480 comp rx=0 tx=0).stop 2024-04-25T00:36:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.526+0000 7fc608200640 1 -- 172.21.15.42:0/185164902 shutdown_connections 2024-04-25T00:36:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.526+0000 7fc608200640 1 --2- 172.21.15.42:0/185164902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc600101390 0x7fc600101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.526+0000 7fc608200640 1 --2- 172.21.15.42:0/185164902 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6001009f0 0x7fc600100e50 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.526+0000 7fc608200640 1 --2- 172.21.15.42:0/185164902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc600100090 0x7fc6001004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.526+0000 7fc608200640 1 -- 172.21.15.42:0/185164902 >> 172.21.15.42:0/185164902 conn(0x7fc6000fbc10 msgr2=0x7fc6000fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.526+0000 7fc608200640 1 -- 172.21.15.42:0/185164902 shutdown_connections 2024-04-25T00:36:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.526+0000 7fc608200640 1 -- 172.21.15.42:0/185164902 wait complete. 2024-04-25T00:36:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.528+0000 7fc608200640 1 Processor -- start 2024-04-25T00:36:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.528+0000 7fc608200640 1 -- start start 2024-04-25T00:36:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.528+0000 7fc608200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc600100090 0x7fc60019e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.528+0000 7fc608200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6001009f0 0x7fc60019e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.528+0000 7fc608200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc600101390 0x7fc6001a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.528+0000 7fc608200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc60010f650 con 0x7fc6001009f0 2024-04-25T00:36:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.528+0000 7fc608200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc60010f4d0 con 0x7fc600100090 2024-04-25T00:36:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.528+0000 7fc608200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc60010f7d0 con 0x7fc600101390 2024-04-25T00:36:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc605800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc600100090 0x7fc60019e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc605800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc600100090 0x7fc60019e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:41154/0 (socket says 172.21.15.42:41154) 2024-04-25T00:36:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc605800640 1 -- 172.21.15.42:0/4131138152 learned_addr learned my addr 172.21.15.42:0/4131138152 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc604e00640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6001009f0 0x7fc60019e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc606200640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc600101390 0x7fc6001a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc605800640 1 -- 172.21.15.42:0/4131138152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc600101390 msgr2=0x7fc6001a5600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc605800640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc600101390 0x7fc6001a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc605800640 1 -- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6001009f0 msgr2=0x7fc60019e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc605800640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6001009f0 0x7fc60019e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc605800640 1 -- 172.21.15.42:0/4131138152 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6001a5c70 con 0x7fc600100090 2024-04-25T00:36:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.529+0000 7fc604e00640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6001009f0 0x7fc60019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.530+0000 7fc605800640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc600100090 0x7fc60019e0a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400d950 tx=0x7fc5f400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.531+0000 7fc5fe000640 1 -- 172.21.15.42:0/4131138152 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4017070 con 0x7fc600100090 2024-04-25T00:36:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.531+0000 7fc5fe000640 1 -- 172.21.15.42:0/4131138152 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f4009c30 con 0x7fc600100090 2024-04-25T00:36:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.531+0000 7fc5fe000640 1 -- 172.21.15.42:0/4131138152 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400f6b0 con 0x7fc600100090 2024-04-25T00:36:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.531+0000 7fc608200640 1 -- 172.21.15.42:0/4131138152 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6001a5f60 con 0x7fc600100090 2024-04-25T00:36:21.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.531+0000 7fc608200640 1 -- 172.21.15.42:0/4131138152 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6001a6370 con 0x7fc600100090 2024-04-25T00:36:21.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.535+0000 7fc60b010740 1 Processor -- start 2024-04-25T00:36:21.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.536+0000 7fc60b010740 1 -- start start 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.536+0000 7fc60b010740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9cca0 0x561cd5f9d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.536+0000 7fc60b010740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9d690 0x561cd5fa2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.536+0000 7fc60b010740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 0x561cd5fa4c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.536+0000 7fc60b010740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561cd5effab0 con 0x561cd5f9cca0 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.536+0000 7fc60b010740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561cd5ea8640 con 0x561cd5fa2800 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.536+0000 7fc60b010740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cd5efba70 con 0x561cd5f9d690 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.536+0000 7fc5fe000640 1 -- 172.21.15.42:0/4131138152 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc5f40109d0 con 0x7fc600100090 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.536+0000 7fc606200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9cca0 0x561cd5f9d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.537+0000 7fc606200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9cca0 0x561cd5f9d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45568/0 (socket says 172.21.15.42:45568) 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.537+0000 7fc606200640 1 -- 172.21.15.42:0/2985292456 learned_addr learned my addr 172.21.15.42:0/2985292456 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.537+0000 7fc604e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 0x561cd5fa4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.537+0000 7fc605800640 1 --2- 172.21.15.42:0/2985292456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9d690 0x561cd5fa2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.537+0000 7fc604e00640 1 -- 172.21.15.42:0/2985292456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9d690 msgr2=0x561cd5fa2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.537+0000 7fc604e00640 1 --2- 172.21.15.42:0/2985292456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9d690 0x561cd5fa2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.537+0000 7fc604e00640 1 -- 172.21.15.42:0/2985292456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9cca0 msgr2=0x561cd5f9d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.537+0000 7fc604e00640 1 --2- 172.21.15.42:0/2985292456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9cca0 0x561cd5f9d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.537+0000 7fc604e00640 1 -- 172.21.15.42:0/2985292456 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd5c14210 con 0x561cd5fa2800 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.537+0000 7fc604e00640 1 --2- 172.21.15.42:0/2985292456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 0x561cd5fa4c90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006010 tx=0x7fc5ec0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=657e487875984caf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.538+0000 7fc5f2a00640 1 -- 172.21.15.42:0/2985292456 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec03f070 con 0x561cd5fa2800 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.538+0000 7fc5f2a00640 1 -- 172.21.15.42:0/2985292456 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec0044b0 con 0x561cd5fa2800 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.538+0000 7fc5f2a00640 1 -- 172.21.15.42:0/2985292456 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec035030 con 0x561cd5fa2800 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.538+0000 7fc5fe000640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc5e80a2d30 0x7fc5e80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.538+0000 7fc5fe000640 1 -- 172.21.15.42:0/4131138152 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc5f40ced10 con 0x7fc600100090 2024-04-25T00:36:21.539 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.538+0000 7fc606200640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc5e80a2d30 0x7fc5e80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.539+0000 7fc60b010740 1 -- 172.21.15.42:0/2985292456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 msgr2=0x561cd5fa4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.539+0000 7fc60b010740 1 --2- 172.21.15.42:0/2985292456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 0x561cd5fa4c90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec006010 tx=0x7fc5ec0040c0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.539+0000 7fc606200640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc5e80a2d30 0x7fc5e80a51f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x561cd5f9def0 tx=0x7fc5f8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.540 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.539+0000 7fc60b010740 1 -- 172.21.15.42:0/2985292456 shutdown_connections 2024-04-25T00:36:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.539+0000 7fc60b010740 1 --2- 172.21.15.42:0/2985292456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 0x561cd5fa4c90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.539+0000 7fc60b010740 1 --2- 172.21.15.42:0/2985292456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9d690 0x561cd5fa2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.539+0000 7fc60b010740 1 --2- 172.21.15.42:0/2985292456 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9cca0 0x561cd5f9d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.539+0000 7fc60b010740 1 -- 172.21.15.42:0/2985292456 >> 172.21.15.42:0/2985292456 conn(0x561cd5f98310 msgr2=0x561cd5f98710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.540+0000 7fc60b010740 1 -- 172.21.15.42:0/2985292456 shutdown_connections 2024-04-25T00:36:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.540+0000 7fc60b010740 1 -- 172.21.15.42:0/2985292456 wait complete. 2024-04-25T00:36:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.540+0000 7fc60b010740 1 -- start start 2024-04-25T00:36:21.541 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.541+0000 7fc60b010740 1 Processor -- start 2024-04-25T00:36:21.542 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.542+0000 7fc60b010740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fa6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.542+0000 7fc60b010740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 0x561cd5fa7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.542+0000 7fc60b010740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 0x561cd5fae3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.542+0000 7fc60b010740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561cd5fa5750 con 0x561cd5f9d690 2024-04-25T00:36:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.542+0000 7fc60b010740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561cd5ebb990 con 0x561cd5fa2800 2024-04-25T00:36:21.543 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.542+0000 7fc60b010740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561cd5fa58d0 con 0x561cd5f9cca0 2024-04-25T00:36:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.542+0000 7fc60b010740 10 client.0 _ll_register_callbacks cb 0x561cd5ebd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:21.544 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.544+0000 7fc606200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 0x561cd5fae3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.544+0000 7fc606200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 0x561cd5fae3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:41172/0 (socket says 172.21.15.42:41172) 2024-04-25T00:36:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.544+0000 7fc606200640 1 -- 172.21.15.42:0/1087118145 learned_addr learned my addr 172.21.15.42:0/1087118145 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.544+0000 7fc605800640 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fa6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.544+0000 7fc604e00640 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 0x561cd5fa7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.545+0000 7fc605800640 1 -- 172.21.15.42:0/1087118145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 msgr2=0x561cd5fae3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.545 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.545+0000 7fc605800640 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 0x561cd5fae3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.545+0000 7fc605800640 1 -- 172.21.15.42:0/1087118145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 msgr2=0x561cd5fa7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.545+0000 7fc605800640 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 0x561cd5fa7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.545+0000 7fc605800640 1 -- 172.21.15.42:0/1087118145 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd5ce0620 con 0x561cd5f9cca0 2024-04-25T00:36:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.545+0000 7fc605800640 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fa6e50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x561cd5f9e100 tx=0x7fc5f40cc7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73fb28cb0c7cb33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.545+0000 7fc5e7400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:36:21.546 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.546+0000 7fc5e7400640 1 -- 172.21.15.42:0/1087118145 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4020030 con 0x561cd5f9cca0 2024-04-25T00:36:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.546+0000 7fc5e7400640 1 -- 172.21.15.42:0/1087118145 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc5f409a990 con 0x561cd5f9cca0 2024-04-25T00:36:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.547+0000 7fc60b010740 1 -- 172.21.15.42:0/1087118145 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5ec0044b0 con 0x561cd5f9cca0 2024-04-25T00:36:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.547+0000 7fc5e7400640 1 -- 172.21.15.42:0/1087118145 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc5f409b080 con 0x561cd5f9cca0 2024-04-25T00:36:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.547+0000 7fc60b010740 10 client.26551 fetch_fsmap learned FSMap version 45 2024-04-25T00:36:21.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.547+0000 7fc60b010740 1 -- 172.21.15.42:0/1087118145 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561cd5fb1420 con 0x561cd5f9cca0 2024-04-25T00:36:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.547+0000 7fc5e7400640 1 -- 172.21.15.42:0/1087118145 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5ec0044b0 con 0x561cd5f9cca0 2024-04-25T00:36:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.547+0000 7fc5e7400640 1 -- 172.21.15.42:0/1087118145 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fc5f40b4220 con 0x561cd5f9cca0 2024-04-25T00:36:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.548+0000 7fc60b010740 10 client.26551 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:36:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.548+0000 7fc60b010740 10 client.26551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:36:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.548+0000 7fc60b010740 20 client.26551 populate_metadata read hostname 'smithi042' 2024-04-25T00:36:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.548+0000 7fc60b010740 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561cd5fb2b40 0x561cd5fb4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.548+0000 7fc60b010740 4 client.26551 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:36:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.548+0000 7fc60b010740 1 -- 172.21.15.42:0/1087118145 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561cd5f0cff0 con 0x561cd5fb2b40 2024-04-25T00:36:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.548+0000 7fc604e00640 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561cd5fb2b40 0x561cd5fb4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.549+0000 7fc604e00640 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561cd5fb2b40 0x561cd5fb4f70 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=918e3dcc5cbfe376 server_cookie=4efa2306d43c0ca2 in_seq=0 out_seq=0 2024-04-25T00:36:21.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.549+0000 7fc5e7400640 10 client.26551 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.551+0000 7fc5e7400640 1 -- 172.21.15.42:0/1087118145 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561cd5f0cff0 con 0x561cd5fb2b40 2024-04-25T00:36:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.551+0000 7fc5e7400640 10 client.26551 handle_command_reply: tid=0 2024-04-25T00:36:21.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.551+0000 7fc60b010740 1 client.26551 shutdown 2024-04-25T00:36:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.551+0000 7fc60b010740 1 -- 172.21.15.42:0/1087118145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 msgr2=0x561cd5fa6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.551+0000 7fc60b010740 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fa6e50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x561cd5f9e100 tx=0x7fc5f40cc7e0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.551+0000 7fc60b010740 1 -- 172.21.15.42:0/1087118145 shutdown_connections 2024-04-25T00:36:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.551+0000 7fc60b010740 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561cd5fb2b40 0x561cd5fb4f70 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.551+0000 7fc5e7400640 0 client.26551 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.551+0000 7fc60b010740 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5fa2800 0x561cd5fae3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.552+0000 7fc60b010740 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 0x561cd5fa7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.552+0000 7fc60b010740 1 --2- 172.21.15.42:0/1087118145 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fa6e50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.552+0000 7fc60b010740 1 -- 172.21.15.42:0/1087118145 >> 172.21.15.42:0/1087118145 conn(0x561cd5f98310 msgr2=0x561cd5f97480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.552+0000 7fc60b010740 1 -- 172.21.15.42:0/1087118145 shutdown_connections 2024-04-25T00:36:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.552+0000 7fc60b010740 1 -- 172.21.15.42:0/1087118145 wait complete. 2024-04-25T00:36:21.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.552+0000 7fc60b010740 20 client.26551 trim_cache size 0 max 16384 2024-04-25T00:36:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.570+0000 7fc60b010740 1 Processor -- start 2024-04-25T00:36:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.570+0000 7fc60b010740 1 -- start start 2024-04-25T00:36:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc60b010740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9cca0 0x561cd5f9b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc60b010740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 0x561cd5fa6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc60b010740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5fa2800 0x561cd5fa6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc60b010740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001b1a50 con 0x561cd5f9d690 2024-04-25T00:36:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc60b010740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001af320 con 0x561cd5f9cca0 2024-04-25T00:36:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc60b010740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001ad190 con 0x561cd5fa2800 2024-04-25T00:36:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc605800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9cca0 0x561cd5f9b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc605800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9cca0 0x561cd5f9b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:41188/0 (socket says 172.21.15.42:41188) 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc605800640 1 -- 172.21.15.42:0/4139391987 learned_addr learned my addr 172.21.15.42:0/4139391987 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc606200640 1 --2- 172.21.15.42:0/4139391987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5fa2800 0x561cd5fa6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc604e00640 1 --2- 172.21.15.42:0/4139391987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 0x561cd5fa6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc605800640 1 -- 172.21.15.42:0/4139391987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5fa2800 msgr2=0x561cd5fa6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc605800640 1 --2- 172.21.15.42:0/4139391987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5fa2800 0x561cd5fa6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc605800640 1 -- 172.21.15.42:0/4139391987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 msgr2=0x561cd5fa6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc605800640 1 --2- 172.21.15.42:0/4139391987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 0x561cd5fa6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc605800640 1 -- 172.21.15.42:0/4139391987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cd5f98140 con 0x561cd5f9cca0 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.571+0000 7fc604e00640 1 --2- 172.21.15.42:0/4139391987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 0x561cd5fa6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.572+0000 7fc605800640 1 --2- 172.21.15.42:0/4139391987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9cca0 0x561cd5f9b930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40b45e0 tx=0x7fc5f40a4320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84b3e292e8f2c982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.572+0000 7fc5f2000640 1 -- 172.21.15.42:0/4139391987 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f40b3030 con 0x561cd5f9cca0 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.572+0000 7fc5f2000640 1 -- 172.21.15.42:0/4139391987 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f40bb070 con 0x561cd5f9cca0 2024-04-25T00:36:21.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.572+0000 7fc5f2000640 1 -- 172.21.15.42:0/4139391987 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4020040 con 0x561cd5f9cca0 2024-04-25T00:36:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.572+0000 7fc60b010740 1 -- 172.21.15.42:0/4139391987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9cca0 msgr2=0x561cd5f9b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.572+0000 7fc60b010740 1 --2- 172.21.15.42:0/4139391987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9cca0 0x561cd5f9b930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc5f40b45e0 tx=0x7fc5f40a4320 comp rx=0 tx=0).stop 2024-04-25T00:36:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.573+0000 7fc60b010740 1 -- 172.21.15.42:0/4139391987 shutdown_connections 2024-04-25T00:36:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.573+0000 7fc60b010740 1 --2- 172.21.15.42:0/4139391987 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5fa2800 0x561cd5fa6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.573+0000 7fc60b010740 1 --2- 172.21.15.42:0/4139391987 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5f9d690 0x561cd5fa6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.573+0000 7fc60b010740 1 --2- 172.21.15.42:0/4139391987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9cca0 0x561cd5f9b930 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.573+0000 7fc60b010740 1 -- 172.21.15.42:0/4139391987 >> 172.21.15.42:0/4139391987 conn(0x561cd5f98310 msgr2=0x561cd5f9b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.573+0000 7fc60b010740 1 -- 172.21.15.42:0/4139391987 shutdown_connections 2024-04-25T00:36:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.573+0000 7fc60b010740 1 -- 172.21.15.42:0/4139391987 wait complete. 2024-04-25T00:36:21.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.573+0000 7fc60b010740 1 -- start start 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.573+0000 7fc60b010740 1 Processor -- start 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc60b010740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fab4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc60b010740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9d690 0x561cd5faba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc60b010740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5fa2800 0x561cd5fac9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc60b010740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001ad7e0 con 0x561cd5fa2800 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc60b010740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001aec50 con 0x561cd5f9d690 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc60b010740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc6001adeb0 con 0x561cd5f9cca0 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc60b010740 10 client.0 _ll_register_callbacks cb 0x7fc5f4097120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc605800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fab4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc605800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fab4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:37902/0 (socket says 172.21.15.42:37902) 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc605800640 1 -- 172.21.15.42:0/4163946392 learned_addr learned my addr 172.21.15.42:0/4163946392 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc606200640 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5fa2800 0x561cd5fac9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc604e00640 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9d690 0x561cd5faba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc605800640 1 -- 172.21.15.42:0/4163946392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9d690 msgr2=0x561cd5faba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc605800640 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9d690 0x561cd5faba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc605800640 1 -- 172.21.15.42:0/4163946392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5fa2800 msgr2=0x561cd5fac9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc605800640 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5fa2800 0x561cd5fac9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc605800640 1 -- 172.21.15.42:0/4163946392 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cd5fa2510 con 0x561cd5f9cca0 2024-04-25T00:36:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.574+0000 7fc605800640 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fab4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc5f409b240 tx=0x7fc5f409ac20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57c6c6b39724ab05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.575+0000 7fc5e7400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:36:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.575+0000 7fc5e7400640 1 -- 172.21.15.42:0/4163946392 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4020050 con 0x561cd5f9cca0 2024-04-25T00:36:21.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.575+0000 7fc5e7400640 1 -- 172.21.15.42:0/4163946392 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc5f4097490 con 0x561cd5f9cca0 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.575+0000 7fc60b010740 1 -- 172.21.15.42:0/4163946392 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5f40bb070 con 0x561cd5f9cca0 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.575+0000 7fc5e7400640 1 -- 172.21.15.42:0/4163946392 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc5f409aea0 con 0x561cd5f9cca0 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.575+0000 7fc60b010740 10 client.26557 fetch_fsmap learned FSMap version 45 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.575+0000 7fc60b010740 1 -- 172.21.15.42:0/4163946392 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561cd5fa2510 con 0x561cd5f9cca0 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.575+0000 7fc5e7400640 1 -- 172.21.15.42:0/4163946392 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5f40bb070 con 0x561cd5f9cca0 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.575+0000 7fc5e7400640 1 -- 172.21.15.42:0/4163946392 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fc5f40104f0 con 0x561cd5f9cca0 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.576+0000 7fc60b010740 10 client.26557 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.576+0000 7fc60b010740 10 client.26557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.576+0000 7fc60b010740 20 client.26557 populate_metadata read hostname 'smithi042' 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.576+0000 7fc60b010740 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561cd5fb2b40 0x561cd601b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.576+0000 7fc60b010740 4 client.26557 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:36:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.576+0000 7fc60b010740 1 -- 172.21.15.42:0/4163946392 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561cd5f07340 con 0x561cd5fb2b40 2024-04-25T00:36:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.576+0000 7fc604e00640 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561cd5fb2b40 0x561cd601b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.576+0000 7fc604e00640 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561cd5fb2b40 0x561cd601b8e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=792da9bc65104a88 server_cookie=e8adad2521d1bc80 in_seq=0 out_seq=0 2024-04-25T00:36:21.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.576+0000 7fc5e7400640 10 client.26557 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:21.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 161356 35.03 MB/sec execute 131 sec latency 410.443 ms 2024-04-25T00:36:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc5e7400640 1 -- 172.21.15.42:0/4163946392 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561cd5f07340 con 0x561cd5fb2b40 2024-04-25T00:36:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc5e7400640 10 client.26557 handle_command_reply: tid=0 2024-04-25T00:36:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc60b010740 1 client.26557 shutdown 2024-04-25T00:36:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc60b010740 1 -- 172.21.15.42:0/4163946392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 msgr2=0x561cd5fab4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc60b010740 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fab4c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc5f409b240 tx=0x7fc5f409ac20 comp rx=0 tx=0).stop 2024-04-25T00:36:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc60b010740 1 -- 172.21.15.42:0/4163946392 shutdown_connections 2024-04-25T00:36:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc60b010740 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561cd5fb2b40 0x561cd601b8e0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc60b010740 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561cd5fa2800 0x561cd5fac9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc60b010740 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561cd5f9d690 0x561cd5faba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc60b010740 1 --2- 172.21.15.42:0/4163946392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561cd5f9cca0 0x561cd5fab4c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc60b010740 1 -- 172.21.15.42:0/4163946392 >> 172.21.15.42:0/4163946392 conn(0x561cd5f98310 msgr2=0x561cd5fa69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.874+0000 7fc5e7400640 0 client.26557 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.875+0000 7fc60b010740 1 -- 172.21.15.42:0/4163946392 shutdown_connections 2024-04-25T00:36:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.875+0000 7fc60b010740 1 -- 172.21.15.42:0/4163946392 wait complete. 2024-04-25T00:36:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.875+0000 7fc60b010740 20 client.26557 trim_cache size 0 max 16384 2024-04-25T00:36:21.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:36:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:36:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:36:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:36:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.876+0000 7fc608200640 1 -- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc5e80a2d30 msgr2=0x7fc5e80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.876+0000 7fc608200640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc5e80a2d30 0x7fc5e80a51f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x561cd5f9def0 tx=0x7fc5f8009500 comp rx=0 tx=0).stop 2024-04-25T00:36:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.876+0000 7fc608200640 1 -- 172.21.15.42:0/4131138152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc600100090 msgr2=0x7fc60019e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.876+0000 7fc608200640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc600100090 0x7fc60019e0a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400d950 tx=0x7fc5f400de10 comp rx=0 tx=0).stop 2024-04-25T00:36:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.876+0000 7fc608200640 1 -- 172.21.15.42:0/4131138152 shutdown_connections 2024-04-25T00:36:21.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.877+0000 7fc608200640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc5e80a2d30 0x7fc5e80a51f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.877+0000 7fc608200640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc600101390 0x7fc6001a5600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.877+0000 7fc608200640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6001009f0 0x7fc60019e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.877+0000 7fc608200640 1 --2- 172.21.15.42:0/4131138152 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc600100090 0x7fc60019e0a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.877+0000 7fc608200640 1 -- 172.21.15.42:0/4131138152 >> 172.21.15.42:0/4131138152 conn(0x7fc6000fbc10 msgr2=0x7fc6000fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.877+0000 7fc608200640 1 -- 172.21.15.42:0/4131138152 shutdown_connections 2024-04-25T00:36:21.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:21.877+0000 7fc608200640 1 -- 172.21.15.42:0/4131138152 wait complete. 2024-04-25T00:36:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:21 smithi136 ceph-mon[32923]: pgmap v918: 97 pgs: 97 active+clean; 632 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.8 MiB/s wr, 627 op/s 2024-04-25T00:36:22.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:21 smithi106 ceph-mon[32510]: pgmap v918: 97 pgs: 97 active+clean; 632 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.8 MiB/s wr, 627 op/s 2024-04-25T00:36:22.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:21 smithi042 ceph-mon[28130]: pgmap v918: 97 pgs: 97 active+clean; 632 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.8 MiB/s wr, 627 op/s 2024-04-25T00:36:22.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 161913 34.98 MB/sec execute 132 sec latency 210.885 ms 2024-04-25T00:36:23.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 162449 34.93 MB/sec execute 133 sec latency 95.460 ms 2024-04-25T00:36:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:23 smithi136 ceph-mon[32923]: pgmap v919: 97 pgs: 97 active+clean; 641 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 716 op/s 2024-04-25T00:36:24.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:23 smithi106 ceph-mon[32510]: pgmap v919: 97 pgs: 97 active+clean; 641 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 716 op/s 2024-04-25T00:36:24.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:23 smithi042 ceph-mon[28130]: pgmap v919: 97 pgs: 97 active+clean; 641 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 716 op/s 2024-04-25T00:36:24.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 163140 34.91 MB/sec execute 134 sec latency 49.113 ms 2024-04-25T00:36:25.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 163941 34.95 MB/sec execute 135 sec latency 90.124 ms 2024-04-25T00:36:26.178 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:25 smithi042 ceph-mon[28130]: pgmap v920: 97 pgs: 97 active+clean; 641 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 446 op/s 2024-04-25T00:36:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:25 smithi136 ceph-mon[32923]: pgmap v920: 97 pgs: 97 active+clean; 641 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 446 op/s 2024-04-25T00:36:26.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:25 smithi106 ceph-mon[32510]: pgmap v920: 97 pgs: 97 active+clean; 641 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.5 MiB/s wr, 446 op/s 2024-04-25T00:36:26.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 164715 35.03 MB/sec execute 136 sec latency 93.259 ms 2024-04-25T00:36:27.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 165411 35.08 MB/sec execute 137 sec latency 56.680 ms 2024-04-25T00:36:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:27 smithi136 ceph-mon[32923]: pgmap v921: 97 pgs: 97 active+clean; 646 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 680 op/s 2024-04-25T00:36:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:27 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:36:28.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:27 smithi106 ceph-mon[32510]: pgmap v921: 97 pgs: 97 active+clean; 646 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 680 op/s 2024-04-25T00:36:28.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:27 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:36:28.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:27 smithi042 ceph-mon[28130]: pgmap v921: 97 pgs: 97 active+clean; 646 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.9 MiB/s wr, 680 op/s 2024-04-25T00:36:28.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:27 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:36:28.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 165977 35.03 MB/sec execute 138 sec latency 143.059 ms 2024-04-25T00:36:29.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 166607 35.03 MB/sec execute 139 sec latency 52.342 ms 2024-04-25T00:36:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:29 smithi136 ceph-mon[32923]: pgmap v922: 97 pgs: 97 active+clean; 648 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-04-25T00:36:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:29 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:29 smithi106 ceph-mon[32510]: pgmap v922: 97 pgs: 97 active+clean; 648 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-04-25T00:36:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:29 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:29 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:29 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:29 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:29 smithi042 ceph-mon[28130]: pgmap v922: 97 pgs: 97 active+clean; 648 MiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 3.7 MiB/s wr, 541 op/s 2024-04-25T00:36:30.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:29 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T00:36:30.760 INFO:tasks.workunit.client.0.smithi042.stdout: 10 167295 34.98 MB/sec execute 140 sec latency 75.932 ms 2024-04-25T00:36:31.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:30 smithi136 ceph-mon[32923]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T00:36:31.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:30 smithi136 ceph-mon[32923]: Unable to set osd_memory_target on smithi136 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T00:36:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:30 smithi106 ceph-mon[32510]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T00:36:31.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:30 smithi106 ceph-mon[32510]: Unable to set osd_memory_target on smithi136 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T00:36:31.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:30 smithi042 ceph-mon[28130]: Adjusting osd_memory_target on smithi136 to 176.2M 2024-04-25T00:36:31.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:30 smithi042 ceph-mon[28130]: Unable to set osd_memory_target on smithi136 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-25T00:36:31.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 167780 34.94 MB/sec execute 141 sec latency 340.970 ms 2024-04-25T00:36:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:31 smithi136 ceph-mon[32923]: pgmap v923: 97 pgs: 97 active+clean; 650 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.5 MiB/s wr, 510 op/s 2024-04-25T00:36:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:31 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:36:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:31 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:36:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:31 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:36:32.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:31 smithi106 ceph-mon[32510]: pgmap v923: 97 pgs: 97 active+clean; 650 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.5 MiB/s wr, 510 op/s 2024-04-25T00:36:32.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:31 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:36:32.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:31 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:36:32.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:31 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:36:32.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:31 smithi042 ceph-mon[28130]: pgmap v923: 97 pgs: 97 active+clean; 650 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.5 MiB/s wr, 510 op/s 2024-04-25T00:36:32.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:31 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:36:32.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:31 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:36:32.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:31 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:36:32.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 168496 34.98 MB/sec execute 142 sec latency 66.289 ms 2024-04-25T00:36:33.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 169014 34.91 MB/sec execute 143 sec latency 229.291 ms 2024-04-25T00:36:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:33 smithi136 ceph-mon[32923]: pgmap v924: 97 pgs: 97 active+clean; 672 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.8 MiB/s wr, 684 op/s 2024-04-25T00:36:34.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:33 smithi106 ceph-mon[32510]: pgmap v924: 97 pgs: 97 active+clean; 672 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.8 MiB/s wr, 684 op/s 2024-04-25T00:36:34.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:33 smithi042 ceph-mon[28130]: pgmap v924: 97 pgs: 97 active+clean; 672 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.8 MiB/s wr, 684 op/s 2024-04-25T00:36:34.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 169636 34.93 MB/sec execute 144 sec latency 377.674 ms 2024-04-25T00:36:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.063+0000 7fc63ba00640 1 -- 172.21.15.42:0/2000551565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc634104bc0 msgr2=0x7fc634105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.063+0000 7fc63ba00640 1 --2- 172.21.15.42:0/2000551565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc634104bc0 0x7fc634105040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc61c009900 tx=0x7fc61c02a410 comp rx=0 tx=0).stop 2024-04-25T00:36:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.063+0000 7fc63ba00640 1 -- 172.21.15.42:0/2000551565 shutdown_connections 2024-04-25T00:36:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.063+0000 7fc63ba00640 1 --2- 172.21.15.42:0/2000551565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc634105580 0x7fc63410be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.063+0000 7fc63ba00640 1 --2- 172.21.15.42:0/2000551565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc634104bc0 0x7fc634105040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.063+0000 7fc63ba00640 1 --2- 172.21.15.42:0/2000551565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc6341039d0 0x7fc634103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.064+0000 7fc63ba00640 1 -- 172.21.15.42:0/2000551565 >> 172.21.15.42:0/2000551565 conn(0x7fc6340fef60 msgr2=0x7fc634101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:35.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.064+0000 7fc63ba00640 1 -- 172.21.15.42:0/2000551565 shutdown_connections 2024-04-25T00:36:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.065+0000 7fc63ba00640 1 -- 172.21.15.42:0/2000551565 wait complete. 2024-04-25T00:36:35.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.066+0000 7fc63ba00640 1 Processor -- start 2024-04-25T00:36:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.067+0000 7fc63ba00640 1 -- start start 2024-04-25T00:36:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.067+0000 7fc63ba00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6341039d0 0x7fc634199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.067+0000 7fc63ba00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc634104bc0 0x7fc63419a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.067+0000 7fc63ba00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc634105580 0x7fc6341a1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.068 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.067+0000 7fc63ba00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc63410f720 con 0x7fc634104bc0 2024-04-25T00:36:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.067+0000 7fc63ba00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc63410f5a0 con 0x7fc634105580 2024-04-25T00:36:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.067+0000 7fc63ba00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc63410f8a0 con 0x7fc6341039d0 2024-04-25T00:36:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.067+0000 7fc639a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc634105580 0x7fc6341a1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc639a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc634105580 0x7fc6341a1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:51012/0 (socket says 172.21.15.42:51012) 2024-04-25T00:36:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc639a00640 1 -- 172.21.15.42:0/608918427 learned_addr learned my addr 172.21.15.42:0/608918427 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc639000640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6341039d0 0x7fc634199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc62be00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc634104bc0 0x7fc63419a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc639a00640 1 -- 172.21.15.42:0/608918427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6341039d0 msgr2=0x7fc634199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc639a00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6341039d0 0x7fc634199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.069 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc639a00640 1 -- 172.21.15.42:0/608918427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc634104bc0 msgr2=0x7fc63419a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc639a00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc634104bc0 0x7fc63419a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc639a00640 1 -- 172.21.15.42:0/608918427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63406b290 con 0x7fc634105580 2024-04-25T00:36:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc639000640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6341039d0 0x7fc634199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.068+0000 7fc639a00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc634105580 0x7fc6341a1210 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00b3f0 tx=0x7fc62c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.069+0000 7fc632000640 1 -- 172.21.15.42:0/608918427 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c015020 con 0x7fc634105580 2024-04-25T00:36:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.069+0000 7fc632000640 1 -- 172.21.15.42:0/608918427 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c011d10 con 0x7fc634105580 2024-04-25T00:36:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.069+0000 7fc632000640 1 -- 172.21.15.42:0/608918427 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62c00f2e0 con 0x7fc634105580 2024-04-25T00:36:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.069+0000 7fc63ba00640 1 -- 172.21.15.42:0/608918427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63406b580 con 0x7fc634105580 2024-04-25T00:36:35.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.069+0000 7fc63ba00640 1 -- 172.21.15.42:0/608918427 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63406bad0 con 0x7fc634105580 2024-04-25T00:36:35.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.075+0000 7fc632000640 1 -- 172.21.15.42:0/608918427 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc62c018020 con 0x7fc634105580 2024-04-25T00:36:35.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.077+0000 7fc63e9a6740 1 Processor -- start 2024-04-25T00:36:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.077+0000 7fc63e9a6740 1 -- start start 2024-04-25T00:36:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.077+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70bbd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.077+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bbdea0 0x55fb70bc29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.077+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bc3010 0x55fb70bc54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.078+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fb70b17ab0 con 0x55fb70bbdea0 2024-04-25T00:36:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.078+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fb70ac0640 con 0x55fb70bc3010 2024-04-25T00:36:35.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.078+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fb70b13a70 con 0x55fb70bbd4b0 2024-04-25T00:36:35.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.078+0000 7fc632000640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc6180a2d00 0x7fc6180a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.078+0000 7fc639a00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc6180a2d00 0x7fc6180a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.079+0000 7fc639000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bbdea0 0x55fb70bc29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.079+0000 7fc62be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70bbd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.079+0000 7fc62be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70bbd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:50914/0 (socket says 172.21.15.42:50914) 2024-04-25T00:36:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.079+0000 7fc639000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bbdea0 0x55fb70bc29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:57090/0 (socket says 172.21.15.42:57090) 2024-04-25T00:36:35.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.079+0000 7fc62be00640 1 -- 172.21.15.42:0/4174455435 learned_addr learned my addr 172.21.15.42:0/4174455435 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.079+0000 7fc632000640 1 -- 172.21.15.42:0/608918427 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc62c0ce860 con 0x7fc634105580 2024-04-25T00:36:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.080+0000 7fc62be00640 1 --2- 172.21.15.42:0/4174455435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bc3010 0x55fb70bc54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.080+0000 7fc639000640 1 -- 172.21.15.42:0/4174455435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 msgr2=0x55fb70bbd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.080+0000 7fc639000640 1 --2- 172.21.15.42:0/4174455435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70bbd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.080+0000 7fc639000640 1 -- 172.21.15.42:0/4174455435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bc3010 msgr2=0x55fb70bc54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.080+0000 7fc639000640 1 --2- 172.21.15.42:0/4174455435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bc3010 0x55fb70bc54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.080+0000 7fc639000640 1 -- 172.21.15.42:0/4174455435 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb7082c210 con 0x55fb70bbdea0 2024-04-25T00:36:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.081+0000 7fc639a00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc6180a2d00 0x7fc6180a51c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0cc390 tx=0x7fc62c011ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.081+0000 7fc639000640 1 --2- 172.21.15.42:0/4174455435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bbdea0 0x55fb70bc29a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc634104a20 tx=0x7fc624009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2e1a87b2e909e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:35.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.082+0000 7fc62a000640 1 -- 172.21.15.42:0/4174455435 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624004d20 con 0x55fb70bbdea0 2024-04-25T00:36:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.082+0000 7fc62a000640 1 -- 172.21.15.42:0/4174455435 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624026070 con 0x55fb70bbdea0 2024-04-25T00:36:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.082+0000 7fc62a000640 1 -- 172.21.15.42:0/4174455435 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624022510 con 0x55fb70bbdea0 2024-04-25T00:36:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.083+0000 7fc63e9a6740 1 -- 172.21.15.42:0/4174455435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bbdea0 msgr2=0x55fb70bc29a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.083+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/4174455435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bbdea0 0x55fb70bc29a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc634104a20 tx=0x7fc624009f90 comp rx=0 tx=0).stop 2024-04-25T00:36:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.083+0000 7fc63e9a6740 1 -- 172.21.15.42:0/4174455435 shutdown_connections 2024-04-25T00:36:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.083+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/4174455435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bc3010 0x55fb70bc54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.084+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/4174455435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bbdea0 0x55fb70bc29a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.084+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/4174455435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70bbd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.084+0000 7fc63e9a6740 1 -- 172.21.15.42:0/4174455435 >> 172.21.15.42:0/4174455435 conn(0x55fb70bb0310 msgr2=0x55fb70bb0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:35.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.084+0000 7fc63e9a6740 1 -- 172.21.15.42:0/4174455435 shutdown_connections 2024-04-25T00:36:35.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.084+0000 7fc63e9a6740 1 -- 172.21.15.42:0/4174455435 wait complete. 2024-04-25T00:36:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.084+0000 7fc63e9a6740 1 -- start start 2024-04-25T00:36:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.085+0000 7fc63e9a6740 1 Processor -- start 2024-04-25T00:36:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.085+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70bb63d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.085+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbdea0 0x55fb70bb6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.085+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 0x55fb70bbb940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.085+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55fb70bb4be0 con 0x55fb70bc3010 2024-04-25T00:36:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.085+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55fb70ad3990 con 0x55fb70bbdea0 2024-04-25T00:36:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.085+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fb70bb4d60 con 0x55fb70bbd4b0 2024-04-25T00:36:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc63e9a6740 10 client.0 _ll_register_callbacks cb 0x55fb70ad5950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:35.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc62be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70bb63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc62be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70bb63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:50918/0 (socket says 172.21.15.42:50918) 2024-04-25T00:36:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc639000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbdea0 0x55fb70bb6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc62be00640 1 -- 172.21.15.42:0/1851769640 learned_addr learned my addr 172.21.15.42:0/1851769640 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc639000640 1 -- 172.21.15.42:0/1851769640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 msgr2=0x55fb70bb63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:35.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc639000640 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70bb63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc639000640 1 -- 172.21.15.42:0/1851769640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 msgr2=0x55fb70bbb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc62be00640 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 0x55fb70bbb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc639000640 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 0x55fb70bbb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc639000640 1 -- 172.21.15.42:0/1851769640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb708f8620 con 0x55fb70bbdea0 2024-04-25T00:36:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.086+0000 7fc62be00640 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 0x55fb70bbb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.087+0000 7fc639000640 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbdea0 0x55fb70bb6910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55fb70bb11e0 tx=0x7fc6240040a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ed65bb19ae505ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:35.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.087+0000 7fc622a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:36:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.087+0000 7fc622a00640 1 -- 172.21.15.42:0/1851769640 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc624022710 con 0x55fb70bbdea0 2024-04-25T00:36:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.087+0000 7fc622a00640 1 -- 172.21.15.42:0/1851769640 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc6240228b0 con 0x55fb70bbdea0 2024-04-25T00:36:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.087+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1851769640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc624026070 con 0x55fb70bbdea0 2024-04-25T00:36:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.088+0000 7fc622a00640 1 -- 172.21.15.42:0/1851769640 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc624022ab0 con 0x55fb70bbdea0 2024-04-25T00:36:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.088+0000 7fc63e9a6740 10 client.26582 fetch_fsmap learned FSMap version 45 2024-04-25T00:36:35.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.088+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1851769640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55fb70bc94f0 con 0x55fb70bbdea0 2024-04-25T00:36:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.088+0000 7fc622a00640 1 -- 172.21.15.42:0/1851769640 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc624026070 con 0x55fb70bbdea0 2024-04-25T00:36:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.088+0000 7fc622a00640 1 -- 172.21.15.42:0/1851769640 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fc624033c30 con 0x55fb70bbdea0 2024-04-25T00:36:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.089+0000 7fc63e9a6740 10 client.26582 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:36:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.089+0000 7fc63e9a6740 10 client.26582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:36:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.089+0000 7fc63e9a6740 20 client.26582 populate_metadata read hostname 'smithi042' 2024-04-25T00:36:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.089+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55fb70bca8f0 0x55fb70bccce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.089+0000 7fc63e9a6740 4 client.26582 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:36:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.089+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1851769640 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb70b24ff0 con 0x55fb70bca8f0 2024-04-25T00:36:35.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.089+0000 7fc62be00640 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55fb70bca8f0 0x55fb70bccce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.090+0000 7fc62be00640 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55fb70bca8f0 0x55fb70bccce0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0692362eb82f7d3 server_cookie=d4c1fb6243b97cb3 in_seq=0 out_seq=0 2024-04-25T00:36:35.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.090+0000 7fc622a00640 10 client.26582 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc622a00640 1 -- 172.21.15.42:0/1851769640 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb70b24ff0 con 0x55fb70bca8f0 2024-04-25T00:36:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc622a00640 10 client.26582 handle_command_reply: tid=0 2024-04-25T00:36:35.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc63e9a6740 1 client.26582 shutdown 2024-04-25T00:36:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1851769640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbdea0 msgr2=0x55fb70bb6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbdea0 0x55fb70bb6910 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55fb70bb11e0 tx=0x7fc6240040a0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1851769640 shutdown_connections 2024-04-25T00:36:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55fb70bca8f0 0x55fb70bccce0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 0x55fb70bbb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbdea0 0x55fb70bb6910 secure :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0x55fb70bb11e0 tx=0x7fc6240040a0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/1851769640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70bb63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.092+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1851769640 >> 172.21.15.42:0/1851769640 conn(0x55fb70bb0310 msgr2=0x55fb70baf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.093+0000 7fc622a00640 0 client.26582 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.093+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1851769640 shutdown_connections 2024-04-25T00:36:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.093+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1851769640 wait complete. 2024-04-25T00:36:35.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.093+0000 7fc63e9a6740 20 client.26582 trim_cache size 0 max 16384 2024-04-25T00:36:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.125+0000 7fc63e9a6740 1 Processor -- start 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.125+0000 7fc63e9a6740 1 -- start start 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.125+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70baafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.125+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 0x55fb70bb3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.126+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bb3760 0x55fb70bb3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.126+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc634072e20 con 0x55fb70bc3010 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.126+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc6340706f0 con 0x55fb70bb3760 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.126+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc63406e540 con 0x55fb70bbd4b0 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.126+0000 7fc62be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70baafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.126+0000 7fc62be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70baafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:50920/0 (socket says 172.21.15.42:50920) 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.126+0000 7fc62be00640 1 -- 172.21.15.42:0/1948041411 learned_addr learned my addr 172.21.15.42:0/1948041411 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.126+0000 7fc639000640 1 --2- 172.21.15.42:0/1948041411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 0x55fb70bb3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.126+0000 7fc62be00640 1 --2- 172.21.15.42:0/1948041411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bb3760 0x55fb70bb3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.127+0000 7fc62be00640 1 -- 172.21.15.42:0/1948041411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bb3760 msgr2=0x55fb70bb3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.127+0000 7fc62be00640 1 --2- 172.21.15.42:0/1948041411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bb3760 0x55fb70bb3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.127+0000 7fc62be00640 1 -- 172.21.15.42:0/1948041411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 msgr2=0x55fb70bb3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.127+0000 7fc62be00640 1 --2- 172.21.15.42:0/1948041411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 0x55fb70bb3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.127+0000 7fc62be00640 1 -- 172.21.15.42:0/1948041411 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb70bbe280 con 0x55fb70bbd4b0 2024-04-25T00:36:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.127+0000 7fc62be00640 1 --2- 172.21.15.42:0/1948041411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70baafb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc61c031440 tx=0x7fc61c02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad81966f777bc05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.127+0000 7fc629600640 1 -- 172.21.15.42:0/1948041411 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c035040 con 0x55fb70bbd4b0 2024-04-25T00:36:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.128+0000 7fc629600640 1 -- 172.21.15.42:0/1948041411 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c04f070 con 0x55fb70bbd4b0 2024-04-25T00:36:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.128+0000 7fc629600640 1 -- 172.21.15.42:0/1948041411 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61c0316b0 con 0x55fb70bbd4b0 2024-04-25T00:36:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.128+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1948041411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 msgr2=0x55fb70baafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.129+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/1948041411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70baafb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc61c031440 tx=0x7fc61c02a410 comp rx=0 tx=0).stop 2024-04-25T00:36:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.130+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1948041411 shutdown_connections 2024-04-25T00:36:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.130+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/1948041411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bb3760 0x55fb70bb3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.130+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/1948041411 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bc3010 0x55fb70bb3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.130+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/1948041411 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bbd4b0 0x55fb70baafb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.130+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1948041411 >> 172.21.15.42:0/1948041411 conn(0x55fb70bb0310 msgr2=0x55fb70bc27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.131+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1948041411 shutdown_connections 2024-04-25T00:36:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.131+0000 7fc63e9a6740 1 -- 172.21.15.42:0/1948041411 wait complete. 2024-04-25T00:36:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.131+0000 7fc63e9a6740 1 -- start start 2024-04-25T00:36:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.132+0000 7fc63e9a6740 1 Processor -- start 2024-04-25T00:36:35.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.132+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bb3760 0x55fb70bddb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbd4b0 0x55fb70bde060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc63e9a6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bc3010 0x55fb70bdc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc63406ebb0 con 0x55fb70bb3760 2024-04-25T00:36:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc634070020 con 0x55fb70bbd4b0 2024-04-25T00:36:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc63e9a6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc63406f280 con 0x55fb70bc3010 2024-04-25T00:36:35.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc63e9a6740 10 client.0 _ll_register_callbacks cb 0x7fc6240093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc62be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bc3010 0x55fb70bdc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc639000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbd4b0 0x55fb70bde060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc62be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bc3010 0x55fb70bdc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:50932/0 (socket says 172.21.15.42:50932) 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc62be00640 1 -- 172.21.15.42:0/970381132 learned_addr learned my addr 172.21.15.42:0/970381132 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc62be00640 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bb3760 0x55fb70bddb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc639000640 1 -- 172.21.15.42:0/970381132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bc3010 msgr2=0x55fb70bdc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc639000640 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bc3010 0x55fb70bdc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.133+0000 7fc639000640 1 -- 172.21.15.42:0/970381132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bb3760 msgr2=0x55fb70bddb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.134+0000 7fc639000640 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bb3760 0x55fb70bddb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.134+0000 7fc639000640 1 -- 172.21.15.42:0/970381132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb70bdca30 con 0x55fb70bbd4b0 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.134+0000 7fc639000640 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbd4b0 0x55fb70bde060 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc624024000 tx=0x7fc624009e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93253cd296e5aed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.134+0000 7fc622a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.134+0000 7fc622a00640 1 -- 172.21.15.42:0/970381132 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc62402ab30 con 0x55fb70bbd4b0 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.134+0000 7fc63e9a6740 1 -- 172.21.15.42:0/970381132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc61c04f070 con 0x55fb70bbd4b0 2024-04-25T00:36:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.134+0000 7fc622a00640 1 -- 172.21.15.42:0/970381132 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc62402acd0 con 0x55fb70bbd4b0 2024-04-25T00:36:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.135+0000 7fc622a00640 1 -- 172.21.15.42:0/970381132 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc624055050 con 0x55fb70bbd4b0 2024-04-25T00:36:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.135+0000 7fc63e9a6740 10 client.26588 fetch_fsmap learned FSMap version 45 2024-04-25T00:36:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.135+0000 7fc63e9a6740 1 -- 172.21.15.42:0/970381132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55fb70c31460 con 0x55fb70bbd4b0 2024-04-25T00:36:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.135+0000 7fc622a00640 1 -- 172.21.15.42:0/970381132 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc61c04f070 con 0x55fb70bbd4b0 2024-04-25T00:36:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.135+0000 7fc622a00640 1 -- 172.21.15.42:0/970381132 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fc624004a10 con 0x55fb70bbd4b0 2024-04-25T00:36:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.136+0000 7fc63e9a6740 10 client.26588 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:36:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.136+0000 7fc63e9a6740 10 client.26588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:36:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.136+0000 7fc63e9a6740 20 client.26588 populate_metadata read hostname 'smithi042' 2024-04-25T00:36:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.136+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55fb70c31d60 0x55fb70c34180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.136+0000 7fc63e9a6740 4 client.26588 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:36:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.136+0000 7fc63e9a6740 1 -- 172.21.15.42:0/970381132 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb70b1f340 con 0x55fb70c31d60 2024-04-25T00:36:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.136+0000 7fc62be00640 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55fb70c31d60 0x55fb70c34180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.137+0000 7fc62be00640 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55fb70c31d60 0x55fb70c34180 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4047fa28e9d251bd server_cookie=4361d7e40bc35f9f in_seq=0 out_seq=0 2024-04-25T00:36:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.137+0000 7fc622a00640 10 client.26588 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:35.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:34 smithi136 ceph-mon[32923]: pgmap v925: 97 pgs: 97 active+clean; 672 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.5 MiB/s wr, 492 op/s 2024-04-25T00:36:35.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:34 smithi106 ceph-mon[32510]: pgmap v925: 97 pgs: 97 active+clean; 672 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.5 MiB/s wr, 492 op/s 2024-04-25T00:36:35.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:34 smithi042 ceph-mon[28130]: pgmap v925: 97 pgs: 97 active+clean; 672 MiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.5 MiB/s wr, 492 op/s 2024-04-25T00:36:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.384+0000 7fc622a00640 1 -- 172.21.15.42:0/970381132 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb70b1f340 con 0x55fb70c31d60 2024-04-25T00:36:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.384+0000 7fc622a00640 10 client.26588 handle_command_reply: tid=0 2024-04-25T00:36:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.384+0000 7fc63e9a6740 1 client.26588 shutdown 2024-04-25T00:36:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc63e9a6740 1 -- 172.21.15.42:0/970381132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbd4b0 msgr2=0x55fb70bde060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbd4b0 0x55fb70bde060 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fc624024000 tx=0x7fc624009e90 comp rx=0 tx=0).stop 2024-04-25T00:36:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc63e9a6740 1 -- 172.21.15.42:0/970381132 shutdown_connections 2024-04-25T00:36:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55fb70c31d60 0x55fb70c34180 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fb70bc3010 0x55fb70bdc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55fb70bbd4b0 0x55fb70bde060 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc63e9a6740 1 --2- 172.21.15.42:0/970381132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55fb70bb3760 0x55fb70bddb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc63e9a6740 1 -- 172.21.15.42:0/970381132 >> 172.21.15.42:0/970381132 conn(0x55fb70bb0310 msgr2=0x55fb70bbf160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc622a00640 0 client.26588 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc63e9a6740 1 -- 172.21.15.42:0/970381132 shutdown_connections 2024-04-25T00:36:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.385+0000 7fc63e9a6740 1 -- 172.21.15.42:0/970381132 wait complete. 2024-04-25T00:36:35.386 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.386+0000 7fc63e9a6740 20 client.26588 trim_cache size 0 max 16384 2024-04-25T00:36:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:36:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:36:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:36:35.387 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 -- 172.21.15.42:0/608918427 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc6180a2d00 msgr2=0x7fc6180a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc6180a2d00 0x7fc6180a51c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fc62c0cc390 tx=0x7fc62c011ec0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 -- 172.21.15.42:0/608918427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc634105580 msgr2=0x7fc6341a1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc634105580 0x7fc6341a1210 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00b3f0 tx=0x7fc62c00b8b0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 -- 172.21.15.42:0/608918427 shutdown_connections 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc6180a2d00 0x7fc6180a51c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc634105580 0x7fc6341a1210 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc634104bc0 0x7fc63419a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 --2- 172.21.15.42:0/608918427 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc6341039d0 0x7fc634199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 -- 172.21.15.42:0/608918427 >> 172.21.15.42:0/608918427 conn(0x7fc6340fef60 msgr2=0x7fc634107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 -- 172.21.15.42:0/608918427 shutdown_connections 2024-04-25T00:36:35.388 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:35.387+0000 7fc63ba00640 1 -- 172.21.15.42:0/608918427 wait complete. 2024-04-25T00:36:35.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 170177 34.93 MB/sec execute 145 sec latency 171.929 ms 2024-04-25T00:36:36.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 170377 34.73 MB/sec execute 146 sec latency 633.675 ms 2024-04-25T00:36:37.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 170525 34.56 MB/sec execute 147 sec latency 965.437 ms 2024-04-25T00:36:37.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:37 smithi106 ceph-mon[32510]: pgmap v926: 97 pgs: 97 active+clean; 667 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 696 op/s 2024-04-25T00:36:37.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:37 smithi042 ceph-mon[28130]: pgmap v926: 97 pgs: 97 active+clean; 667 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 696 op/s 2024-04-25T00:36:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:37 smithi136 ceph-mon[32923]: pgmap v926: 97 pgs: 97 active+clean; 667 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.8 MiB/s wr, 696 op/s 2024-04-25T00:36:38.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 170906 34.42 MB/sec execute 148 sec latency 135.478 ms 2024-04-25T00:36:39.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 171649 34.50 MB/sec execute 149 sec latency 91.325 ms 2024-04-25T00:36:39.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:39 smithi106 ceph-mon[32510]: pgmap v927: 97 pgs: 97 active+clean; 689 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.2 MiB/s wr, 632 op/s 2024-04-25T00:36:39.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:39 smithi042 ceph-mon[28130]: pgmap v927: 97 pgs: 97 active+clean; 689 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.2 MiB/s wr, 632 op/s 2024-04-25T00:36:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:39 smithi136 ceph-mon[32923]: pgmap v927: 97 pgs: 97 active+clean; 689 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.2 MiB/s wr, 632 op/s 2024-04-25T00:36:40.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 172470 34.52 MB/sec execute 150 sec latency 37.273 ms 2024-04-25T00:36:41.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 173258 34.59 MB/sec execute 151 sec latency 98.923 ms 2024-04-25T00:36:41.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:41 smithi106 ceph-mon[32510]: pgmap v928: 97 pgs: 97 active+clean; 683 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 746 op/s 2024-04-25T00:36:41.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:41 smithi042 ceph-mon[28130]: pgmap v928: 97 pgs: 97 active+clean; 683 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 746 op/s 2024-04-25T00:36:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:41 smithi136 ceph-mon[32923]: pgmap v928: 97 pgs: 97 active+clean; 683 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 12 MiB/s wr, 746 op/s 2024-04-25T00:36:42.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 174013 34.68 MB/sec execute 152 sec latency 51.519 ms 2024-04-25T00:36:43.761 INFO:tasks.workunit.client.0.smithi042.stdout: 10 174639 34.63 MB/sec execute 153 sec latency 48.848 ms 2024-04-25T00:36:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:43 smithi106 ceph-mon[32510]: pgmap v929: 97 pgs: 97 active+clean; 638 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.01k op/s 2024-04-25T00:36:43.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:43 smithi042 ceph-mon[28130]: pgmap v929: 97 pgs: 97 active+clean; 638 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.01k op/s 2024-04-25T00:36:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:43 smithi136 ceph-mon[32923]: pgmap v929: 97 pgs: 97 active+clean; 638 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 17 MiB/s wr, 1.01k op/s 2024-04-25T00:36:44.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 175367 34.71 MB/sec execute 154 sec latency 46.736 ms 2024-04-25T00:36:45.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 176173 34.68 MB/sec execute 155 sec latency 57.443 ms 2024-04-25T00:36:45.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:45 smithi106 ceph-mon[32510]: pgmap v930: 97 pgs: 97 active+clean; 643 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 870 op/s 2024-04-25T00:36:45.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:45 smithi042 ceph-mon[28130]: pgmap v930: 97 pgs: 97 active+clean; 643 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 870 op/s 2024-04-25T00:36:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:45 smithi136 ceph-mon[32923]: pgmap v930: 97 pgs: 97 active+clean; 643 MiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 15 MiB/s wr, 870 op/s 2024-04-25T00:36:46.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 176922 34.77 MB/sec execute 156 sec latency 77.259 ms 2024-04-25T00:36:47.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 177652 34.78 MB/sec execute 157 sec latency 50.545 ms 2024-04-25T00:36:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:47 smithi106 ceph-mon[32510]: pgmap v931: 97 pgs: 97 active+clean; 655 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-04-25T00:36:47.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:47 smithi042 ceph-mon[28130]: pgmap v931: 97 pgs: 97 active+clean; 655 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-04-25T00:36:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:47 smithi136 ceph-mon[32923]: pgmap v931: 97 pgs: 97 active+clean; 655 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 16 MiB/s wr, 1.02k op/s 2024-04-25T00:36:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.558+0000 7fc071800640 1 -- 172.21.15.42:0/199093629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc06c10da60 msgr2=0x7fc06c10dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.558+0000 7fc071800640 1 --2- 172.21.15.42:0/199093629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc06c10da60 0x7fc06c10dee0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc064009ca0 tx=0x7fc06402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.559+0000 7fc071800640 1 -- 172.21.15.42:0/199093629 shutdown_connections 2024-04-25T00:36:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.559+0000 7fc071800640 1 --2- 172.21.15.42:0/199093629 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc06c10da60 0x7fc06c10dee0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.559+0000 7fc071800640 1 --2- 172.21.15.42:0/199093629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc06c10b0b0 0x7fc06c10d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.559+0000 7fc071800640 1 --2- 172.21.15.42:0/199093629 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06c108740 0x7fc06c10ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.559+0000 7fc071800640 1 -- 172.21.15.42:0/199093629 >> 172.21.15.42:0/199093629 conn(0x7fc06c06e7b0 msgr2=0x7fc06c070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.559+0000 7fc071800640 1 -- 172.21.15.42:0/199093629 shutdown_connections 2024-04-25T00:36:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.559+0000 7fc071800640 1 -- 172.21.15.42:0/199093629 wait complete. 2024-04-25T00:36:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.560+0000 7fc071800640 1 Processor -- start 2024-04-25T00:36:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.561+0000 7fc071800640 1 -- start start 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.561+0000 7fc071800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06c108740 0x7fc06c1a67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.561+0000 7fc071800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc06c10b0b0 0x7fc06c1a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.561+0000 7fc071800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc06c10da60 0x7fc06c1add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.561+0000 7fc071800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c11c280 con 0x7fc06c108740 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.563+0000 7fc071800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c11c100 con 0x7fc06c10da60 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.563+0000 7fc071800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c11c400 con 0x7fc06c10b0b0 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.563+0000 7fc06a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc06c10b0b0 0x7fc06c1a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.562+0000 7fc06b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc06c10da60 0x7fc06c1add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.563+0000 7fc06b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc06c10da60 0x7fc06c1add50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60294/0 (socket says 172.21.15.42:60294) 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.563+0000 7fc06b400640 1 -- 172.21.15.42:0/3247188375 learned_addr learned my addr 172.21.15.42:0/3247188375 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.564+0000 7fc06b400640 1 -- 172.21.15.42:0/3247188375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc06c10b0b0 msgr2=0x7fc06c1a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.564+0000 7fc06b400640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc06c10b0b0 0x7fc06c1a6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.564+0000 7fc06b400640 1 -- 172.21.15.42:0/3247188375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06c108740 msgr2=0x7fc06c1a67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.563+0000 7fc06aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06c108740 0x7fc06c1a67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.565+0000 7fc06b400640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06c108740 0x7fc06c1a67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.565+0000 7fc06b400640 1 -- 172.21.15.42:0/3247188375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06c10ffa0 con 0x7fc06c10da60 2024-04-25T00:36:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.565+0000 7fc06aa00640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06c108740 0x7fc06c1a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:36:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.565+0000 7fc06b400640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc06c10da60 0x7fc06c1add50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc064009430 tx=0x7fc0640319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.567+0000 7fc063400640 1 -- 172.21.15.42:0/3247188375 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064003b70 con 0x7fc06c10da60 2024-04-25T00:36:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.567+0000 7fc063400640 1 -- 172.21.15.42:0/3247188375 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc064003d10 con 0x7fc06c10da60 2024-04-25T00:36:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.568+0000 7fc071800640 1 -- 172.21.15.42:0/3247188375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06c110230 con 0x7fc06c10da60 2024-04-25T00:36:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.568+0000 7fc063400640 1 -- 172.21.15.42:0/3247188375 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06402fc50 con 0x7fc06c10da60 2024-04-25T00:36:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.568+0000 7fc071800640 1 -- 172.21.15.42:0/3247188375 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06c1107d0 con 0x7fc06c10da60 2024-04-25T00:36:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.574+0000 7fc0745fc740 1 Processor -- start 2024-04-25T00:36:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.574+0000 7fc063400640 1 -- 172.21.15.42:0/3247188375 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc064004040 con 0x7fc06c10da60 2024-04-25T00:36:48.574 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.574+0000 7fc0745fc740 1 -- start start 2024-04-25T00:36:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.575+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 0x55decc4860c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.575+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc486690 0x55decc48b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.575+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 0x55decc48dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.575+0000 7fc0745fc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55decc3e8ab0 con 0x55decc486690 2024-04-25T00:36:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.575+0000 7fc0745fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55decc391640 con 0x55decc48b800 2024-04-25T00:36:48.575 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.575+0000 7fc0745fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55decc3e4a70 con 0x55decc485ca0 2024-04-25T00:36:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.575+0000 7fc06a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 0x55decc4860c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.575+0000 7fc06aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc486690 0x55decc48b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.575+0000 7fc06aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc486690 0x55decc48b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34782/0 (socket says 172.21.15.42:34782) 2024-04-25T00:36:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.575+0000 7fc06aa00640 1 -- 172.21.15.42:0/1825550295 learned_addr learned my addr 172.21.15.42:0/1825550295 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.576+0000 7fc06a000640 1 --2- 172.21.15.42:0/1825550295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 0x55decc48dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.576 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.576+0000 7fc06a000640 1 -- 172.21.15.42:0/1825550295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 msgr2=0x55decc48dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.576+0000 7fc06a000640 1 --2- 172.21.15.42:0/1825550295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 0x55decc48dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.576+0000 7fc06a000640 1 -- 172.21.15.42:0/1825550295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc486690 msgr2=0x55decc48b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.576+0000 7fc06a000640 1 --2- 172.21.15.42:0/1825550295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc486690 0x55decc48b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.576+0000 7fc06a000640 1 -- 172.21.15.42:0/1825550295 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55decc0fd210 con 0x55decc485ca0 2024-04-25T00:36:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.577+0000 7fc063400640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc0500a2d30 0x7fc0500a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.577+0000 7fc063400640 1 -- 172.21.15.42:0/3247188375 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc0640f0730 con 0x7fc06c10da60 2024-04-25T00:36:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.577+0000 7fc06a000640 1 --2- 172.21.15.42:0/1825550295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 0x55decc4860c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc06c1a7cb0 tx=0x7fc0580096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b3b43a632c22c67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.578+0000 7fc06aa00640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc0500a2d30 0x7fc0500a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.578+0000 7fc057e00640 1 -- 172.21.15.42:0/1825550295 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058004030 con 0x55decc485ca0 2024-04-25T00:36:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.578+0000 7fc057e00640 1 -- 172.21.15.42:0/1825550295 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0580041d0 con 0x55decc485ca0 2024-04-25T00:36:48.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.578+0000 7fc057e00640 1 -- 172.21.15.42:0/1825550295 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05800f6f0 con 0x55decc485ca0 2024-04-25T00:36:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.579+0000 7fc06aa00640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc0500a2d30 0x7fc0500a51f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55decc486a90 tx=0x7fc05c009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.579+0000 7fc0745fc740 1 -- 172.21.15.42:0/1825550295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 msgr2=0x55decc4860c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.579+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1825550295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 0x55decc4860c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc06c1a7cb0 tx=0x7fc0580096a0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.579+0000 7fc0745fc740 1 -- 172.21.15.42:0/1825550295 shutdown_connections 2024-04-25T00:36:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.579+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1825550295 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 0x55decc48dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.579+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1825550295 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc486690 0x55decc48b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.579+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1825550295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 0x55decc4860c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.579+0000 7fc0745fc740 1 -- 172.21.15.42:0/1825550295 >> 172.21.15.42:0/1825550295 conn(0x55decc481310 msgr2=0x55decc481710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.580+0000 7fc0745fc740 1 -- 172.21.15.42:0/1825550295 shutdown_connections 2024-04-25T00:36:48.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.580+0000 7fc0745fc740 1 -- 172.21.15.42:0/1825550295 wait complete. 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.580+0000 7fc0745fc740 1 -- start start 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.580+0000 7fc0745fc740 1 Processor -- start 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 0x55decc48ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc486690 0x55decc4904c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc48b800 0x55decc497500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc0745fc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55decc48e7b0 con 0x55decc48b800 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc0745fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55decc3a4990 con 0x55decc486690 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc0745fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55decc48e930 con 0x55decc485ca0 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc0745fc740 10 client.0 _ll_register_callbacks cb 0x55decc3a6950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc06a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc486690 0x55decc4904c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc06a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc486690 0x55decc4904c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60316/0 (socket says 172.21.15.42:60316) 2024-04-25T00:36:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc06a000640 1 -- 172.21.15.42:0/1175695124 learned_addr learned my addr 172.21.15.42:0/1175695124 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc06b400640 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc48b800 0x55decc497500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc06aa00640 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 0x55decc48ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc06a000640 1 -- 172.21.15.42:0/1175695124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 msgr2=0x55decc48ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc06a000640 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 0x55decc48ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc06a000640 1 -- 172.21.15.42:0/1175695124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc48b800 msgr2=0x55decc497500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.581+0000 7fc06a000640 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc48b800 0x55decc497500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.582+0000 7fc06a000640 1 -- 172.21.15.42:0/1175695124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55decc1c9620 con 0x55decc486690 2024-04-25T00:36:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.582+0000 7fc06aa00640 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 0x55decc48ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:48.583 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.582+0000 7fc06b400640 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc48b800 0x55decc497500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:36:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.582+0000 7fc06a000640 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc486690 0x55decc4904c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc0580097a0 tx=0x7fc058003ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8de4e7b3a6be317a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.584 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.582+0000 7fc054c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.582+0000 7fc054c00640 1 -- 172.21.15.42:0/1175695124 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc058004380 con 0x55decc486690 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.582+0000 7fc054c00640 1 -- 172.21.15.42:0/1175695124 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc058019050 con 0x55decc486690 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.582+0000 7fc0745fc740 1 -- 172.21.15.42:0/1175695124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0580041d0 con 0x55decc486690 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc054c00640 1 -- 172.21.15.42:0/1175695124 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc058005040 con 0x55decc486690 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc0745fc740 10 client.26600 fetch_fsmap learned FSMap version 45 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc0745fc740 1 -- 172.21.15.42:0/1175695124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55decc49a4e0 con 0x55decc486690 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc054c00640 1 -- 172.21.15.42:0/1175695124 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc0580041d0 con 0x55decc486690 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc054c00640 1 -- 172.21.15.42:0/1175695124 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fc058004c70 con 0x55decc486690 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc0745fc740 10 client.26600 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc0745fc740 10 client.26600 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc0745fc740 20 client.26600 populate_metadata read hostname 'smithi042' 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55decc49b9b0 0x55decc49dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc0745fc740 4 client.26600 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:36:48.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.583+0000 7fc0745fc740 1 -- 172.21.15.42:0/1175695124 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55decc3f5ff0 con 0x55decc49b9b0 2024-04-25T00:36:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.584+0000 7fc06aa00640 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55decc49b9b0 0x55decc49dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.585+0000 7fc06aa00640 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55decc49b9b0 0x55decc49dda0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfd248404dc55a38 server_cookie=37f1338c191a4726 in_seq=0 out_seq=0 2024-04-25T00:36:48.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.585+0000 7fc054c00640 10 client.26600 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.587+0000 7fc054c00640 1 -- 172.21.15.42:0/1175695124 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55decc3f5ff0 con 0x55decc49b9b0 2024-04-25T00:36:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.587+0000 7fc054c00640 10 client.26600 handle_command_reply: tid=0 2024-04-25T00:36:48.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.587+0000 7fc0745fc740 1 client.26600 shutdown 2024-04-25T00:36:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc0745fc740 1 -- 172.21.15.42:0/1175695124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc486690 msgr2=0x55decc4904c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.588 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc486690 0x55decc4904c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc0580097a0 tx=0x7fc058003ad0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc0745fc740 1 -- 172.21.15.42:0/1175695124 shutdown_connections 2024-04-25T00:36:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55decc49b9b0 0x55decc49dda0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc48b800 0x55decc497500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc486690 0x55decc4904c0 secure :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc0580097a0 tx=0x7fc058003ad0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1175695124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc485ca0 0x55decc48ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc0745fc740 1 -- 172.21.15.42:0/1175695124 >> 172.21.15.42:0/1175695124 conn(0x55decc481310 msgr2=0x55decc480480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc054c00640 0 client.26600 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc0745fc740 1 -- 172.21.15.42:0/1175695124 shutdown_connections 2024-04-25T00:36:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.588+0000 7fc0745fc740 1 -- 172.21.15.42:0/1175695124 wait complete. 2024-04-25T00:36:48.589 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.589+0000 7fc0745fc740 20 client.26600 trim_cache size 0 max 16384 2024-04-25T00:36:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc0745fc740 1 Processor -- start 2024-04-25T00:36:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc0745fc740 1 -- start start 2024-04-25T00:36:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc485ca0 0x55decc484220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc48b800 0x55decc4974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc4880c0 0x55decc484760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc0745fc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1c29f0 con 0x55decc485ca0 2024-04-25T00:36:48.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc0745fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1c02c0 con 0x55decc4880c0 2024-04-25T00:36:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc0745fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1174e0 con 0x55decc48b800 2024-04-25T00:36:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc06a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc48b800 0x55decc4974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc06aa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc485ca0 0x55decc484220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc06a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc48b800 0x55decc4974c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33114/0 (socket says 172.21.15.42:33114) 2024-04-25T00:36:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.621+0000 7fc06a000640 1 -- 172.21.15.42:0/1420541107 learned_addr learned my addr 172.21.15.42:0/1420541107 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:48.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.622+0000 7fc06b400640 1 --2- 172.21.15.42:0/1420541107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc4880c0 0x55decc484760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.622+0000 7fc06b400640 1 -- 172.21.15.42:0/1420541107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc48b800 msgr2=0x55decc4974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.622+0000 7fc06b400640 1 --2- 172.21.15.42:0/1420541107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc48b800 0x55decc4974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.622+0000 7fc06b400640 1 -- 172.21.15.42:0/1420541107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc485ca0 msgr2=0x55decc484220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.622+0000 7fc06b400640 1 --2- 172.21.15.42:0/1420541107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc485ca0 0x55decc484220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.622+0000 7fc06b400640 1 -- 172.21.15.42:0/1420541107 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55decc481140 con 0x55decc4880c0 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.622+0000 7fc06b400640 1 --2- 172.21.15.42:0/1420541107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc4880c0 0x55decc484760 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc0640ee200 tx=0x7fc0640095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1bde5c98d7e52d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.622+0000 7fc057400640 1 -- 172.21.15.42:0/1420541107 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0640d5070 con 0x55decc4880c0 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.623+0000 7fc057400640 1 -- 172.21.15.42:0/1420541107 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc064036030 con 0x55decc4880c0 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.623+0000 7fc057400640 1 -- 172.21.15.42:0/1420541107 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0640ee310 con 0x55decc4880c0 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.623+0000 7fc0745fc740 1 -- 172.21.15.42:0/1420541107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc4880c0 msgr2=0x55decc484760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.623+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1420541107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc4880c0 0x55decc484760 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc0640ee200 tx=0x7fc0640095c0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.623+0000 7fc0745fc740 1 -- 172.21.15.42:0/1420541107 shutdown_connections 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.623+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1420541107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc4880c0 0x55decc484760 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.623+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1420541107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc48b800 0x55decc4974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.623+0000 7fc0745fc740 1 --2- 172.21.15.42:0/1420541107 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc485ca0 0x55decc484220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.623+0000 7fc0745fc740 1 -- 172.21.15.42:0/1420541107 >> 172.21.15.42:0/1420541107 conn(0x55decc481310 msgr2=0x55decc4879b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.624+0000 7fc0745fc740 1 -- 172.21.15.42:0/1420541107 shutdown_connections 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.624+0000 7fc0745fc740 1 -- 172.21.15.42:0/1420541107 wait complete. 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.624+0000 7fc0745fc740 1 -- start start 2024-04-25T00:36:48.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.624+0000 7fc0745fc740 1 Processor -- start 2024-04-25T00:36:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.624+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc485ca0 0x55decc494dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.624+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 0x55decc495300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.624+0000 7fc0745fc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc496270 0x55decc4966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.624+0000 7fc0745fc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c117b50 con 0x55decc485ca0 2024-04-25T00:36:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.624+0000 7fc0745fc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c1bfbf0 con 0x55decc48b800 2024-04-25T00:36:48.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.624+0000 7fc0745fc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc06c118220 con 0x55decc496270 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc0745fc740 10 client.0 _ll_register_callbacks cb 0x7fc05c0094c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc496270 0x55decc4966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc496270 0x55decc4966f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:33116/0 (socket says 172.21.15.42:33116) 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06b400640 1 -- 172.21.15.42:0/4109099533 learned_addr learned my addr 172.21.15.42:0/4109099533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06a000640 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 0x55decc495300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06aa00640 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc485ca0 0x55decc494dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06b400640 1 -- 172.21.15.42:0/4109099533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 msgr2=0x55decc495300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06b400640 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 0x55decc495300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06b400640 1 -- 172.21.15.42:0/4109099533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc485ca0 msgr2=0x55decc494dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06b400640 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc485ca0 0x55decc494dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06b400640 1 -- 172.21.15.42:0/4109099533 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55decc496c30 con 0x55decc496270 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.625+0000 7fc06a000640 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 0x55decc495300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.626+0000 7fc06b400640 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc496270 0x55decc4966f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc0640bcef0 tx=0x7fc064030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7c57160481a4327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:36:48.629 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.626+0000 7fc054c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.626+0000 7fc054c00640 1 -- 172.21.15.42:0/4109099533 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0640d5030 con 0x55decc496270 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.626+0000 7fc054c00640 1 -- 172.21.15.42:0/4109099533 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc064004740 con 0x55decc496270 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.626+0000 7fc0745fc740 1 -- 172.21.15.42:0/4109099533 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc064036030 con 0x55decc496270 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.626+0000 7fc054c00640 1 -- 172.21.15.42:0/4109099533 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc0640bd660 con 0x55decc496270 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.626+0000 7fc0745fc740 10 client.26593 fetch_fsmap learned FSMap version 45 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.626+0000 7fc0745fc740 1 -- 172.21.15.42:0/4109099533 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55decc5023f0 con 0x55decc496270 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.626+0000 7fc054c00640 1 -- 172.21.15.42:0/4109099533 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc064036030 con 0x55decc496270 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.627+0000 7fc054c00640 1 -- 172.21.15.42:0/4109099533 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fc0640da070 con 0x55decc496270 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.627+0000 7fc0745fc740 10 client.26593 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.627+0000 7fc0745fc740 10 client.26593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.627+0000 7fc0745fc740 20 client.26593 populate_metadata read hostname 'smithi042' 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.627+0000 7fc0745fc740 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55decc502d70 0x55decc505190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.627+0000 7fc0745fc740 4 client.26593 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:36:48.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.627+0000 7fc0745fc740 1 -- 172.21.15.42:0/4109099533 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55decc3f0340 con 0x55decc502d70 2024-04-25T00:36:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.627+0000 7fc06aa00640 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55decc502d70 0x55decc505190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:36:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.628+0000 7fc06aa00640 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55decc502d70 0x55decc505190 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19132fe6910f2780 server_cookie=107e713bf87ec1ea in_seq=0 out_seq=0 2024-04-25T00:36:48.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.628+0000 7fc054c00640 10 client.26593 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:48.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 178296 34.78 MB/sec execute 158 sec latency 93.678 ms 2024-04-25T00:36:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.898+0000 7fc054c00640 1 -- 172.21.15.42:0/4109099533 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55decc3f0340 con 0x55decc502d70 2024-04-25T00:36:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.898+0000 7fc054c00640 10 client.26593 handle_command_reply: tid=0 2024-04-25T00:36:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.898+0000 7fc0745fc740 1 client.26593 shutdown 2024-04-25T00:36:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.898+0000 7fc0745fc740 1 -- 172.21.15.42:0/4109099533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc496270 msgr2=0x55decc4966f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.898+0000 7fc0745fc740 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc496270 0x55decc4966f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc0640bcef0 tx=0x7fc064030000 comp rx=0 tx=0).stop 2024-04-25T00:36:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.899+0000 7fc0745fc740 1 -- 172.21.15.42:0/4109099533 shutdown_connections 2024-04-25T00:36:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.899+0000 7fc0745fc740 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55decc502d70 0x55decc505190 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.899+0000 7fc0745fc740 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55decc496270 0x55decc4966f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.899+0000 7fc0745fc740 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55decc48b800 0x55decc495300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.899+0000 7fc0745fc740 1 --2- 172.21.15.42:0/4109099533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55decc485ca0 0x55decc494dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.899+0000 7fc0745fc740 1 -- 172.21.15.42:0/4109099533 >> 172.21.15.42:0/4109099533 conn(0x55decc481310 msgr2=0x55decc48fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:48.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.899+0000 7fc054c00640 0 client.26593 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:36:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.899+0000 7fc0745fc740 1 -- 172.21.15.42:0/4109099533 shutdown_connections 2024-04-25T00:36:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.902+0000 7fc0745fc740 1 -- 172.21.15.42:0/4109099533 wait complete. 2024-04-25T00:36:48.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.902+0000 7fc0745fc740 20 client.26593 trim_cache size 0 max 16384 2024-04-25T00:36:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:36:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:36:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:36:48.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:36:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.905+0000 7fc071800640 1 -- 172.21.15.42:0/3247188375 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc0500a2d30 msgr2=0x7fc0500a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.905+0000 7fc071800640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc0500a2d30 0x7fc0500a51f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x55decc486a90 tx=0x7fc05c009620 comp rx=0 tx=0).stop 2024-04-25T00:36:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.905+0000 7fc071800640 1 -- 172.21.15.42:0/3247188375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc06c10da60 msgr2=0x7fc06c1add50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:36:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.905+0000 7fc071800640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc06c10da60 0x7fc06c1add50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc064009430 tx=0x7fc0640319f0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.905+0000 7fc071800640 1 -- 172.21.15.42:0/3247188375 shutdown_connections 2024-04-25T00:36:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.905+0000 7fc071800640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc0500a2d30 0x7fc0500a51f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.905+0000 7fc071800640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc06c10da60 0x7fc06c1add50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.905+0000 7fc071800640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc06c10b0b0 0x7fc06c1a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.905+0000 7fc071800640 1 --2- 172.21.15.42:0/3247188375 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc06c108740 0x7fc06c1a67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:36:48.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.905+0000 7fc071800640 1 -- 172.21.15.42:0/3247188375 >> 172.21.15.42:0/3247188375 conn(0x7fc06c06e7b0 msgr2=0x7fc06c10bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:36:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.906+0000 7fc071800640 1 -- 172.21.15.42:0/3247188375 shutdown_connections 2024-04-25T00:36:48.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:36:48.906+0000 7fc071800640 1 -- 172.21.15.42:0/3247188375 wait complete. 2024-04-25T00:36:49.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 178982 34.81 MB/sec execute 159 sec latency 178.866 ms 2024-04-25T00:36:49.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:49 smithi106 ceph-mon[32510]: pgmap v932: 97 pgs: 97 active+clean; 658 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 857 op/s 2024-04-25T00:36:49.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:49 smithi042 ceph-mon[28130]: pgmap v932: 97 pgs: 97 active+clean; 658 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 857 op/s 2024-04-25T00:36:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:49 smithi136 ceph-mon[32923]: pgmap v932: 97 pgs: 97 active+clean; 658 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 857 op/s 2024-04-25T00:36:50.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 179700 34.85 MB/sec execute 160 sec latency 49.035 ms 2024-04-25T00:36:51.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 180286 34.78 MB/sec execute 161 sec latency 69.505 ms 2024-04-25T00:36:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:51 smithi106 ceph-mon[32510]: pgmap v933: 97 pgs: 97 active+clean; 664 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 851 op/s 2024-04-25T00:36:51.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:51 smithi042 ceph-mon[28130]: pgmap v933: 97 pgs: 97 active+clean; 664 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 851 op/s 2024-04-25T00:36:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:51 smithi136 ceph-mon[32923]: pgmap v933: 97 pgs: 97 active+clean; 664 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 12 MiB/s wr, 851 op/s 2024-04-25T00:36:52.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 181110 34.85 MB/sec execute 162 sec latency 70.626 ms 2024-04-25T00:36:53.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 181929 34.88 MB/sec execute 163 sec latency 59.545 ms 2024-04-25T00:36:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:53 smithi106 ceph-mon[32510]: pgmap v934: 97 pgs: 97 active+clean; 666 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.4 MiB/s wr, 843 op/s 2024-04-25T00:36:53.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:53 smithi042 ceph-mon[28130]: pgmap v934: 97 pgs: 97 active+clean; 666 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.4 MiB/s wr, 843 op/s 2024-04-25T00:36:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:53 smithi136 ceph-mon[32923]: pgmap v934: 97 pgs: 97 active+clean; 666 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 9.4 MiB/s wr, 843 op/s 2024-04-25T00:36:54.762 INFO:tasks.workunit.client.0.smithi042.stdout: 10 182626 34.99 MB/sec execute 164 sec latency 50.569 ms 2024-04-25T00:36:55.763 INFO:tasks.workunit.client.0.smithi042.stdout: 10 183070 34.91 MB/sec execute 165 sec latency 65.917 ms 2024-04-25T00:36:55.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:55 smithi106 ceph-mon[32510]: pgmap v935: 97 pgs: 97 active+clean; 666 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.8 MiB/s wr, 558 op/s 2024-04-25T00:36:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:55 smithi042 ceph-mon[28130]: pgmap v935: 97 pgs: 97 active+clean; 666 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.8 MiB/s wr, 558 op/s 2024-04-25T00:36:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:55 smithi136 ceph-mon[32923]: pgmap v935: 97 pgs: 97 active+clean; 666 MiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.8 MiB/s wr, 558 op/s 2024-04-25T00:36:56.763 INFO:tasks.workunit.client.0.smithi042.stdout: 10 183566 34.78 MB/sec execute 166 sec latency 85.753 ms 2024-04-25T00:36:57.763 INFO:tasks.workunit.client.0.smithi042.stdout: 10 183803 34.66 MB/sec execute 167 sec latency 489.035 ms 2024-04-25T00:36:57.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:57 smithi106 ceph-mon[32510]: pgmap v936: 97 pgs: 97 active+clean; 677 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 634 op/s 2024-04-25T00:36:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:57 smithi042 ceph-mon[28130]: pgmap v936: 97 pgs: 97 active+clean; 677 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 634 op/s 2024-04-25T00:36:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:57 smithi136 ceph-mon[32923]: pgmap v936: 97 pgs: 97 active+clean; 677 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.5 MiB/s wr, 634 op/s 2024-04-25T00:36:58.763 INFO:tasks.workunit.client.0.smithi042.stdout: 10 184187 34.59 MB/sec execute 168 sec latency 235.555 ms 2024-04-25T00:36:59.763 INFO:tasks.workunit.client.0.smithi042.stdout: 10 184802 34.56 MB/sec execute 169 sec latency 521.332 ms 2024-04-25T00:36:59.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:36:59 smithi106 ceph-mon[32510]: pgmap v937: 97 pgs: 97 active+clean; 692 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 587 op/s 2024-04-25T00:36:59.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:36:59 smithi042 ceph-mon[28130]: pgmap v937: 97 pgs: 97 active+clean; 692 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 587 op/s 2024-04-25T00:37:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:36:59 smithi136 ceph-mon[32923]: pgmap v937: 97 pgs: 97 active+clean; 692 MiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.8 MiB/s wr, 587 op/s 2024-04-25T00:37:00.763 INFO:tasks.workunit.client.0.smithi042.stdout: 10 185423 34.60 MB/sec execute 170 sec latency 88.176 ms 2024-04-25T00:37:01.763 INFO:tasks.workunit.client.0.smithi042.stdout: 10 185996 34.63 MB/sec execute 171 sec latency 205.578 ms 2024-04-25T00:37:01.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:01 smithi106 ceph-mon[32510]: pgmap v938: 97 pgs: 97 active+clean; 694 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.1 MiB/s wr, 635 op/s 2024-04-25T00:37:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:01 smithi042 ceph-mon[28130]: pgmap v938: 97 pgs: 97 active+clean; 694 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.1 MiB/s wr, 635 op/s 2024-04-25T00:37:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:01 smithi136 ceph-mon[32923]: pgmap v938: 97 pgs: 97 active+clean; 694 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 6.1 MiB/s wr, 635 op/s 2024-04-25T00:37:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.097+0000 7f2019e00640 1 -- 172.21.15.42:0/1756190194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20141006a0 msgr2=0x7f2014100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.097+0000 7f2019e00640 1 --2- 172.21.15.42:0/1756190194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20141006a0 0x7f2014100ac0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f200401a910 tx=0x7f200403b420 comp rx=0 tx=0).stop 2024-04-25T00:37:02.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.098+0000 7f2019e00640 1 -- 172.21.15.42:0/1756190194 shutdown_connections 2024-04-25T00:37:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.098+0000 7f2019e00640 1 --2- 172.21.15.42:0/1756190194 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2014102250 0x7f2014108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.098+0000 7f2019e00640 1 --2- 172.21.15.42:0/1756190194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2014101890 0x7f2014101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.098+0000 7f2019e00640 1 --2- 172.21.15.42:0/1756190194 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20141006a0 0x7f2014100ac0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.098+0000 7f2019e00640 1 -- 172.21.15.42:0/1756190194 >> 172.21.15.42:0/1756190194 conn(0x7f20140fbc10 msgr2=0x7f20140fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.099+0000 7f2019e00640 1 -- 172.21.15.42:0/1756190194 shutdown_connections 2024-04-25T00:37:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.099+0000 7f2019e00640 1 -- 172.21.15.42:0/1756190194 wait complete. 2024-04-25T00:37:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.100+0000 7f2019e00640 1 Processor -- start 2024-04-25T00:37:02.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.100+0000 7f2019e00640 1 -- start start 2024-04-25T00:37:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2019e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20141006a0 0x7f20141912c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2019e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2014101890 0x7f2014191800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2019e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2014102250 0x7f2014198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2019e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f201406ddb0 con 0x7f2014101890 2024-04-25T00:37:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2019e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f201406dc30 con 0x7f2014102250 2024-04-25T00:37:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2019e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f201406df30 con 0x7f20141006a0 2024-04-25T00:37:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2013400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2014102250 0x7f2014198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2012a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20141006a0 0x7f20141912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2013400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2014102250 0x7f2014198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43920/0 (socket says 172.21.15.42:43920) 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2012a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20141006a0 0x7f20141912c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:55690/0 (socket says 172.21.15.42:55690) 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.101+0000 7f2013400640 1 -- 172.21.15.42:0/3902138183 learned_addr learned my addr 172.21.15.42:0/3902138183 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.102+0000 7f2012a00640 1 -- 172.21.15.42:0/3902138183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2014102250 msgr2=0x7f2014198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.102+0000 7f2012a00640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2014102250 0x7f2014198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.102+0000 7f2012a00640 1 -- 172.21.15.42:0/3902138183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2014101890 msgr2=0x7f2014191800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.102+0000 7f2012a00640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2014101890 0x7f2014191800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.102+0000 7f2012a00640 1 -- 172.21.15.42:0/3902138183 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2014198e90 con 0x7f20141006a0 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.102+0000 7f2013400640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2014102250 0x7f2014198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.102+0000 7f2012a00640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20141006a0 0x7f20141912c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f200401a860 tx=0x7f2004004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.103+0000 7f2018a00640 1 -- 172.21.15.42:0/3902138183 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200404f020 con 0x7f20141006a0 2024-04-25T00:37:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.103+0000 7f2019e00640 1 -- 172.21.15.42:0/3902138183 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2014102ab0 con 0x7f20141006a0 2024-04-25T00:37:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.103+0000 7f2019e00640 1 -- 172.21.15.42:0/3902138183 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2014102ff0 con 0x7f20141006a0 2024-04-25T00:37:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.103+0000 7f2018a00640 1 -- 172.21.15.42:0/3902138183 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2004004320 con 0x7f20141006a0 2024-04-25T00:37:02.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.104+0000 7f2018a00640 1 -- 172.21.15.42:0/3902138183 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2004046070 con 0x7f20141006a0 2024-04-25T00:37:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.107+0000 7f201cbac740 1 Processor -- start 2024-04-25T00:37:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.107+0000 7f201cbac740 1 -- start start 2024-04-25T00:37:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f201cbac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe46d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f201cbac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe46d690 0x561fbe472190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f201cbac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe472800 0x561fbe474c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f201cbac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561fbe3cfab0 con 0x561fbe46d690 2024-04-25T00:37:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f201cbac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561fbe378640 con 0x561fbe46cca0 2024-04-25T00:37:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f201cbac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561fbe3cba70 con 0x561fbe472800 2024-04-25T00:37:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f2013400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe46d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f2012000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe472800 0x561fbe474c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f2013400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe46d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43930/0 (socket says 172.21.15.42:43930) 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f2013400640 1 -- 172.21.15.42:0/3337298283 learned_addr learned my addr 172.21.15.42:0/3337298283 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f2013400640 1 -- 172.21.15.42:0/3337298283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe472800 msgr2=0x561fbe474c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f2013400640 1 --2- 172.21.15.42:0/3337298283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe472800 0x561fbe474c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f2013400640 1 -- 172.21.15.42:0/3337298283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe46d690 msgr2=0x561fbe472190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f2013400640 1 --2- 172.21.15.42:0/3337298283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe46d690 0x561fbe472190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.108+0000 7f2013400640 1 -- 172.21.15.42:0/3337298283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fbe0e4210 con 0x561fbe46cca0 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.109+0000 7f2013400640 1 --2- 172.21.15.42:0/3337298283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe46d0c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f200c009f90 tx=0x7f200c009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adeeaf19ea9bab1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.109+0000 7f1fffe00640 1 -- 172.21.15.42:0/3337298283 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c01e600 con 0x561fbe46cca0 2024-04-25T00:37:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.109+0000 7f1fffe00640 1 -- 172.21.15.42:0/3337298283 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c01ec80 con 0x561fbe46cca0 2024-04-25T00:37:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.109+0000 7f1fffe00640 1 -- 172.21.15.42:0/3337298283 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c01dc50 con 0x561fbe46cca0 2024-04-25T00:37:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.109+0000 7f2018a00640 1 -- 172.21.15.42:0/3902138183 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2004052020 con 0x7f20141006a0 2024-04-25T00:37:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.110+0000 7f201cbac740 1 -- 172.21.15.42:0/3337298283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 msgr2=0x561fbe46d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:02.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.110+0000 7f201cbac740 1 --2- 172.21.15.42:0/3337298283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe46d0c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f200c009f90 tx=0x7f200c009500 comp rx=0 tx=0).stop 2024-04-25T00:37:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.110+0000 7f201cbac740 1 -- 172.21.15.42:0/3337298283 shutdown_connections 2024-04-25T00:37:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.110+0000 7f201cbac740 1 --2- 172.21.15.42:0/3337298283 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe472800 0x561fbe474c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.110+0000 7f201cbac740 1 --2- 172.21.15.42:0/3337298283 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe46d690 0x561fbe472190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.110+0000 7f201cbac740 1 --2- 172.21.15.42:0/3337298283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe46d0c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.110+0000 7f201cbac740 1 -- 172.21.15.42:0/3337298283 >> 172.21.15.42:0/3337298283 conn(0x561fbe468310 msgr2=0x561fbe468710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.110+0000 7f201cbac740 1 -- 172.21.15.42:0/3337298283 shutdown_connections 2024-04-25T00:37:02.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.110+0000 7f201cbac740 1 -- 172.21.15.42:0/3337298283 wait complete. 2024-04-25T00:37:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.111+0000 7f201cbac740 1 -- start start 2024-04-25T00:37:02.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.111+0000 7f201cbac740 1 Processor -- start 2024-04-25T00:37:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f201cbac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe46cca0 0x561fbe476fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f201cbac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe4774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f201cbac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe472800 0x561fbe47e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f201cbac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561fbe4757b0 con 0x561fbe46cca0 2024-04-25T00:37:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f201cbac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561fbe38b990 con 0x561fbe472800 2024-04-25T00:37:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f201cbac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561fbe475930 con 0x561fbe46d690 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f2018a00640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ff80a2b40 0x7f1ff80a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f201cbac740 10 client.0 _ll_register_callbacks cb 0x561fbe38d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f2018a00640 1 -- 172.21.15.42:0/3902138183 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f20041007b0 con 0x7f20141006a0 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f2012000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe472800 0x561fbe47e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f2012000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe472800 0x561fbe47e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43940/0 (socket says 172.21.15.42:43940) 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f2012000640 1 -- 172.21.15.42:0/2567412014 learned_addr learned my addr 172.21.15.42:0/2567412014 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f2013400640 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe46cca0 0x561fbe476fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f2012a00640 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe4774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f2012000640 1 -- 172.21.15.42:0/2567412014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 msgr2=0x561fbe4774e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f2012000640 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe4774e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.112+0000 7f2012000640 1 -- 172.21.15.42:0/2567412014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe46cca0 msgr2=0x561fbe476fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.113+0000 7f2012000640 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe46cca0 0x561fbe476fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.113+0000 7f2012000640 1 -- 172.21.15.42:0/2567412014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fbe1b0620 con 0x561fbe472800 2024-04-25T00:37:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.113+0000 7f2013400640 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe46cca0 0x561fbe476fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.113+0000 7f2013400640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ff80a2b40 0x7f1ff80a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.113+0000 7f2012000640 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe472800 0x561fbe47e520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f20000116f0 tx=0x7f2000011bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edfd449e6556b792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.113+0000 7f1ffcc00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.113+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2567412014 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200000f920 con 0x561fbe472800 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.114+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2567412014 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f200000fac0 con 0x561fbe472800 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.114+0000 7f201cbac740 1 -- 172.21.15.42:0/2567412014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f200c01ec80 con 0x561fbe472800 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.114+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2567412014 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f200000fcc0 con 0x561fbe472800 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.114+0000 7f201cbac740 10 client.26618 fetch_fsmap learned FSMap version 45 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.114+0000 7f2013400640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ff80a2b40 0x7f1ff80a5000 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f200c01d480 tx=0x7f200c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.114+0000 7f201cbac740 1 -- 172.21.15.42:0/2567412014 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561fbe4813b0 con 0x561fbe472800 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.114+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2567412014 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c01ec80 con 0x561fbe472800 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.114+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2567412014 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f2000018070 con 0x561fbe472800 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.115+0000 7f201cbac740 10 client.26618 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:37:02.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.115+0000 7f201cbac740 10 client.26618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:37:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.115+0000 7f201cbac740 20 client.26618 populate_metadata read hostname 'smithi042' 2024-04-25T00:37:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.115+0000 7f201cbac740 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561fbe482a30 0x561fbe484e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.115+0000 7f201cbac740 4 client.26618 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.115+0000 7f201cbac740 1 -- 172.21.15.42:0/2567412014 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fbe3dcff0 con 0x561fbe482a30 2024-04-25T00:37:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.115+0000 7f2012a00640 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561fbe482a30 0x561fbe484e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.116+0000 7f2012a00640 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561fbe482a30 0x561fbe484e20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d40427396965a41 server_cookie=9df8a23ce4e68a37 in_seq=0 out_seq=0 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.116+0000 7f1ffcc00640 10 client.26618 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.117+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2567412014 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561fbe3dcff0 con 0x561fbe482a30 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.117+0000 7f1ffcc00640 10 client.26618 handle_command_reply: tid=0 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.117+0000 7f201cbac740 1 client.26618 shutdown 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.117+0000 7f201cbac740 1 -- 172.21.15.42:0/2567412014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe472800 msgr2=0x561fbe47e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.117+0000 7f201cbac740 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe472800 0x561fbe47e520 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f20000116f0 tx=0x7f2000011bb0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.118+0000 7f201cbac740 1 -- 172.21.15.42:0/2567412014 shutdown_connections 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.118+0000 7f201cbac740 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561fbe482a30 0x561fbe484e20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.118+0000 7f201cbac740 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe472800 0x561fbe47e520 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.118+0000 7f201cbac740 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe4774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.118+0000 7f201cbac740 1 --2- 172.21.15.42:0/2567412014 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe46cca0 0x561fbe476fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.118+0000 7f201cbac740 1 -- 172.21.15.42:0/2567412014 >> 172.21.15.42:0/2567412014 conn(0x561fbe468310 msgr2=0x561fbe467480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.118+0000 7f1ffcc00640 0 client.26618 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.118+0000 7f201cbac740 1 -- 172.21.15.42:0/2567412014 shutdown_connections 2024-04-25T00:37:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.118+0000 7f201cbac740 1 -- 172.21.15.42:0/2567412014 wait complete. 2024-04-25T00:37:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.118+0000 7f201cbac740 20 client.26618 trim_cache size 0 max 16384 2024-04-25T00:37:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f201cbac740 1 Processor -- start 2024-04-25T00:37:02.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f201cbac740 1 -- start start 2024-04-25T00:37:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f201cbac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 0x561fbe476fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f201cbac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe4774e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f201cbac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f201cbac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f20141a0890 con 0x561fbe468310 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f201cbac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f201419e160 con 0x561fbe46cca0 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f201cbac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f201419bf70 con 0x561fbe46d690 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f2013400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f2013400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:55720/0 (socket says 172.21.15.42:55720) 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.148+0000 7f2013400640 1 -- 172.21.15.42:0/3199355369 learned_addr learned my addr 172.21.15.42:0/3199355369 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.149+0000 7f2012a00640 1 --2- 172.21.15.42:0/3199355369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 0x561fbe476fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.149+0000 7f2012000640 1 --2- 172.21.15.42:0/3199355369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe4774e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.149+0000 7f2013400640 1 -- 172.21.15.42:0/3199355369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 msgr2=0x561fbe4774e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.149+0000 7f2013400640 1 --2- 172.21.15.42:0/3199355369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe4774e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.149+0000 7f2013400640 1 -- 172.21.15.42:0/3199355369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 msgr2=0x561fbe476fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.149+0000 7f2013400640 1 --2- 172.21.15.42:0/3199355369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 0x561fbe476fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.149+0000 7f2013400640 1 -- 172.21.15.42:0/3199355369 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fbe468140 con 0x561fbe46d690 2024-04-25T00:37:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.149+0000 7f2012a00640 1 --2- 172.21.15.42:0/3199355369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 0x561fbe476fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:02.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.149+0000 7f2013400640 1 --2- 172.21.15.42:0/3199355369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47e440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f200c01d740 tx=0x7f200c02c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c61e237eb0ebd03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.150+0000 7f1fff400640 1 -- 172.21.15.42:0/3199355369 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c01dbf0 con 0x561fbe46d690 2024-04-25T00:37:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.150+0000 7f1fff400640 1 -- 172.21.15.42:0/3199355369 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c038070 con 0x561fbe46d690 2024-04-25T00:37:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.151+0000 7f1fff400640 1 -- 172.21.15.42:0/3199355369 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c004650 con 0x561fbe46d690 2024-04-25T00:37:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.152+0000 7f201cbac740 1 -- 172.21.15.42:0/3199355369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 msgr2=0x561fbe47e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:02.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.152+0000 7f201cbac740 1 --2- 172.21.15.42:0/3199355369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47e440 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f200c01d740 tx=0x7f200c02c000 comp rx=0 tx=0).stop 2024-04-25T00:37:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.152+0000 7f201cbac740 1 -- 172.21.15.42:0/3199355369 shutdown_connections 2024-04-25T00:37:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.152+0000 7f201cbac740 1 --2- 172.21.15.42:0/3199355369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47e440 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.152+0000 7f201cbac740 1 --2- 172.21.15.42:0/3199355369 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe4774e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.152+0000 7f201cbac740 1 --2- 172.21.15.42:0/3199355369 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 0x561fbe476fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.152+0000 7f201cbac740 1 -- 172.21.15.42:0/3199355369 >> 172.21.15.42:0/3199355369 conn(0x561fbe472340 msgr2=0x561fbe4686f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.153+0000 7f201cbac740 1 -- 172.21.15.42:0/3199355369 shutdown_connections 2024-04-25T00:37:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.153+0000 7f201cbac740 1 -- 172.21.15.42:0/3199355369 wait complete. 2024-04-25T00:37:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.153+0000 7f201cbac740 1 -- start start 2024-04-25T00:37:02.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.154+0000 7f201cbac740 1 Processor -- start 2024-04-25T00:37:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.154+0000 7f201cbac740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 0x561fbe47b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.154+0000 7f201cbac740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe47bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.154+0000 7f201cbac740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.154+0000 7f201cbac740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f201419c620 con 0x561fbe468310 2024-04-25T00:37:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.154+0000 7f201cbac740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f201419da90 con 0x561fbe46cca0 2024-04-25T00:37:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.154+0000 7f201cbac740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f201419ccf0 con 0x561fbe46d690 2024-04-25T00:37:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.154+0000 7f201cbac740 10 client.0 _ll_register_callbacks cb 0x7f2000009710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:02.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.155+0000 7f2012000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe47bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.155+0000 7f2012000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe47bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43950/0 (socket says 172.21.15.42:43950) 2024-04-25T00:37:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.155+0000 7f2012000640 1 -- 172.21.15.42:0/2533565493 learned_addr learned my addr 172.21.15.42:0/2533565493 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.155+0000 7f2013400640 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.155+0000 7f2012a00640 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 0x561fbe47b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.155+0000 7f2013400640 1 -- 172.21.15.42:0/2533565493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 msgr2=0x561fbe47bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.155+0000 7f2013400640 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe47bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.155+0000 7f2013400640 1 -- 172.21.15.42:0/2533565493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 msgr2=0x561fbe47b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.155+0000 7f2013400640 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 0x561fbe47b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.155+0000 7f2013400640 1 -- 172.21.15.42:0/2533565493 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fbe47d310 con 0x561fbe46d690 2024-04-25T00:37:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.156+0000 7f2013400640 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47cd20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f200c00bbe0 tx=0x7f200c01dc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a2b107619dfc240 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.156+0000 7f1ffcc00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:37:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.156+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2533565493 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c04a060 con 0x561fbe46d690 2024-04-25T00:37:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.156+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2533565493 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f200c003b70 con 0x561fbe46d690 2024-04-25T00:37:02.157 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.156+0000 7f201cbac740 1 -- 172.21.15.42:0/2533565493 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f200c038070 con 0x561fbe46d690 2024-04-25T00:37:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.157+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2533565493 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f200c003d70 con 0x561fbe46d690 2024-04-25T00:37:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.157+0000 7f201cbac740 10 client.26608 fetch_fsmap learned FSMap version 45 2024-04-25T00:37:02.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.157+0000 7f201cbac740 1 -- 172.21.15.42:0/2533565493 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561fbe47e590 con 0x561fbe46d690 2024-04-25T00:37:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.158+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2533565493 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f200c038070 con 0x561fbe46d690 2024-04-25T00:37:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.159+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2533565493 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f200c045480 con 0x561fbe46d690 2024-04-25T00:37:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.159+0000 7f201cbac740 10 client.26608 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:37:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.159+0000 7f201cbac740 10 client.26608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:37:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.159+0000 7f201cbac740 20 client.26608 populate_metadata read hostname 'smithi042' 2024-04-25T00:37:02.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.159+0000 7f201cbac740 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561fbe472800 0x561fbe4eb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.159+0000 7f201cbac740 4 client.26608 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:37:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.159+0000 7f201cbac740 1 -- 172.21.15.42:0/2533565493 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561fbe3d7340 con 0x561fbe472800 2024-04-25T00:37:02.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.160+0000 7f2012a00640 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561fbe472800 0x561fbe4eb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.160+0000 7f2012a00640 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561fbe472800 0x561fbe4eb150 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01fe7721cabf4fc server_cookie=183d8726fdf260ca in_seq=0 out_seq=0 2024-04-25T00:37:02.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.160+0000 7f1ffcc00640 10 client.26608 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f1ffcc00640 1 -- 172.21.15.42:0/2533565493 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561fbe3d7340 con 0x561fbe472800 2024-04-25T00:37:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f1ffcc00640 10 client.26608 handle_command_reply: tid=0 2024-04-25T00:37:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f201cbac740 1 client.26608 shutdown 2024-04-25T00:37:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f201cbac740 1 -- 172.21.15.42:0/2533565493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 msgr2=0x561fbe47cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f201cbac740 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47cd20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f200c00bbe0 tx=0x7f200c01dc20 comp rx=0 tx=0).stop 2024-04-25T00:37:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f201cbac740 1 -- 172.21.15.42:0/2533565493 shutdown_connections 2024-04-25T00:37:02.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f201cbac740 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561fbe472800 0x561fbe4eb150 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f1ffcc00640 0 client.26608 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f201cbac740 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561fbe46d690 0x561fbe47cd20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f201cbac740 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561fbe46cca0 0x561fbe47bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f201cbac740 1 --2- 172.21.15.42:0/2533565493 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561fbe468310 0x561fbe47b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.448+0000 7f201cbac740 1 -- 172.21.15.42:0/2533565493 >> 172.21.15.42:0/2533565493 conn(0x561fbe472340 msgr2=0x561fbe46e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.449+0000 7f201cbac740 1 -- 172.21.15.42:0/2533565493 shutdown_connections 2024-04-25T00:37:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.449+0000 7f201cbac740 1 -- 172.21.15.42:0/2533565493 wait complete. 2024-04-25T00:37:02.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.449+0000 7f201cbac740 20 client.26608 trim_cache size 0 max 16384 2024-04-25T00:37:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:37:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:37:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:37:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:37:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.450+0000 7f2019e00640 1 -- 172.21.15.42:0/3902138183 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ff80a2b40 msgr2=0x7f1ff80a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.450+0000 7f2019e00640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ff80a2b40 0x7f1ff80a5000 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f200c01d480 tx=0x7f200c009500 comp rx=0 tx=0).stop 2024-04-25T00:37:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.450+0000 7f2019e00640 1 -- 172.21.15.42:0/3902138183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20141006a0 msgr2=0x7f20141912c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.450+0000 7f2019e00640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20141006a0 0x7f20141912c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f200401a860 tx=0x7f2004004ce0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.450+0000 7f2019e00640 1 -- 172.21.15.42:0/3902138183 shutdown_connections 2024-04-25T00:37:02.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.450+0000 7f2019e00640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1ff80a2b40 0x7f1ff80a5000 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.450+0000 7f2019e00640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2014102250 0x7f2014198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.450+0000 7f2019e00640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f2014101890 0x7f2014191800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.450+0000 7f2019e00640 1 --2- 172.21.15.42:0/3902138183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f20141006a0 0x7f20141912c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.450+0000 7f2019e00640 1 -- 172.21.15.42:0/3902138183 >> 172.21.15.42:0/3902138183 conn(0x7f20140fbc10 msgr2=0x7f20140fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:02.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.452+0000 7f2019e00640 1 -- 172.21.15.42:0/3902138183 shutdown_connections 2024-04-25T00:37:02.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:02.453+0000 7f2019e00640 1 -- 172.21.15.42:0/3902138183 wait complete. 2024-04-25T00:37:02.764 INFO:tasks.workunit.client.0.smithi042.stdout: 10 186310 34.50 MB/sec execute 172 sec latency 194.265 ms 2024-04-25T00:37:03.764 INFO:tasks.workunit.client.0.smithi042.stdout: 10 186441 34.35 MB/sec execute 173 sec latency 781.053 ms 2024-04-25T00:37:03.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:03 smithi042 ceph-mon[28130]: pgmap v939: 97 pgs: 97 active+clean; 693 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.7 MiB/s wr, 622 op/s 2024-04-25T00:37:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:03 smithi136 ceph-mon[32923]: pgmap v939: 97 pgs: 97 active+clean; 693 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.7 MiB/s wr, 622 op/s 2024-04-25T00:37:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:03 smithi106 ceph-mon[32510]: pgmap v939: 97 pgs: 97 active+clean; 693 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 8.7 MiB/s wr, 622 op/s 2024-04-25T00:37:04.764 INFO:tasks.workunit.client.0.smithi042.stdout: 10 186854 34.25 MB/sec execute 174 sec latency 173.786 ms 2024-04-25T00:37:05.764 INFO:tasks.workunit.client.0.smithi042.stdout: 10 187588 34.21 MB/sec execute 175 sec latency 73.238 ms 2024-04-25T00:37:05.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:05 smithi042 ceph-mon[28130]: pgmap v940: 97 pgs: 97 active+clean; 687 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.3 MiB/s wr, 544 op/s 2024-04-25T00:37:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:05 smithi136 ceph-mon[32923]: pgmap v940: 97 pgs: 97 active+clean; 687 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.3 MiB/s wr, 544 op/s 2024-04-25T00:37:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:05 smithi106 ceph-mon[32510]: pgmap v940: 97 pgs: 97 active+clean; 687 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.3 MiB/s wr, 544 op/s 2024-04-25T00:37:06.765 INFO:tasks.workunit.client.0.smithi042.stdout: 10 188555 34.34 MB/sec execute 176 sec latency 42.815 ms 2024-04-25T00:37:07.765 INFO:tasks.workunit.client.0.smithi042.stdout: 10 189355 34.49 MB/sec execute 177 sec latency 38.693 ms 2024-04-25T00:37:07.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:07 smithi042 ceph-mon[28130]: pgmap v941: 97 pgs: 97 active+clean; 661 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 852 op/s 2024-04-25T00:37:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:07 smithi136 ceph-mon[32923]: pgmap v941: 97 pgs: 97 active+clean; 661 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 852 op/s 2024-04-25T00:37:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:07 smithi106 ceph-mon[32510]: pgmap v941: 97 pgs: 97 active+clean; 661 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 13 MiB/s wr, 852 op/s 2024-04-25T00:37:08.765 INFO:tasks.workunit.client.0.smithi042.stdout: 10 189817 34.39 MB/sec execute 178 sec latency 65.064 ms 2024-04-25T00:37:09.765 INFO:tasks.workunit.client.0.smithi042.stdout: 10 190376 34.34 MB/sec execute 179 sec latency 84.339 ms 2024-04-25T00:37:09.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:09 smithi042 ceph-mon[28130]: pgmap v942: 97 pgs: 97 active+clean; 646 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 871 op/s 2024-04-25T00:37:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:09 smithi136 ceph-mon[32923]: pgmap v942: 97 pgs: 97 active+clean; 646 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 871 op/s 2024-04-25T00:37:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:09 smithi106 ceph-mon[32510]: pgmap v942: 97 pgs: 97 active+clean; 646 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 15 MiB/s wr, 871 op/s 2024-04-25T00:37:10.765 INFO:tasks.workunit.client.0.smithi042.stdout: 10 191268 34.38 MB/sec execute 180 sec latency 72.318 ms 2024-04-25T00:37:11.765 INFO:tasks.workunit.client.0.smithi042.stdout: 10 192191 34.48 MB/sec execute 181 sec latency 106.364 ms 2024-04-25T00:37:11.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:11 smithi042 ceph-mon[28130]: pgmap v943: 97 pgs: 97 active+clean; 644 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 807 op/s 2024-04-25T00:37:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:11 smithi136 ceph-mon[32923]: pgmap v943: 97 pgs: 97 active+clean; 644 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 807 op/s 2024-04-25T00:37:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:11 smithi106 ceph-mon[32510]: pgmap v943: 97 pgs: 97 active+clean; 644 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 14 MiB/s wr, 807 op/s 2024-04-25T00:37:12.765 INFO:tasks.workunit.client.0.smithi042.stdout: 10 192931 34.60 MB/sec execute 182 sec latency 72.907 ms 2024-04-25T00:37:13.765 INFO:tasks.workunit.client.0.smithi042.stdout: 10 193341 34.50 MB/sec execute 183 sec latency 65.082 ms 2024-04-25T00:37:13.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:13 smithi042 ceph-mon[28130]: pgmap v944: 97 pgs: 97 active+clean; 654 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 842 op/s 2024-04-25T00:37:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:13 smithi136 ceph-mon[32923]: pgmap v944: 97 pgs: 97 active+clean; 654 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 842 op/s 2024-04-25T00:37:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:13 smithi106 ceph-mon[32510]: pgmap v944: 97 pgs: 97 active+clean; 654 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 13 MiB/s wr, 842 op/s 2024-04-25T00:37:14.765 INFO:tasks.workunit.client.0.smithi042.stdout: 10 194027 34.47 MB/sec execute 184 sec latency 46.130 ms 2024-04-25T00:37:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.631+0000 7fa25da00640 1 -- 172.21.15.42:0/2823886693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2581017a0 msgr2=0x7fa258101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.631+0000 7fa25da00640 1 --2- 172.21.15.42:0/2823886693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2581017a0 0x7fa258101bc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa248009860 tx=0x7fa24802a340 comp rx=0 tx=0).stop 2024-04-25T00:37:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.631+0000 7fa25da00640 1 -- 172.21.15.42:0/2823886693 shutdown_connections 2024-04-25T00:37:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.631+0000 7fa25da00640 1 --2- 172.21.15.42:0/2823886693 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa258103350 0x7fa258109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.631+0000 7fa25da00640 1 --2- 172.21.15.42:0/2823886693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258102990 0x7fa258102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.631+0000 7fa25da00640 1 --2- 172.21.15.42:0/2823886693 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2581017a0 0x7fa258101bc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.631+0000 7fa25da00640 1 -- 172.21.15.42:0/2823886693 >> 172.21.15.42:0/2823886693 conn(0x7fa2580fcd30 msgr2=0x7fa2580ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.631+0000 7fa25da00640 1 -- 172.21.15.42:0/2823886693 shutdown_connections 2024-04-25T00:37:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.631+0000 7fa25da00640 1 -- 172.21.15.42:0/2823886693 wait complete. 2024-04-25T00:37:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.632+0000 7fa25da00640 1 Processor -- start 2024-04-25T00:37:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.632+0000 7fa25da00640 1 -- start start 2024-04-25T00:37:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa25da00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2581017a0 0x7fa2580754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa25da00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa258102990 0x7fa258073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa25da00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258103350 0x7fa258074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa25da00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa25810d3f0 con 0x7fa2581017a0 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa25da00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa25810d270 con 0x7fa258102990 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa25da00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa25810d570 con 0x7fa258103350 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa257400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258103350 0x7fa258074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa256a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2581017a0 0x7fa2580754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa256000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa258102990 0x7fa258073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa257400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258103350 0x7fa258074070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47908/0 (socket says 172.21.15.42:47908) 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa256a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2581017a0 0x7fa2580754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:45500/0 (socket says 172.21.15.42:45500) 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa257400640 1 -- 172.21.15.42:0/466636670 learned_addr learned my addr 172.21.15.42:0/466636670 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.633+0000 7fa256000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa258102990 0x7fa258073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56944/0 (socket says 172.21.15.42:56944) 2024-04-25T00:37:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.634+0000 7fa256000640 1 -- 172.21.15.42:0/466636670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258103350 msgr2=0x7fa258074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.634+0000 7fa256000640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258103350 0x7fa258074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.634+0000 7fa256000640 1 -- 172.21.15.42:0/466636670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2581017a0 msgr2=0x7fa2580754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.634+0000 7fa256000640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2581017a0 0x7fa2580754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.634+0000 7fa256000640 1 -- 172.21.15.42:0/466636670 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa258074900 con 0x7fa258102990 2024-04-25T00:37:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.634+0000 7fa256000640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa258102990 0x7fa258073b30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa24400e790 tx=0x7fa24400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.634+0000 7fa24f400640 1 -- 172.21.15.42:0/466636670 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24400eea0 con 0x7fa258102990 2024-04-25T00:37:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.634+0000 7fa24f400640 1 -- 172.21.15.42:0/466636670 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa24400ccc0 con 0x7fa258102990 2024-04-25T00:37:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.634+0000 7fa24f400640 1 -- 172.21.15.42:0/466636670 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa244018780 con 0x7fa258102990 2024-04-25T00:37:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.634+0000 7fa25da00640 1 -- 172.21.15.42:0/466636670 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2581a1ad0 con 0x7fa258102990 2024-04-25T00:37:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.635+0000 7fa25da00640 1 -- 172.21.15.42:0/466636670 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2581a2070 con 0x7fa258102990 2024-04-25T00:37:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.639+0000 7fa2607ec740 1 Processor -- start 2024-04-25T00:37:15.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.639+0000 7fa2607ec740 1 -- start start 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.639+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a2ca0 0x55acb33a30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.639+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 0x55acb33a8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb33aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa2607ec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55acb3305ab0 con 0x55acb33a2ca0 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa2607ec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55acb32ae640 con 0x55acb33a3690 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa2607ec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55acb3301a70 con 0x55acb33a8800 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa256a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 0x55acb33a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa256a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 0x55acb33a8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56954/0 (socket says 172.21.15.42:56954) 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa256a00640 1 -- 172.21.15.42:0/3586133812 learned_addr learned my addr 172.21.15.42:0/3586133812 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa257400640 1 --2- 172.21.15.42:0/3586133812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a2ca0 0x55acb33a30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa256a00640 1 -- 172.21.15.42:0/3586133812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 msgr2=0x55acb33aac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa256a00640 1 --2- 172.21.15.42:0/3586133812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb33aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa256a00640 1 -- 172.21.15.42:0/3586133812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a2ca0 msgr2=0x55acb33a30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa256a00640 1 --2- 172.21.15.42:0/3586133812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a2ca0 0x55acb33a30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa24f400640 1 -- 172.21.15.42:0/466636670 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa244010430 con 0x7fa258102990 2024-04-25T00:37:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.640+0000 7fa256a00640 1 -- 172.21.15.42:0/3586133812 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb301a210 con 0x55acb33a3690 2024-04-25T00:37:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.641+0000 7fa256a00640 1 --2- 172.21.15.42:0/3586133812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 0x55acb33a8190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa248009830 tx=0x7fa2480095a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96ad11259dcf8b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.641+0000 7fa243e00640 1 -- 172.21.15.42:0/3586133812 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248037040 con 0x55acb33a3690 2024-04-25T00:37:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.641+0000 7fa243e00640 1 -- 172.21.15.42:0/3586133812 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa248042070 con 0x55acb33a3690 2024-04-25T00:37:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.642+0000 7fa243e00640 1 -- 172.21.15.42:0/3586133812 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248005320 con 0x55acb33a3690 2024-04-25T00:37:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.642+0000 7fa2607ec740 1 -- 172.21.15.42:0/3586133812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 msgr2=0x55acb33a8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.642+0000 7fa2607ec740 1 --2- 172.21.15.42:0/3586133812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 0x55acb33a8190 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa248009830 tx=0x7fa2480095a0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.643+0000 7fa24f400640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa23c0a2d30 0x7fa23c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.643+0000 7fa24f400640 1 -- 172.21.15.42:0/466636670 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fa2440cf190 con 0x7fa258102990 2024-04-25T00:37:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.643+0000 7fa2607ec740 1 -- 172.21.15.42:0/3586133812 shutdown_connections 2024-04-25T00:37:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.643+0000 7fa2607ec740 1 --2- 172.21.15.42:0/3586133812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb33aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.643+0000 7fa2607ec740 1 --2- 172.21.15.42:0/3586133812 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 0x55acb33a8190 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.643+0000 7fa2607ec740 1 --2- 172.21.15.42:0/3586133812 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a2ca0 0x55acb33a30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.643+0000 7fa2607ec740 1 -- 172.21.15.42:0/3586133812 >> 172.21.15.42:0/3586133812 conn(0x55acb339e310 msgr2=0x55acb339e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.643+0000 7fa257400640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa23c0a2d30 0x7fa23c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.644+0000 7fa2607ec740 1 -- 172.21.15.42:0/3586133812 shutdown_connections 2024-04-25T00:37:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.644+0000 7fa2607ec740 1 -- 172.21.15.42:0/3586133812 wait complete. 2024-04-25T00:37:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.645+0000 7fa257400640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa23c0a2d30 0x7fa23c0a51f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55acb33a3ef0 tx=0x7fa250009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.645+0000 7fa2607ec740 1 -- start start 2024-04-25T00:37:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.645+0000 7fa2607ec740 1 Processor -- start 2024-04-25T00:37:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.646+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a2ca0 0x55acb33acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.646+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 0x55acb33ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.646+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a8800 0x55acb33b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.646+0000 7fa2607ec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55acb33ab7f0 con 0x55acb33a3690 2024-04-25T00:37:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.646+0000 7fa2607ec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55acb32c1990 con 0x55acb33a8800 2024-04-25T00:37:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.646+0000 7fa2607ec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55acb33ab970 con 0x55acb33a2ca0 2024-04-25T00:37:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.646+0000 7fa2607ec740 10 client.0 _ll_register_callbacks cb 0x55acb32c3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.646+0000 7fa257400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a8800 0x55acb33b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.646+0000 7fa257400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a8800 0x55acb33b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:56958/0 (socket says 172.21.15.42:56958) 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.646+0000 7fa257400640 1 -- 172.21.15.42:0/2924532065 learned_addr learned my addr 172.21.15.42:0/2924532065 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.647+0000 7fa257400640 1 -- 172.21.15.42:0/2924532065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a2ca0 msgr2=0x55acb33acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.647+0000 7fa256000640 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 0x55acb33ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.647+0000 7fa257400640 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a2ca0 0x55acb33acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.647+0000 7fa257400640 1 -- 172.21.15.42:0/2924532065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 msgr2=0x55acb33ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.647+0000 7fa257400640 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 0x55acb33ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.647+0000 7fa257400640 1 -- 172.21.15.42:0/2924532065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb30e6620 con 0x55acb33a8800 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.647+0000 7fa256000640 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 0x55acb33ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.647+0000 7fa257400640 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a8800 0x55acb33b4490 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa25000faf0 tx=0x7fa250004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c92fe3bca964a8d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.647+0000 7fa240c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:37:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.647+0000 7fa240c00640 1 -- 172.21.15.42:0/2924532065 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25002d060 con 0x55acb33a8800 2024-04-25T00:37:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.648+0000 7fa240c00640 1 -- 172.21.15.42:0/2924532065 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fa250017030 con 0x55acb33a8800 2024-04-25T00:37:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.648+0000 7fa2607ec740 1 -- 172.21.15.42:0/2924532065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa248042070 con 0x55acb33a8800 2024-04-25T00:37:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.648+0000 7fa240c00640 1 -- 172.21.15.42:0/2924532065 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa25001d640 con 0x55acb33a8800 2024-04-25T00:37:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.648+0000 7fa2607ec740 10 client.26636 fetch_fsmap learned FSMap version 45 2024-04-25T00:37:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.648+0000 7fa2607ec740 1 -- 172.21.15.42:0/2924532065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55acb33b7530 con 0x55acb33a8800 2024-04-25T00:37:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.648+0000 7fa240c00640 1 -- 172.21.15.42:0/2924532065 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa248042070 con 0x55acb33a8800 2024-04-25T00:37:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.648+0000 7fa240c00640 1 -- 172.21.15.42:0/2924532065 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fa250028b50 con 0x55acb33a8800 2024-04-25T00:37:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.648+0000 7fa2607ec740 10 client.26636 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:37:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.648+0000 7fa2607ec740 10 client.26636 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:37:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.649+0000 7fa2607ec740 20 client.26636 populate_metadata read hostname 'smithi042' 2024-04-25T00:37:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.649+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55acb33b8c70 0x55acb33bb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.649+0000 7fa2607ec740 4 client.26636 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.649+0000 7fa2607ec740 1 -- 172.21.15.42:0/2924532065 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acb3312ff0 con 0x55acb33b8c70 2024-04-25T00:37:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.649+0000 7fa256a00640 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55acb33b8c70 0x55acb33bb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.649+0000 7fa256a00640 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55acb33b8c70 0x55acb33bb080 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16641b8633062100 server_cookie=f45578a2dd7414e1 in_seq=0 out_seq=0 2024-04-25T00:37:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.649+0000 7fa240c00640 10 client.26636 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:15.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.651+0000 7fa240c00640 1 -- 172.21.15.42:0/2924532065 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55acb3312ff0 con 0x55acb33b8c70 2024-04-25T00:37:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.651+0000 7fa240c00640 10 client.26636 handle_command_reply: tid=0 2024-04-25T00:37:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.651+0000 7fa2607ec740 1 client.26636 shutdown 2024-04-25T00:37:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.652+0000 7fa2607ec740 1 -- 172.21.15.42:0/2924532065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a8800 msgr2=0x55acb33b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.652+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a8800 0x55acb33b4490 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa25000faf0 tx=0x7fa250004a40 comp rx=0 tx=0).stop 2024-04-25T00:37:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.652+0000 7fa2607ec740 1 -- 172.21.15.42:0/2924532065 shutdown_connections 2024-04-25T00:37:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.652+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55acb33b8c70 0x55acb33bb080 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.652+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a8800 0x55acb33b4490 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.652+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 0x55acb33ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.652+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2924532065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a2ca0 0x55acb33acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.652+0000 7fa2607ec740 1 -- 172.21.15.42:0/2924532065 >> 172.21.15.42:0/2924532065 conn(0x55acb339e310 msgr2=0x55acb339d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.652+0000 7fa240c00640 0 client.26636 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.653+0000 7fa2607ec740 1 -- 172.21.15.42:0/2924532065 shutdown_connections 2024-04-25T00:37:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.653+0000 7fa2607ec740 1 -- 172.21.15.42:0/2924532065 wait complete. 2024-04-25T00:37:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.653+0000 7fa2607ec740 20 client.26636 trim_cache size 0 max 16384 2024-04-25T00:37:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.678+0000 7fa2607ec740 1 Processor -- start 2024-04-25T00:37:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.678+0000 7fa2607ec740 1 -- start start 2024-04-25T00:37:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.678+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a2ca0 0x55acb339cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.678+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 0x55acb33b4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.678+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb339d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa2607ec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581ad670 con 0x55acb33a3690 2024-04-25T00:37:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa2607ec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581aaf40 con 0x55acb33a2ca0 2024-04-25T00:37:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa2607ec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581a8d90 con 0x55acb33a8800 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa257400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb339d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa256a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a2ca0 0x55acb339cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa257400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb339d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47944/0 (socket says 172.21.15.42:47944) 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa257400640 1 -- 172.21.15.42:0/1477420721 learned_addr learned my addr 172.21.15.42:0/1477420721 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa256000640 1 --2- 172.21.15.42:0/1477420721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 0x55acb33b4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa256a00640 1 -- 172.21.15.42:0/1477420721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 msgr2=0x55acb339d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa256a00640 1 --2- 172.21.15.42:0/1477420721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb339d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa256a00640 1 -- 172.21.15.42:0/1477420721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 msgr2=0x55acb33b4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa256a00640 1 --2- 172.21.15.42:0/1477420721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 0x55acb33b4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa256a00640 1 -- 172.21.15.42:0/1477420721 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acb339dbb0 con 0x55acb33a2ca0 2024-04-25T00:37:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.679+0000 7fa257400640 1 --2- 172.21.15.42:0/1477420721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb339d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:37:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.680+0000 7fa256a00640 1 --2- 172.21.15.42:0/1477420721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a2ca0 0x55acb339cbf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55acb339f1e0 tx=0x7fa248004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f426365e20f457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.680+0000 7fa243400640 1 -- 172.21.15.42:0/1477420721 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24804a070 con 0x55acb33a2ca0 2024-04-25T00:37:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.680+0000 7fa243400640 1 -- 172.21.15.42:0/1477420721 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa248042030 con 0x55acb33a2ca0 2024-04-25T00:37:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.680+0000 7fa243400640 1 -- 172.21.15.42:0/1477420721 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa248037040 con 0x55acb33a2ca0 2024-04-25T00:37:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.682+0000 7fa2607ec740 1 -- 172.21.15.42:0/1477420721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a2ca0 msgr2=0x55acb339cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.682+0000 7fa2607ec740 1 --2- 172.21.15.42:0/1477420721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a2ca0 0x55acb339cbf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55acb339f1e0 tx=0x7fa248004600 comp rx=0 tx=0).stop 2024-04-25T00:37:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.682+0000 7fa2607ec740 1 -- 172.21.15.42:0/1477420721 shutdown_connections 2024-04-25T00:37:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.682+0000 7fa2607ec740 1 --2- 172.21.15.42:0/1477420721 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb339d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.682+0000 7fa2607ec740 1 --2- 172.21.15.42:0/1477420721 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a3690 0x55acb33b4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.682+0000 7fa2607ec740 1 --2- 172.21.15.42:0/1477420721 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a2ca0 0x55acb339cbf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.682+0000 7fa2607ec740 1 -- 172.21.15.42:0/1477420721 >> 172.21.15.42:0/1477420721 conn(0x55acb339e310 msgr2=0x55acb33a48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.682+0000 7fa2607ec740 1 -- 172.21.15.42:0/1477420721 shutdown_connections 2024-04-25T00:37:15.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.683+0000 7fa2607ec740 1 -- 172.21.15.42:0/1477420721 wait complete. 2024-04-25T00:37:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.683+0000 7fa2607ec740 1 -- start start 2024-04-25T00:37:15.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.684+0000 7fa2607ec740 1 Processor -- start 2024-04-25T00:37:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a2ca0 0x55acb33b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 0x55acb33b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa2607ec740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb33b2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa2607ec740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581a9400 con 0x55acb33a2ca0 2024-04-25T00:37:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa2607ec740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581aa870 con 0x55acb33a3690 2024-04-25T00:37:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa2607ec740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa2581a9ad0 con 0x55acb33a8800 2024-04-25T00:37:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa2607ec740 10 client.0 _ll_register_callbacks cb 0x7fa250009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa257400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb33b2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa257400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb33b2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:47952/0 (socket says 172.21.15.42:47952) 2024-04-25T00:37:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa257400640 1 -- 172.21.15.42:0/2476516740 learned_addr learned my addr 172.21.15.42:0/2476516740 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.685+0000 7fa256a00640 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a2ca0 0x55acb33b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.686+0000 7fa257400640 1 -- 172.21.15.42:0/2476516740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 msgr2=0x55acb33b1f80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:37:15.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.686+0000 7fa257400640 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 0x55acb33b1f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.686+0000 7fa257400640 1 -- 172.21.15.42:0/2476516740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a2ca0 msgr2=0x55acb33b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.686+0000 7fa257400640 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a2ca0 0x55acb33b1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.686+0000 7fa257400640 1 -- 172.21.15.42:0/2476516740 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acb33b3580 con 0x55acb33a8800 2024-04-25T00:37:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.686+0000 7fa257400640 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb33b2f90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa25000faf0 tx=0x7fa250009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cb00a5b803188ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.686+0000 7fa240c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:37:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.686+0000 7fa240c00640 1 -- 172.21.15.42:0/2476516740 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250038070 con 0x55acb33a8800 2024-04-25T00:37:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.687+0000 7fa240c00640 1 -- 172.21.15.42:0/2476516740 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fa250004cc0 con 0x55acb33a8800 2024-04-25T00:37:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.687+0000 7fa2607ec740 1 -- 172.21.15.42:0/2476516740 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa248042030 con 0x55acb33a8800 2024-04-25T00:37:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.687+0000 7fa240c00640 1 -- 172.21.15.42:0/2476516740 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa250033970 con 0x55acb33a8800 2024-04-25T00:37:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.687+0000 7fa2607ec740 10 client.26620 fetch_fsmap learned FSMap version 45 2024-04-25T00:37:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.687+0000 7fa2607ec740 1 -- 172.21.15.42:0/2476516740 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55acb341f200 con 0x55acb33a8800 2024-04-25T00:37:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.688+0000 7fa240c00640 1 -- 172.21.15.42:0/2476516740 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa248042030 con 0x55acb33a8800 2024-04-25T00:37:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.688+0000 7fa240c00640 1 -- 172.21.15.42:0/2476516740 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fa250028420 con 0x55acb33a8800 2024-04-25T00:37:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.688+0000 7fa2607ec740 10 client.26620 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:37:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.688+0000 7fa2607ec740 10 client.26620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:37:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.688+0000 7fa2607ec740 20 client.26620 populate_metadata read hostname 'smithi042' 2024-04-25T00:37:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.688+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55acb341faf0 0x55acb3421f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.688+0000 7fa2607ec740 4 client.26620 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:37:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.688+0000 7fa2607ec740 1 -- 172.21.15.42:0/2476516740 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55acb330d340 con 0x55acb341faf0 2024-04-25T00:37:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.689+0000 7fa256a00640 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55acb341faf0 0x55acb3421f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.689+0000 7fa256a00640 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55acb341faf0 0x55acb3421f10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4154ba983dfb018c server_cookie=f296b3703d663afe in_seq=0 out_seq=0 2024-04-25T00:37:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.689+0000 7fa240c00640 10 client.26620 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:15.768 INFO:tasks.workunit.client.0.smithi042.stdout: 10 194949 34.52 MB/sec execute 185 sec latency 71.378 ms 2024-04-25T00:37:15.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:15 smithi042 ceph-mon[28130]: pgmap v945: 97 pgs: 97 active+clean; 658 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.9 MiB/s wr, 731 op/s 2024-04-25T00:37:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa240c00640 1 -- 172.21.15.42:0/2476516740 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55acb330d340 con 0x55acb341faf0 2024-04-25T00:37:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa240c00640 10 client.26620 handle_command_reply: tid=0 2024-04-25T00:37:15.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 client.26620 shutdown 2024-04-25T00:37:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 -- 172.21.15.42:0/2476516740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 msgr2=0x55acb33b2f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb33b2f90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fa25000faf0 tx=0x7fa250009f60 comp rx=0 tx=0).stop 2024-04-25T00:37:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 -- 172.21.15.42:0/2476516740 shutdown_connections 2024-04-25T00:37:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55acb341faf0 0x55acb3421f10 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55acb33a8800 0x55acb33b2f90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55acb33a3690 0x55acb33b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa240c00640 0 client.26620 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 --2- 172.21.15.42:0/2476516740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55acb33a2ca0 0x55acb33b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 -- 172.21.15.42:0/2476516740 >> 172.21.15.42:0/2476516740 conn(0x55acb339e310 msgr2=0x55acb33a48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 -- 172.21.15.42:0/2476516740 shutdown_connections 2024-04-25T00:37:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.961+0000 7fa2607ec740 1 -- 172.21.15.42:0/2476516740 wait complete. 2024-04-25T00:37:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.962+0000 7fa2607ec740 20 client.26620 trim_cache size 0 max 16384 2024-04-25T00:37:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:37:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:37:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:37:15.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:37:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 -- 172.21.15.42:0/466636670 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa23c0a2d30 msgr2=0x7fa23c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa23c0a2d30 0x7fa23c0a51f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x55acb33a3ef0 tx=0x7fa250009500 comp rx=0 tx=0).stop 2024-04-25T00:37:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 -- 172.21.15.42:0/466636670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa258102990 msgr2=0x7fa258073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa258102990 0x7fa258073b30 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa24400e790 tx=0x7fa24400ec50 comp rx=0 tx=0).stop 2024-04-25T00:37:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 -- 172.21.15.42:0/466636670 shutdown_connections 2024-04-25T00:37:15.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa23c0a2d30 0x7fa23c0a51f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa258103350 0x7fa258074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa258102990 0x7fa258073b30 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 --2- 172.21.15.42:0/466636670 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa2581017a0 0x7fa2580754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 -- 172.21.15.42:0/466636670 >> 172.21.15.42:0/466636670 conn(0x7fa2580fcd30 msgr2=0x7fa258105c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 -- 172.21.15.42:0/466636670 shutdown_connections 2024-04-25T00:37:15.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:15.963+0000 7fa25da00640 1 -- 172.21.15.42:0/466636670 wait complete. 2024-04-25T00:37:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:15 smithi136 ceph-mon[32923]: pgmap v945: 97 pgs: 97 active+clean; 658 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.9 MiB/s wr, 731 op/s 2024-04-25T00:37:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:15 smithi106 ceph-mon[32510]: pgmap v945: 97 pgs: 97 active+clean; 658 MiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 9.9 MiB/s wr, 731 op/s 2024-04-25T00:37:16.767 INFO:tasks.workunit.client.0.smithi042.stdout: 10 195776 34.60 MB/sec execute 186 sec latency 175.252 ms 2024-04-25T00:37:17.767 INFO:tasks.workunit.client.0.smithi042.stdout: 10 196513 34.71 MB/sec execute 187 sec latency 53.893 ms 2024-04-25T00:37:17.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:17 smithi042 ceph-mon[28130]: pgmap v946: 97 pgs: 97 active+clean; 658 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 779 op/s 2024-04-25T00:37:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:17 smithi136 ceph-mon[32923]: pgmap v946: 97 pgs: 97 active+clean; 658 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 779 op/s 2024-04-25T00:37:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:17 smithi106 ceph-mon[32510]: pgmap v946: 97 pgs: 97 active+clean; 658 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 779 op/s 2024-04-25T00:37:18.767 INFO:tasks.workunit.client.0.smithi042.stdout: 10 196860 34.59 MB/sec execute 188 sec latency 217.139 ms 2024-04-25T00:37:19.767 INFO:tasks.workunit.client.0.smithi042.stdout: 10 197047 34.44 MB/sec execute 189 sec latency 431.746 ms 2024-04-25T00:37:19.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:19 smithi042 ceph-mon[28130]: pgmap v947: 97 pgs: 97 active+clean; 667 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.1 MiB/s wr, 608 op/s 2024-04-25T00:37:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:19 smithi136 ceph-mon[32923]: pgmap v947: 97 pgs: 97 active+clean; 667 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.1 MiB/s wr, 608 op/s 2024-04-25T00:37:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:19 smithi106 ceph-mon[32510]: pgmap v947: 97 pgs: 97 active+clean; 667 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 6.1 MiB/s wr, 608 op/s 2024-04-25T00:37:20.767 INFO:tasks.workunit.client.0.smithi042.stdout: 10 197678 34.43 MB/sec execute 190 sec latency 99.012 ms 2024-04-25T00:37:21.767 INFO:tasks.workunit.client.0.smithi042.stdout: 10 198573 34.44 MB/sec execute 191 sec latency 56.920 ms 2024-04-25T00:37:21.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:21 smithi042 ceph-mon[28130]: pgmap v948: 97 pgs: 97 active+clean; 670 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 558 op/s 2024-04-25T00:37:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:21 smithi136 ceph-mon[32923]: pgmap v948: 97 pgs: 97 active+clean; 670 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 558 op/s 2024-04-25T00:37:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:21 smithi106 ceph-mon[32510]: pgmap v948: 97 pgs: 97 active+clean; 670 MiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.2 MiB/s wr, 558 op/s 2024-04-25T00:37:22.767 INFO:tasks.workunit.client.0.smithi042.stdout: 10 199449 34.56 MB/sec execute 192 sec latency 43.820 ms 2024-04-25T00:37:23.772 INFO:tasks.workunit.client.0.smithi042.stdout: 10 200045 34.59 MB/sec execute 193 sec latency 175.246 ms 2024-04-25T00:37:23.889 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:23 smithi106 ceph-mon[32510]: pgmap v949: 97 pgs: 97 active+clean; 677 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.5 MiB/s wr, 675 op/s 2024-04-25T00:37:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:23 smithi136 ceph-mon[32923]: pgmap v949: 97 pgs: 97 active+clean; 677 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.5 MiB/s wr, 675 op/s 2024-04-25T00:37:24.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:23 smithi042 ceph-mon[28130]: pgmap v949: 97 pgs: 97 active+clean; 677 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 6.5 MiB/s wr, 675 op/s 2024-04-25T00:37:24.772 INFO:tasks.workunit.client.0.smithi042.stdout: 10 200143 34.46 MB/sec execute 194 sec latency 913.349 ms 2024-04-25T00:37:25.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 200434 34.35 MB/sec execute 195 sec latency 103.337 ms 2024-04-25T00:37:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:25 smithi136 ceph-mon[32923]: pgmap v950: 97 pgs: 97 active+clean; 678 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.2 MiB/s wr, 567 op/s 2024-04-25T00:37:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:25 smithi106 ceph-mon[32510]: pgmap v950: 97 pgs: 97 active+clean; 678 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.2 MiB/s wr, 567 op/s 2024-04-25T00:37:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:25 smithi042 ceph-mon[28130]: pgmap v950: 97 pgs: 97 active+clean; 678 MiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 6.2 MiB/s wr, 567 op/s 2024-04-25T00:37:26.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 200784 34.24 MB/sec execute 196 sec latency 86.353 ms 2024-04-25T00:37:27.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 201395 34.21 MB/sec execute 197 sec latency 69.284 ms 2024-04-25T00:37:27.928 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:27 smithi042 ceph-mon[28130]: pgmap v951: 97 pgs: 97 active+clean; 705 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 795 op/s 2024-04-25T00:37:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:27 smithi136 ceph-mon[32923]: pgmap v951: 97 pgs: 97 active+clean; 705 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 795 op/s 2024-04-25T00:37:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:27 smithi106 ceph-mon[32510]: pgmap v951: 97 pgs: 97 active+clean; 705 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 12 MiB/s wr, 795 op/s 2024-04-25T00:37:28.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 202269 34.22 MB/sec execute 198 sec latency 45.944 ms 2024-04-25T00:37:29.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.124+0000 7f414be00640 1 -- 172.21.15.42:0/3963727166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f414c1049f0 msgr2=0x7f414c104e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.124+0000 7f414be00640 1 --2- 172.21.15.42:0/3963727166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f414c1049f0 0x7f414c104e70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f4138009900 tx=0x7f413802a410 comp rx=0 tx=0).stop 2024-04-25T00:37:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.124+0000 7f414be00640 1 -- 172.21.15.42:0/3963727166 shutdown_connections 2024-04-25T00:37:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.124+0000 7f414be00640 1 --2- 172.21.15.42:0/3963727166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f414c1053b0 0x7f414c10bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.124+0000 7f414be00640 1 --2- 172.21.15.42:0/3963727166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f414c1049f0 0x7f414c104e70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.124+0000 7f414be00640 1 --2- 172.21.15.42:0/3963727166 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f414c103800 0x7f414c103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.124+0000 7f414be00640 1 -- 172.21.15.42:0/3963727166 >> 172.21.15.42:0/3963727166 conn(0x7f414c0fed70 msgr2=0x7f414c1011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.124+0000 7f414be00640 1 -- 172.21.15.42:0/3963727166 shutdown_connections 2024-04-25T00:37:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.124+0000 7f414be00640 1 -- 172.21.15.42:0/3963727166 wait complete. 2024-04-25T00:37:29.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.126+0000 7f414be00640 1 Processor -- start 2024-04-25T00:37:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.126+0000 7f414be00640 1 -- start start 2024-04-25T00:37:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.126+0000 7f414be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f414c103800 0x7f414c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.126+0000 7f414be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f414c1049f0 0x7f414c19a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.126+0000 7f414be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f414c1053b0 0x7f414c1a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f414c10f4b0 con 0x7f414c103800 2024-04-25T00:37:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f414c10f330 con 0x7f414c1049f0 2024-04-25T00:37:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f414c10f630 con 0x7f414c1053b0 2024-04-25T00:37:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f414c1049f0 0x7f414c19a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414a000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f414c1049f0 0x7f414c19a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:41546/0 (socket says 172.21.15.42:41546) 2024-04-25T00:37:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414a000640 1 -- 172.21.15.42:0/2869499146 learned_addr learned my addr 172.21.15.42:0/2869499146 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414b400640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f414c1053b0 0x7f414c1a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414aa00640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f414c103800 0x7f414c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414a000640 1 -- 172.21.15.42:0/2869499146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f414c1053b0 msgr2=0x7f414c1a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414a000640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f414c1053b0 0x7f414c1a10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414a000640 1 -- 172.21.15.42:0/2869499146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f414c103800 msgr2=0x7f414c199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414a000640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f414c103800 0x7f414c199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.127+0000 7f414a000640 1 -- 172.21.15.42:0/2869499146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c1a1710 con 0x7f414c1049f0 2024-04-25T00:37:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.128+0000 7f414a000640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f414c1049f0 0x7f414c19a030 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4138031440 tx=0x7f4138032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.128+0000 7f4143400640 1 -- 172.21.15.42:0/2869499146 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413803f030 con 0x7f414c1049f0 2024-04-25T00:37:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.128+0000 7f414be00640 1 -- 172.21.15.42:0/2869499146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c1a19a0 con 0x7f414c1049f0 2024-04-25T00:37:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.128+0000 7f4143400640 1 -- 172.21.15.42:0/2869499146 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4138003ce0 con 0x7f414c1049f0 2024-04-25T00:37:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.128+0000 7f4143400640 1 -- 172.21.15.42:0/2869499146 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138035070 con 0x7f414c1049f0 2024-04-25T00:37:29.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.128+0000 7f414be00640 1 -- 172.21.15.42:0/2869499146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c1a1e90 con 0x7f414c1049f0 2024-04-25T00:37:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.134+0000 7f41549e2740 1 Processor -- start 2024-04-25T00:37:29.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.134+0000 7f41549e2740 1 -- start start 2024-04-25T00:37:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f41549e2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 0x561ed4e34a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f41549e2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 0x561ed4e39b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f4143400640 1 -- 172.21.15.42:0/2869499146 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4138042030 con 0x7f414c1049f0 2024-04-25T00:37:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f41549e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 0x561ed4e3c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f41549e2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561ed4d8eab0 con 0x561ed4e35050 2024-04-25T00:37:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f41549e2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561ed4d37640 con 0x561ed4e34660 2024-04-25T00:37:29.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f41549e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561ed4d8aa70 con 0x561ed4e3a1c0 2024-04-25T00:37:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f414b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 0x561ed4e34a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f414a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 0x561ed4e3c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f414b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 0x561ed4e34a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:41550/0 (socket says 172.21.15.42:41550) 2024-04-25T00:37:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.135+0000 7f414b400640 1 -- 172.21.15.42:0/741095010 learned_addr learned my addr 172.21.15.42:0/741095010 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.136+0000 7f414aa00640 1 --2- 172.21.15.42:0/741095010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 0x561ed4e39b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.136+0000 7f414a000640 1 -- 172.21.15.42:0/741095010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 msgr2=0x561ed4e34a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.136+0000 7f414a000640 1 --2- 172.21.15.42:0/741095010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 0x561ed4e34a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.136+0000 7f414a000640 1 -- 172.21.15.42:0/741095010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 msgr2=0x561ed4e39b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.136+0000 7f414a000640 1 --2- 172.21.15.42:0/741095010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 0x561ed4e39b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.136+0000 7f414a000640 1 -- 172.21.15.42:0/741095010 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ed4aa3210 con 0x561ed4e3a1c0 2024-04-25T00:37:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.137+0000 7f414a000640 1 --2- 172.21.15.42:0/741095010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 0x561ed4e3c650 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f41380eca40 tx=0x7f41380f7440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd2108b31304342c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.137+0000 7f4137e00640 1 -- 172.21.15.42:0/741095010 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41380fe070 con 0x561ed4e3a1c0 2024-04-25T00:37:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.137+0000 7f4137e00640 1 -- 172.21.15.42:0/741095010 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41380f7c20 con 0x561ed4e3a1c0 2024-04-25T00:37:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.137+0000 7f4137e00640 1 -- 172.21.15.42:0/741095010 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41380f3600 con 0x561ed4e3a1c0 2024-04-25T00:37:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.138+0000 7f4143400640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41300a2d80 0x7f41300a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.138+0000 7f4143400640 1 -- 172.21.15.42:0/2869499146 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f413803ba90 con 0x7f414c1049f0 2024-04-25T00:37:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.138+0000 7f414b400640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41300a2d80 0x7f41300a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.138+0000 7f41549e2740 1 -- 172.21.15.42:0/741095010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 msgr2=0x561ed4e3c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.138+0000 7f41549e2740 1 --2- 172.21.15.42:0/741095010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 0x561ed4e3c650 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f41380eca40 tx=0x7f41380f7440 comp rx=0 tx=0).stop 2024-04-25T00:37:29.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.139+0000 7f414b400640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41300a2d80 0x7f41300a5240 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x561ed4e358b0 tx=0x7f4144009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.140+0000 7f41549e2740 1 -- 172.21.15.42:0/741095010 shutdown_connections 2024-04-25T00:37:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.140+0000 7f41549e2740 1 --2- 172.21.15.42:0/741095010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 0x561ed4e3c650 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.140+0000 7f41549e2740 1 --2- 172.21.15.42:0/741095010 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 0x561ed4e39b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.140+0000 7f41549e2740 1 --2- 172.21.15.42:0/741095010 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 0x561ed4e34a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.140+0000 7f41549e2740 1 -- 172.21.15.42:0/741095010 >> 172.21.15.42:0/741095010 conn(0x561ed4e2fcd0 msgr2=0x561ed4e300d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.140+0000 7f41549e2740 1 -- 172.21.15.42:0/741095010 shutdown_connections 2024-04-25T00:37:29.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.141+0000 7f41549e2740 1 -- 172.21.15.42:0/741095010 wait complete. 2024-04-25T00:37:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.141+0000 7f41549e2740 1 -- start start 2024-04-25T00:37:29.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.142+0000 7f41549e2740 1 Processor -- start 2024-04-25T00:37:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.142+0000 7f41549e2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 0x561ed4e3e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.142+0000 7f41549e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e35050 0x561ed4e3eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.142+0000 7f41549e2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e45b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.142+0000 7f41549e2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x561ed4e3cf30 con 0x561ed4e34660 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.142+0000 7f41549e2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x561ed4d4a990 con 0x561ed4e3a1c0 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.142+0000 7f41549e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561ed4e3d0b0 con 0x561ed4e35050 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.142+0000 7f41549e2740 10 client.0 _ll_register_callbacks cb 0x561ed4d4c950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.142+0000 7f414b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e45b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.143+0000 7f414b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e45b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:41554/0 (socket says 172.21.15.42:41554) 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.143+0000 7f414b400640 1 -- 172.21.15.42:0/2883509600 learned_addr learned my addr 172.21.15.42:0/2883509600 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.143+0000 7f414aa00640 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 0x561ed4e3e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.143+0000 7f414a000640 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e35050 0x561ed4e3eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.143+0000 7f414b400640 1 -- 172.21.15.42:0/2883509600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e35050 msgr2=0x561ed4e3eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.143+0000 7f414b400640 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e35050 0x561ed4e3eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.143+0000 7f414b400640 1 -- 172.21.15.42:0/2883509600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 msgr2=0x561ed4e3e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.143+0000 7f414b400640 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 0x561ed4e3e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.143+0000 7f414b400640 1 -- 172.21.15.42:0/2883509600 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ed4b6f620 con 0x561ed4e3a1c0 2024-04-25T00:37:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.143+0000 7f414a000640 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e35050 0x561ed4e3eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.144+0000 7f414b400640 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e45b90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f414400f700 tx=0x7f4144004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5cec947bec98307 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.144+0000 7f4134c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:37:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.144+0000 7f4134c00640 1 -- 172.21.15.42:0/2883509600 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144003970 con 0x561ed4e3a1c0 2024-04-25T00:37:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.144+0000 7f41549e2740 1 -- 172.21.15.42:0/2883509600 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41380f7c20 con 0x561ed4e3a1c0 2024-04-25T00:37:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.145+0000 7f4134c00640 1 -- 172.21.15.42:0/2883509600 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f4144017030 con 0x561ed4e3a1c0 2024-04-25T00:37:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.145+0000 7f4134c00640 1 -- 172.21.15.42:0/2883509600 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4144003b10 con 0x561ed4e3a1c0 2024-04-25T00:37:29.147 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.145+0000 7f41549e2740 10 client.26651 fetch_fsmap learned FSMap version 45 2024-04-25T00:37:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.145+0000 7f41549e2740 1 -- 172.21.15.42:0/2883509600 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561ed4e48c60 con 0x561ed4e3a1c0 2024-04-25T00:37:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.146+0000 7f4134c00640 1 -- 172.21.15.42:0/2883509600 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41380f7c20 con 0x561ed4e3a1c0 2024-04-25T00:37:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.146+0000 7f4134c00640 1 -- 172.21.15.42:0/2883509600 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f4144004460 con 0x561ed4e3a1c0 2024-04-25T00:37:29.148 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.146+0000 7f41549e2740 10 client.26651 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:37:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.146+0000 7f41549e2740 10 client.26651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:37:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.146+0000 7f41549e2740 20 client.26651 populate_metadata read hostname 'smithi042' 2024-04-25T00:37:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.146+0000 7f41549e2740 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561ed4e4a2a0 0x561ed4e4c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.146+0000 7f41549e2740 4 client.26651 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.146+0000 7f41549e2740 1 -- 172.21.15.42:0/2883509600 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ed4d9bff0 con 0x561ed4e4a2a0 2024-04-25T00:37:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.147+0000 7f414aa00640 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561ed4e4a2a0 0x561ed4e4c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.148+0000 7f414aa00640 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561ed4e4a2a0 0x561ed4e4c760 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acf586d8a2fc82af server_cookie=22f888a949c6a305 in_seq=0 out_seq=0 2024-04-25T00:37:29.149 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.148+0000 7f4134c00640 10 client.26651 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.149+0000 7f4134c00640 1 -- 172.21.15.42:0/2883509600 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561ed4d9bff0 con 0x561ed4e4a2a0 2024-04-25T00:37:29.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.149+0000 7f4134c00640 10 client.26651 handle_command_reply: tid=0 2024-04-25T00:37:29.151 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.149+0000 7f41549e2740 1 client.26651 shutdown 2024-04-25T00:37:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.150+0000 7f41549e2740 1 -- 172.21.15.42:0/2883509600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 msgr2=0x561ed4e45b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.150+0000 7f41549e2740 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e45b90 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f414400f700 tx=0x7f4144004ab0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.150+0000 7f41549e2740 1 -- 172.21.15.42:0/2883509600 shutdown_connections 2024-04-25T00:37:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.150+0000 7f41549e2740 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561ed4e4a2a0 0x561ed4e4c760 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.150+0000 7f41549e2740 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e45b90 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.150+0000 7f41549e2740 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e35050 0x561ed4e3eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.150+0000 7f4134c00640 0 client.26651 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.150+0000 7f41549e2740 1 --2- 172.21.15.42:0/2883509600 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 0x561ed4e3e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.152 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.150+0000 7f41549e2740 1 -- 172.21.15.42:0/2883509600 >> 172.21.15.42:0/2883509600 conn(0x561ed4e2fcd0 msgr2=0x561ed4e26620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.151+0000 7f41549e2740 1 -- 172.21.15.42:0/2883509600 shutdown_connections 2024-04-25T00:37:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.151+0000 7f41549e2740 1 -- 172.21.15.42:0/2883509600 wait complete. 2024-04-25T00:37:29.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.151+0000 7f41549e2740 20 client.26651 trim_cache size 0 max 16384 2024-04-25T00:37:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.176+0000 7f41549e2740 1 Processor -- start 2024-04-25T00:37:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.176+0000 7f41549e2740 1 -- start start 2024-04-25T00:37:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f41549e2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 0x561ed4e39880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f41549e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e35050 0x561ed4e45b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f41549e2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e329b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f41549e2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f412c010c70 con 0x561ed4e34660 2024-04-25T00:37:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f41549e2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f412c00e540 con 0x561ed4e3a1c0 2024-04-25T00:37:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f41549e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f412c00c390 con 0x561ed4e35050 2024-04-25T00:37:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f414b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e329b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f414b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e329b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:41560/0 (socket says 172.21.15.42:41560) 2024-04-25T00:37:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f414b400640 1 -- 172.21.15.42:0/56675435 learned_addr learned my addr 172.21.15.42:0/56675435 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f414aa00640 1 --2- 172.21.15.42:0/56675435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 0x561ed4e39880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.177+0000 7f414b400640 1 -- 172.21.15.42:0/56675435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e35050 msgr2=0x561ed4e45b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.178+0000 7f414b400640 1 --2- 172.21.15.42:0/56675435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e35050 0x561ed4e45b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.178+0000 7f414b400640 1 -- 172.21.15.42:0/56675435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 msgr2=0x561ed4e39880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.178+0000 7f414b400640 1 --2- 172.21.15.42:0/56675435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 0x561ed4e39880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.178+0000 7f414b400640 1 -- 172.21.15.42:0/56675435 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ed4e4cde0 con 0x561ed4e3a1c0 2024-04-25T00:37:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.178+0000 7f414aa00640 1 --2- 172.21.15.42:0/56675435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 0x561ed4e39880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.178+0000 7f414b400640 1 --2- 172.21.15.42:0/56675435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e329b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4144009d90 tx=0x7f4144011470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9422948704026784 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.179+0000 7f4137400640 1 -- 172.21.15.42:0/56675435 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4144038030 con 0x561ed4e3a1c0 2024-04-25T00:37:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.179+0000 7f4137400640 1 -- 172.21.15.42:0/56675435 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41440336c0 con 0x561ed4e3a1c0 2024-04-25T00:37:29.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.179+0000 7f4137400640 1 -- 172.21.15.42:0/56675435 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414401fca0 con 0x561ed4e3a1c0 2024-04-25T00:37:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.180+0000 7f41549e2740 1 -- 172.21.15.42:0/56675435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 msgr2=0x561ed4e329b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.180+0000 7f41549e2740 1 --2- 172.21.15.42:0/56675435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e329b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4144009d90 tx=0x7f4144011470 comp rx=0 tx=0).stop 2024-04-25T00:37:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.180+0000 7f41549e2740 1 -- 172.21.15.42:0/56675435 shutdown_connections 2024-04-25T00:37:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.180+0000 7f41549e2740 1 --2- 172.21.15.42:0/56675435 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e3a1c0 0x561ed4e329b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.180+0000 7f41549e2740 1 --2- 172.21.15.42:0/56675435 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e35050 0x561ed4e45b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.180+0000 7f41549e2740 1 --2- 172.21.15.42:0/56675435 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e34660 0x561ed4e39880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.180+0000 7f41549e2740 1 -- 172.21.15.42:0/56675435 >> 172.21.15.42:0/56675435 conn(0x561ed4e2fcd0 msgr2=0x561ed4e22110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.182 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.181+0000 7f41549e2740 1 -- 172.21.15.42:0/56675435 shutdown_connections 2024-04-25T00:37:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.181+0000 7f41549e2740 1 -- 172.21.15.42:0/56675435 wait complete. 2024-04-25T00:37:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.181+0000 7f41549e2740 1 -- start start 2024-04-25T00:37:29.183 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.182+0000 7f41549e2740 1 Processor -- start 2024-04-25T00:37:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.182+0000 7f41549e2740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 0x561ed4e45590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.182+0000 7f41549e2740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 0x561ed4e45ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.182+0000 7f41549e2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 0x561ed4e43c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.182+0000 7f41549e2740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f412c00ca00 con 0x561ed4e35050 2024-04-25T00:37:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.182+0000 7f41549e2740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f412c00de70 con 0x561ed4e34660 2024-04-25T00:37:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.182+0000 7f41549e2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f412c00d0d0 con 0x561ed4e3a1c0 2024-04-25T00:37:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.182+0000 7f41549e2740 10 client.0 _ll_register_callbacks cb 0x7f4144009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:29.184 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 0x561ed4e45ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 0x561ed4e45ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:34964/0 (socket says 172.21.15.42:34964) 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414a000640 1 -- 172.21.15.42:0/1075566059 learned_addr learned my addr 172.21.15.42:0/1075566059 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414b400640 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 0x561ed4e43c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414aa00640 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 0x561ed4e45590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414a000640 1 -- 172.21.15.42:0/1075566059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 msgr2=0x561ed4e43c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414a000640 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 0x561ed4e43c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414a000640 1 -- 172.21.15.42:0/1075566059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 msgr2=0x561ed4e45590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414a000640 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 0x561ed4e45590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414a000640 1 -- 172.21.15.42:0/1075566059 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ed4e33c10 con 0x561ed4e35050 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414b400640 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 0x561ed4e43c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:29.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.183+0000 7f414aa00640 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 0x561ed4e45590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.184+0000 7f414a000640 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 0x561ed4e45ad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f41380b7c30 tx=0x7f4138005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92f949040ca62f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.184+0000 7f4134c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:37:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.184+0000 7f4134c00640 1 -- 172.21.15.42:0/1075566059 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138035030 con 0x561ed4e35050 2024-04-25T00:37:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.184+0000 7f41549e2740 1 -- 172.21.15.42:0/1075566059 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41440336c0 con 0x561ed4e35050 2024-04-25T00:37:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.184+0000 7f4134c00640 1 -- 172.21.15.42:0/1075566059 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f41380f25f0 con 0x561ed4e35050 2024-04-25T00:37:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.184+0000 7f4134c00640 1 -- 172.21.15.42:0/1075566059 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4138049070 con 0x561ed4e35050 2024-04-25T00:37:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.184+0000 7f41549e2740 10 client.16440 fetch_fsmap learned FSMap version 45 2024-04-25T00:37:29.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.184+0000 7f41549e2740 1 -- 172.21.15.42:0/1075566059 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x561ed4e35d20 con 0x561ed4e35050 2024-04-25T00:37:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.185+0000 7f4134c00640 1 -- 172.21.15.42:0/1075566059 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41440336c0 con 0x561ed4e35050 2024-04-25T00:37:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.185+0000 7f4134c00640 1 -- 172.21.15.42:0/1075566059 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f413803bcd0 con 0x561ed4e35050 2024-04-25T00:37:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.185+0000 7f41549e2740 10 client.16440 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:37:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.185+0000 7f41549e2740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:37:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.185+0000 7f41549e2740 20 client.16440 populate_metadata read hostname 'smithi042' 2024-04-25T00:37:29.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.185+0000 7f41549e2740 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561ed4ec2850 0x561ed4ec4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.185+0000 7f41549e2740 4 client.16440 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:37:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.185+0000 7f41549e2740 1 -- 172.21.15.42:0/1075566059 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ed4d96340 con 0x561ed4ec2850 2024-04-25T00:37:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.185+0000 7f414aa00640 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561ed4ec2850 0x561ed4ec4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.186+0000 7f414aa00640 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561ed4ec2850 0x561ed4ec4c70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0b4ca14d7c8348 server_cookie=ac70a520fd125c9c in_seq=0 out_seq=0 2024-04-25T00:37:29.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.186+0000 7f4134c00640 10 client.16440 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.416+0000 7f4134c00640 1 -- 172.21.15.42:0/1075566059 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561ed4d96340 con 0x561ed4ec2850 2024-04-25T00:37:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.416+0000 7f4134c00640 10 client.16440 handle_command_reply: tid=0 2024-04-25T00:37:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.417+0000 7f41549e2740 1 client.16440 shutdown 2024-04-25T00:37:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.417+0000 7f41549e2740 1 -- 172.21.15.42:0/1075566059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 msgr2=0x561ed4e45ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.417+0000 7f41549e2740 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 0x561ed4e45ad0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f41380b7c30 tx=0x7f4138005b50 comp rx=0 tx=0).stop 2024-04-25T00:37:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.417+0000 7f41549e2740 1 -- 172.21.15.42:0/1075566059 shutdown_connections 2024-04-25T00:37:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.417+0000 7f41549e2740 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x561ed4ec2850 0x561ed4ec4c70 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.417+0000 7f41549e2740 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561ed4e3a1c0 0x561ed4e43c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.417+0000 7f41549e2740 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x561ed4e35050 0x561ed4e45ad0 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f41380b7c30 tx=0x7f4138005b50 comp rx=0 tx=0).stop 2024-04-25T00:37:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.417+0000 7f41549e2740 1 --2- 172.21.15.42:0/1075566059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x561ed4e34660 0x561ed4e45590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.417+0000 7f41549e2740 1 -- 172.21.15.42:0/1075566059 >> 172.21.15.42:0/1075566059 conn(0x561ed4e2fcd0 msgr2=0x561ed4e36230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.417+0000 7f4134c00640 0 client.16440 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.418+0000 7f41549e2740 1 -- 172.21.15.42:0/1075566059 shutdown_connections 2024-04-25T00:37:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.418+0000 7f41549e2740 1 -- 172.21.15.42:0/1075566059 wait complete. 2024-04-25T00:37:29.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.419+0000 7f41549e2740 20 client.16440 trim_cache size 0 max 16384 2024-04-25T00:37:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:37:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:37:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:37:29.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:37:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.421+0000 7f414be00640 1 -- 172.21.15.42:0/2869499146 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41300a2d80 msgr2=0x7f41300a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.421+0000 7f414be00640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41300a2d80 0x7f41300a5240 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x561ed4e358b0 tx=0x7f4144009500 comp rx=0 tx=0).stop 2024-04-25T00:37:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.421+0000 7f414be00640 1 -- 172.21.15.42:0/2869499146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f414c1049f0 msgr2=0x7f414c19a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:29.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.421+0000 7f414be00640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f414c1049f0 0x7f414c19a030 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f4138031440 tx=0x7f4138032ce0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.422+0000 7f414be00640 1 -- 172.21.15.42:0/2869499146 shutdown_connections 2024-04-25T00:37:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.422+0000 7f414be00640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41300a2d80 0x7f41300a5240 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.422+0000 7f414be00640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f414c1053b0 0x7f414c1a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.422+0000 7f414be00640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f414c1049f0 0x7f414c19a030 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.422+0000 7f414be00640 1 --2- 172.21.15.42:0/2869499146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f414c103800 0x7f414c199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:29.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.422+0000 7f414be00640 1 -- 172.21.15.42:0/2869499146 >> 172.21.15.42:0/2869499146 conn(0x7f414c0fed70 msgr2=0x7f414c107cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.422+0000 7f414be00640 1 -- 172.21.15.42:0/2869499146 shutdown_connections 2024-04-25T00:37:29.424 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:29.422+0000 7f414be00640 1 -- 172.21.15.42:0/2869499146 wait complete. 2024-04-25T00:37:29.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 203017 34.29 MB/sec execute 199 sec latency 136.435 ms 2024-04-25T00:37:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:29 smithi136 ceph-mon[32923]: pgmap v952: 97 pgs: 97 active+clean; 701 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 779 op/s 2024-04-25T00:37:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:29 smithi106 ceph-mon[32510]: pgmap v952: 97 pgs: 97 active+clean; 701 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 779 op/s 2024-04-25T00:37:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:29 smithi042 ceph-mon[28130]: pgmap v952: 97 pgs: 97 active+clean; 701 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 13 MiB/s wr, 779 op/s 2024-04-25T00:37:30.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 203733 34.39 MB/sec execute 200 sec latency 69.418 ms 2024-04-25T00:37:31.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 204180 34.33 MB/sec execute 201 sec latency 72.744 ms 2024-04-25T00:37:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:31 smithi136 ceph-mon[32923]: pgmap v953: 97 pgs: 97 active+clean; 693 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 818 op/s 2024-04-25T00:37:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:31 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:37:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:31 smithi106 ceph-mon[32510]: pgmap v953: 97 pgs: 97 active+clean; 693 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 818 op/s 2024-04-25T00:37:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:31 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:37:32.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:31 smithi042 ceph-mon[28130]: pgmap v953: 97 pgs: 97 active+clean; 693 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 13 MiB/s wr, 818 op/s 2024-04-25T00:37:32.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:31 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:37:32.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 204647 34.22 MB/sec execute 202 sec latency 93.119 ms 2024-04-25T00:37:33.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 205444 34.23 MB/sec execute 203 sec latency 50.334 ms 2024-04-25T00:37:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:33 smithi136 ceph-mon[32923]: pgmap v954: 97 pgs: 97 active+clean; 672 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 810 op/s 2024-04-25T00:37:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:33 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:37:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:33 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:37:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:33 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:37:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:33 smithi106 ceph-mon[32510]: pgmap v954: 97 pgs: 97 active+clean; 672 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 810 op/s 2024-04-25T00:37:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:33 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:37:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:33 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:37:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:33 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:37:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:33 smithi042 ceph-mon[28130]: pgmap v954: 97 pgs: 97 active+clean; 672 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 810 op/s 2024-04-25T00:37:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:33 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:37:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:33 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:37:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:33 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:37:34.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 206315 34.31 MB/sec execute 204 sec latency 105.811 ms 2024-04-25T00:37:35.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 207174 34.45 MB/sec execute 205 sec latency 39.471 ms 2024-04-25T00:37:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:35 smithi136 ceph-mon[32923]: pgmap v955: 97 pgs: 97 active+clean; 671 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-25T00:37:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:35 smithi106 ceph-mon[32510]: pgmap v955: 97 pgs: 97 active+clean; 671 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-25T00:37:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:35 smithi042 ceph-mon[28130]: pgmap v955: 97 pgs: 97 active+clean; 671 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 693 op/s 2024-04-25T00:37:36.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 207664 34.42 MB/sec execute 206 sec latency 59.533 ms 2024-04-25T00:37:37.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 208179 34.32 MB/sec execute 207 sec latency 53.727 ms 2024-04-25T00:37:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:37 smithi136 ceph-mon[32923]: pgmap v956: 97 pgs: 97 active+clean; 671 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 818 op/s 2024-04-25T00:37:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:37 smithi106 ceph-mon[32510]: pgmap v956: 97 pgs: 97 active+clean; 671 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 818 op/s 2024-04-25T00:37:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:37 smithi042 ceph-mon[28130]: pgmap v956: 97 pgs: 97 active+clean; 671 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 11 MiB/s wr, 818 op/s 2024-04-25T00:37:38.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 209091 34.36 MB/sec execute 208 sec latency 58.073 ms 2024-04-25T00:37:39.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 210024 34.46 MB/sec execute 209 sec latency 61.639 ms 2024-04-25T00:37:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:39 smithi136 ceph-mon[32923]: pgmap v957: 97 pgs: 97 active+clean; 675 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 604 op/s 2024-04-25T00:37:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:39 smithi106 ceph-mon[32510]: pgmap v957: 97 pgs: 97 active+clean; 675 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 604 op/s 2024-04-25T00:37:40.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:39 smithi042 ceph-mon[28130]: pgmap v957: 97 pgs: 97 active+clean; 675 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 604 op/s 2024-04-25T00:37:40.774 INFO:tasks.workunit.client.0.smithi042.stdout: 10 210742 34.53 MB/sec execute 210 sec latency 63.244 ms 2024-04-25T00:37:41.773 INFO:tasks.workunit.client.0.smithi042.stdout: 10 211249 34.49 MB/sec execute 211 sec latency 98.132 ms 2024-04-25T00:37:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:41 smithi136 ceph-mon[32923]: pgmap v958: 97 pgs: 97 active+clean; 677 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 531 op/s 2024-04-25T00:37:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:41 smithi106 ceph-mon[32510]: pgmap v958: 97 pgs: 97 active+clean; 677 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 531 op/s 2024-04-25T00:37:42.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:41 smithi042 ceph-mon[28130]: pgmap v958: 97 pgs: 97 active+clean; 677 MiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.7 MiB/s wr, 531 op/s 2024-04-25T00:37:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.627+0000 7fe596200640 1 -- 172.21.15.42:0/2312652774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe590109650 msgr2=0x7fe59010bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.630 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.627+0000 7fe596200640 1 --2- 172.21.15.42:0/2312652774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe590109650 0x7fe59010bae0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe5880133d0 tx=0x7fe588032ad0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.627+0000 7fe596200640 1 -- 172.21.15.42:0/2312652774 shutdown_connections 2024-04-25T00:37:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.627+0000 7fe596200640 1 --2- 172.21.15.42:0/2312652774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe590109650 0x7fe59010bae0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.627+0000 7fe596200640 1 --2- 172.21.15.42:0/2312652774 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590074120 0x7fe590074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.631 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.627+0000 7fe596200640 1 --2- 172.21.15.42:0/2312652774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590075700 0x7fe590073b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.627+0000 7fe596200640 1 -- 172.21.15.42:0/2312652774 >> 172.21.15.42:0/2312652774 conn(0x7fe5900fcd30 msgr2=0x7fe5900ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:42.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.631+0000 7fe596200640 1 -- 172.21.15.42:0/2312652774 shutdown_connections 2024-04-25T00:37:42.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.633+0000 7fe596200640 1 -- 172.21.15.42:0/2312652774 wait complete. 2024-04-25T00:37:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.634+0000 7fe596200640 1 Processor -- start 2024-04-25T00:37:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.634+0000 7fe596200640 1 -- start start 2024-04-25T00:37:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.634+0000 7fe596200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590074120 0x7fe59006f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.634+0000 7fe596200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590075700 0x7fe59006f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.635+0000 7fe58ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590075700 0x7fe59006f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.635+0000 7fe58ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590075700 0x7fe59006f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48496/0 (socket says 172.21.15.42:48496) 2024-04-25T00:37:42.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.635+0000 7fe58ea00640 1 -- 172.21.15.42:0/1862843587 learned_addr learned my addr 172.21.15.42:0/1862843587 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.636+0000 7fe58f400640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590074120 0x7fe59006f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.636+0000 7fe596200640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe590109650 0x7fe590072730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.636+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe590103e50 con 0x7fe590074120 2024-04-25T00:37:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.636+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe590103cd0 con 0x7fe590109650 2024-04-25T00:37:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.636+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe590103fd0 con 0x7fe590075700 2024-04-25T00:37:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.637+0000 7fe58fe00640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe590109650 0x7fe590072730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.637+0000 7fe58f400640 1 -- 172.21.15.42:0/1862843587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590075700 msgr2=0x7fe59006f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.637+0000 7fe58f400640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590075700 0x7fe59006f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.637+0000 7fe58f400640 1 -- 172.21.15.42:0/1862843587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe590109650 msgr2=0x7fe590072730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.637+0000 7fe58f400640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe590109650 0x7fe590072730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.637+0000 7fe58f400640 1 -- 172.21.15.42:0/1862843587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe590072da0 con 0x7fe590074120 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.637+0000 7fe58ea00640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590075700 0x7fe59006f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.637+0000 7fe58fe00640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe590109650 0x7fe590072730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.638+0000 7fe58f400640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590074120 0x7fe59006f1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe58000d6d0 tx=0x7fe58000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.642+0000 7fe587e00640 1 -- 172.21.15.42:0/1862843587 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580017070 con 0x7fe590074120 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.642+0000 7fe587e00640 1 -- 172.21.15.42:0/1862843587 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe580010460 con 0x7fe590074120 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.642+0000 7fe587e00640 1 -- 172.21.15.42:0/1862843587 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58000f620 con 0x7fe590074120 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.642+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe590073090 con 0x7fe590074120 2024-04-25T00:37:42.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.642+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe590073590 con 0x7fe590074120 2024-04-25T00:37:42.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.651+0000 7fe587e00640 1 -- 172.21.15.42:0/1862843587 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe58001a070 con 0x7fe590074120 2024-04-25T00:37:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.655+0000 7fe598fd4740 1 Processor -- start 2024-04-25T00:37:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.656+0000 7fe598fd4740 1 -- start start 2024-04-25T00:37:42.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.656+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 0x555a8e9300c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.656+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e935190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.656+0000 7fe587e00640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5740a2da0 0x7fe5740a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.657+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e937c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.657+0000 7fe598fd4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555a8e892ab0 con 0x555a8e92fca0 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.657+0000 7fe598fd4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555a8e83b640 con 0x555a8e935800 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.657+0000 7fe598fd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555a8e88ea70 con 0x555a8e930690 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.657+0000 7fe58fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 0x555a8e9300c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.657+0000 7fe58fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e937c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.657+0000 7fe58fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e937c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:50444/0 (socket says 172.21.15.42:50444) 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.657+0000 7fe58fe00640 1 -- 172.21.15.42:0/2392654249 learned_addr learned my addr 172.21.15.42:0/2392654249 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:42.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.657+0000 7fe587e00640 1 -- 172.21.15.42:0/1862843587 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fe5800ce7d0 con 0x7fe590074120 2024-04-25T00:37:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.657+0000 7fe58f400640 1 --2- 172.21.15.42:0/2392654249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e935190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.658+0000 7fe58fe00640 1 -- 172.21.15.42:0/2392654249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 msgr2=0x555a8e935190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.658+0000 7fe58fe00640 1 --2- 172.21.15.42:0/2392654249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e935190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.658+0000 7fe58fe00640 1 -- 172.21.15.42:0/2392654249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 msgr2=0x555a8e9300c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.658+0000 7fe58fe00640 1 --2- 172.21.15.42:0/2392654249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 0x555a8e9300c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.658+0000 7fe58fe00640 1 -- 172.21.15.42:0/2392654249 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8e5a7210 con 0x555a8e935800 2024-04-25T00:37:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.659+0000 7fe58ea00640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5740a2da0 0x7fe5740a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.664 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.659+0000 7fe58fe00640 1 --2- 172.21.15.42:0/2392654249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e937c90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe588013210 tx=0x7fe588014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc2e3e700f546c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.660+0000 7fe57be00640 1 -- 172.21.15.42:0/2392654249 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58804a040 con 0x555a8e935800 2024-04-25T00:37:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.660+0000 7fe57be00640 1 -- 172.21.15.42:0/2392654249 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe588034e70 con 0x555a8e935800 2024-04-25T00:37:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.660+0000 7fe57be00640 1 -- 172.21.15.42:0/2392654249 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe58803a040 con 0x555a8e935800 2024-04-25T00:37:42.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.661+0000 7fe598fd4740 1 -- 172.21.15.42:0/2392654249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 msgr2=0x555a8e937c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.661+0000 7fe598fd4740 1 --2- 172.21.15.42:0/2392654249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e937c90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe588013210 tx=0x7fe588014000 comp rx=0 tx=0).stop 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.661+0000 7fe58ea00640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5740a2da0 0x7fe5740a5260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe5900706b0 tx=0x7fe57c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.662+0000 7fe598fd4740 1 -- 172.21.15.42:0/2392654249 shutdown_connections 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.662+0000 7fe598fd4740 1 --2- 172.21.15.42:0/2392654249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e937c90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.662+0000 7fe598fd4740 1 --2- 172.21.15.42:0/2392654249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e935190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.662+0000 7fe598fd4740 1 --2- 172.21.15.42:0/2392654249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 0x555a8e9300c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.662+0000 7fe598fd4740 1 -- 172.21.15.42:0/2392654249 >> 172.21.15.42:0/2392654249 conn(0x555a8e92b310 msgr2=0x555a8e92b710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.663+0000 7fe598fd4740 1 -- 172.21.15.42:0/2392654249 shutdown_connections 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.663+0000 7fe598fd4740 1 -- 172.21.15.42:0/2392654249 wait complete. 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.663+0000 7fe598fd4740 1 -- start start 2024-04-25T00:37:42.667 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.664+0000 7fe598fd4740 1 Processor -- start 2024-04-25T00:37:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e92fca0 0x555a8e939f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e93a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.668 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 0x555a8e9414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe598fd4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x555a8e938750 con 0x555a8e935800 2024-04-25T00:37:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe598fd4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x555a8e84e990 con 0x555a8e92fca0 2024-04-25T00:37:42.669 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe598fd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555a8e9388d0 con 0x555a8e930690 2024-04-25T00:37:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe58f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e92fca0 0x555a8e939f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe58f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e92fca0 0x555a8e939f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:50452/0 (socket says 172.21.15.42:50452) 2024-04-25T00:37:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe58f400640 1 -- 172.21.15.42:0/3114161161 learned_addr learned my addr 172.21.15.42:0/3114161161 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe598fd4740 10 client.0 _ll_register_callbacks cb 0x555a8e850950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:42.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.665+0000 7fe58ea00640 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e93a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.666+0000 7fe58fe00640 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 0x555a8e9414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.666+0000 7fe58ea00640 1 -- 172.21.15.42:0/3114161161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e92fca0 msgr2=0x555a8e939f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.666+0000 7fe58ea00640 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e92fca0 0x555a8e939f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.666+0000 7fe58ea00640 1 -- 172.21.15.42:0/3114161161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 msgr2=0x555a8e9414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.666+0000 7fe58ea00640 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 0x555a8e9414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.666+0000 7fe58ea00640 1 -- 172.21.15.42:0/3114161161 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8e673620 con 0x555a8e930690 2024-04-25T00:37:42.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.666+0000 7fe58fe00640 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 0x555a8e9414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.666+0000 7fe58ea00640 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e93a470 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00afd0 tx=0x7fe57c004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7876036b3606c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.666+0000 7fe58f400640 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e92fca0 0x555a8e939f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.666+0000 7fe578c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:37:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.667+0000 7fe578c00640 1 -- 172.21.15.42:0/3114161161 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c016070 con 0x555a8e930690 2024-04-25T00:37:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.667+0000 7fe578c00640 1 -- 172.21.15.42:0/3114161161 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fe57c027680 con 0x555a8e930690 2024-04-25T00:37:42.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.667+0000 7fe598fd4740 1 -- 172.21.15.42:0/3114161161 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe588034e70 con 0x555a8e930690 2024-04-25T00:37:42.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.671+0000 7fe578c00640 1 -- 172.21.15.42:0/3114161161 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe57c027880 con 0x555a8e930690 2024-04-25T00:37:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.671+0000 7fe598fd4740 10 client.26644 fetch_fsmap learned FSMap version 45 2024-04-25T00:37:42.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.672+0000 7fe598fd4740 1 -- 172.21.15.42:0/3114161161 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555a8e944560 con 0x555a8e930690 2024-04-25T00:37:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.672+0000 7fe578c00640 1 -- 172.21.15.42:0/3114161161 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe588034e70 con 0x555a8e930690 2024-04-25T00:37:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.672+0000 7fe578c00640 1 -- 172.21.15.42:0/3114161161 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fe57c031ca0 con 0x555a8e930690 2024-04-25T00:37:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.673+0000 7fe598fd4740 10 client.26644 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:37:42.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.673+0000 7fe598fd4740 10 client.26644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:37:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.673+0000 7fe598fd4740 20 client.26644 populate_metadata read hostname 'smithi042' 2024-04-25T00:37:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.673+0000 7fe598fd4740 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555a8e945be0 0x555a8e948050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.673+0000 7fe598fd4740 4 client.26644 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.673+0000 7fe598fd4740 1 -- 172.21.15.42:0/3114161161 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555a8e89fff0 con 0x555a8e945be0 2024-04-25T00:37:42.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.673+0000 7fe58f400640 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555a8e945be0 0x555a8e948050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.673+0000 7fe58f400640 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555a8e945be0 0x555a8e948050 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=863c468c25885db4 server_cookie=4a2fc61ee43274a8 in_seq=0 out_seq=0 2024-04-25T00:37:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.673+0000 7fe578c00640 10 client.26644 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.674+0000 7fe578c00640 1 -- 172.21.15.42:0/3114161161 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555a8e89fff0 con 0x555a8e945be0 2024-04-25T00:37:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.674+0000 7fe578c00640 10 client.26644 handle_command_reply: tid=0 2024-04-25T00:37:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.674+0000 7fe598fd4740 1 client.26644 shutdown 2024-04-25T00:37:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 1 -- 172.21.15.42:0/3114161161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 msgr2=0x555a8e93a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e93a470 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe57c00afd0 tx=0x7fe57c004730 comp rx=0 tx=0).stop 2024-04-25T00:37:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 1 -- 172.21.15.42:0/3114161161 shutdown_connections 2024-04-25T00:37:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555a8e945be0 0x555a8e948050 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 0x555a8e9414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e93a470 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 1 --2- 172.21.15.42:0/3114161161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e92fca0 0x555a8e939f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 1 -- 172.21.15.42:0/3114161161 >> 172.21.15.42:0/3114161161 conn(0x555a8e92b310 msgr2=0x555a8e92a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe578c00640 0 client.26644 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 1 -- 172.21.15.42:0/3114161161 shutdown_connections 2024-04-25T00:37:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 1 -- 172.21.15.42:0/3114161161 wait complete. 2024-04-25T00:37:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.675+0000 7fe598fd4740 20 client.26644 trim_cache size 0 max 16384 2024-04-25T00:37:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.701+0000 7fe598fd4740 1 Processor -- start 2024-04-25T00:37:42.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe598fd4740 1 -- start start 2024-04-25T00:37:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 0x555a8e925fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e941470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e947a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe598fd4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901b1a80 con 0x555a8e92fca0 2024-04-25T00:37:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe598fd4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901af350 con 0x555a8e935800 2024-04-25T00:37:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe598fd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901ad1a0 con 0x555a8e930690 2024-04-25T00:37:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe58f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 0x555a8e925fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe58f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 0x555a8e925fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47052/0 (socket says 172.21.15.42:47052) 2024-04-25T00:37:42.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe58f400640 1 -- 172.21.15.42:0/4012810544 learned_addr learned my addr 172.21.15.42:0/4012810544 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.702+0000 7fe58fe00640 1 --2- 172.21.15.42:0/4012810544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e947a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.703+0000 7fe58f400640 1 -- 172.21.15.42:0/4012810544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 msgr2=0x555a8e941470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:37:42.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.703+0000 7fe58f400640 1 --2- 172.21.15.42:0/4012810544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e941470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.703+0000 7fe58f400640 1 -- 172.21.15.42:0/4012810544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 msgr2=0x555a8e947a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.703+0000 7fe58f400640 1 --2- 172.21.15.42:0/4012810544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e947a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.703+0000 7fe58f400640 1 -- 172.21.15.42:0/4012810544 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a8e92b140 con 0x555a8e92fca0 2024-04-25T00:37:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.703+0000 7fe58fe00640 1 --2- 172.21.15.42:0/4012810544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e947a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.704+0000 7fe58f400640 1 --2- 172.21.15.42:0/4012810544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 0x555a8e925fd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe580009610 tx=0x7fe5800bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9eeb2c8447fc7e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.704+0000 7fe57b400640 1 -- 172.21.15.42:0/4012810544 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580021040 con 0x555a8e92fca0 2024-04-25T00:37:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.704+0000 7fe57b400640 1 -- 172.21.15.42:0/4012810544 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58009f410 con 0x555a8e92fca0 2024-04-25T00:37:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.704+0000 7fe57b400640 1 -- 172.21.15.42:0/4012810544 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe580097780 con 0x555a8e92fca0 2024-04-25T00:37:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.705+0000 7fe598fd4740 1 -- 172.21.15.42:0/4012810544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 msgr2=0x555a8e925fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.705+0000 7fe598fd4740 1 --2- 172.21.15.42:0/4012810544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 0x555a8e925fd0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fe580009610 tx=0x7fe5800bb040 comp rx=0 tx=0).stop 2024-04-25T00:37:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.706+0000 7fe598fd4740 1 -- 172.21.15.42:0/4012810544 shutdown_connections 2024-04-25T00:37:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.706+0000 7fe598fd4740 1 --2- 172.21.15.42:0/4012810544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e935800 0x555a8e947a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.706+0000 7fe598fd4740 1 --2- 172.21.15.42:0/4012810544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e930690 0x555a8e941470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.706+0000 7fe598fd4740 1 --2- 172.21.15.42:0/4012810544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e92fca0 0x555a8e925fd0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.706+0000 7fe598fd4740 1 -- 172.21.15.42:0/4012810544 >> 172.21.15.42:0/4012810544 conn(0x555a8e92b310 msgr2=0x555a8e92b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.706+0000 7fe598fd4740 1 -- 172.21.15.42:0/4012810544 shutdown_connections 2024-04-25T00:37:42.709 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.706+0000 7fe598fd4740 1 -- 172.21.15.42:0/4012810544 wait complete. 2024-04-25T00:37:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.706+0000 7fe598fd4740 1 -- start start 2024-04-25T00:37:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.707+0000 7fe598fd4740 1 Processor -- start 2024-04-25T00:37:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.708+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e92fca0 0x555a8e93e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.708+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e930690 0x555a8e93ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.708+0000 7fe598fd4740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 0x555a8e93fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.708+0000 7fe598fd4740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901ad810 con 0x555a8e935800 2024-04-25T00:37:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.708+0000 7fe598fd4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901aec80 con 0x555a8e930690 2024-04-25T00:37:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.708+0000 7fe598fd4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5901adee0 con 0x555a8e92fca0 2024-04-25T00:37:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.708+0000 7fe598fd4740 10 client.0 _ll_register_callbacks cb 0x7fe57c005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:42.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 0x555a8e93fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e930690 0x555a8e93ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e930690 0x555a8e93ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:50480/0 (socket says 172.21.15.42:50480) 2024-04-25T00:37:42.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58ea00640 1 -- 172.21.15.42:0/1540839509 learned_addr learned my addr 172.21.15.42:0/1540839509 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:42.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58f400640 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e92fca0 0x555a8e93e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58ea00640 1 -- 172.21.15.42:0/1540839509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e92fca0 msgr2=0x555a8e93e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58ea00640 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e92fca0 0x555a8e93e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58ea00640 1 -- 172.21.15.42:0/1540839509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 msgr2=0x555a8e93fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58ea00640 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 0x555a8e93fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.723 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58ea00640 1 -- 172.21.15.42:0/1540839509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a8e940270 con 0x555a8e930690 2024-04-25T00:37:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.709+0000 7fe58fe00640 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 0x555a8e93fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:37:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.710+0000 7fe58ea00640 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e930690 0x555a8e93ec70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe57c01c610 tx=0x7fe57c027690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5c2946a9c353482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.710+0000 7fe578c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:37:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.711+0000 7fe578c00640 1 -- 172.21.15.42:0/1540839509 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe57c027a20 con 0x555a8e930690 2024-04-25T00:37:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.711+0000 7fe578c00640 1 -- 172.21.15.42:0/1540839509 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fe57c027bc0 con 0x555a8e930690 2024-04-25T00:37:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.711+0000 7fe598fd4740 1 -- 172.21.15.42:0/1540839509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe58009f410 con 0x555a8e930690 2024-04-25T00:37:42.724 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.711+0000 7fe578c00640 1 -- 172.21.15.42:0/1540839509 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe57c027dc0 con 0x555a8e930690 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.711+0000 7fe598fd4740 10 client.26672 fetch_fsmap learned FSMap version 45 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.711+0000 7fe598fd4740 1 -- 172.21.15.42:0/1540839509 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x555a8e941520 con 0x555a8e930690 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.712+0000 7fe578c00640 1 -- 172.21.15.42:0/1540839509 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe58009f410 con 0x555a8e930690 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.712+0000 7fe578c00640 1 -- 172.21.15.42:0/1540839509 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fe57c044420 con 0x555a8e930690 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.712+0000 7fe598fd4740 10 client.26672 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.712+0000 7fe598fd4740 10 client.26672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.712+0000 7fe598fd4740 20 client.26672 populate_metadata read hostname 'smithi042' 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.712+0000 7fe598fd4740 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555a8e9bd6c0 0x555a8e9bfae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.712+0000 7fe598fd4740 4 client.26672 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.712+0000 7fe598fd4740 1 -- 172.21.15.42:0/1540839509 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555a8e89a340 con 0x555a8e9bd6c0 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.713+0000 7fe58f400640 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555a8e9bd6c0 0x555a8e9bfae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.715+0000 7fe58f400640 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555a8e9bd6c0 0x555a8e9bfae0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a299b323271b6518 server_cookie=b5426707dfeda76e in_seq=0 out_seq=0 2024-04-25T00:37:42.725 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.715+0000 7fe578c00640 10 client.26672 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:42.775 INFO:tasks.workunit.client.0.smithi042.stdout: 10 211836 34.44 MB/sec execute 212 sec latency 74.401 ms 2024-04-25T00:37:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe578c00640 1 -- 172.21.15.42:0/1540839509 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555a8e89a340 con 0x555a8e9bd6c0 2024-04-25T00:37:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe578c00640 10 client.26672 handle_command_reply: tid=0 2024-04-25T00:37:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe598fd4740 1 client.26672 shutdown 2024-04-25T00:37:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe598fd4740 1 -- 172.21.15.42:0/1540839509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e930690 msgr2=0x555a8e93ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe598fd4740 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e930690 0x555a8e93ec70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fe57c01c610 tx=0x7fe57c027690 comp rx=0 tx=0).stop 2024-04-25T00:37:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe598fd4740 1 -- 172.21.15.42:0/1540839509 shutdown_connections 2024-04-25T00:37:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe598fd4740 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x555a8e9bd6c0 0x555a8e9bfae0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe598fd4740 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x555a8e935800 0x555a8e93fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe598fd4740 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x555a8e930690 0x555a8e93ec70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe598fd4740 1 --2- 172.21.15.42:0/1540839509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555a8e92fca0 0x555a8e93e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe598fd4740 1 -- 172.21.15.42:0/1540839509 >> 172.21.15.42:0/1540839509 conn(0x555a8e92b310 msgr2=0x555a8e925fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.968+0000 7fe578c00640 0 client.26672 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.969+0000 7fe598fd4740 1 -- 172.21.15.42:0/1540839509 shutdown_connections 2024-04-25T00:37:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.969+0000 7fe598fd4740 1 -- 172.21.15.42:0/1540839509 wait complete. 2024-04-25T00:37:42.971 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.969+0000 7fe598fd4740 20 client.26672 trim_cache size 0 max 16384 2024-04-25T00:37:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:37:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:37:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:37:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:37:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.970+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5740a2da0 msgr2=0x7fe5740a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.972 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.971+0000 7fe596200640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5740a2da0 0x7fe5740a5260 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fe5900706b0 tx=0x7fe57c00a380 comp rx=0 tx=0).stop 2024-04-25T00:37:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.971+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590074120 msgr2=0x7fe59006f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.971+0000 7fe596200640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590074120 0x7fe59006f1a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fe58000d6d0 tx=0x7fe58000db90 comp rx=0 tx=0).stop 2024-04-25T00:37:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.971+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 shutdown_connections 2024-04-25T00:37:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.971+0000 7fe596200640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5740a2da0 0x7fe5740a5260 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.971+0000 7fe596200640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe590109650 0x7fe590072730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.971+0000 7fe596200640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe590075700 0x7fe59006f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.971+0000 7fe596200640 1 --2- 172.21.15.42:0/1862843587 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe590074120 0x7fe59006f1a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.971+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 >> 172.21.15.42:0/1862843587 conn(0x7fe5900fcd30 msgr2=0x7fe5901047e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.972+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 shutdown_connections 2024-04-25T00:37:42.973 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:42.972+0000 7fe596200640 1 -- 172.21.15.42:0/1862843587 wait complete. 2024-04-25T00:37:43.775 INFO:tasks.workunit.client.0.smithi042.stdout: 10 212606 34.44 MB/sec execute 213 sec latency 178.158 ms 2024-04-25T00:37:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:43 smithi136 ceph-mon[32923]: pgmap v959: 97 pgs: 97 active+clean; 682 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 500 op/s 2024-04-25T00:37:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:43 smithi106 ceph-mon[32510]: pgmap v959: 97 pgs: 97 active+clean; 682 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 500 op/s 2024-04-25T00:37:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:43 smithi042 ceph-mon[28130]: pgmap v959: 97 pgs: 97 active+clean; 682 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.9 MiB/s wr, 500 op/s 2024-04-25T00:37:44.775 INFO:tasks.workunit.client.0.smithi042.stdout: 10 213177 34.42 MB/sec execute 214 sec latency 370.358 ms 2024-04-25T00:37:45.775 INFO:tasks.workunit.client.0.smithi042.stdout: 10 213254 34.28 MB/sec execute 215 sec latency 799.987 ms 2024-04-25T00:37:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:45 smithi136 ceph-mon[32923]: pgmap v960: 97 pgs: 97 active+clean; 683 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.0 MiB/s wr, 467 op/s 2024-04-25T00:37:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:45 smithi106 ceph-mon[32510]: pgmap v960: 97 pgs: 97 active+clean; 683 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.0 MiB/s wr, 467 op/s 2024-04-25T00:37:46.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:45 smithi042 ceph-mon[28130]: pgmap v960: 97 pgs: 97 active+clean; 683 MiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 4.0 MiB/s wr, 467 op/s 2024-04-25T00:37:46.775 INFO:tasks.workunit.client.0.smithi042.stdout: 10 213894 34.33 MB/sec execute 216 sec latency 188.437 ms 2024-04-25T00:37:47.775 INFO:tasks.workunit.client.0.smithi042.stdout: 10 214451 34.34 MB/sec execute 217 sec latency 77.858 ms 2024-04-25T00:37:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:47 smithi136 ceph-mon[32923]: pgmap v961: 97 pgs: 97 active+clean; 695 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 645 op/s 2024-04-25T00:37:48.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:47 smithi106 ceph-mon[32510]: pgmap v961: 97 pgs: 97 active+clean; 695 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 645 op/s 2024-04-25T00:37:48.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:47 smithi042 ceph-mon[28130]: pgmap v961: 97 pgs: 97 active+clean; 695 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 8.3 MiB/s wr, 645 op/s 2024-04-25T00:37:48.775 INFO:tasks.workunit.client.0.smithi042.stdout: 10 214950 34.27 MB/sec execute 218 sec latency 112.613 ms 2024-04-25T00:37:49.775 INFO:tasks.workunit.client.0.smithi042.stdout: 10 215695 34.29 MB/sec execute 219 sec latency 45.295 ms 2024-04-25T00:37:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:49 smithi136 ceph-mon[32923]: pgmap v962: 97 pgs: 97 active+clean; 698 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 706 op/s 2024-04-25T00:37:50.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:49 smithi106 ceph-mon[32510]: pgmap v962: 97 pgs: 97 active+clean; 698 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 706 op/s 2024-04-25T00:37:50.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:49 smithi042 ceph-mon[28130]: pgmap v962: 97 pgs: 97 active+clean; 698 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 706 op/s 2024-04-25T00:37:50.775 INFO:tasks.workunit.client.0.smithi042.stdout: 10 216057 34.21 MB/sec execute 220 sec latency 363.650 ms 2024-04-25T00:37:51.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 216700 34.22 MB/sec execute 221 sec latency 244.773 ms 2024-04-25T00:37:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:51 smithi136 ceph-mon[32923]: pgmap v963: 97 pgs: 97 active+clean; 692 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 10 MiB/s wr, 696 op/s 2024-04-25T00:37:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:51 smithi106 ceph-mon[32510]: pgmap v963: 97 pgs: 97 active+clean; 692 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 10 MiB/s wr, 696 op/s 2024-04-25T00:37:52.130 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:51 smithi042 ceph-mon[28130]: pgmap v963: 97 pgs: 97 active+clean; 692 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 10 MiB/s wr, 696 op/s 2024-04-25T00:37:52.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 217253 34.21 MB/sec execute 222 sec latency 126.211 ms 2024-04-25T00:37:53.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 217811 34.22 MB/sec execute 223 sec latency 83.964 ms 2024-04-25T00:37:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:53 smithi136 ceph-mon[32923]: pgmap v964: 97 pgs: 97 active+clean; 708 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 892 op/s 2024-04-25T00:37:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:53 smithi106 ceph-mon[32510]: pgmap v964: 97 pgs: 97 active+clean; 708 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 892 op/s 2024-04-25T00:37:54.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:53 smithi042 ceph-mon[28130]: pgmap v964: 97 pgs: 97 active+clean; 708 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 14 MiB/s wr, 892 op/s 2024-04-25T00:37:54.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 218358 34.19 MB/sec execute 224 sec latency 63.263 ms 2024-04-25T00:37:55.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 219000 34.15 MB/sec execute 225 sec latency 113.317 ms 2024-04-25T00:37:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:55 smithi136 ceph-mon[32923]: pgmap v965: 97 pgs: 97 active+clean; 708 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 786 op/s 2024-04-25T00:37:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:55 smithi106 ceph-mon[32510]: pgmap v965: 97 pgs: 97 active+clean; 708 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 786 op/s 2024-04-25T00:37:56.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:55 smithi042 ceph-mon[28130]: pgmap v965: 97 pgs: 97 active+clean; 708 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 786 op/s 2024-04-25T00:37:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.139+0000 7f04d5600640 1 -- 172.21.15.42:0/2060933372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04d01004c0 msgr2=0x7f04d01008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.139+0000 7f04d5600640 1 --2- 172.21.15.42:0/2060933372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04d01004c0 0x7f04d01008e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f04c0009900 tx=0x7f04c002a420 comp rx=0 tx=0).stop 2024-04-25T00:37:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.139+0000 7f04d5600640 1 -- 172.21.15.42:0/2060933372 shutdown_connections 2024-04-25T00:37:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.139+0000 7f04d5600640 1 --2- 172.21.15.42:0/2060933372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04d0102070 0x7f04d0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.139+0000 7f04d5600640 1 --2- 172.21.15.42:0/2060933372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d01016b0 0x7f04d0101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.139+0000 7f04d5600640 1 --2- 172.21.15.42:0/2060933372 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04d01004c0 0x7f04d01008e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.139+0000 7f04d5600640 1 -- 172.21.15.42:0/2060933372 >> 172.21.15.42:0/2060933372 conn(0x7f04d00fba30 msgr2=0x7f04d00fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.139+0000 7f04d5600640 1 -- 172.21.15.42:0/2060933372 shutdown_connections 2024-04-25T00:37:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.139+0000 7f04d5600640 1 -- 172.21.15.42:0/2060933372 wait complete. 2024-04-25T00:37:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.141+0000 7f04d5600640 1 Processor -- start 2024-04-25T00:37:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.141+0000 7f04d5600640 1 -- start start 2024-04-25T00:37:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.141+0000 7f04d5600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d01004c0 0x7f04d0199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.141+0000 7f04d5600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04d01016b0 0x7f04d019a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.141+0000 7f04d5600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04d0102070 0x7f04d01a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.141+0000 7f04d5600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f04d010c170 con 0x7f04d0102070 2024-04-25T00:37:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.141+0000 7f04d5600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04d010bff0 con 0x7f04d01016b0 2024-04-25T00:37:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.141+0000 7f04d5600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f04d010c2f0 con 0x7f04d01004c0 2024-04-25T00:37:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04cf400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04d01016b0 0x7f04d019a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d01004c0 0x7f04d0199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d01004c0 0x7f04d0199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:40442/0 (socket says 172.21.15.42:40442) 2024-04-25T00:37:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04cf400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04d01016b0 0x7f04d019a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43622/0 (socket says 172.21.15.42:43622) 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04cfe00640 1 -- 172.21.15.42:0/35440040 learned_addr learned my addr 172.21.15.42:0/35440040 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04d4c00640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04d0102070 0x7f04d01a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04cfe00640 1 -- 172.21.15.42:0/35440040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04d01016b0 msgr2=0x7f04d019a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04cfe00640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04d01016b0 0x7f04d019a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04cfe00640 1 -- 172.21.15.42:0/35440040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04d0102070 msgr2=0x7f04d01a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04cfe00640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04d0102070 0x7f04d01a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.142+0000 7f04cfe00640 1 -- 172.21.15.42:0/35440040 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d01a1730 con 0x7f04d01004c0 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.143+0000 7f04cfe00640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d01004c0 0x7f04d0199b60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f04c002aa10 tx=0x7f04c0032bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.143+0000 7f04ccc00640 1 -- 172.21.15.42:0/35440040 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c003f030 con 0x7f04d01004c0 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.143+0000 7f04ccc00640 1 -- 172.21.15.42:0/35440040 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c0003ad0 con 0x7f04d01004c0 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.143+0000 7f04d5600640 1 -- 172.21.15.42:0/35440040 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d01a19c0 con 0x7f04d01004c0 2024-04-25T00:37:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.143+0000 7f04ccc00640 1 -- 172.21.15.42:0/35440040 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0035070 con 0x7f04d01004c0 2024-04-25T00:37:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.143+0000 7f04d5600640 1 -- 172.21.15.42:0/35440040 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d01a1f00 con 0x7f04d01004c0 2024-04-25T00:37:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.148+0000 7f04d9b7a740 1 Processor -- start 2024-04-25T00:37:56.150 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.149+0000 7f04d9b7a740 1 -- start start 2024-04-25T00:37:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.149+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b8511fe50 0x556b85120270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.149+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b85125340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.149+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b85127e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.149+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556b85082ab0 con 0x556b85120840 2024-04-25T00:37:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.149+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556b8502b640 con 0x556b851259b0 2024-04-25T00:37:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.149+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556b8507ea70 con 0x556b8511fe50 2024-04-25T00:37:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.150+0000 7f04ccc00640 1 -- 172.21.15.42:0/35440040 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f04c0042030 con 0x7f04d01004c0 2024-04-25T00:37:56.153 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.150+0000 7f04cf400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b85127e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.150+0000 7f04cf400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b85127e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43636/0 (socket says 172.21.15.42:43636) 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.150+0000 7f04cf400640 1 -- 172.21.15.42:0/1203695259 learned_addr learned my addr 172.21.15.42:0/1203695259 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.150+0000 7f04cf400640 1 -- 172.21.15.42:0/1203695259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b8511fe50 msgr2=0x556b85120270 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.150+0000 7f04cf400640 1 --2- 172.21.15.42:0/1203695259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b8511fe50 0x556b85120270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.150+0000 7f04cf400640 1 -- 172.21.15.42:0/1203695259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 msgr2=0x556b85125340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.150+0000 7f04cfe00640 1 --2- 172.21.15.42:0/1203695259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b85125340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.151+0000 7f04cf400640 1 --2- 172.21.15.42:0/1203695259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b85125340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.151+0000 7f04cf400640 1 -- 172.21.15.42:0/1203695259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b84d97210 con 0x556b851259b0 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.151+0000 7f04cfe00640 1 --2- 172.21.15.42:0/1203695259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b85125340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.151+0000 7f04cf400640 1 --2- 172.21.15.42:0/1203695259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b85127e40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00bcb0 tx=0x7f04bc0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c18490e6c67d7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.152+0000 7f04c5200640 1 -- 172.21.15.42:0/1203695259 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc02d480 con 0x556b851259b0 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.152+0000 7f04c5200640 1 -- 172.21.15.42:0/1203695259 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04bc022930 con 0x556b851259b0 2024-04-25T00:37:56.154 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.152+0000 7f04c5200640 1 -- 172.21.15.42:0/1203695259 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc02ba20 con 0x556b851259b0 2024-04-25T00:37:56.155 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.152+0000 7f04ccc00640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f04b40a2cc0 0x7f04b40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.153+0000 7f04ccc00640 1 -- 172.21.15.42:0/35440040 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f04c00ef700 con 0x7f04d01004c0 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.153+0000 7f04d4c00640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f04b40a2cc0 0x7f04b40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.153+0000 7f04d9b7a740 1 -- 172.21.15.42:0/1203695259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 msgr2=0x556b85127e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.153+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/1203695259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b85127e40 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00bcb0 tx=0x7f04bc0107a0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.154+0000 7f04d9b7a740 1 -- 172.21.15.42:0/1203695259 shutdown_connections 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.154+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/1203695259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b85127e40 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.154+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/1203695259 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b85125340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.154+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/1203695259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b8511fe50 0x556b85120270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.154+0000 7f04d9b7a740 1 -- 172.21.15.42:0/1203695259 >> 172.21.15.42:0/1203695259 conn(0x556b8511b4c0 msgr2=0x556b8511b8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.154+0000 7f04d9b7a740 1 -- 172.21.15.42:0/1203695259 shutdown_connections 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.154+0000 7f04d4c00640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f04b40a2cc0 0x7f04b40a5180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x556b851210a0 tx=0x7f04c8009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.156 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.154+0000 7f04d9b7a740 1 -- 172.21.15.42:0/1203695259 wait complete. 2024-04-25T00:37:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.155+0000 7f04d9b7a740 1 -- start start 2024-04-25T00:37:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.155+0000 7f04d9b7a740 1 Processor -- start 2024-04-25T00:37:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b8511fe50 0x556b85129fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b8512a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b851259b0 0x556b8512f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556b85128820 con 0x556b85120840 2024-04-25T00:37:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556b8503e990 con 0x556b8511fe50 2024-04-25T00:37:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556b851289a0 con 0x556b851259b0 2024-04-25T00:37:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04d9b7a740 10 client.0 _ll_register_callbacks cb 0x556b85040950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:56.158 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b8511fe50 0x556b85129fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04cfe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b8511fe50 0x556b85129fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43642/0 (socket says 172.21.15.42:43642) 2024-04-25T00:37:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04cfe00640 1 -- 172.21.15.42:0/3103706044 learned_addr learned my addr 172.21.15.42:0/3103706044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04cfe00640 1 -- 172.21.15.42:0/3103706044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b851259b0 msgr2=0x556b8512f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04cfe00640 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b851259b0 0x556b8512f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04cfe00640 1 -- 172.21.15.42:0/3103706044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 msgr2=0x556b8512a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.156+0000 7f04cfe00640 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b8512a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.159 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.157+0000 7f04cfe00640 1 -- 172.21.15.42:0/3103706044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b84e63620 con 0x556b8511fe50 2024-04-25T00:37:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.157+0000 7f04cfe00640 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b8511fe50 0x556b85129fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f04c00bbf70 tx=0x7f04c003be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8d3c0c268dc9e36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.157+0000 7f04b9600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:37:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.157+0000 7f04b9600640 1 -- 172.21.15.42:0/3103706044 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c0049050 con 0x556b8511fe50 2024-04-25T00:37:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.157+0000 7f04d9b7a740 1 -- 172.21.15.42:0/3103706044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04bc022930 con 0x556b8511fe50 2024-04-25T00:37:56.160 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.157+0000 7f04b9600640 1 -- 172.21.15.42:0/3103706044 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f04c0003c80 con 0x556b8511fe50 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.157+0000 7f04b9600640 1 -- 172.21.15.42:0/3103706044 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f04c00c02e0 con 0x556b8511fe50 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.157+0000 7f04d9b7a740 10 client.26684 fetch_fsmap learned FSMap version 45 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.158+0000 7f04d9b7a740 1 -- 172.21.15.42:0/3103706044 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556b851345f0 con 0x556b8511fe50 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.158+0000 7f04b9600640 1 -- 172.21.15.42:0/3103706044 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c009ae50 con 0x556b8511fe50 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.158+0000 7f04b9600640 1 -- 172.21.15.42:0/3103706044 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f04c00c4710 con 0x556b8511fe50 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.159+0000 7f04d9b7a740 10 client.26684 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.159+0000 7f04d9b7a740 10 client.26684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.159+0000 7f04d9b7a740 20 client.26684 populate_metadata read hostname 'smithi042' 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.159+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556b85135d30 0x556b851381d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.159+0000 7f04d9b7a740 4 client.26684 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:37:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.159+0000 7f04d9b7a740 1 -- 172.21.15.42:0/3103706044 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b8508fff0 con 0x556b85135d30 2024-04-25T00:37:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.159+0000 7f04cf400640 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556b85135d30 0x556b851381d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.160+0000 7f04b9600640 10 client.26684 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.160+0000 7f04cf400640 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556b85135d30 0x556b851381d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f488938892fd4bec server_cookie=f50f50222c729efb in_seq=0 out_seq=0 2024-04-25T00:37:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.161+0000 7f04b9600640 1 -- 172.21.15.42:0/3103706044 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b8508fff0 con 0x556b85135d30 2024-04-25T00:37:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.161+0000 7f04b9600640 10 client.26684 handle_command_reply: tid=0 2024-04-25T00:37:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.161+0000 7f04d9b7a740 1 client.26684 shutdown 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04d9b7a740 1 -- 172.21.15.42:0/3103706044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b8511fe50 msgr2=0x556b85129fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b8511fe50 0x556b85129fe0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f04c00bbf70 tx=0x7f04c003be40 comp rx=0 tx=0).stop 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04d9b7a740 1 -- 172.21.15.42:0/3103706044 shutdown_connections 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556b85135d30 0x556b851381d0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b851259b0 0x556b8512f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b8512a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/3103706044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b8511fe50 0x556b85129fe0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04d9b7a740 1 -- 172.21.15.42:0/3103706044 >> 172.21.15.42:0/3103706044 conn(0x556b8511b4c0 msgr2=0x556b8511a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04b9600640 0 client.26684 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04d9b7a740 1 -- 172.21.15.42:0/3103706044 shutdown_connections 2024-04-25T00:37:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.162+0000 7f04d9b7a740 1 -- 172.21.15.42:0/3103706044 wait complete. 2024-04-25T00:37:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.163+0000 7f04d9b7a740 20 client.26684 trim_cache size 0 max 16384 2024-04-25T00:37:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.192+0000 7f04d9b7a740 1 Processor -- start 2024-04-25T00:37:56.193 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.192+0000 7f04d9b7a740 1 -- start start 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.192+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b8511fe50 0x556b85116110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.192+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b85129bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.192+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b8512a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.192+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01a90c0 con 0x556b85120840 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.192+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01a6990 con 0x556b851259b0 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.192+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01a4800 con 0x556b8511fe50 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04d4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b8512a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04d4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b8512a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43650/0 (socket says 172.21.15.42:43650) 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04d4c00640 1 -- 172.21.15.42:0/661485326 learned_addr learned my addr 172.21.15.42:0/661485326 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04cfe00640 1 --2- 172.21.15.42:0/661485326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b8511fe50 0x556b85116110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.195 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04cf400640 1 --2- 172.21.15.42:0/661485326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b85129bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04d4c00640 1 -- 172.21.15.42:0/661485326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b8511fe50 msgr2=0x556b85116110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04d4c00640 1 --2- 172.21.15.42:0/661485326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b8511fe50 0x556b85116110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04d4c00640 1 -- 172.21.15.42:0/661485326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 msgr2=0x556b85129bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04d4c00640 1 --2- 172.21.15.42:0/661485326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b85129bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04d4c00640 1 -- 172.21.15.42:0/661485326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8511b2f0 con 0x556b851259b0 2024-04-25T00:37:56.196 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.193+0000 7f04cf400640 1 --2- 172.21.15.42:0/661485326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b85129bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:37:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.194+0000 7f04d4c00640 1 --2- 172.21.15.42:0/661485326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b8512a130 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f04c8003bc0 tx=0x7f04c8013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60a89926371c6260 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.194+0000 7f04bbe00640 1 -- 172.21.15.42:0/661485326 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c801e420 con 0x556b851259b0 2024-04-25T00:37:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.194+0000 7f04bbe00640 1 -- 172.21.15.42:0/661485326 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c801eaa0 con 0x556b851259b0 2024-04-25T00:37:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.194+0000 7f04bbe00640 1 -- 172.21.15.42:0/661485326 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c8026c90 con 0x556b851259b0 2024-04-25T00:37:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.195+0000 7f04d9b7a740 1 -- 172.21.15.42:0/661485326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 msgr2=0x556b8512a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.197 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.195+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/661485326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b8512a130 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f04c8003bc0 tx=0x7f04c8013040 comp rx=0 tx=0).stop 2024-04-25T00:37:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.196+0000 7f04d9b7a740 1 -- 172.21.15.42:0/661485326 shutdown_connections 2024-04-25T00:37:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.196+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/661485326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b8512a130 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.196+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/661485326 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b85120840 0x556b85129bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.196+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/661485326 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b8511fe50 0x556b85116110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.198 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.196+0000 7f04d9b7a740 1 -- 172.21.15.42:0/661485326 >> 172.21.15.42:0/661485326 conn(0x556b8511b4c0 msgr2=0x556b85128da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.196+0000 7f04d9b7a740 1 -- 172.21.15.42:0/661485326 shutdown_connections 2024-04-25T00:37:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.196+0000 7f04d9b7a740 1 -- 172.21.15.42:0/661485326 wait complete. 2024-04-25T00:37:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.197+0000 7f04d9b7a740 1 -- start start 2024-04-25T00:37:56.199 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.197+0000 7f04d9b7a740 1 Processor -- start 2024-04-25T00:37:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b8511fe50 0x556b85147350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.200 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b85120840 0x556b85147890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04d9b7a740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b851488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01a4e50 con 0x556b8511fe50 2024-04-25T00:37:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04cf400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b85120840 0x556b85147890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04d4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b851488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04d4c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b851488a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:43656/0 (socket says 172.21.15.42:43656) 2024-04-25T00:37:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04d4c00640 1 -- 172.21.15.42:0/2938414657 learned_addr learned my addr 172.21.15.42:0/2938414657 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:37:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04d9b7a740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01a62c0 con 0x556b851259b0 2024-04-25T00:37:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04d9b7a740 1 -- 172.21.15.42:0/2938414657 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f04d01a5520 con 0x556b85120840 2024-04-25T00:37:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.199+0000 7f04d9b7a740 10 client.0 _ll_register_callbacks cb 0x7f04c0005b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:37:56.201 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.200+0000 7f04cfe00640 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b8511fe50 0x556b85147350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.200+0000 7f04cf400640 1 -- 172.21.15.42:0/2938414657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 msgr2=0x556b851488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.200+0000 7f04cf400640 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b851488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.200+0000 7f04cf400640 1 -- 172.21.15.42:0/2938414657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b8511fe50 msgr2=0x556b85147350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:37:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.200+0000 7f04cf400640 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b8511fe50 0x556b85147350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.200+0000 7f04cf400640 1 -- 172.21.15.42:0/2938414657 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b85148e90 con 0x556b85120840 2024-04-25T00:37:56.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.201+0000 7f04cf400640 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b85120840 0x556b85147890 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00b5f0 tx=0x7f04bc005d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ba09cceecb05d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:37:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.201+0000 7f04b9600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:37:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.202+0000 7f04b9600640 1 -- 172.21.15.42:0/2938414657 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc00bea0 con 0x556b85120840 2024-04-25T00:37:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.202+0000 7f04b9600640 1 -- 172.21.15.42:0/2938414657 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f04bc022ae0 con 0x556b85120840 2024-04-25T00:37:56.203 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.202+0000 7f04d9b7a740 1 -- 172.21.15.42:0/2938414657 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04c801eaa0 con 0x556b85120840 2024-04-25T00:37:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.202+0000 7f04b9600640 1 -- 172.21.15.42:0/2938414657 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f04bc022ce0 con 0x556b85120840 2024-04-25T00:37:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.202+0000 7f04d9b7a740 10 client.26656 fetch_fsmap learned FSMap version 45 2024-04-25T00:37:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.202+0000 7f04d9b7a740 1 -- 172.21.15.42:0/2938414657 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556b8519c4b0 con 0x556b85120840 2024-04-25T00:37:56.204 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.203+0000 7f04b9600640 1 -- 172.21.15.42:0/2938414657 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f04c801eaa0 con 0x556b85120840 2024-04-25T00:37:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.203+0000 7f04b9600640 1 -- 172.21.15.42:0/2938414657 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f04bc03a550 con 0x556b85120840 2024-04-25T00:37:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.203+0000 7f04d9b7a740 10 client.26656 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:37:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.203+0000 7f04d9b7a740 10 client.26656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:37:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.203+0000 7f04d9b7a740 20 client.26656 populate_metadata read hostname 'smithi042' 2024-04-25T00:37:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.203+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556b8519cbb0 0x556b8519efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:37:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.203+0000 7f04d9b7a740 4 client.26656 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:37:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.203+0000 7f04d9b7a740 1 -- 172.21.15.42:0/2938414657 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b8508a340 con 0x556b8519cbb0 2024-04-25T00:37:56.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.204+0000 7f04cfe00640 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556b8519cbb0 0x556b8519efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:37:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.205+0000 7f04cfe00640 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556b8519cbb0 0x556b8519efd0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa3e922fdae8a4b3 server_cookie=4998f4d58b81584e in_seq=0 out_seq=0 2024-04-25T00:37:56.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.205+0000 7f04b9600640 10 client.26656 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.456+0000 7f04b9600640 1 -- 172.21.15.42:0/2938414657 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556b8508a340 con 0x556b8519cbb0 2024-04-25T00:37:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.456+0000 7f04b9600640 10 client.26656 handle_command_reply: tid=0 2024-04-25T00:37:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.456+0000 7f04d9b7a740 1 client.26656 shutdown 2024-04-25T00:37:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.456+0000 7f04d9b7a740 1 -- 172.21.15.42:0/2938414657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b85120840 msgr2=0x556b85147890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.456+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b85120840 0x556b85147890 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f04bc00b5f0 tx=0x7f04bc005d40 comp rx=0 tx=0).stop 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.457+0000 7f04d9b7a740 1 -- 172.21.15.42:0/2938414657 shutdown_connections 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.457+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556b8519cbb0 0x556b8519efd0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.457+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556b851259b0 0x556b851488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.457+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b85120840 0x556b85147890 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.457+0000 7f04d9b7a740 1 --2- 172.21.15.42:0/2938414657 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556b8511fe50 0x556b85147350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.457+0000 7f04b9600640 0 client.26656 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.457+0000 7f04d9b7a740 1 -- 172.21.15.42:0/2938414657 >> 172.21.15.42:0/2938414657 conn(0x556b8511b4c0 msgr2=0x556b85128da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.457+0000 7f04d9b7a740 1 -- 172.21.15.42:0/2938414657 shutdown_connections 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.457+0000 7f04d9b7a740 1 -- 172.21.15.42:0/2938414657 wait complete. 2024-04-25T00:37:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.457+0000 7f04d9b7a740 20 client.26656 trim_cache size 0 max 16384 2024-04-25T00:37:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:37:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:37:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:37:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:37:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.458+0000 7f04d5600640 1 -- 172.21.15.42:0/35440040 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f04b40a2cc0 msgr2=0x7f04b40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.458+0000 7f04d5600640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f04b40a2cc0 0x7f04b40a5180 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x556b851210a0 tx=0x7f04c8009500 comp rx=0 tx=0).stop 2024-04-25T00:37:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.458+0000 7f04d5600640 1 -- 172.21.15.42:0/35440040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d01004c0 msgr2=0x7f04d0199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:37:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.458+0000 7f04d5600640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d01004c0 0x7f04d0199b60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f04c002aa10 tx=0x7f04c0032bf0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.459+0000 7f04d5600640 1 -- 172.21.15.42:0/35440040 shutdown_connections 2024-04-25T00:37:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.459+0000 7f04d5600640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f04b40a2cc0 0x7f04b40a5180 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.459+0000 7f04d5600640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f04d0102070 0x7f04d01a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.459+0000 7f04d5600640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04d01016b0 0x7f04d019a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.459+0000 7f04d5600640 1 --2- 172.21.15.42:0/35440040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04d01004c0 0x7f04d0199b60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:37:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.459+0000 7f04d5600640 1 -- 172.21.15.42:0/35440040 >> 172.21.15.42:0/35440040 conn(0x7f04d00fba30 msgr2=0x7f04d0068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:37:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.460+0000 7f04d5600640 1 -- 172.21.15.42:0/35440040 shutdown_connections 2024-04-25T00:37:56.461 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:37:56.460+0000 7f04d5600640 1 -- 172.21.15.42:0/35440040 wait complete. 2024-04-25T00:37:56.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 219504 34.12 MB/sec execute 226 sec latency 260.169 ms 2024-04-25T00:37:57.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 220376 34.19 MB/sec execute 227 sec latency 49.346 ms 2024-04-25T00:37:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:57 smithi136 ceph-mon[32923]: pgmap v966: 97 pgs: 97 active+clean; 679 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 890 op/s 2024-04-25T00:37:58.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:57 smithi106 ceph-mon[32510]: pgmap v966: 97 pgs: 97 active+clean; 679 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 890 op/s 2024-04-25T00:37:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:57 smithi042 ceph-mon[28130]: pgmap v966: 97 pgs: 97 active+clean; 679 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 15 MiB/s wr, 890 op/s 2024-04-25T00:37:58.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 221183 34.28 MB/sec execute 228 sec latency 45.220 ms 2024-04-25T00:37:59.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 221775 34.25 MB/sec execute 229 sec latency 63.228 ms 2024-04-25T00:38:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:37:59 smithi136 ceph-mon[32923]: pgmap v967: 97 pgs: 97 active+clean; 679 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 796 op/s 2024-04-25T00:38:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:37:59 smithi106 ceph-mon[32510]: pgmap v967: 97 pgs: 97 active+clean; 679 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 796 op/s 2024-04-25T00:38:00.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:37:59 smithi042 ceph-mon[28130]: pgmap v967: 97 pgs: 97 active+clean; 679 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 796 op/s 2024-04-25T00:38:00.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 222401 34.21 MB/sec execute 230 sec latency 52.233 ms 2024-04-25T00:38:01.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 223234 34.25 MB/sec execute 231 sec latency 49.227 ms 2024-04-25T00:38:02.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:01 smithi136 ceph-mon[32923]: pgmap v968: 97 pgs: 97 active+clean; 685 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.4 MiB/s wr, 667 op/s 2024-04-25T00:38:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:01 smithi106 ceph-mon[32510]: pgmap v968: 97 pgs: 97 active+clean; 685 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.4 MiB/s wr, 667 op/s 2024-04-25T00:38:02.391 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:01 smithi042 ceph-mon[28130]: pgmap v968: 97 pgs: 97 active+clean; 685 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 8.4 MiB/s wr, 667 op/s 2024-04-25T00:38:02.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 224135 34.32 MB/sec execute 232 sec latency 39.491 ms 2024-04-25T00:38:03.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 224918 34.40 MB/sec execute 233 sec latency 67.223 ms 2024-04-25T00:38:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:03 smithi136 ceph-mon[32923]: pgmap v969: 97 pgs: 97 active+clean; 687 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 739 op/s 2024-04-25T00:38:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:03 smithi106 ceph-mon[32510]: pgmap v969: 97 pgs: 97 active+clean; 687 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 739 op/s 2024-04-25T00:38:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:03 smithi042 ceph-mon[28130]: pgmap v969: 97 pgs: 97 active+clean; 687 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.4 MiB/s wr, 739 op/s 2024-04-25T00:38:04.776 INFO:tasks.workunit.client.0.smithi042.stdout: 10 225538 34.39 MB/sec execute 234 sec latency 53.082 ms 2024-04-25T00:38:05.777 INFO:tasks.workunit.client.0.smithi042.stdout: 10 225842 34.30 MB/sec execute 235 sec latency 367.877 ms 2024-04-25T00:38:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:05 smithi106 ceph-mon[32510]: pgmap v970: 97 pgs: 97 active+clean; 688 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.4 MiB/s wr, 537 op/s 2024-04-25T00:38:06.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:05 smithi042 ceph-mon[28130]: pgmap v970: 97 pgs: 97 active+clean; 688 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.4 MiB/s wr, 537 op/s 2024-04-25T00:38:06.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:05 smithi136 ceph-mon[32923]: pgmap v970: 97 pgs: 97 active+clean; 688 MiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 4.4 MiB/s wr, 537 op/s 2024-04-25T00:38:06.777 INFO:tasks.workunit.client.0.smithi042.stdout: 10 225882 34.17 MB/sec execute 236 sec latency 1133.533 ms 2024-04-25T00:38:07.777 INFO:tasks.workunit.client.0.smithi042.stdout: 10 226266 34.09 MB/sec execute 237 sec latency 220.889 ms 2024-04-25T00:38:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:07 smithi106 ceph-mon[32510]: pgmap v971: 97 pgs: 97 active+clean; 723 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.7 MiB/s wr, 716 op/s 2024-04-25T00:38:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:07 smithi042 ceph-mon[28130]: pgmap v971: 97 pgs: 97 active+clean; 723 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.7 MiB/s wr, 716 op/s 2024-04-25T00:38:08.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:07 smithi136 ceph-mon[32923]: pgmap v971: 97 pgs: 97 active+clean; 723 MiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.7 MiB/s wr, 716 op/s 2024-04-25T00:38:08.777 INFO:tasks.workunit.client.0.smithi042.stdout: 10 226894 34.08 MB/sec execute 238 sec latency 92.662 ms 2024-04-25T00:38:09.777 INFO:tasks.workunit.client.0.smithi042.stdout: 10 227852 34.11 MB/sec execute 239 sec latency 52.936 ms 2024-04-25T00:38:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.895+0000 7ffabc600640 1 -- 172.21.15.42:0/2583183775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab4100990 msgr2=0x7ffab4100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.895+0000 7ffabc600640 1 --2- 172.21.15.42:0/2583183775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab4100990 0x7ffab4100db0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8009900 tx=0x7ffaa802a400 comp rx=0 tx=0).stop 2024-04-25T00:38:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.895+0000 7ffabc600640 1 -- 172.21.15.42:0/2583183775 shutdown_connections 2024-04-25T00:38:09.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.895+0000 7ffabc600640 1 --2- 172.21.15.42:0/2583183775 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab40ffe20 0x7ffab4101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.895+0000 7ffabc600640 1 --2- 172.21.15.42:0/2583183775 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab40ff460 0x7ffab40ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.895+0000 7ffabc600640 1 --2- 172.21.15.42:0/2583183775 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab4100990 0x7ffab4100db0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.895+0000 7ffabc600640 1 -- 172.21.15.42:0/2583183775 >> 172.21.15.42:0/2583183775 conn(0x7ffab40faf60 msgr2=0x7ffab40fd380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.896+0000 7ffabc600640 1 -- 172.21.15.42:0/2583183775 shutdown_connections 2024-04-25T00:38:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.896+0000 7ffabc600640 1 -- 172.21.15.42:0/2583183775 wait complete. 2024-04-25T00:38:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.897+0000 7ffabc600640 1 Processor -- start 2024-04-25T00:38:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.897+0000 7ffabc600640 1 -- start start 2024-04-25T00:38:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffabc600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab40ff460 0x7ffab41068a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffabc600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab40ffe20 0x7ffab4106de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffabc600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab4100990 0x7ffab4103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffabc600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffab410f690 con 0x7ffab40ffe20 2024-04-25T00:38:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffabc600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffab410f510 con 0x7ffab40ff460 2024-04-25T00:38:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffabc600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffab410f810 con 0x7ffab4100990 2024-04-25T00:38:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffab9c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab40ff460 0x7ffab41068a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffaba600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab4100990 0x7ffab4103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffab9c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab40ff460 0x7ffab41068a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58102/0 (socket says 172.21.15.42:58102) 2024-04-25T00:38:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffaba600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab4100990 0x7ffab4103410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:36476/0 (socket says 172.21.15.42:36476) 2024-04-25T00:38:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.898+0000 7ffab9c00640 1 -- 172.21.15.42:0/2713941270 learned_addr learned my addr 172.21.15.42:0/2713941270 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.899+0000 7ffab9200640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab40ffe20 0x7ffab4106de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.899+0000 7ffaba600640 1 -- 172.21.15.42:0/2713941270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab40ff460 msgr2=0x7ffab41068a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.899+0000 7ffaba600640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab40ff460 0x7ffab41068a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.899+0000 7ffaba600640 1 -- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab40ffe20 msgr2=0x7ffab4106de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.899+0000 7ffaba600640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab40ffe20 0x7ffab4106de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.899+0000 7ffaba600640 1 -- 172.21.15.42:0/2713941270 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffab4103cd0 con 0x7ffab4100990 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.899+0000 7ffab9c00640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab40ff460 0x7ffab41068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.899+0000 7ffab9200640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab40ffe20 0x7ffab4106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.899+0000 7ffaba600640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab4100990 0x7ffab4103410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009fd0 tx=0x7ffaac00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.900+0000 7ffab2000640 1 -- 172.21.15.42:0/2713941270 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac00ce20 con 0x7ffab4100990 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.900+0000 7ffab2000640 1 -- 172.21.15.42:0/2713941270 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaac005020 con 0x7ffab4100990 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.900+0000 7ffab2000640 1 -- 172.21.15.42:0/2713941270 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac010720 con 0x7ffab4100990 2024-04-25T00:38:09.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.900+0000 7ffabc600640 1 -- 172.21.15.42:0/2713941270 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffab41a6040 con 0x7ffab4100990 2024-04-25T00:38:09.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.900+0000 7ffabc600640 1 -- 172.21.15.42:0/2713941270 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffab41a6450 con 0x7ffab4100990 2024-04-25T00:38:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.918+0000 7ffab2000640 1 -- 172.21.15.42:0/2713941270 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffaac005540 con 0x7ffab4100990 2024-04-25T00:38:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.929+0000 7ffab2000640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa9c0a2cb0 0x7ffa9c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.929+0000 7ffab2000640 1 -- 172.21.15.42:0/2713941270 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ffaac0cfa40 con 0x7ffab4100990 2024-04-25T00:38:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.931+0000 7ffab9c00640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa9c0a2cb0 0x7ffa9c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.934+0000 7ffab9c00640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa9c0a2cb0 0x7ffa9c0a5170 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80096c0 tx=0x7ffaa80315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.937+0000 7ffabf3eb740 1 Processor -- start 2024-04-25T00:38:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.938+0000 7ffabf3eb740 1 -- start start 2024-04-25T00:38:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.938+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4aca0 0x56315ac4b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.938+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4b690 0x56315ac50190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.938+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 0x56315ac52c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.938+0000 7ffabf3eb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56315abadab0 con 0x56315ac4aca0 2024-04-25T00:38:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.938+0000 7ffabf3eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56315ab56640 con 0x56315ac50800 2024-04-25T00:38:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.938+0000 7ffabf3eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56315aba9a70 con 0x56315ac4b690 2024-04-25T00:38:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.939+0000 7ffab9200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4b690 0x56315ac50190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.939+0000 7ffab9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4aca0 0x56315ac4b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.939+0000 7ffab9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4aca0 0x56315ac4b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59898/0 (socket says 172.21.15.42:59898) 2024-04-25T00:38:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.939+0000 7ffab9c00640 1 -- 172.21.15.42:0/2844990714 learned_addr learned my addr 172.21.15.42:0/2844990714 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.940+0000 7ffab9200640 1 -- 172.21.15.42:0/2844990714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 msgr2=0x56315ac52c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.940+0000 7ffab9200640 1 --2- 172.21.15.42:0/2844990714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 0x56315ac52c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.940+0000 7ffab9200640 1 -- 172.21.15.42:0/2844990714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4aca0 msgr2=0x56315ac4b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.941+0000 7ffab9200640 1 --2- 172.21.15.42:0/2844990714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4aca0 0x56315ac4b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.941+0000 7ffab9200640 1 -- 172.21.15.42:0/2844990714 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315a8c2210 con 0x56315ac4b690 2024-04-25T00:38:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.941+0000 7ffab9200640 1 --2- 172.21.15.42:0/2844990714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4b690 0x56315ac50190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0005fd0 tx=0x7ffaa000f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b131b73ba9252ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.941+0000 7ffaa6a00640 1 -- 172.21.15.42:0/2844990714 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa0005020 con 0x56315ac4b690 2024-04-25T00:38:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.941+0000 7ffaa6a00640 1 -- 172.21.15.42:0/2844990714 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa00047a0 con 0x56315ac4b690 2024-04-25T00:38:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.942+0000 7ffaa6a00640 1 -- 172.21.15.42:0/2844990714 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa00116a0 con 0x56315ac4b690 2024-04-25T00:38:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.943+0000 7ffabf3eb740 1 -- 172.21.15.42:0/2844990714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4b690 msgr2=0x56315ac50190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.943+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/2844990714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4b690 0x56315ac50190 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0005fd0 tx=0x7ffaa000f730 comp rx=0 tx=0).stop 2024-04-25T00:38:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.947+0000 7ffabf3eb740 1 -- 172.21.15.42:0/2844990714 shutdown_connections 2024-04-25T00:38:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.947+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/2844990714 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 0x56315ac52c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.947+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/2844990714 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4b690 0x56315ac50190 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.947+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/2844990714 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4aca0 0x56315ac4b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.947+0000 7ffabf3eb740 1 -- 172.21.15.42:0/2844990714 >> 172.21.15.42:0/2844990714 conn(0x56315ac46310 msgr2=0x56315ac46710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:09.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.947+0000 7ffabf3eb740 1 -- 172.21.15.42:0/2844990714 shutdown_connections 2024-04-25T00:38:09.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.947+0000 7ffabf3eb740 1 -- 172.21.15.42:0/2844990714 wait complete. 2024-04-25T00:38:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.948+0000 7ffabf3eb740 1 -- start start 2024-04-25T00:38:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.948+0000 7ffabf3eb740 1 Processor -- start 2024-04-25T00:38:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.949+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac65f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.949+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac4b690 0x56315ac66460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.949+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 0x56315ac6d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.949+0000 7ffabf3eb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56315ac647c0 con 0x56315ac50800 2024-04-25T00:38:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.949+0000 7ffabf3eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56315ab69990 con 0x56315ac4b690 2024-04-25T00:38:09.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.949+0000 7ffabf3eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56315ac64940 con 0x56315ac4aca0 2024-04-25T00:38:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.949+0000 7ffab9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac65f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.949+0000 7ffab9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac65f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:36496/0 (socket says 172.21.15.42:36496) 2024-04-25T00:38:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.949+0000 7ffab9c00640 1 -- 172.21.15.42:0/3074518327 learned_addr learned my addr 172.21.15.42:0/3074518327 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.949+0000 7ffabf3eb740 10 client.0 _ll_register_callbacks cb 0x56315ab6b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.951+0000 7ffaba600640 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 0x56315ac6d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.953 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.951+0000 7ffab9c00640 1 -- 172.21.15.42:0/3074518327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac4b690 msgr2=0x56315ac66460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:38:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.951+0000 7ffab9c00640 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac4b690 0x56315ac66460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.951+0000 7ffab9c00640 1 -- 172.21.15.42:0/3074518327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 msgr2=0x56315ac6d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.951+0000 7ffab9c00640 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 0x56315ac6d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.951+0000 7ffab9c00640 1 -- 172.21.15.42:0/3074518327 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315a98e620 con 0x56315ac4aca0 2024-04-25T00:38:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.952+0000 7ffab9c00640 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac65f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8004770 tx=0x7ffaa8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7994bfe8437032d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.953+0000 7ffa9b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:38:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.953+0000 7ffa9b400640 1 -- 172.21.15.42:0/3074518327 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa8070070 con 0x56315ac4aca0 2024-04-25T00:38:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.954+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3074518327 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaa00047a0 con 0x56315ac4aca0 2024-04-25T00:38:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.954+0000 7ffa9b400640 1 -- 172.21.15.42:0/3074518327 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ffaa8003ac0 con 0x56315ac4aca0 2024-04-25T00:38:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.954+0000 7ffa9b400640 1 -- 172.21.15.42:0/3074518327 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ffaa807c050 con 0x56315ac4aca0 2024-04-25T00:38:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.954+0000 7ffabf3eb740 10 client.26668 fetch_fsmap learned FSMap version 45 2024-04-25T00:38:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.954+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3074518327 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56315ac53580 con 0x56315ac4aca0 2024-04-25T00:38:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.955+0000 7ffa9b400640 1 -- 172.21.15.42:0/3074518327 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa8003cc0 con 0x56315ac4aca0 2024-04-25T00:38:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.957+0000 7ffa9b400640 1 -- 172.21.15.42:0/3074518327 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7ffaa806a040 con 0x56315ac4aca0 2024-04-25T00:38:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.957+0000 7ffabf3eb740 10 client.26668 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:38:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.957+0000 7ffabf3eb740 10 client.26668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:38:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.957+0000 7ffabf3eb740 20 client.26668 populate_metadata read hostname 'smithi042' 2024-04-25T00:38:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.957+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56315ac55130 0x56315ac575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.957+0000 7ffabf3eb740 4 client.26668 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:38:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.957+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3074518327 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffaa00047a0 con 0x56315ac55130 2024-04-25T00:38:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.958+0000 7ffab9200640 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56315ac55130 0x56315ac575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.961+0000 7ffa9b400640 10 client.26668 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:09.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.961+0000 7ffab9200640 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56315ac55130 0x56315ac575d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e8b0c9c1ba732ba server_cookie=b4d128584c1db9bf in_seq=0 out_seq=0 2024-04-25T00:38:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.968+0000 7ffa9b400640 1 -- 172.21.15.42:0/3074518327 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7ffaa00047a0 con 0x56315ac55130 2024-04-25T00:38:09.969 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.968+0000 7ffa9b400640 10 client.26668 handle_command_reply: tid=0 2024-04-25T00:38:09.970 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.969+0000 7ffabf3eb740 1 client.26668 shutdown 2024-04-25T00:38:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.977+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3074518327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 msgr2=0x56315ac65f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.977+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac65f20 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffaa8004770 tx=0x7ffaa8009260 comp rx=0 tx=0).stop 2024-04-25T00:38:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.977+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3074518327 shutdown_connections 2024-04-25T00:38:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.977+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56315ac55130 0x56315ac575d0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.977+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 0x56315ac6d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.977+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac4b690 0x56315ac66460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.977+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3074518327 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac65f20 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.977+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3074518327 >> 172.21.15.42:0/3074518327 conn(0x56315ac46310 msgr2=0x56315ac45480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:09.979 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.977+0000 7ffa9b400640 0 client.26668 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:09.980 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.978+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3074518327 shutdown_connections 2024-04-25T00:38:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.979+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3074518327 wait complete. 2024-04-25T00:38:09.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:09.980+0000 7ffabf3eb740 20 client.26668 trim_cache size 0 max 16384 2024-04-25T00:38:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.032+0000 7ffabf3eb740 1 Processor -- start 2024-04-25T00:38:10.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.033+0000 7ffabf3eb740 1 -- start start 2024-04-25T00:38:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.034+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac49ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.034+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4b690 0x56315ac6d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.034+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 0x56315ac44bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.034+0000 7ffabf3eb740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41a9260 con 0x56315ac4b690 2024-04-25T00:38:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.034+0000 7ffabf3eb740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41a6b30 con 0x56315ac50800 2024-04-25T00:38:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.034+0000 7ffabf3eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41a0260 con 0x56315ac4aca0 2024-04-25T00:38:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.034+0000 7ffab9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac49ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.035+0000 7ffaba600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 0x56315ac44bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.035+0000 7ffaba600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 0x56315ac44bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58122/0 (socket says 172.21.15.42:58122) 2024-04-25T00:38:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.035+0000 7ffaba600640 1 -- 172.21.15.42:0/1826590780 learned_addr learned my addr 172.21.15.42:0/1826590780 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.035+0000 7ffab9200640 1 --2- 172.21.15.42:0/1826590780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4b690 0x56315ac6d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.035+0000 7ffaba600640 1 -- 172.21.15.42:0/1826590780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 msgr2=0x56315ac49ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.035+0000 7ffaba600640 1 --2- 172.21.15.42:0/1826590780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac49ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.035+0000 7ffaba600640 1 -- 172.21.15.42:0/1826590780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4b690 msgr2=0x56315ac6d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.035+0000 7ffaba600640 1 --2- 172.21.15.42:0/1826590780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4b690 0x56315ac6d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.035+0000 7ffaba600640 1 -- 172.21.15.42:0/1826590780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315ac44890 con 0x56315ac50800 2024-04-25T00:38:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.036+0000 7ffaba600640 1 --2- 172.21.15.42:0/1826590780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 0x56315ac44bf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ffaac098730 tx=0x7ffaac09c1d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=201ddf869205149f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.037+0000 7ffaa6000640 1 -- 172.21.15.42:0/1826590780 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0a0680 con 0x56315ac50800 2024-04-25T00:38:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.037+0000 7ffaa6000640 1 -- 172.21.15.42:0/1826590780 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaac0a4f40 con 0x56315ac50800 2024-04-25T00:38:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.037+0000 7ffaa6000640 1 -- 172.21.15.42:0/1826590780 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0052f0 con 0x56315ac50800 2024-04-25T00:38:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.040+0000 7ffabf3eb740 1 -- 172.21.15.42:0/1826590780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 msgr2=0x56315ac44bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.040+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/1826590780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 0x56315ac44bf0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ffaac098730 tx=0x7ffaac09c1d0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.040+0000 7ffabf3eb740 1 -- 172.21.15.42:0/1826590780 shutdown_connections 2024-04-25T00:38:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.041+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/1826590780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac50800 0x56315ac44bf0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.041+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/1826590780 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac4b690 0x56315ac6d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.041+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/1826590780 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac49ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.041+0000 7ffabf3eb740 1 -- 172.21.15.42:0/1826590780 >> 172.21.15.42:0/1826590780 conn(0x56315ac46310 msgr2=0x56315ac40fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.041+0000 7ffabf3eb740 1 -- 172.21.15.42:0/1826590780 shutdown_connections 2024-04-25T00:38:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.041+0000 7ffabf3eb740 1 -- 172.21.15.42:0/1826590780 wait complete. 2024-04-25T00:38:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.042+0000 7ffabf3eb740 1 -- start start 2024-04-25T00:38:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.042+0000 7ffabf3eb740 1 Processor -- start 2024-04-25T00:38:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.044+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac6d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.044+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac4b690 0x56315ac69cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.044+0000 7ffabf3eb740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 0x56315ac6a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.044+0000 7ffab9200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac4b690 0x56315ac69cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffab9200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac4b690 0x56315ac69cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58138/0 (socket says 172.21.15.42:58138) 2024-04-25T00:38:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffab9200640 1 -- 172.21.15.42:0/3282898006 learned_addr learned my addr 172.21.15.42:0/3282898006 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.044+0000 7ffab9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac6d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41a08d0 con 0x56315ac50800 2024-04-25T00:38:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41a1d40 con 0x56315ac4b690 2024-04-25T00:38:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffab41a0fa0 con 0x56315ac4aca0 2024-04-25T00:38:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffaba600640 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 0x56315ac6a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffabf3eb740 10 client.0 _ll_register_callbacks cb 0x7ffaa8040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffaba600640 1 -- 172.21.15.42:0/3282898006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 msgr2=0x56315ac6d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffaba600640 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac6d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffaba600640 1 -- 172.21.15.42:0/3282898006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac4b690 msgr2=0x56315ac69cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffaba600640 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac4b690 0x56315ac69cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.045+0000 7ffaba600640 1 -- 172.21.15.42:0/3282898006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315ac6a890 con 0x56315ac50800 2024-04-25T00:38:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.047+0000 7ffaba600640 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 0x56315ac6a1f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x56315ac6da60 tx=0x7ffaac09c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1de5460c771f056e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.047+0000 7ffa9b400640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.048+0000 7ffa9b400640 1 -- 172.21.15.42:0/3282898006 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaac0b9070 con 0x56315ac50800 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.048+0000 7ffa9b400640 1 -- 172.21.15.42:0/3282898006 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ffaac0a0170 con 0x56315ac50800 2024-04-25T00:38:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.048+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaac0a4f40 con 0x56315ac50800 2024-04-25T00:38:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.051+0000 7ffa9b400640 1 -- 172.21.15.42:0/3282898006 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ffaac0a0680 con 0x56315ac50800 2024-04-25T00:38:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.051+0000 7ffabf3eb740 10 client.16479 fetch_fsmap learned FSMap version 45 2024-04-25T00:38:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.051+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56315acc76d0 con 0x56315ac50800 2024-04-25T00:38:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.052+0000 7ffa9b400640 1 -- 172.21.15.42:0/3282898006 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaac0a4f40 con 0x56315ac50800 2024-04-25T00:38:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.053+0000 7ffa9b400640 1 -- 172.21.15.42:0/3282898006 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7ffaac020070 con 0x56315ac50800 2024-04-25T00:38:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.054+0000 7ffabf3eb740 10 client.16479 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:38:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.054+0000 7ffabf3eb740 10 client.16479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:38:10.055 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.054+0000 7ffabf3eb740 20 client.16479 populate_metadata read hostname 'smithi042' 2024-04-25T00:38:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.054+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56315acc7ee0 0x56315acca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.055+0000 7ffabf3eb740 4 client.16479 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:38:10.056 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.055+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56315abbaff0 con 0x56315acc7ee0 2024-04-25T00:38:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.055+0000 7ffab9c00640 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56315acc7ee0 0x56315acca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.056+0000 7ffab9c00640 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56315acc7ee0 0x56315acca300 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d6f9a9f7e63a6a2 server_cookie=21a7736a6a79d10 in_seq=0 out_seq=0 2024-04-25T00:38:10.057 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.056+0000 7ffa9b400640 10 client.16479 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:09 smithi106 ceph-mon[32510]: pgmap v972: 97 pgs: 97 active+clean; 741 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-04-25T00:38:10.214 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:09 smithi042 ceph-mon[28130]: pgmap v972: 97 pgs: 97 active+clean; 741 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-04-25T00:38:10.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:09 smithi136 ceph-mon[32923]: pgmap v972: 97 pgs: 97 active+clean; 741 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 11 MiB/s wr, 655 op/s 2024-04-25T00:38:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.334+0000 7ffa9b400640 1 -- 172.21.15.42:0/3282898006 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56315abbaff0 con 0x56315acc7ee0 2024-04-25T00:38:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.334+0000 7ffa9b400640 10 client.16479 handle_command_reply: tid=0 2024-04-25T00:38:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.334+0000 7ffabf3eb740 1 client.16479 shutdown 2024-04-25T00:38:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.334+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 msgr2=0x56315ac6a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.336 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.334+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 0x56315ac6a1f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x56315ac6da60 tx=0x7ffaac09c450 comp rx=0 tx=0).stop 2024-04-25T00:38:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.335+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 shutdown_connections 2024-04-25T00:38:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.335+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56315acc7ee0 0x56315acca300 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.335+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56315ac50800 0x56315ac6a1f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.335+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56315ac4b690 0x56315ac69cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.335+0000 7ffabf3eb740 1 --2- 172.21.15.42:0/3282898006 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56315ac4aca0 0x56315ac6d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.335+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 >> 172.21.15.42:0/3282898006 conn(0x56315ac46310 msgr2=0x56315ac44bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.337 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.335+0000 7ffa9b400640 0 client.16479 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.336+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 shutdown_connections 2024-04-25T00:38:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.336+0000 7ffabf3eb740 1 -- 172.21.15.42:0/3282898006 wait complete. 2024-04-25T00:38:10.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.337+0000 7ffabf3eb740 20 client.16479 trim_cache size 0 max 16384 2024-04-25T00:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:38:10.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:38:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.338+0000 7ffabc600640 1 -- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa9c0a2cb0 msgr2=0x7ffa9c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.338+0000 7ffabc600640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa9c0a2cb0 0x7ffa9c0a5170 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80096c0 tx=0x7ffaa80315b0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.338+0000 7ffabc600640 1 -- 172.21.15.42:0/2713941270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab4100990 msgr2=0x7ffab4103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.338+0000 7ffabc600640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab4100990 0x7ffab4103410 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ffaac009fd0 tx=0x7ffaac00ef10 comp rx=0 tx=0).stop 2024-04-25T00:38:10.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.338+0000 7ffabc600640 1 -- 172.21.15.42:0/2713941270 shutdown_connections 2024-04-25T00:38:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.338+0000 7ffabc600640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ffa9c0a2cb0 0x7ffa9c0a5170 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.338+0000 7ffabc600640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffab4100990 0x7ffab4103410 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.338+0000 7ffabc600640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ffab40ffe20 0x7ffab4106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.338+0000 7ffabc600640 1 --2- 172.21.15.42:0/2713941270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffab40ff460 0x7ffab41068a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.339+0000 7ffabc600640 1 -- 172.21.15.42:0/2713941270 >> 172.21.15.42:0/2713941270 conn(0x7ffab40faf60 msgr2=0x7ffab40fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.339+0000 7ffabc600640 1 -- 172.21.15.42:0/2713941270 shutdown_connections 2024-04-25T00:38:10.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:10.339+0000 7ffabc600640 1 -- 172.21.15.42:0/2713941270 wait complete. 2024-04-25T00:38:10.778 INFO:tasks.workunit.client.0.smithi042.stdout: 10 228346 34.14 MB/sec execute 240 sec latency 256.137 ms 2024-04-25T00:38:11.778 INFO:tasks.workunit.client.0.smithi042.stdout: 10 229016 34.20 MB/sec execute 241 sec latency 66.576 ms 2024-04-25T00:38:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:11 smithi106 ceph-mon[32510]: pgmap v973: 97 pgs: 97 active+clean; 734 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 622 op/s 2024-04-25T00:38:12.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:11 smithi042 ceph-mon[28130]: pgmap v973: 97 pgs: 97 active+clean; 734 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 622 op/s 2024-04-25T00:38:12.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:11 smithi136 ceph-mon[32923]: pgmap v973: 97 pgs: 97 active+clean; 734 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 12 MiB/s wr, 622 op/s 2024-04-25T00:38:12.778 INFO:tasks.workunit.client.0.smithi042.stdout: 10 229449 34.13 MB/sec execute 242 sec latency 75.245 ms 2024-04-25T00:38:13.778 INFO:tasks.workunit.client.0.smithi042.stdout: 10 229975 34.07 MB/sec execute 243 sec latency 79.590 ms 2024-04-25T00:38:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:13 smithi106 ceph-mon[32510]: pgmap v974: 97 pgs: 97 active+clean; 683 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 866 op/s 2024-04-25T00:38:14.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:13 smithi042 ceph-mon[28130]: pgmap v974: 97 pgs: 97 active+clean; 683 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 866 op/s 2024-04-25T00:38:14.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:13 smithi136 ceph-mon[32923]: pgmap v974: 97 pgs: 97 active+clean; 683 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 15 MiB/s wr, 866 op/s 2024-04-25T00:38:14.778 INFO:tasks.workunit.client.0.smithi042.stdout: 10 230702 34.07 MB/sec execute 244 sec latency 117.855 ms 2024-04-25T00:38:15.779 INFO:tasks.workunit.client.0.smithi042.stdout: 10 231532 34.11 MB/sec execute 245 sec latency 145.830 ms 2024-04-25T00:38:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:15 smithi106 ceph-mon[32510]: pgmap v975: 97 pgs: 97 active+clean; 676 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 790 op/s 2024-04-25T00:38:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:15 smithi042 ceph-mon[28130]: pgmap v975: 97 pgs: 97 active+clean; 676 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 790 op/s 2024-04-25T00:38:16.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:15 smithi136 ceph-mon[32923]: pgmap v975: 97 pgs: 97 active+clean; 676 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 790 op/s 2024-04-25T00:38:16.779 INFO:tasks.workunit.client.0.smithi042.stdout: 10 232154 34.16 MB/sec execute 246 sec latency 323.271 ms 2024-04-25T00:38:17.780 INFO:tasks.workunit.client.0.smithi042.stdout: 10 232645 34.15 MB/sec execute 247 sec latency 90.542 ms 2024-04-25T00:38:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:17 smithi106 ceph-mon[32510]: pgmap v976: 97 pgs: 97 active+clean; 678 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 896 op/s 2024-04-25T00:38:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:17 smithi042 ceph-mon[28130]: pgmap v976: 97 pgs: 97 active+clean; 678 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 896 op/s 2024-04-25T00:38:18.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:17 smithi136 ceph-mon[32923]: pgmap v976: 97 pgs: 97 active+clean; 678 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 17 MiB/s wr, 896 op/s 2024-04-25T00:38:18.780 INFO:tasks.workunit.client.0.smithi042.stdout: 10 233124 34.08 MB/sec execute 248 sec latency 62.774 ms 2024-04-25T00:38:19.780 INFO:tasks.workunit.client.0.smithi042.stdout: 10 233709 34.05 MB/sec execute 249 sec latency 73.530 ms 2024-04-25T00:38:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:19 smithi106 ceph-mon[32510]: pgmap v977: 97 pgs: 97 active+clean; 679 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-04-25T00:38:20.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:19 smithi042 ceph-mon[28130]: pgmap v977: 97 pgs: 97 active+clean; 679 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-04-25T00:38:20.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:19 smithi136 ceph-mon[32923]: pgmap v977: 97 pgs: 97 active+clean; 679 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 14 MiB/s wr, 801 op/s 2024-04-25T00:38:20.780 INFO:tasks.workunit.client.0.smithi042.stdout: 10 234624 34.08 MB/sec execute 250 sec latency 93.099 ms 2024-04-25T00:38:21.780 INFO:tasks.workunit.client.0.smithi042.stdout: 10 235331 34.15 MB/sec execute 251 sec latency 53.178 ms 2024-04-25T00:38:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:21 smithi106 ceph-mon[32510]: pgmap v978: 97 pgs: 97 active+clean; 673 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 767 op/s 2024-04-25T00:38:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:21 smithi042 ceph-mon[28130]: pgmap v978: 97 pgs: 97 active+clean; 673 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 767 op/s 2024-04-25T00:38:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:21 smithi136 ceph-mon[32923]: pgmap v978: 97 pgs: 97 active+clean; 673 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 11 MiB/s wr, 767 op/s 2024-04-25T00:38:22.782 INFO:tasks.workunit.client.0.smithi042.stdout: 10 235986 34.19 MB/sec execute 252 sec latency 60.736 ms 2024-04-25T00:38:23.782 INFO:tasks.workunit.client.0.smithi042.stdout: 10 236395 34.12 MB/sec execute 253 sec latency 84.877 ms 2024-04-25T00:38:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:23 smithi106 ceph-mon[32510]: pgmap v979: 97 pgs: 97 active+clean; 664 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 844 op/s 2024-04-25T00:38:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:23 smithi042 ceph-mon[28130]: pgmap v979: 97 pgs: 97 active+clean; 664 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 844 op/s 2024-04-25T00:38:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:23 smithi136 ceph-mon[32923]: pgmap v979: 97 pgs: 97 active+clean; 664 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 10 MiB/s wr, 844 op/s 2024-04-25T00:38:24.782 INFO:tasks.workunit.client.0.smithi042.stdout: 10 236880 34.04 MB/sec execute 254 sec latency 78.204 ms 2024-04-25T00:38:25.782 INFO:tasks.workunit.client.0.smithi042.stdout: 10 237626 34.09 MB/sec execute 255 sec latency 75.732 ms 2024-04-25T00:38:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:25 smithi106 ceph-mon[32510]: pgmap v980: 97 pgs: 97 active+clean; 663 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.8 MiB/s wr, 532 op/s 2024-04-25T00:38:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:25 smithi042 ceph-mon[28130]: pgmap v980: 97 pgs: 97 active+clean; 663 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.8 MiB/s wr, 532 op/s 2024-04-25T00:38:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:25 smithi136 ceph-mon[32923]: pgmap v980: 97 pgs: 97 active+clean; 663 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 6.8 MiB/s wr, 532 op/s 2024-04-25T00:38:26.782 INFO:tasks.workunit.client.0.smithi042.stdout: 10 238500 34.12 MB/sec execute 256 sec latency 40.778 ms 2024-04-25T00:38:27.783 INFO:tasks.workunit.client.0.smithi042.stdout: 10 239249 34.20 MB/sec execute 257 sec latency 52.252 ms 2024-04-25T00:38:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:27 smithi106 ceph-mon[32510]: pgmap v981: 97 pgs: 97 active+clean; 666 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.8 MiB/s wr, 533 op/s 2024-04-25T00:38:28.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:27 smithi042 ceph-mon[28130]: pgmap v981: 97 pgs: 97 active+clean; 666 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.8 MiB/s wr, 533 op/s 2024-04-25T00:38:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:27 smithi136 ceph-mon[32923]: pgmap v981: 97 pgs: 97 active+clean; 666 MiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 6.8 MiB/s wr, 533 op/s 2024-04-25T00:38:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.439+0000 7f4204200640 1 -- 172.21.15.42:0/3220595481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc075780 msgr2=0x7f41fc073bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.439+0000 7f4204200640 1 --2- 172.21.15.42:0/3220595481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc075780 0x7f41fc073bd0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f41e8009930 tx=0x7f41e802a440 comp rx=0 tx=0).stop 2024-04-25T00:38:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.439+0000 7f4204200640 1 -- 172.21.15.42:0/3220595481 shutdown_connections 2024-04-25T00:38:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.439+0000 7f4204200640 1 --2- 172.21.15.42:0/3220595481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41fc10dda0 0x7f41fc110190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.439+0000 7f4204200640 1 --2- 172.21.15.42:0/3220595481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41fc0741a0 0x7f41fc074620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.439+0000 7f4204200640 1 --2- 172.21.15.42:0/3220595481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc075780 0x7f41fc073bd0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.439+0000 7f4204200640 1 -- 172.21.15.42:0/3220595481 >> 172.21.15.42:0/3220595481 conn(0x7f41fc0fcd60 msgr2=0x7f41fc0ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:28.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.440+0000 7f4204200640 1 -- 172.21.15.42:0/3220595481 shutdown_connections 2024-04-25T00:38:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.440+0000 7f4204200640 1 -- 172.21.15.42:0/3220595481 wait complete. 2024-04-25T00:38:28.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.441+0000 7f4204200640 1 Processor -- start 2024-04-25T00:38:28.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.441+0000 7f4204200640 1 -- start start 2024-04-25T00:38:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.442+0000 7f4204200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41fc0741a0 0x7f41fc19e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.442+0000 7f4204200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41fc075780 0x7f41fc19e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.442+0000 7f4204200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc10dda0 0x7f41fc1a55c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.442+0000 7f4204200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc113af0 con 0x7f41fc075780 2024-04-25T00:38:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.442+0000 7f4204200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc113970 con 0x7f41fc10dda0 2024-04-25T00:38:28.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.442+0000 7f4204200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc113c70 con 0x7f41fc0741a0 2024-04-25T00:38:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.443+0000 7f4202200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc10dda0 0x7f41fc1a55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.443+0000 7f4202200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc10dda0 0x7f41fc1a55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34540/0 (socket says 172.21.15.42:34540) 2024-04-25T00:38:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.443+0000 7f4202200640 1 -- 172.21.15.42:0/1175191273 learned_addr learned my addr 172.21.15.42:0/1175191273 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.443+0000 7f4200e00640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41fc075780 0x7f41fc19e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.444+0000 7f4201800640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41fc0741a0 0x7f41fc19e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.444+0000 7f4201800640 1 -- 172.21.15.42:0/1175191273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc10dda0 msgr2=0x7f41fc1a55c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.444+0000 7f4201800640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc10dda0 0x7f41fc1a55c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.444+0000 7f4201800640 1 -- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41fc075780 msgr2=0x7f41fc19e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.444+0000 7f4201800640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41fc075780 0x7f41fc19e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.445+0000 7f4201800640 1 -- 172.21.15.42:0/1175191273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc06b260 con 0x7f41fc0741a0 2024-04-25T00:38:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.445+0000 7f4202200640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc10dda0 0x7f41fc1a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.445+0000 7f4200e00640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41fc075780 0x7f41fc19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.445+0000 7f4201800640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41fc0741a0 0x7f41fc19e010 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f41e802a960 tx=0x7f41e8032e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.446+0000 7f41fa000640 1 -- 172.21.15.42:0/1175191273 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e803f030 con 0x7f41fc0741a0 2024-04-25T00:38:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.446+0000 7f41fa000640 1 -- 172.21.15.42:0/1175191273 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e8003ce0 con 0x7f41fc0741a0 2024-04-25T00:38:28.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.446+0000 7f41fa000640 1 -- 172.21.15.42:0/1175191273 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8035070 con 0x7f41fc0741a0 2024-04-25T00:38:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.446+0000 7f4204200640 1 -- 172.21.15.42:0/1175191273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc06b4f0 con 0x7f41fc0741a0 2024-04-25T00:38:28.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.447+0000 7f4204200640 1 -- 172.21.15.42:0/1175191273 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc06b9e0 con 0x7f41fc0741a0 2024-04-25T00:38:28.452 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.452+0000 7f41fa000640 1 -- 172.21.15.42:0/1175191273 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f41e8042030 con 0x7f41fc0741a0 2024-04-25T00:38:28.453 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.453+0000 7f41fa000640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2cb0 0x7f41e40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.454+0000 7f41fa000640 1 -- 172.21.15.42:0/1175191273 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f41e80ef240 con 0x7f41fc0741a0 2024-04-25T00:38:28.454 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.454+0000 7f4200e00640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2cb0 0x7f41e40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.455 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.455+0000 7f4200e00640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2cb0 0x7f41e40a5170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f41fc19f520 tx=0x7f41f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.471+0000 7f420703d740 1 Processor -- start 2024-04-25T00:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.471+0000 7f420703d740 1 -- start start 2024-04-25T00:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.472+0000 7f420703d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.472+0000 7f420703d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 0x557c5e1d0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.472+0000 7f420703d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1d2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.472+0000 7f420703d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557c5e12dab0 con 0x557c5e1caca0 2024-04-25T00:38:28.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.472+0000 7f420703d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557c5e0d6640 con 0x557c5e1d0800 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.472+0000 7f420703d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557c5e129a70 con 0x557c5e1cb690 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.473+0000 7f4202200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1d2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.473+0000 7f4202200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1d2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34544/0 (socket says 172.21.15.42:34544) 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.473+0000 7f4202200640 1 -- 172.21.15.42:0/1186829108 learned_addr learned my addr 172.21.15.42:0/1186829108 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.473+0000 7f4200e00640 1 --2- 172.21.15.42:0/1186829108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 0x557c5e1d0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.473+0000 7f4201800640 1 --2- 172.21.15.42:0/1186829108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.473+0000 7f4202200640 1 -- 172.21.15.42:0/1186829108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 msgr2=0x557c5e1d0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.473+0000 7f4202200640 1 --2- 172.21.15.42:0/1186829108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 0x557c5e1d0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.473+0000 7f4202200640 1 -- 172.21.15.42:0/1186829108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 msgr2=0x557c5e1cb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.473+0000 7f4202200640 1 --2- 172.21.15.42:0/1186829108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1cb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.473+0000 7f4202200640 1 -- 172.21.15.42:0/1186829108 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5de42210 con 0x557c5e1d0800 2024-04-25T00:38:28.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.474+0000 7f4202200640 1 --2- 172.21.15.42:0/1186829108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1d2c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f41f400e350 tx=0x7f41f400e810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6254fc9ae9d5c144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.474+0000 7f41eea00640 1 -- 172.21.15.42:0/1186829108 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4014070 con 0x557c5e1d0800 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.474+0000 7f41eea00640 1 -- 172.21.15.42:0/1186829108 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f4004d20 con 0x557c5e1d0800 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.474+0000 7f41eea00640 1 -- 172.21.15.42:0/1186829108 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f40057a0 con 0x557c5e1d0800 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.475+0000 7f420703d740 1 -- 172.21.15.42:0/1186829108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 msgr2=0x557c5e1d2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.475+0000 7f420703d740 1 --2- 172.21.15.42:0/1186829108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1d2c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f41f400e350 tx=0x7f41f400e810 comp rx=0 tx=0).stop 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.475+0000 7f420703d740 1 -- 172.21.15.42:0/1186829108 shutdown_connections 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.475+0000 7f420703d740 1 --2- 172.21.15.42:0/1186829108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1d2c90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.475+0000 7f420703d740 1 --2- 172.21.15.42:0/1186829108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 0x557c5e1d0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.475+0000 7f420703d740 1 --2- 172.21.15.42:0/1186829108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.475+0000 7f420703d740 1 -- 172.21.15.42:0/1186829108 >> 172.21.15.42:0/1186829108 conn(0x557c5e1c6310 msgr2=0x557c5e1c6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.475+0000 7f420703d740 1 -- 172.21.15.42:0/1186829108 shutdown_connections 2024-04-25T00:38:28.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.475+0000 7f420703d740 1 -- 172.21.15.42:0/1186829108 wait complete. 2024-04-25T00:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.476+0000 7f420703d740 1 -- start start 2024-04-25T00:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.476+0000 7f420703d740 1 Processor -- start 2024-04-25T00:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.477+0000 7f420703d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1d4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.477+0000 7f420703d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 0x557c5e1d5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.477+0000 7f420703d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dc470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.477+0000 7f420703d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557c5e1d37b0 con 0x557c5e1caca0 2024-04-25T00:38:28.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.477+0000 7f420703d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557c5e0e9990 con 0x557c5e1d0800 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.477+0000 7f420703d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557c5e1d3930 con 0x557c5e1cb690 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.477+0000 7f420703d740 10 client.0 _ll_register_callbacks cb 0x557c5e0eb950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.477+0000 7f4202200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.477+0000 7f4202200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dc470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34548/0 (socket says 172.21.15.42:34548) 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.478+0000 7f4202200640 1 -- 172.21.15.42:0/2591377230 learned_addr learned my addr 172.21.15.42:0/2591377230 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.478+0000 7f4201800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1d4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.478+0000 7f4200e00640 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 0x557c5e1d5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.478+0000 7f4202200640 1 -- 172.21.15.42:0/2591377230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 msgr2=0x557c5e1d5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.478+0000 7f4202200640 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 0x557c5e1d5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.478+0000 7f4202200640 1 -- 172.21.15.42:0/2591377230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 msgr2=0x557c5e1d4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.478+0000 7f4202200640 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1d4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.478+0000 7f4202200640 1 -- 172.21.15.42:0/2591377230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c5df0e620 con 0x557c5e1d0800 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.478+0000 7f4200e00640 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 0x557c5e1d5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:28.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.478+0000 7f4201800640 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1d4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.479+0000 7f4202200640 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dc470 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41f4004c70 tx=0x7f41f4004ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb8e7f2002756e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.479+0000 7f41e3400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:38:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.479+0000 7f41e3400640 1 -- 172.21.15.42:0/2591377230 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4014070 con 0x557c5e1d0800 2024-04-25T00:38:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.479+0000 7f41e3400640 1 -- 172.21.15.42:0/2591377230 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f41f4003d80 con 0x557c5e1d0800 2024-04-25T00:38:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.479+0000 7f420703d740 1 -- 172.21.15.42:0/2591377230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f4004d20 con 0x557c5e1d0800 2024-04-25T00:38:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.479+0000 7f41e3400640 1 -- 172.21.15.42:0/2591377230 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f41f4004ed0 con 0x557c5e1d0800 2024-04-25T00:38:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.480+0000 7f420703d740 10 client.26714 fetch_fsmap learned FSMap version 45 2024-04-25T00:38:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.480+0000 7f420703d740 1 -- 172.21.15.42:0/2591377230 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557c5e1df450 con 0x557c5e1d0800 2024-04-25T00:38:28.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.480+0000 7f41e3400640 1 -- 172.21.15.42:0/2591377230 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f4004d20 con 0x557c5e1d0800 2024-04-25T00:38:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.481+0000 7f41e3400640 1 -- 172.21.15.42:0/2591377230 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f41f4010700 con 0x557c5e1d0800 2024-04-25T00:38:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.481+0000 7f420703d740 10 client.26714 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:38:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.481+0000 7f420703d740 10 client.26714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:38:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.481+0000 7f420703d740 20 client.26714 populate_metadata read hostname 'smithi042' 2024-04-25T00:38:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.481+0000 7f420703d740 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557c5e1e0850 0x557c5e1e2c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.481+0000 7f420703d740 4 client.26714 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:38:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.481+0000 7f420703d740 1 -- 172.21.15.42:0/2591377230 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c5e13aff0 con 0x557c5e1e0850 2024-04-25T00:38:28.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.482+0000 7f4201800640 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557c5e1e0850 0x557c5e1e2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.482+0000 7f4201800640 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557c5e1e0850 0x557c5e1e2c40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f3e45214573eed server_cookie=6a6fdfb283d0b685 in_seq=0 out_seq=0 2024-04-25T00:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.482+0000 7f41e3400640 10 client.26714 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.483+0000 7f41e3400640 1 -- 172.21.15.42:0/2591377230 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557c5e13aff0 con 0x557c5e1e0850 2024-04-25T00:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.483+0000 7f41e3400640 10 client.26714 handle_command_reply: tid=0 2024-04-25T00:38:28.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.483+0000 7f420703d740 1 client.26714 shutdown 2024-04-25T00:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.483+0000 7f420703d740 1 -- 172.21.15.42:0/2591377230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 msgr2=0x557c5e1dc470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.483+0000 7f420703d740 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dc470 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f41f4004c70 tx=0x7f41f4004ca0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.484+0000 7f420703d740 1 -- 172.21.15.42:0/2591377230 shutdown_connections 2024-04-25T00:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.484+0000 7f420703d740 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557c5e1e0850 0x557c5e1e2c40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.484+0000 7f420703d740 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dc470 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.484+0000 7f420703d740 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1cb690 0x557c5e1d5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.484+0000 7f41e3400640 0 client.26714 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.484+0000 7f420703d740 1 --2- 172.21.15.42:0/2591377230 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1d4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.484+0000 7f420703d740 1 -- 172.21.15.42:0/2591377230 >> 172.21.15.42:0/2591377230 conn(0x557c5e1c6310 msgr2=0x557c5e1c5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.484+0000 7f420703d740 1 -- 172.21.15.42:0/2591377230 shutdown_connections 2024-04-25T00:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.484+0000 7f420703d740 1 -- 172.21.15.42:0/2591377230 wait complete. 2024-04-25T00:38:28.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.484+0000 7f420703d740 20 client.26714 trim_cache size 0 max 16384 2024-04-25T00:38:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.505+0000 7f420703d740 1 Processor -- start 2024-04-25T00:38:28.506 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.506+0000 7f420703d740 1 -- start start 2024-04-25T00:38:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.506+0000 7f420703d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1c9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.506+0000 7f420703d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1cb690 0x557c5e1dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.506+0000 7f420703d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1d0800 0x557c5e1e7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.507 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.506+0000 7f420703d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1ba240 con 0x557c5e1caca0 2024-04-25T00:38:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.506+0000 7f420703d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b7b10 con 0x557c5e1cb690 2024-04-25T00:38:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.506+0000 7f420703d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc072790 con 0x557c5e1d0800 2024-04-25T00:38:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.507+0000 7f4200e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1cb690 0x557c5e1dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.507+0000 7f4200e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1cb690 0x557c5e1dc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34558/0 (socket says 172.21.15.42:34558) 2024-04-25T00:38:28.508 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.507+0000 7f4200e00640 1 -- 172.21.15.42:0/3250829546 learned_addr learned my addr 172.21.15.42:0/3250829546 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.507+0000 7f4200e00640 1 -- 172.21.15.42:0/3250829546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1d0800 msgr2=0x557c5e1e7480 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:38:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.508+0000 7f4200e00640 1 --2- 172.21.15.42:0/3250829546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1d0800 0x557c5e1e7480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.508+0000 7f4200e00640 1 -- 172.21.15.42:0/3250829546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 msgr2=0x557c5e1c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.508+0000 7f4200e00640 1 --2- 172.21.15.42:0/3250829546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.508+0000 7f4200e00640 1 -- 172.21.15.42:0/3250829546 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c5e1e2fb0 con 0x557c5e1cb690 2024-04-25T00:38:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.508+0000 7f4200e00640 1 --2- 172.21.15.42:0/3250829546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1cb690 0x557c5e1dc430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f41f000db20 tx=0x7f41f0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=437b3a111056c7bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.508+0000 7f41ee000640 1 -- 172.21.15.42:0/3250829546 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0014c80 con 0x557c5e1cb690 2024-04-25T00:38:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.509+0000 7f41ee000640 1 -- 172.21.15.42:0/3250829546 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f0027070 con 0x557c5e1cb690 2024-04-25T00:38:28.509 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.509+0000 7f41ee000640 1 -- 172.21.15.42:0/3250829546 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f00124d0 con 0x557c5e1cb690 2024-04-25T00:38:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.509+0000 7f420703d740 1 -- 172.21.15.42:0/3250829546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1cb690 msgr2=0x557c5e1dc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.509+0000 7f420703d740 1 --2- 172.21.15.42:0/3250829546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1cb690 0x557c5e1dc430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f41f000db20 tx=0x7f41f0013000 comp rx=0 tx=0).stop 2024-04-25T00:38:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.510+0000 7f420703d740 1 -- 172.21.15.42:0/3250829546 shutdown_connections 2024-04-25T00:38:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.510+0000 7f420703d740 1 --2- 172.21.15.42:0/3250829546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1d0800 0x557c5e1e7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.510+0000 7f420703d740 1 --2- 172.21.15.42:0/3250829546 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1cb690 0x557c5e1dc430 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.510+0000 7f420703d740 1 --2- 172.21.15.42:0/3250829546 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1caca0 0x557c5e1c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.510 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.510+0000 7f420703d740 1 -- 172.21.15.42:0/3250829546 >> 172.21.15.42:0/3250829546 conn(0x557c5e1c6310 msgr2=0x557c5e1c8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.511+0000 7f420703d740 1 -- 172.21.15.42:0/3250829546 shutdown_connections 2024-04-25T00:38:28.511 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.511+0000 7f420703d740 1 -- 172.21.15.42:0/3250829546 wait complete. 2024-04-25T00:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.512+0000 7f420703d740 1 -- start start 2024-04-25T00:38:28.512 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.512+0000 7f420703d740 1 Processor -- start 2024-04-25T00:38:28.513 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.513+0000 7f420703d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1caca0 0x557c5e1d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.513+0000 7f420703d740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1cb690 0x557c5e1d9f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.513+0000 7f420703d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.513+0000 7f420703d740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc072e00 con 0x557c5e1cb690 2024-04-25T00:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.513+0000 7f4202200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.513+0000 7f4202200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34568/0 (socket says 172.21.15.42:34568) 2024-04-25T00:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.513+0000 7f4202200640 1 -- 172.21.15.42:0/3930304964 learned_addr learned my addr 172.21.15.42:0/3930304964 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f420703d740 1 -- 172.21.15.42:0/3930304964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1b7440 con 0x557c5e1d0800 2024-04-25T00:38:28.514 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f420703d740 1 -- 172.21.15.42:0/3930304964 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc0734d0 con 0x557c5e1caca0 2024-04-25T00:38:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f420703d740 10 client.0 _ll_register_callbacks cb 0x7f41f0009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f4200e00640 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1cb690 0x557c5e1d9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.515 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f4201800640 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1caca0 0x557c5e1d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f4202200640 1 -- 172.21.15.42:0/3930304964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1caca0 msgr2=0x557c5e1d99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f4202200640 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1caca0 0x557c5e1d99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f4202200640 1 -- 172.21.15.42:0/3930304964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1cb690 msgr2=0x557c5e1d9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f4202200640 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1cb690 0x557c5e1d9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f4202200640 1 -- 172.21.15.42:0/3930304964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c5e1db420 con 0x557c5e1d0800 2024-04-25T00:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f4202200640 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dae30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f41f400e830 tx=0x7f41f4005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a8bef0f47c865dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.514+0000 7f41e3400640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.515+0000 7f4201800640 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1caca0 0x557c5e1d99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.515+0000 7f41e3400640 1 -- 172.21.15.42:0/3930304964 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4014030 con 0x557c5e1d0800 2024-04-25T00:38:28.516 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.515+0000 7f41e3400640 1 -- 172.21.15.42:0/3930304964 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f41f4010d50 con 0x557c5e1d0800 2024-04-25T00:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.515+0000 7f420703d740 1 -- 172.21.15.42:0/3930304964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f0027070 con 0x557c5e1d0800 2024-04-25T00:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.515+0000 7f41e3400640 1 -- 172.21.15.42:0/3930304964 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f41f4005820 con 0x557c5e1d0800 2024-04-25T00:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.515+0000 7f420703d740 10 client.26720 fetch_fsmap learned FSMap version 45 2024-04-25T00:38:28.517 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.515+0000 7f420703d740 1 -- 172.21.15.42:0/3930304964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557c5e1dc730 con 0x557c5e1d0800 2024-04-25T00:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.516+0000 7f41e3400640 1 -- 172.21.15.42:0/3930304964 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f0027070 con 0x557c5e1d0800 2024-04-25T00:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.517+0000 7f41e3400640 1 -- 172.21.15.42:0/3930304964 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f41f4019450 con 0x557c5e1d0800 2024-04-25T00:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.517+0000 7f420703d740 10 client.26720 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.517+0000 7f420703d740 10 client.26720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.517+0000 7f420703d740 20 client.26720 populate_metadata read hostname 'smithi042' 2024-04-25T00:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.517+0000 7f420703d740 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557c5e2473c0 0x557c5e2497e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.517+0000 7f420703d740 4 client.26720 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.517+0000 7f420703d740 1 -- 172.21.15.42:0/3930304964 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557c5e135340 con 0x557c5e2473c0 2024-04-25T00:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.517+0000 7f4201800640 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557c5e2473c0 0x557c5e2497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:28.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.518+0000 7f4201800640 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557c5e2473c0 0x557c5e2497e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55451ad702b72536 server_cookie=fdf1dfda2d7b5b48 in_seq=0 out_seq=0 2024-04-25T00:38:28.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.518+0000 7f41e3400640 10 client.26720 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:28.782 INFO:tasks.workunit.client.0.smithi042.stdout: 10 239689 34.15 MB/sec execute 258 sec latency 162.796 ms 2024-04-25T00:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f41e3400640 1 -- 172.21.15.42:0/3930304964 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557c5e135340 con 0x557c5e2473c0 2024-04-25T00:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f41e3400640 10 client.26720 handle_command_reply: tid=0 2024-04-25T00:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f420703d740 1 client.26720 shutdown 2024-04-25T00:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f420703d740 1 -- 172.21.15.42:0/3930304964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 msgr2=0x557c5e1dae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f420703d740 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dae30 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f41f400e830 tx=0x7f41f4005b50 comp rx=0 tx=0).stop 2024-04-25T00:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f420703d740 1 -- 172.21.15.42:0/3930304964 shutdown_connections 2024-04-25T00:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f420703d740 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557c5e2473c0 0x557c5e2497e0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f420703d740 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557c5e1d0800 0x557c5e1dae30 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f420703d740 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557c5e1cb690 0x557c5e1d9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f420703d740 1 --2- 172.21.15.42:0/3930304964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557c5e1caca0 0x557c5e1d99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f41e3400640 0 client.26720 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f420703d740 1 -- 172.21.15.42:0/3930304964 >> 172.21.15.42:0/3930304964 conn(0x557c5e1c6310 msgr2=0x557c5e1c9880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.838+0000 7f420703d740 1 -- 172.21.15.42:0/3930304964 shutdown_connections 2024-04-25T00:38:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.839+0000 7f420703d740 1 -- 172.21.15.42:0/3930304964 wait complete. 2024-04-25T00:38:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.839+0000 7f420703d740 20 client.26720 trim_cache size 0 max 16384 2024-04-25T00:38:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:38:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:38:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:38:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:38:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.840+0000 7f4204200640 1 -- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2cb0 msgr2=0x7f41e40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.840+0000 7f4204200640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2cb0 0x7f41e40a5170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f41fc19f520 tx=0x7f41f0008040 comp rx=0 tx=0).stop 2024-04-25T00:38:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.840+0000 7f4204200640 1 -- 172.21.15.42:0/1175191273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41fc0741a0 msgr2=0x7f41fc19e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.840+0000 7f4204200640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41fc0741a0 0x7f41fc19e010 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f41e802a960 tx=0x7f41e8032e20 comp rx=0 tx=0).stop 2024-04-25T00:38:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.840+0000 7f4204200640 1 -- 172.21.15.42:0/1175191273 shutdown_connections 2024-04-25T00:38:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.840+0000 7f4204200640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2cb0 0x7f41e40a5170 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.840+0000 7f4204200640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc10dda0 0x7f41fc1a55c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.840+0000 7f4204200640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f41fc075780 0x7f41fc19e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.840+0000 7f4204200640 1 --2- 172.21.15.42:0/1175191273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f41fc0741a0 0x7f41fc19e010 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.840+0000 7f4204200640 1 -- 172.21.15.42:0/1175191273 >> 172.21.15.42:0/1175191273 conn(0x7f41fc0fcd60 msgr2=0x7f41fc101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.841+0000 7f4204200640 1 -- 172.21.15.42:0/1175191273 shutdown_connections 2024-04-25T00:38:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:28.841+0000 7f4204200640 1 -- 172.21.15.42:0/1175191273 wait complete. 2024-04-25T00:38:29.782 INFO:tasks.workunit.client.0.smithi042.stdout: 10 240114 34.08 MB/sec execute 259 sec latency 190.978 ms 2024-04-25T00:38:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:29 smithi042 ceph-mon[28130]: pgmap v982: 97 pgs: 97 active+clean; 678 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 5.9 MiB/s wr, 566 op/s 2024-04-25T00:38:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:29 smithi136 ceph-mon[32923]: pgmap v982: 97 pgs: 97 active+clean; 678 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 5.9 MiB/s wr, 566 op/s 2024-04-25T00:38:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:29 smithi106 ceph-mon[32510]: pgmap v982: 97 pgs: 97 active+clean; 678 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 5.9 MiB/s wr, 566 op/s 2024-04-25T00:38:30.782 INFO:tasks.workunit.client.0.smithi042.stdout: 10 240775 34.05 MB/sec execute 260 sec latency 159.777 ms 2024-04-25T00:38:31.782 INFO:tasks.workunit.client.0.smithi042.stdout: 10 241641 34.15 MB/sec execute 261 sec latency 54.620 ms 2024-04-25T00:38:32.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:31 smithi042 ceph-mon[28130]: pgmap v983: 97 pgs: 97 active+clean; 679 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 467 op/s 2024-04-25T00:38:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:31 smithi136 ceph-mon[32923]: pgmap v983: 97 pgs: 97 active+clean; 679 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 467 op/s 2024-04-25T00:38:32.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:31 smithi106 ceph-mon[32510]: pgmap v983: 97 pgs: 97 active+clean; 679 MiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 467 op/s 2024-04-25T00:38:32.782 INFO:tasks.workunit.client.0.smithi042.stdout: 10 242407 34.16 MB/sec execute 262 sec latency 49.354 ms 2024-04-25T00:38:33.784 INFO:tasks.workunit.client.0.smithi042.stdout: 10 242989 34.17 MB/sec execute 263 sec latency 48.323 ms 2024-04-25T00:38:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:33 smithi042 ceph-mon[28130]: pgmap v984: 97 pgs: 97 active+clean; 679 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 548 op/s 2024-04-25T00:38:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:33 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:38:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:33 smithi136 ceph-mon[32923]: pgmap v984: 97 pgs: 97 active+clean; 679 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 548 op/s 2024-04-25T00:38:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:33 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:38:34.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:33 smithi106 ceph-mon[32510]: pgmap v984: 97 pgs: 97 active+clean; 679 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.9 MiB/s wr, 548 op/s 2024-04-25T00:38:34.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:33 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:38:34.784 INFO:tasks.workunit.client.0.smithi042.stdout: 10 243478 34.15 MB/sec execute 264 sec latency 76.543 ms 2024-04-25T00:38:35.784 INFO:tasks.workunit.client.0.smithi042.stdout: 10 244110 34.12 MB/sec execute 265 sec latency 91.029 ms 2024-04-25T00:38:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:35 smithi042 ceph-mon[28130]: pgmap v985: 97 pgs: 97 active+clean; 681 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 481 op/s 2024-04-25T00:38:36.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:35 smithi136 ceph-mon[32923]: pgmap v985: 97 pgs: 97 active+clean; 681 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 481 op/s 2024-04-25T00:38:36.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:35 smithi106 ceph-mon[32510]: pgmap v985: 97 pgs: 97 active+clean; 681 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.4 MiB/s wr, 481 op/s 2024-04-25T00:38:36.784 INFO:tasks.workunit.client.0.smithi042.stdout: 10 244837 34.13 MB/sec execute 266 sec latency 87.489 ms 2024-04-25T00:38:37.784 INFO:tasks.workunit.client.0.smithi042.stdout: 10 245656 34.18 MB/sec execute 267 sec latency 45.926 ms 2024-04-25T00:38:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:37 smithi042 ceph-mon[28130]: pgmap v986: 97 pgs: 97 active+clean; 680 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 551 op/s 2024-04-25T00:38:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:38:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:38:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:38:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:38:38.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:38:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:37 smithi136 ceph-mon[32923]: pgmap v986: 97 pgs: 97 active+clean; 680 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 551 op/s 2024-04-25T00:38:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:38:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:38:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:38:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:38:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:38:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:37 smithi106 ceph-mon[32510]: pgmap v986: 97 pgs: 97 active+clean; 680 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.0 MiB/s wr, 551 op/s 2024-04-25T00:38:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:38:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:38:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:38:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:38:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:38:38.784 INFO:tasks.workunit.client.0.smithi042.stdout: 10 246348 34.20 MB/sec execute 268 sec latency 36.952 ms 2024-04-25T00:38:39.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 247015 34.22 MB/sec execute 269 sec latency 47.071 ms 2024-04-25T00:38:40.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:39 smithi042 ceph-mon[28130]: pgmap v987: 97 pgs: 97 active+clean; 671 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 582 op/s 2024-04-25T00:38:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:39 smithi136 ceph-mon[32923]: pgmap v987: 97 pgs: 97 active+clean; 671 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 582 op/s 2024-04-25T00:38:40.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:39 smithi106 ceph-mon[32510]: pgmap v987: 97 pgs: 97 active+clean; 671 MiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 582 op/s 2024-04-25T00:38:40.784 INFO:tasks.workunit.client.0.smithi042.stdout: 10 247603 34.18 MB/sec execute 270 sec latency 53.951 ms 2024-04-25T00:38:41.784 INFO:tasks.workunit.client.0.smithi042.stdout: 10 248323 34.21 MB/sec execute 271 sec latency 96.155 ms 2024-04-25T00:38:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.114+0000 7f1007400640 1 -- 172.21.15.42:0/294449057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10001039d0 msgr2=0x7f1000103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:42.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.114+0000 7f1007400640 1 --2- 172.21.15.42:0/294449057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10001039d0 0x7f1000103df0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009900 tx=0x7f0ff402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.115+0000 7f1007400640 1 -- 172.21.15.42:0/294449057 shutdown_connections 2024-04-25T00:38:42.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.115+0000 7f1007400640 1 --2- 172.21.15.42:0/294449057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1000105580 0x7f100010be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.115+0000 7f1007400640 1 --2- 172.21.15.42:0/294449057 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1000104bc0 0x7f1000105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.115+0000 7f1007400640 1 --2- 172.21.15.42:0/294449057 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10001039d0 0x7f1000103df0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.115+0000 7f1007400640 1 -- 172.21.15.42:0/294449057 >> 172.21.15.42:0/294449057 conn(0x7f10000fef60 msgr2=0x7f1000101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.115+0000 7f1007400640 1 -- 172.21.15.42:0/294449057 shutdown_connections 2024-04-25T00:38:42.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.115+0000 7f1007400640 1 -- 172.21.15.42:0/294449057 wait complete. 2024-04-25T00:38:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.116+0000 7f1007400640 1 Processor -- start 2024-04-25T00:38:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.117+0000 7f1007400640 1 -- start start 2024-04-25T00:38:42.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.117+0000 7f1007400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10001039d0 0x7f1000199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.117+0000 7f1007400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1000104bc0 0x7f100019a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.117+0000 7f1007400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1000105580 0x7f10001a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f1007400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f100010f6a0 con 0x7f10001039d0 2024-04-25T00:38:42.118 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f1007400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f100010f520 con 0x7f1000104bc0 2024-04-25T00:38:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f1007400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f100010f820 con 0x7f1000105580 2024-04-25T00:38:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1000104bc0 0x7f100019a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f1004a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10001039d0 0x7f1000199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1000104bc0 0x7f100019a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60908/0 (socket says 172.21.15.42:60908) 2024-04-25T00:38:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f1005400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1000105580 0x7f10001a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f0fffe00640 1 -- 172.21.15.42:0/12958332 learned_addr learned my addr 172.21.15.42:0/12958332 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f1005400640 1 -- 172.21.15.42:0/12958332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1000104bc0 msgr2=0x7f100019a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f1005400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1000104bc0 0x7f100019a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f1005400640 1 -- 172.21.15.42:0/12958332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10001039d0 msgr2=0x7f1000199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f1005400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10001039d0 0x7f1000199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.118+0000 7f1005400640 1 -- 172.21.15.42:0/12958332 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10001a1850 con 0x7f1000105580 2024-04-25T00:38:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.119+0000 7f1005400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1000105580 0x7f10001a11b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009fd0 tx=0x7f0ff800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.119+0000 7f0ffd600640 1 -- 172.21.15.42:0/12958332 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800ce20 con 0x7f1000105580 2024-04-25T00:38:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.119+0000 7f0ffd600640 1 -- 172.21.15.42:0/12958332 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff8005020 con 0x7f1000105580 2024-04-25T00:38:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.119+0000 7f1007400640 1 -- 172.21.15.42:0/12958332 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10001a1b40 con 0x7f1000105580 2024-04-25T00:38:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.119+0000 7f1007400640 1 -- 172.21.15.42:0/12958332 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10001a20b0 con 0x7f1000105580 2024-04-25T00:38:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.119+0000 7f0ffd600640 1 -- 172.21.15.42:0/12958332 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8010720 con 0x7f1000105580 2024-04-25T00:38:42.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.124+0000 7f100a21f740 1 Processor -- start 2024-04-25T00:38:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.125+0000 7f0ffd600640 1 -- 172.21.15.42:0/12958332 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0ff8004a40 con 0x7f1000105580 2024-04-25T00:38:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.125+0000 7f100a21f740 1 -- start start 2024-04-25T00:38:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.126+0000 7f100a21f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 0x55de665f90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.126+0000 7f100a21f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de665fe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.126+0000 7f100a21f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de66600c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.126+0000 7f100a21f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55de6655bab0 con 0x55de665fe800 2024-04-25T00:38:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.126+0000 7f100a21f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55de66504640 con 0x55de665f9690 2024-04-25T00:38:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.126+0000 7f100a21f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55de66557a70 con 0x55de665f8ca0 2024-04-25T00:38:42.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:41 smithi042 ceph-mon[28130]: pgmap v988: 97 pgs: 97 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.7 MiB/s wr, 516 op/s 2024-04-25T00:38:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.127+0000 7f1004a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de665fe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.127+0000 7f1004a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de665fe190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60910/0 (socket says 172.21.15.42:60910) 2024-04-25T00:38:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.127+0000 7f1004a00640 1 -- 172.21.15.42:0/3608051485 learned_addr learned my addr 172.21.15.42:0/3608051485 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.127+0000 7f0fffe00640 1 --2- 172.21.15.42:0/3608051485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de66600c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.127+0000 7f0fffe00640 1 --2- 172.21.15.42:0/3608051485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 0x55de665f90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.127+0000 7f0ffd600640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0fe80a2cb0 0x7f0fe80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.128+0000 7f0ffd600640 1 -- 172.21.15.42:0/12958332 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f0ff80cf610 con 0x7f1000105580 2024-04-25T00:38:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.128+0000 7f1004a00640 1 -- 172.21.15.42:0/3608051485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 msgr2=0x55de665f90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.128+0000 7f1004a00640 1 --2- 172.21.15.42:0/3608051485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 0x55de665f90c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.128+0000 7f1004a00640 1 -- 172.21.15.42:0/3608051485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 msgr2=0x55de66600c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.128+0000 7f1004a00640 1 --2- 172.21.15.42:0/3608051485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de66600c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.128+0000 7f1004a00640 1 -- 172.21.15.42:0/3608051485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de66270210 con 0x55de665f9690 2024-04-25T00:38:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.128+0000 7f0fffe00640 1 --2- 172.21.15.42:0/3608051485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de66600c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:38:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.129+0000 7f1005400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0fe80a2cb0 0x7f0fe80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.129+0000 7f1004a00640 1 --2- 172.21.15.42:0/3608051485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de665fe190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005ab0 tx=0x7f0ff40040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47f534d346ea8e66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.130+0000 7f0ff1c00640 1 -- 172.21.15.42:0/3608051485 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4035030 con 0x55de665f9690 2024-04-25T00:38:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.130+0000 7f0ff1c00640 1 -- 172.21.15.42:0/3608051485 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff4005520 con 0x55de665f9690 2024-04-25T00:38:42.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.130+0000 7f0ff1c00640 1 -- 172.21.15.42:0/3608051485 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff40316b0 con 0x55de665f9690 2024-04-25T00:38:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.131+0000 7f100a21f740 1 -- 172.21.15.42:0/3608051485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 msgr2=0x55de665fe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.131+0000 7f100a21f740 1 --2- 172.21.15.42:0/3608051485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de665fe190 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4005ab0 tx=0x7f0ff40040c0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.131+0000 7f1005400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0fe80a2cb0 0x7f0fe80a5170 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8098780 tx=0x7f0ff80bc040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.131+0000 7f100a21f740 1 -- 172.21.15.42:0/3608051485 shutdown_connections 2024-04-25T00:38:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.131+0000 7f100a21f740 1 --2- 172.21.15.42:0/3608051485 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de66600c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.131+0000 7f100a21f740 1 --2- 172.21.15.42:0/3608051485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de665fe190 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.131+0000 7f100a21f740 1 --2- 172.21.15.42:0/3608051485 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 0x55de665f90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.131+0000 7f100a21f740 1 -- 172.21.15.42:0/3608051485 >> 172.21.15.42:0/3608051485 conn(0x55de665f4310 msgr2=0x55de665f4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.131+0000 7f100a21f740 1 -- 172.21.15.42:0/3608051485 shutdown_connections 2024-04-25T00:38:42.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.131+0000 7f100a21f740 1 -- 172.21.15.42:0/3608051485 wait complete. 2024-04-25T00:38:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.132+0000 7f100a21f740 1 -- start start 2024-04-25T00:38:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.132+0000 7f100a21f740 1 Processor -- start 2024-04-25T00:38:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.133+0000 7f100a21f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665f8ca0 0x55de66602f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.133+0000 7f100a21f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de66603480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.133+0000 7f100a21f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665fe800 0x55de6660a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.133+0000 7f100a21f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55de66601750 con 0x55de665f8ca0 2024-04-25T00:38:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.133+0000 7f100a21f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55de66517990 con 0x55de665f9690 2024-04-25T00:38:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.133+0000 7f100a21f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55de666018d0 con 0x55de665fe800 2024-04-25T00:38:42.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.133+0000 7f100a21f740 10 client.0 _ll_register_callbacks cb 0x55de66519950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.133+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665f8ca0 0x55de66602f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.134+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665f8ca0 0x55de66602f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:35460/0 (socket says 172.21.15.42:35460) 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.134+0000 7f0fffe00640 1 -- 172.21.15.42:0/1712128168 learned_addr learned my addr 172.21.15.42:0/1712128168 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.134+0000 7f1004a00640 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de66603480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.134+0000 7f0fffe00640 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665fe800 0x55de6660a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.134+0000 7f1004a00640 1 -- 172.21.15.42:0/1712128168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665fe800 msgr2=0x55de6660a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.134+0000 7f1004a00640 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665fe800 0x55de6660a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.135+0000 7f1004a00640 1 -- 172.21.15.42:0/1712128168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665f8ca0 msgr2=0x55de66602f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.135+0000 7f1004a00640 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665f8ca0 0x55de66602f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.135+0000 7f1004a00640 1 -- 172.21.15.42:0/1712128168 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de6633c620 con 0x55de665f9690 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.135+0000 7f0fffe00640 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665fe800 0x55de6660a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.135+0000 7f1004a00640 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de66603480 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55de665f51e0 tx=0x7f0ff40045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df4e266c0a441669 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:42.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.135+0000 7f0fe6000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:38:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.135+0000 7f0fe6000640 1 -- 172.21.15.42:0/1712128168 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4035040 con 0x55de665f9690 2024-04-25T00:38:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.136+0000 7f0fe6000640 1 -- 172.21.15.42:0/1712128168 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f0ff40051b0 con 0x55de665f9690 2024-04-25T00:38:42.137 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.136+0000 7f100a21f740 1 -- 172.21.15.42:0/1712128168 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff4005520 con 0x55de665f9690 2024-04-25T00:38:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.136+0000 7f0fe6000640 1 -- 172.21.15.42:0/1712128168 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0ff40316b0 con 0x55de665f9690 2024-04-25T00:38:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.136+0000 7f100a21f740 10 client.26732 fetch_fsmap learned FSMap version 45 2024-04-25T00:38:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.136+0000 7f100a21f740 1 -- 172.21.15.42:0/1712128168 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55de6661e4b0 con 0x55de665f9690 2024-04-25T00:38:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.136+0000 7f0fe6000640 1 -- 172.21.15.42:0/1712128168 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff4005520 con 0x55de665f9690 2024-04-25T00:38:42.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.137+0000 7f0fe6000640 1 -- 172.21.15.42:0/1712128168 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f0ff4054030 con 0x55de665f9690 2024-04-25T00:38:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.137+0000 7f100a21f740 10 client.26732 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:38:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.137+0000 7f100a21f740 10 client.26732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:38:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.137+0000 7f100a21f740 20 client.26732 populate_metadata read hostname 'smithi042' 2024-04-25T00:38:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.137+0000 7f100a21f740 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55de6661f800 0x55de66621bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.137+0000 7f100a21f740 4 client.26732 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:38:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.137+0000 7f100a21f740 1 -- 172.21.15.42:0/1712128168 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de66568ff0 con 0x55de6661f800 2024-04-25T00:38:42.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.138+0000 7f0fffe00640 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55de6661f800 0x55de66621bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.138+0000 7f0fffe00640 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55de6661f800 0x55de66621bf0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2d3f4efa79e205a server_cookie=8d73137ec63fbab3 in_seq=0 out_seq=0 2024-04-25T00:38:42.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.139+0000 7f0fe6000640 10 client.26732 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f0fe6000640 1 -- 172.21.15.42:0/1712128168 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de66568ff0 con 0x55de6661f800 2024-04-25T00:38:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f0fe6000640 10 client.26732 handle_command_reply: tid=0 2024-04-25T00:38:42.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f100a21f740 1 client.26732 shutdown 2024-04-25T00:38:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f100a21f740 1 -- 172.21.15.42:0/1712128168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 msgr2=0x55de66603480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f100a21f740 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de66603480 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x55de665f51e0 tx=0x7f0ff40045a0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f100a21f740 1 -- 172.21.15.42:0/1712128168 shutdown_connections 2024-04-25T00:38:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f100a21f740 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55de6661f800 0x55de66621bf0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f100a21f740 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665fe800 0x55de6660a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f100a21f740 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de66603480 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f100a21f740 1 --2- 172.21.15.42:0/1712128168 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665f8ca0 0x55de66602f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.141+0000 7f100a21f740 1 -- 172.21.15.42:0/1712128168 >> 172.21.15.42:0/1712128168 conn(0x55de665f4310 msgr2=0x55de665f3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.142+0000 7f0fe6000640 0 client.26732 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:42.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.142+0000 7f100a21f740 1 -- 172.21.15.42:0/1712128168 shutdown_connections 2024-04-25T00:38:42.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.142+0000 7f100a21f740 1 -- 172.21.15.42:0/1712128168 wait complete. 2024-04-25T00:38:42.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.142+0000 7f100a21f740 20 client.26732 trim_cache size 0 max 16384 2024-04-25T00:38:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.162+0000 7f100a21f740 1 Processor -- start 2024-04-25T00:38:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.162+0000 7f100a21f740 1 -- start start 2024-04-25T00:38:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f100a21f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f8ca0 0x55de665f7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f100a21f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f9690 0x55de6660a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f100a21f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de665f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f100a21f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe0010c60 con 0x55de665fe800 2024-04-25T00:38:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f100a21f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe000e530 con 0x55de665f8ca0 2024-04-25T00:38:42.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f100a21f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe000c380 con 0x55de665f9690 2024-04-25T00:38:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f8ca0 0x55de665f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f1004a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f9690 0x55de6660a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f1004a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f9690 0x55de6660a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44514/0 (socket says 172.21.15.42:44514) 2024-04-25T00:38:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f8ca0 0x55de665f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60924/0 (socket says 172.21.15.42:60924) 2024-04-25T00:38:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f1004a00640 1 -- 172.21.15.42:0/1094403486 learned_addr learned my addr 172.21.15.42:0/1094403486 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.163+0000 7f0fffe00640 1 --2- 172.21.15.42:0/1094403486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de665f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.164+0000 7f1004a00640 1 -- 172.21.15.42:0/1094403486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f8ca0 msgr2=0x55de665f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.164+0000 7f1004a00640 1 --2- 172.21.15.42:0/1094403486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f8ca0 0x55de665f7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.166 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.164+0000 7f1004a00640 1 -- 172.21.15.42:0/1094403486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 msgr2=0x55de665f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.164+0000 7f1004a00640 1 --2- 172.21.15.42:0/1094403486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de665f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.164+0000 7f1004a00640 1 -- 172.21.15.42:0/1094403486 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de665ef620 con 0x55de665f9690 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.164+0000 7f1004a00640 1 --2- 172.21.15.42:0/1094403486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f9690 0x55de6660a480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40314e0 tx=0x7f0ff4005a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=170a2183d8149571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.164+0000 7f0ff1200640 1 -- 172.21.15.42:0/1094403486 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4035040 con 0x55de665f9690 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.164+0000 7f0ff1200640 1 -- 172.21.15.42:0/1094403486 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff4004fb0 con 0x55de665f9690 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.164+0000 7f0ff1200640 1 -- 172.21.15.42:0/1094403486 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff403b980 con 0x55de665f9690 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.165+0000 7f100a21f740 1 -- 172.21.15.42:0/1094403486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f9690 msgr2=0x55de6660a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.165+0000 7f100a21f740 1 --2- 172.21.15.42:0/1094403486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f9690 0x55de6660a480 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0ff40314e0 tx=0x7f0ff4005a20 comp rx=0 tx=0).stop 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.165+0000 7f100a21f740 1 -- 172.21.15.42:0/1094403486 shutdown_connections 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.165+0000 7f100a21f740 1 --2- 172.21.15.42:0/1094403486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de665f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.165+0000 7f100a21f740 1 --2- 172.21.15.42:0/1094403486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f9690 0x55de6660a480 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.165+0000 7f100a21f740 1 --2- 172.21.15.42:0/1094403486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f8ca0 0x55de665f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.165+0000 7f100a21f740 1 -- 172.21.15.42:0/1094403486 >> 172.21.15.42:0/1094403486 conn(0x55de665f4310 msgr2=0x55de665fa910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.165+0000 7f100a21f740 1 -- 172.21.15.42:0/1094403486 shutdown_connections 2024-04-25T00:38:42.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.165+0000 7f100a21f740 1 -- 172.21.15.42:0/1094403486 wait complete. 2024-04-25T00:38:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.166+0000 7f100a21f740 1 -- start start 2024-04-25T00:38:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.166+0000 7f100a21f740 1 Processor -- start 2024-04-25T00:38:42.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f100a21f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 0x55de66607e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f100a21f740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de666083d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f100a21f740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de666093e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f100a21f740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe000c9f0 con 0x55de665fe800 2024-04-25T00:38:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f100a21f740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe000de60 con 0x55de665f9690 2024-04-25T00:38:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f100a21f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe000d0c0 con 0x55de665f8ca0 2024-04-25T00:38:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f100a21f740 10 client.0 _ll_register_callbacks cb 0x7f0fec0098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 0x55de66607e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f0fffe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 0x55de66607e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44520/0 (socket says 172.21.15.42:44520) 2024-04-25T00:38:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f0fffe00640 1 -- 172.21.15.42:0/250752223 learned_addr learned my addr 172.21.15.42:0/250752223 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:42.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f1004a00640 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de666083d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f0fffe00640 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de666093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f1004a00640 1 -- 172.21.15.42:0/250752223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 msgr2=0x55de66607e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f1004a00640 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 0x55de66607e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f1004a00640 1 -- 172.21.15.42:0/250752223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 msgr2=0x55de666093e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f1004a00640 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de666093e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.167+0000 7f1004a00640 1 -- 172.21.15.42:0/250752223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de666099d0 con 0x55de665f9690 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.168+0000 7f0fffe00640 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de666093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.168+0000 7f1004a00640 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de666083d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55de665fded0 tx=0x7f0ff4031400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac9b77ae7331a22f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.168+0000 7f0fe6000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.168+0000 7f0fe6000640 1 -- 172.21.15.42:0/250752223 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4035040 con 0x55de665f9690 2024-04-25T00:38:42.173 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.168+0000 7f0fe6000640 1 -- 172.21.15.42:0/250752223 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f0ff4031c60 con 0x55de665f9690 2024-04-25T00:38:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.168+0000 7f100a21f740 1 -- 172.21.15.42:0/250752223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ff4004fb0 con 0x55de665f9690 2024-04-25T00:38:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.169+0000 7f0fe6000640 1 -- 172.21.15.42:0/250752223 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0ff403b6e0 con 0x55de665f9690 2024-04-25T00:38:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.169+0000 7f100a21f740 10 client.26738 fetch_fsmap learned FSMap version 45 2024-04-25T00:38:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.169+0000 7f100a21f740 1 -- 172.21.15.42:0/250752223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55de66685fe0 con 0x55de665f9690 2024-04-25T00:38:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.169+0000 7f0fe6000640 1 -- 172.21.15.42:0/250752223 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ff4004fb0 con 0x55de665f9690 2024-04-25T00:38:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.169+0000 7f0fe6000640 1 -- 172.21.15.42:0/250752223 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f0ff405e070 con 0x55de665f9690 2024-04-25T00:38:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.170+0000 7f100a21f740 10 client.26738 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:38:42.174 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.170+0000 7f100a21f740 10 client.26738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:38:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.170+0000 7f100a21f740 20 client.26738 populate_metadata read hostname 'smithi042' 2024-04-25T00:38:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.170+0000 7f100a21f740 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55de66686840 0x55de66688c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.170+0000 7f100a21f740 4 client.26738 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:38:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.170+0000 7f100a21f740 1 -- 172.21.15.42:0/250752223 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de66563340 con 0x55de66686840 2024-04-25T00:38:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.170+0000 7f0fffe00640 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55de66686840 0x55de66688c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.171+0000 7f0fffe00640 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55de66686840 0x55de66688c60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c9343249deebd41 server_cookie=669fe935fe91bd1f in_seq=0 out_seq=0 2024-04-25T00:38:42.175 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:42.171+0000 7f0fe6000640 10 client.26738 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:41 smithi136 ceph-mon[32923]: pgmap v988: 97 pgs: 97 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.7 MiB/s wr, 516 op/s 2024-04-25T00:38:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:41 smithi106 ceph-mon[32510]: pgmap v988: 97 pgs: 97 active+clean; 678 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 3.7 MiB/s wr, 516 op/s 2024-04-25T00:38:42.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 248592 34.14 MB/sec execute 272 sec latency 433.346 ms 2024-04-25T00:38:43.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.685+0000 7f0fe6000640 1 -- 172.21.15.42:0/250752223 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55de66563340 con 0x55de66686840 2024-04-25T00:38:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.685+0000 7f0fe6000640 10 client.26738 handle_command_reply: tid=0 2024-04-25T00:38:43.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.685+0000 7f100a21f740 1 client.26738 shutdown 2024-04-25T00:38:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.685+0000 7f100a21f740 1 -- 172.21.15.42:0/250752223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 msgr2=0x55de666083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.685+0000 7f100a21f740 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de666083d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55de665fded0 tx=0x7f0ff4031400 comp rx=0 tx=0).stop 2024-04-25T00:38:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.686+0000 7f100a21f740 1 -- 172.21.15.42:0/250752223 shutdown_connections 2024-04-25T00:38:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.686+0000 7f100a21f740 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55de66686840 0x55de66688c60 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.686+0000 7f100a21f740 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55de665fe800 0x55de666093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.686+0000 7f100a21f740 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55de665f9690 0x55de666083d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.686+0000 7f100a21f740 1 --2- 172.21.15.42:0/250752223 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de665f8ca0 0x55de66607e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.686+0000 7f100a21f740 1 -- 172.21.15.42:0/250752223 >> 172.21.15.42:0/250752223 conn(0x55de665f4310 msgr2=0x55de665f7260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:43.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.686+0000 7f0fe6000640 0 client.26738 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.686+0000 7f100a21f740 1 -- 172.21.15.42:0/250752223 shutdown_connections 2024-04-25T00:38:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.686+0000 7f100a21f740 1 -- 172.21.15.42:0/250752223 wait complete. 2024-04-25T00:38:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.687+0000 7f100a21f740 20 client.26738 trim_cache size 0 max 16384 2024-04-25T00:38:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:38:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:38:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:38:43.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:38:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.688+0000 7f1007400640 1 -- 172.21.15.42:0/12958332 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0fe80a2cb0 msgr2=0x7f0fe80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.688+0000 7f1007400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0fe80a2cb0 0x7f0fe80a5170 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8098780 tx=0x7f0ff80bc040 comp rx=0 tx=0).stop 2024-04-25T00:38:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.688+0000 7f1007400640 1 -- 172.21.15.42:0/12958332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1000105580 msgr2=0x7f10001a11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:43.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.688+0000 7f1007400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1000105580 0x7f10001a11b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8009fd0 tx=0x7f0ff800ef10 comp rx=0 tx=0).stop 2024-04-25T00:38:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.689+0000 7f1007400640 1 -- 172.21.15.42:0/12958332 shutdown_connections 2024-04-25T00:38:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.689+0000 7f1007400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f0fe80a2cb0 0x7f0fe80a5170 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.689+0000 7f1007400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1000105580 0x7f10001a11b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.689+0000 7f1007400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1000104bc0 0x7f100019a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.689+0000 7f1007400640 1 --2- 172.21.15.42:0/12958332 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10001039d0 0x7f1000199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.689+0000 7f1007400640 1 -- 172.21.15.42:0/12958332 >> 172.21.15.42:0/12958332 conn(0x7f10000fef60 msgr2=0x7f1000107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:43.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.690+0000 7f1007400640 1 -- 172.21.15.42:0/12958332 shutdown_connections 2024-04-25T00:38:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:43.690+0000 7f1007400640 1 -- 172.21.15.42:0/12958332 wait complete. 2024-04-25T00:38:43.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 248616 34.02 MB/sec execute 273 sec latency 750.123 ms 2024-04-25T00:38:44.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:43 smithi136 ceph-mon[32923]: pgmap v989: 97 pgs: 97 active+clean; 685 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 583 op/s 2024-04-25T00:38:44.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:43 smithi106 ceph-mon[32510]: pgmap v989: 97 pgs: 97 active+clean; 685 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 583 op/s 2024-04-25T00:38:44.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:43 smithi042 ceph-mon[28130]: pgmap v989: 97 pgs: 97 active+clean; 685 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.4 MiB/s wr, 583 op/s 2024-04-25T00:38:44.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 248760 33.92 MB/sec execute 274 sec latency 1484.097 ms 2024-04-25T00:38:45.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 249169 33.86 MB/sec execute 275 sec latency 154.248 ms 2024-04-25T00:38:46.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:45 smithi136 ceph-mon[32923]: pgmap v990: 97 pgs: 97 active+clean; 698 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.9 MiB/s wr, 493 op/s 2024-04-25T00:38:46.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:45 smithi106 ceph-mon[32510]: pgmap v990: 97 pgs: 97 active+clean; 698 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.9 MiB/s wr, 493 op/s 2024-04-25T00:38:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:45 smithi042 ceph-mon[28130]: pgmap v990: 97 pgs: 97 active+clean; 698 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.9 MiB/s wr, 493 op/s 2024-04-25T00:38:46.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 249804 33.86 MB/sec execute 276 sec latency 77.359 ms 2024-04-25T00:38:47.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 250606 33.91 MB/sec execute 277 sec latency 50.501 ms 2024-04-25T00:38:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:47 smithi136 ceph-mon[32923]: pgmap v991: 97 pgs: 97 active+clean; 696 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 781 op/s 2024-04-25T00:38:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:47 smithi106 ceph-mon[32510]: pgmap v991: 97 pgs: 97 active+clean; 696 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 781 op/s 2024-04-25T00:38:48.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:47 smithi042 ceph-mon[28130]: pgmap v991: 97 pgs: 97 active+clean; 696 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 13 MiB/s wr, 781 op/s 2024-04-25T00:38:48.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 251195 33.93 MB/sec execute 278 sec latency 76.991 ms 2024-04-25T00:38:49.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 251681 33.88 MB/sec execute 279 sec latency 190.904 ms 2024-04-25T00:38:50.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:49 smithi136 ceph-mon[32923]: pgmap v992: 97 pgs: 97 active+clean; 675 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 15 MiB/s wr, 874 op/s 2024-04-25T00:38:50.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:49 smithi106 ceph-mon[32510]: pgmap v992: 97 pgs: 97 active+clean; 675 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 15 MiB/s wr, 874 op/s 2024-04-25T00:38:50.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:49 smithi042 ceph-mon[28130]: pgmap v992: 97 pgs: 97 active+clean; 675 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 15 MiB/s wr, 874 op/s 2024-04-25T00:38:50.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 252235 33.86 MB/sec execute 280 sec latency 74.678 ms 2024-04-25T00:38:51.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 252998 33.87 MB/sec execute 281 sec latency 48.553 ms 2024-04-25T00:38:52.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:51 smithi136 ceph-mon[32923]: pgmap v993: 97 pgs: 97 active+clean; 669 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 893 op/s 2024-04-25T00:38:52.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:51 smithi106 ceph-mon[32510]: pgmap v993: 97 pgs: 97 active+clean; 669 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 893 op/s 2024-04-25T00:38:52.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:51 smithi042 ceph-mon[28130]: pgmap v993: 97 pgs: 97 active+clean; 669 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 15 MiB/s wr, 893 op/s 2024-04-25T00:38:52.785 INFO:tasks.workunit.client.0.smithi042.stdout: 10 253788 33.91 MB/sec execute 282 sec latency 52.720 ms 2024-04-25T00:38:53.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 254573 33.97 MB/sec execute 283 sec latency 37.519 ms 2024-04-25T00:38:54.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:53 smithi136 ceph-mon[32923]: pgmap v994: 97 pgs: 97 active+clean; 672 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 895 op/s 2024-04-25T00:38:54.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:53 smithi106 ceph-mon[32510]: pgmap v994: 97 pgs: 97 active+clean; 672 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 895 op/s 2024-04-25T00:38:54.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:53 smithi042 ceph-mon[28130]: pgmap v994: 97 pgs: 97 active+clean; 672 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 15 MiB/s wr, 895 op/s 2024-04-25T00:38:54.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 255169 33.95 MB/sec execute 284 sec latency 64.369 ms 2024-04-25T00:38:55.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 255657 33.91 MB/sec execute 285 sec latency 66.095 ms 2024-04-25T00:38:56.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:55 smithi136 ceph-mon[32923]: pgmap v995: 97 pgs: 97 active+clean; 675 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 822 op/s 2024-04-25T00:38:56.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:55 smithi106 ceph-mon[32510]: pgmap v995: 97 pgs: 97 active+clean; 675 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 822 op/s 2024-04-25T00:38:56.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:55 smithi042 ceph-mon[28130]: pgmap v995: 97 pgs: 97 active+clean; 675 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 822 op/s 2024-04-25T00:38:56.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 256374 33.92 MB/sec execute 286 sec latency 63.501 ms 2024-04-25T00:38:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.948+0000 7fc24d800640 1 -- 172.21.15.42:0/110245475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc248108740 msgr2=0x7fc24810ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.948+0000 7fc24d800640 1 --2- 172.21.15.42:0/110245475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc248108740 0x7fc24810ab70 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc24001acb0 tx=0x7fc24003b8e0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.950+0000 7fc24d800640 1 -- 172.21.15.42:0/110245475 shutdown_connections 2024-04-25T00:38:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.950+0000 7fc24d800640 1 --2- 172.21.15.42:0/110245475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24810da60 0x7fc24810dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.950+0000 7fc24d800640 1 --2- 172.21.15.42:0/110245475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc24810b0b0 0x7fc24810d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.950+0000 7fc24d800640 1 --2- 172.21.15.42:0/110245475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc248108740 0x7fc24810ab70 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.950+0000 7fc24d800640 1 -- 172.21.15.42:0/110245475 >> 172.21.15.42:0/110245475 conn(0x7fc24806e7b0 msgr2=0x7fc248070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:56.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.950+0000 7fc24d800640 1 -- 172.21.15.42:0/110245475 shutdown_connections 2024-04-25T00:38:56.951 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.950+0000 7fc24d800640 1 -- 172.21.15.42:0/110245475 wait complete. 2024-04-25T00:38:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.951+0000 7fc24d800640 1 Processor -- start 2024-04-25T00:38:56.952 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.952+0000 7fc24d800640 1 -- start start 2024-04-25T00:38:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.955+0000 7fc24d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc248108740 0x7fc24819dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.955 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.955+0000 7fc24d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24810b0b0 0x7fc24819e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.955+0000 7fc24d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24810da60 0x7fc2481a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.955+0000 7fc24d800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc248111a80 con 0x7fc248108740 2024-04-25T00:38:56.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.955+0000 7fc24d800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc248111900 con 0x7fc24810b0b0 2024-04-25T00:38:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.955+0000 7fc24d800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc248111c00 con 0x7fc24810da60 2024-04-25T00:38:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.956+0000 7fc247400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24810da60 0x7fc2481a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.956+0000 7fc247400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24810da60 0x7fc2481a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:40272/0 (socket says 172.21.15.42:40272) 2024-04-25T00:38:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.956+0000 7fc247400640 1 -- 172.21.15.42:0/1793256862 learned_addr learned my addr 172.21.15.42:0/1793256862 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.956+0000 7fc246a00640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc248108740 0x7fc24819dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.957+0000 7fc247400640 1 -- 172.21.15.42:0/1793256862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24810b0b0 msgr2=0x7fc24819e530 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:38:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.957+0000 7fc247400640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24810b0b0 0x7fc24819e530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.957+0000 7fc247400640 1 -- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc248108740 msgr2=0x7fc24819dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.957+0000 7fc247400640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc248108740 0x7fc24819dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.957+0000 7fc247400640 1 -- 172.21.15.42:0/1793256862 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2481a5c10 con 0x7fc24810da60 2024-04-25T00:38:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.957+0000 7fc246a00640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc248108740 0x7fc24819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:56.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.958+0000 7fc247400640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24810da60 0x7fc2481a55a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc23400e970 tx=0x7fc23400ed70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.958+0000 7fc23f400640 1 -- 172.21.15.42:0/1793256862 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23400ccc0 con 0x7fc24810da60 2024-04-25T00:38:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.958+0000 7fc23f400640 1 -- 172.21.15.42:0/1793256862 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc23400ce60 con 0x7fc24810da60 2024-04-25T00:38:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.958+0000 7fc23f400640 1 -- 172.21.15.42:0/1793256862 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234010490 con 0x7fc24810da60 2024-04-25T00:38:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.958+0000 7fc24d800640 1 -- 172.21.15.42:0/1793256862 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2481a5f00 con 0x7fc24810da60 2024-04-25T00:38:56.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.958+0000 7fc24d800640 1 -- 172.21.15.42:0/1793256862 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2481a63f0 con 0x7fc24810da60 2024-04-25T00:38:56.981 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.981+0000 7fc250788740 1 Processor -- start 2024-04-25T00:38:56.982 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.982+0000 7fc23f400640 1 -- 172.21.15.42:0/1793256862 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc234005000 con 0x7fc24810da60 2024-04-25T00:38:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.982+0000 7fc250788740 1 -- start start 2024-04-25T00:38:56.983 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.983+0000 7fc250788740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.983+0000 7fc250788740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 0x55c1d8ef9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.983+0000 7fc250788740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef9800 0x55c1d8efbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.983+0000 7fc250788740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8e56ab0 con 0x55c1d8ef9800 2024-04-25T00:38:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.983+0000 7fc250788740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8dff640 con 0x55c1d8ef3ca0 2024-04-25T00:38:56.984 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.983+0000 7fc250788740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8e52a70 con 0x55c1d8ef4690 2024-04-25T00:38:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.984+0000 7fc246000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.984+0000 7fc246000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:55262/0 (socket says 172.21.15.42:55262) 2024-04-25T00:38:56.985 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.984+0000 7fc246000640 1 -- 172.21.15.42:0/935543398 learned_addr learned my addr 172.21.15.42:0/935543398 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.985+0000 7fc246000640 1 --2- 172.21.15.42:0/935543398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef9800 0x55c1d8efbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.986+0000 7fc246a00640 1 --2- 172.21.15.42:0/935543398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 0x55c1d8ef9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.987 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.986+0000 7fc246a00640 1 -- 172.21.15.42:0/935543398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 msgr2=0x55c1d8ef40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:56.988 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.986+0000 7fc246a00640 1 --2- 172.21.15.42:0/935543398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.989 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.986+0000 7fc246a00640 1 -- 172.21.15.42:0/935543398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef9800 msgr2=0x55c1d8efbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.986+0000 7fc23f400640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc22c0a2da0 0x7fc22c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.987+0000 7fc23f400640 1 -- 172.21.15.42:0/1793256862 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc2340cf560 con 0x7fc24810da60 2024-04-25T00:38:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.987+0000 7fc246a00640 1 --2- 172.21.15.42:0/935543398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef9800 0x55c1d8efbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.987+0000 7fc246a00640 1 -- 172.21.15.42:0/935543398 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d8b6b210 con 0x55c1d8ef4690 2024-04-25T00:38:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.987+0000 7fc246a00640 1 --2- 172.21.15.42:0/935543398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 0x55c1d8ef9190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc24001acb0 tx=0x7fc24001a810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d4c03c55e0f053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.988+0000 7fc233e00640 1 -- 172.21.15.42:0/935543398 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2400428a0 con 0x55c1d8ef4690 2024-04-25T00:38:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.988+0000 7fc233e00640 1 -- 172.21.15.42:0/935543398 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc240047040 con 0x55c1d8ef4690 2024-04-25T00:38:56.991 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.988+0000 7fc233e00640 1 -- 172.21.15.42:0/935543398 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240004670 con 0x55c1d8ef4690 2024-04-25T00:38:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.989+0000 7fc250788740 1 -- 172.21.15.42:0/935543398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 msgr2=0x55c1d8ef9190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.989+0000 7fc250788740 1 --2- 172.21.15.42:0/935543398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 0x55c1d8ef9190 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc24001acb0 tx=0x7fc24001a810 comp rx=0 tx=0).stop 2024-04-25T00:38:56.992 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.991+0000 7fc246000640 1 --2- 172.21.15.42:0/935543398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef9800 0x55c1d8efbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:38:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.993+0000 7fc246000640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc22c0a2da0 0x7fc22c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:56.993 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.993+0000 7fc250788740 1 -- 172.21.15.42:0/935543398 shutdown_connections 2024-04-25T00:38:56.994 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.993+0000 7fc250788740 1 --2- 172.21.15.42:0/935543398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef9800 0x55c1d8efbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.994+0000 7fc250788740 1 --2- 172.21.15.42:0/935543398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 0x55c1d8ef9190 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.994+0000 7fc250788740 1 --2- 172.21.15.42:0/935543398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:56.995 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.994+0000 7fc250788740 1 -- 172.21.15.42:0/935543398 >> 172.21.15.42:0/935543398 conn(0x55c1d8eef310 msgr2=0x55c1d8eef710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.997+0000 7fc250788740 1 -- 172.21.15.42:0/935543398 shutdown_connections 2024-04-25T00:38:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.997+0000 7fc246000640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc22c0a2da0 0x7fc22c0a5260 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55c1d8ef4ef0 tx=0x7fc2380046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.997+0000 7fc250788740 1 -- 172.21.15.42:0/935543398 wait complete. 2024-04-25T00:38:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.998+0000 7fc250788740 1 -- start start 2024-04-25T00:38:56.999 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:56.998+0000 7fc250788740 1 Processor -- start 2024-04-25T00:38:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.000+0000 7fc250788740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8efdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.003 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.003+0000 7fc250788740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef4690 0x55c1d8efe3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.003+0000 7fc250788740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef9800 0x55c1d8f05420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.004+0000 7fc250788740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8efc750 con 0x55c1d8ef4690 2024-04-25T00:38:57.004 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.004+0000 7fc250788740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8e12990 con 0x55c1d8ef9800 2024-04-25T00:38:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.004+0000 7fc250788740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8efc8d0 con 0x55c1d8ef3ca0 2024-04-25T00:38:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.004+0000 7fc250788740 10 client.0 _ll_register_callbacks cb 0x55c1d8e14950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.004+0000 7fc247400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef9800 0x55c1d8f05420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.004+0000 7fc247400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef9800 0x55c1d8f05420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:55276/0 (socket says 172.21.15.42:55276) 2024-04-25T00:38:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.004+0000 7fc247400640 1 -- 172.21.15.42:0/347925779 learned_addr learned my addr 172.21.15.42:0/347925779 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.005+0000 7fc247400640 1 -- 172.21.15.42:0/347925779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef3ca0 msgr2=0x55c1d8efdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.005+0000 7fc247400640 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8efdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.005+0000 7fc247400640 1 -- 172.21.15.42:0/347925779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef4690 msgr2=0x55c1d8efe3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:57.005 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.005+0000 7fc247400640 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef4690 0x55c1d8efe3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.005+0000 7fc247400640 1 -- 172.21.15.42:0/347925779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d8c37620 con 0x55c1d8ef9800 2024-04-25T00:38:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.005+0000 7fc247400640 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef9800 0x55c1d8f05420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc23409bd80 tx=0x7fc23409c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876b50bb0f58228 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:57.006 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.005+0000 7fc230c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:38:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.006+0000 7fc230c00640 1 -- 172.21.15.42:0/347925779 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340a4a60 con 0x55c1d8ef9800 2024-04-25T00:38:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.006+0000 7fc230c00640 1 -- 172.21.15.42:0/347925779 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc234098460 con 0x55c1d8ef9800 2024-04-25T00:38:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.006+0000 7fc250788740 1 -- 172.21.15.42:0/347925779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc240047040 con 0x55c1d8ef9800 2024-04-25T00:38:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.007+0000 7fc230c00640 1 -- 172.21.15.42:0/347925779 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc2340052f0 con 0x55c1d8ef9800 2024-04-25T00:38:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.007+0000 7fc250788740 10 client.26747 fetch_fsmap learned FSMap version 45 2024-04-25T00:38:57.007 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.007+0000 7fc250788740 1 -- 172.21.15.42:0/347925779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c1d8f081d0 con 0x55c1d8ef9800 2024-04-25T00:38:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.007+0000 7fc230c00640 1 -- 172.21.15.42:0/347925779 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc240047040 con 0x55c1d8ef9800 2024-04-25T00:38:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.007+0000 7fc230c00640 1 -- 172.21.15.42:0/347925779 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fc23409b240 con 0x55c1d8ef9800 2024-04-25T00:38:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.008+0000 7fc250788740 10 client.26747 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:38:57.008 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.008+0000 7fc250788740 10 client.26747 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:38:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.008+0000 7fc250788740 20 client.26747 populate_metadata read hostname 'smithi042' 2024-04-25T00:38:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.008+0000 7fc250788740 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c1d8f09980 0x55c1d8f0bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.008+0000 7fc250788740 4 client.26747 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:38:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.008+0000 7fc250788740 1 -- 172.21.15.42:0/347925779 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1d8e63ff0 con 0x55c1d8f09980 2024-04-25T00:38:57.009 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.009+0000 7fc246a00640 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c1d8f09980 0x55c1d8f0bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.011+0000 7fc246a00640 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c1d8f09980 0x55c1d8f0bdd0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787e4452f177fcd5 server_cookie=a32d58fedfa8a9a2 in_seq=0 out_seq=0 2024-04-25T00:38:57.011 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.011+0000 7fc230c00640 10 client.26747 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.014+0000 7fc230c00640 1 -- 172.21.15.42:0/347925779 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1d8e63ff0 con 0x55c1d8f09980 2024-04-25T00:38:57.015 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.014+0000 7fc230c00640 10 client.26747 handle_command_reply: tid=0 2024-04-25T00:38:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.015+0000 7fc250788740 1 client.26747 shutdown 2024-04-25T00:38:57.016 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.015+0000 7fc250788740 1 -- 172.21.15.42:0/347925779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef9800 msgr2=0x55c1d8f05420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.015+0000 7fc250788740 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef9800 0x55c1d8f05420 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc23409bd80 tx=0x7fc23409c4f0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.015+0000 7fc250788740 1 -- 172.21.15.42:0/347925779 shutdown_connections 2024-04-25T00:38:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.015+0000 7fc250788740 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c1d8f09980 0x55c1d8f0bdd0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.015+0000 7fc250788740 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef9800 0x55c1d8f05420 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.015+0000 7fc250788740 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef4690 0x55c1d8efe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.015+0000 7fc250788740 1 --2- 172.21.15.42:0/347925779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8efdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.015+0000 7fc250788740 1 -- 172.21.15.42:0/347925779 >> 172.21.15.42:0/347925779 conn(0x55c1d8eef310 msgr2=0x55c1d8eee480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.015+0000 7fc230c00640 0 client.26747 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.016+0000 7fc250788740 1 -- 172.21.15.42:0/347925779 shutdown_connections 2024-04-25T00:38:57.017 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.016+0000 7fc250788740 1 -- 172.21.15.42:0/347925779 wait complete. 2024-04-25T00:38:57.018 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.016+0000 7fc250788740 20 client.26747 trim_cache size 0 max 16384 2024-04-25T00:38:57.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.072+0000 7fc250788740 1 Processor -- start 2024-04-25T00:38:57.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.075+0000 7fc250788740 1 -- start start 2024-04-25T00:38:57.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.075+0000 7fc250788740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.076+0000 7fc250788740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 0x55c1d8f053e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.076+0000 7fc246a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.076+0000 7fc246a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:55284/0 (socket says 172.21.15.42:55284) 2024-04-25T00:38:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.076+0000 7fc246a00640 1 -- 172.21.15.42:0/1638054174 learned_addr learned my addr 172.21.15.42:0/1638054174 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:57.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.077+0000 7fc250788740 1 --2- 172.21.15.42:0/1638054174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef9800 0x55c1d8ef5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.077+0000 7fc246000640 1 --2- 172.21.15.42:0/1638054174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 0x55c1d8f053e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.078+0000 7fc250788740 1 -- 172.21.15.42:0/1638054174 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481b1a60 con 0x55c1d8ef9800 2024-04-25T00:38:57.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.078+0000 7fc250788740 1 -- 172.21.15.42:0/1638054174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481af330 con 0x55c1d8ef3ca0 2024-04-25T00:38:57.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.079+0000 7fc250788740 1 -- 172.21.15.42:0/1638054174 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481ad180 con 0x55c1d8ef4690 2024-04-25T00:38:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.080+0000 7fc246a00640 1 -- 172.21.15.42:0/1638054174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 msgr2=0x55c1d8f053e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.080+0000 7fc246a00640 1 --2- 172.21.15.42:0/1638054174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 0x55c1d8f053e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.080+0000 7fc246a00640 1 -- 172.21.15.42:0/1638054174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef9800 msgr2=0x55c1d8ef5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.080+0000 7fc246a00640 1 --2- 172.21.15.42:0/1638054174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef9800 0x55c1d8ef5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.080+0000 7fc246a00640 1 -- 172.21.15.42:0/1638054174 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d8eef140 con 0x55c1d8ef3ca0 2024-04-25T00:38:57.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.081+0000 7fc246a00640 1 --2- 172.21.15.42:0/1638054174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef5910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc24003be00 tx=0x7fc240000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68f6bf8a1b58afcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.082+0000 7fc233400640 1 -- 172.21.15.42:0/1638054174 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240064070 con 0x55c1d8ef3ca0 2024-04-25T00:38:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.082+0000 7fc233400640 1 -- 172.21.15.42:0/1638054174 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc240047040 con 0x55c1d8ef3ca0 2024-04-25T00:38:57.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.082+0000 7fc233400640 1 -- 172.21.15.42:0/1638054174 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240058040 con 0x55c1d8ef3ca0 2024-04-25T00:38:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.083+0000 7fc250788740 1 -- 172.21.15.42:0/1638054174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 msgr2=0x55c1d8ef5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.083+0000 7fc250788740 1 --2- 172.21.15.42:0/1638054174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef5910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc24003be00 tx=0x7fc240000f40 comp rx=0 tx=0).stop 2024-04-25T00:38:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.084+0000 7fc250788740 1 -- 172.21.15.42:0/1638054174 shutdown_connections 2024-04-25T00:38:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.084+0000 7fc250788740 1 --2- 172.21.15.42:0/1638054174 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef9800 0x55c1d8ef5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.084+0000 7fc250788740 1 --2- 172.21.15.42:0/1638054174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef4690 0x55c1d8f053e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.084+0000 7fc250788740 1 --2- 172.21.15.42:0/1638054174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8ef5910 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.084+0000 7fc250788740 1 -- 172.21.15.42:0/1638054174 >> 172.21.15.42:0/1638054174 conn(0x55c1d8eef310 msgr2=0x55c1d8ef8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.084+0000 7fc250788740 1 -- 172.21.15.42:0/1638054174 shutdown_connections 2024-04-25T00:38:57.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.085+0000 7fc250788740 1 -- 172.21.15.42:0/1638054174 wait complete. 2024-04-25T00:38:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.086+0000 7fc250788740 1 -- start start 2024-04-25T00:38:57.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.086+0000 7fc250788740 1 Processor -- start 2024-04-25T00:38:57.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.089+0000 7fc250788740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8f02dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.089+0000 7fc250788740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef4690 0x55c1d8f03310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.089+0000 7fc250788740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef9800 0x55c1d8f04320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.091+0000 7fc247400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef9800 0x55c1d8f04320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.091+0000 7fc247400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef9800 0x55c1d8f04320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:40310/0 (socket says 172.21.15.42:40310) 2024-04-25T00:38:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.091+0000 7fc247400640 1 -- 172.21.15.42:0/4183122650 learned_addr learned my addr 172.21.15.42:0/4183122650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:38:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.091+0000 7fc246000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef4690 0x55c1d8f03310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.091+0000 7fc246a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8f02dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.091+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481ad7f0 con 0x55c1d8ef4690 2024-04-25T00:38:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.091+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481aec60 con 0x55c1d8ef3ca0 2024-04-25T00:38:57.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.092+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc2481adec0 con 0x55c1d8ef9800 2024-04-25T00:38:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.093+0000 7fc250788740 10 client.0 _ll_register_callbacks cb 0x7fc234004950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:38:57.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.093+0000 7fc246000640 1 -- 172.21.15.42:0/4183122650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef9800 msgr2=0x55c1d8f04320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.093+0000 7fc246000640 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef9800 0x55c1d8f04320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.093+0000 7fc246000640 1 -- 172.21.15.42:0/4183122650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 msgr2=0x55c1d8f02dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:38:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.093+0000 7fc246000640 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8f02dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.093+0000 7fc246000640 1 -- 172.21.15.42:0/4183122650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d8f04910 con 0x55c1d8ef4690 2024-04-25T00:38:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.093+0000 7fc246a00640 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8f02dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:38:57.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.094+0000 7fc246000640 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef4690 0x55c1d8f03310 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55c1d8eff380 tx=0x7fc238000f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=894a65cfc19cf4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:38:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.094+0000 7fc230c00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:38:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.094+0000 7fc230c00640 1 -- 172.21.15.42:0/4183122650 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc238029060 con 0x55c1d8ef4690 2024-04-25T00:38:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.094+0000 7fc230c00640 1 -- 172.21.15.42:0/4183122650 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fc238012040 con 0x55c1d8ef4690 2024-04-25T00:38:57.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.094+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc240047040 con 0x55c1d8ef4690 2024-04-25T00:38:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.096+0000 7fc230c00640 1 -- 172.21.15.42:0/4183122650 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fc238016d60 con 0x55c1d8ef4690 2024-04-25T00:38:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.096+0000 7fc250788740 10 client.16512 fetch_fsmap learned FSMap version 45 2024-04-25T00:38:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.096+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c1d8f70350 con 0x55c1d8ef4690 2024-04-25T00:38:57.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.096+0000 7fc230c00640 1 -- 172.21.15.42:0/4183122650 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc240047040 con 0x55c1d8ef4690 2024-04-25T00:38:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.096+0000 7fc230c00640 1 -- 172.21.15.42:0/4183122650 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fc23802e420 con 0x55c1d8ef4690 2024-04-25T00:38:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.097+0000 7fc250788740 10 client.16512 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:38:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.097+0000 7fc250788740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:38:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.097+0000 7fc250788740 20 client.16512 populate_metadata read hostname 'smithi042' 2024-04-25T00:38:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.097+0000 7fc250788740 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c1d8f70d60 0x55c1d8f73180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:38:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.097+0000 7fc250788740 4 client.16512 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:38:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.097+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1d8e5e340 con 0x55c1d8f70d60 2024-04-25T00:38:57.098 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.097+0000 7fc246a00640 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c1d8f70d60 0x55c1d8f73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:38:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.098+0000 7fc246a00640 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c1d8f70d60 0x55c1d8f73180 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee99f23ebf79c872 server_cookie=1553cb0f8b21f882 in_seq=0 out_seq=0 2024-04-25T00:38:57.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.098+0000 7fc230c00640 10 client.16512 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:57.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 256496 33.81 MB/sec execute 287 sec latency 543.096 ms 2024-04-25T00:38:57.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.930+0000 7fc230c00640 1 -- 172.21.15.42:0/4183122650 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1d8e5e340 con 0x55c1d8f70d60 2024-04-25T00:38:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc230c00640 10 client.16512 handle_command_reply: tid=0 2024-04-25T00:38:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc250788740 1 client.16512 shutdown 2024-04-25T00:38:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef4690 msgr2=0x55c1d8f03310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc250788740 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef4690 0x55c1d8f03310 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55c1d8eff380 tx=0x7fc238000f60 comp rx=0 tx=0).stop 2024-04-25T00:38:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 shutdown_connections 2024-04-25T00:38:57.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc250788740 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c1d8f70d60 0x55c1d8f73180 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc250788740 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8ef9800 0x55c1d8f04320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc250788740 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c1d8ef4690 0x55c1d8f03310 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc250788740 1 --2- 172.21.15.42:0/4183122650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c1d8ef3ca0 0x55c1d8f02dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 >> 172.21.15.42:0/4183122650 conn(0x55c1d8eef310 msgr2=0x55c1d8efdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc230c00640 0 client.16512 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:38:57.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.931+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 shutdown_connections 2024-04-25T00:38:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.932+0000 7fc250788740 1 -- 172.21.15.42:0/4183122650 wait complete. 2024-04-25T00:38:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:38:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:38:57.935 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:38:57.936 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:38:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.932+0000 7fc250788740 20 client.16512 trim_cache size 0 max 16384 2024-04-25T00:38:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 -- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc22c0a2da0 msgr2=0x7fc22c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.937 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc22c0a2da0 0x7fc22c0a5260 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x55c1d8ef4ef0 tx=0x7fc2380046c0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 -- 172.21.15.42:0/1793256862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24810da60 msgr2=0x7fc2481a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:38:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24810da60 0x7fc2481a55a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc23400e970 tx=0x7fc23400ed70 comp rx=0 tx=0).stop 2024-04-25T00:38:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 -- 172.21.15.42:0/1793256862 shutdown_connections 2024-04-25T00:38:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fc22c0a2da0 0x7fc22c0a5260 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc24810da60 0x7fc2481a55a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.938 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24810b0b0 0x7fc24819e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 --2- 172.21.15.42:0/1793256862 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fc248108740 0x7fc24819dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:38:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 -- 172.21.15.42:0/1793256862 >> 172.21.15.42:0/1793256862 conn(0x7fc24806e7b0 msgr2=0x7fc248108d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:38:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 -- 172.21.15.42:0/1793256862 shutdown_connections 2024-04-25T00:38:57.939 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:38:57.933+0000 7fc24d800640 1 -- 172.21.15.42:0/1793256862 wait complete. 2024-04-25T00:38:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:57 smithi136 ceph-mon[32923]: pgmap v996: 97 pgs: 97 active+clean; 668 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 873 op/s 2024-04-25T00:38:58.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:57 smithi106 ceph-mon[32510]: pgmap v996: 97 pgs: 97 active+clean; 668 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 873 op/s 2024-04-25T00:38:58.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:57 smithi042 ceph-mon[28130]: pgmap v996: 97 pgs: 97 active+clean; 668 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 873 op/s 2024-04-25T00:38:58.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 256955 33.78 MB/sec execute 288 sec latency 385.675 ms 2024-04-25T00:38:59.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 257690 33.81 MB/sec execute 289 sec latency 130.303 ms 2024-04-25T00:39:00.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:38:59 smithi136 ceph-mon[32923]: pgmap v997: 97 pgs: 97 active+clean; 680 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.4 MiB/s wr, 596 op/s 2024-04-25T00:39:00.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:38:59 smithi106 ceph-mon[32510]: pgmap v997: 97 pgs: 97 active+clean; 680 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.4 MiB/s wr, 596 op/s 2024-04-25T00:39:00.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:38:59 smithi042 ceph-mon[28130]: pgmap v997: 97 pgs: 97 active+clean; 680 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 7.4 MiB/s wr, 596 op/s 2024-04-25T00:39:00.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 258437 33.86 MB/sec execute 290 sec latency 56.016 ms 2024-04-25T00:39:01.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 258937 33.84 MB/sec execute 291 sec latency 55.798 ms 2024-04-25T00:39:02.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:01 smithi136 ceph-mon[32923]: pgmap v998: 97 pgs: 97 active+clean; 663 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.2 MiB/s wr, 495 op/s 2024-04-25T00:39:02.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:01 smithi106 ceph-mon[32510]: pgmap v998: 97 pgs: 97 active+clean; 663 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.2 MiB/s wr, 495 op/s 2024-04-25T00:39:02.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:01 smithi042 ceph-mon[28130]: pgmap v998: 97 pgs: 97 active+clean; 663 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 5.2 MiB/s wr, 495 op/s 2024-04-25T00:39:02.786 INFO:tasks.workunit.client.0.smithi042.stdout: 10 259549 33.83 MB/sec execute 292 sec latency 64.740 ms 2024-04-25T00:39:03.787 INFO:tasks.workunit.client.0.smithi042.stdout: 10 260352 33.81 MB/sec execute 293 sec latency 57.572 ms 2024-04-25T00:39:04.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:03 smithi136 ceph-mon[32923]: pgmap v999: 97 pgs: 97 active+clean; 665 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.8 MiB/s wr, 717 op/s 2024-04-25T00:39:04.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:03 smithi106 ceph-mon[32510]: pgmap v999: 97 pgs: 97 active+clean; 665 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.8 MiB/s wr, 717 op/s 2024-04-25T00:39:04.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:03 smithi042 ceph-mon[28130]: pgmap v999: 97 pgs: 97 active+clean; 665 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 9.8 MiB/s wr, 717 op/s 2024-04-25T00:39:04.787 INFO:tasks.workunit.client.0.smithi042.stdout: 10 261278 33.91 MB/sec execute 294 sec latency 49.144 ms 2024-04-25T00:39:05.787 INFO:tasks.workunit.client.0.smithi042.stdout: 10 262025 33.94 MB/sec execute 295 sec latency 63.539 ms 2024-04-25T00:39:06.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:05 smithi136 ceph-mon[32923]: pgmap v1000: 97 pgs: 97 active+clean; 659 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.1 MiB/s wr, 647 op/s 2024-04-25T00:39:06.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:05 smithi106 ceph-mon[32510]: pgmap v1000: 97 pgs: 97 active+clean; 659 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.1 MiB/s wr, 647 op/s 2024-04-25T00:39:06.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:05 smithi042 ceph-mon[28130]: pgmap v1000: 97 pgs: 97 active+clean; 659 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.1 MiB/s wr, 647 op/s 2024-04-25T00:39:06.787 INFO:tasks.workunit.client.0.smithi042.stdout: 10 262567 33.94 MB/sec execute 296 sec latency 71.933 ms 2024-04-25T00:39:07.787 INFO:tasks.workunit.client.0.smithi042.stdout: 10 263165 33.90 MB/sec execute 297 sec latency 57.917 ms 2024-04-25T00:39:08.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:07 smithi136 ceph-mon[32923]: pgmap v1001: 97 pgs: 97 active+clean; 668 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 753 op/s 2024-04-25T00:39:08.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:07 smithi106 ceph-mon[32510]: pgmap v1001: 97 pgs: 97 active+clean; 668 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 753 op/s 2024-04-25T00:39:08.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:07 smithi042 ceph-mon[28130]: pgmap v1001: 97 pgs: 97 active+clean; 668 MiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 10 MiB/s wr, 753 op/s 2024-04-25T00:39:08.787 INFO:tasks.workunit.client.0.smithi042.stdout: 10 263949 33.91 MB/sec execute 298 sec latency 44.558 ms 2024-04-25T00:39:09.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:09 smithi106 ceph-mon[32510]: pgmap v1002: 97 pgs: 97 active+clean; 664 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.6 MiB/s wr, 706 op/s 2024-04-25T00:39:09.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:09 smithi042 ceph-mon[28130]: pgmap v1002: 97 pgs: 97 active+clean; 664 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.6 MiB/s wr, 706 op/s 2024-04-25T00:39:09.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:09 smithi136 ceph-mon[32923]: pgmap v1002: 97 pgs: 97 active+clean; 664 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.6 MiB/s wr, 706 op/s 2024-04-25T00:39:09.787 INFO:tasks.workunit.client.0.smithi042.stdout: 10 264760 33.97 MB/sec execute 299 sec latency 62.454 ms 2024-04-25T00:39:10.787 INFO:tasks.workunit.client.0.smithi042.stdout: 10 265443 33.99 MB/sec execute 300 sec latency 70.503 ms 2024-04-25T00:39:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.224+0000 7ff496c00640 1 -- 172.21.15.42:0/697176908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff490103800 msgr2=0x7ff490103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.224+0000 7ff496c00640 1 --2- 172.21.15.42:0/697176908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff490103800 0x7ff490103c20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff484009be0 tx=0x7ff48402a6f0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.227+0000 7ff496c00640 1 -- 172.21.15.42:0/697176908 shutdown_connections 2024-04-25T00:39:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.227+0000 7ff496c00640 1 --2- 172.21.15.42:0/697176908 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4901053b0 0x7ff49010bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.227+0000 7ff496c00640 1 --2- 172.21.15.42:0/697176908 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4901049f0 0x7ff490104e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.227+0000 7ff496c00640 1 --2- 172.21.15.42:0/697176908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff490103800 0x7ff490103c20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.227+0000 7ff496c00640 1 -- 172.21.15.42:0/697176908 >> 172.21.15.42:0/697176908 conn(0x7ff4900fed70 msgr2=0x7ff4901011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.227+0000 7ff496c00640 1 -- 172.21.15.42:0/697176908 shutdown_connections 2024-04-25T00:39:11.228 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.227+0000 7ff496c00640 1 -- 172.21.15.42:0/697176908 wait complete. 2024-04-25T00:39:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.228+0000 7ff496c00640 1 Processor -- start 2024-04-25T00:39:11.229 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.229+0000 7ff496c00640 1 -- start start 2024-04-25T00:39:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.229+0000 7ff496c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff490103800 0x7ff49019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.229+0000 7ff496c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4901049f0 0x7ff49019e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.230+0000 7ff495800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff490103800 0x7ff49019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.230 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.230+0000 7ff495800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff490103800 0x7ff49019df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53992/0 (socket says 172.21.15.42:53992) 2024-04-25T00:39:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.230+0000 7ff496c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4901053b0 0x7ff4901a5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.230+0000 7ff494e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4901049f0 0x7ff49019e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.231 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.230+0000 7ff494e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4901049f0 0x7ff49019e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48834/0 (socket says 172.21.15.42:48834) 2024-04-25T00:39:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.231+0000 7ff496200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4901053b0 0x7ff4901a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.231+0000 7ff496200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4901053b0 0x7ff4901a5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48170/0 (socket says 172.21.15.42:48170) 2024-04-25T00:39:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.231+0000 7ff495800640 1 -- 172.21.15.42:0/1423019533 learned_addr learned my addr 172.21.15.42:0/1423019533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.231+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4900765d0 con 0x7ff4901053b0 2024-04-25T00:39:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.231+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff490076450 con 0x7ff490103800 2024-04-25T00:39:11.232 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.231+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff490076750 con 0x7ff4901049f0 2024-04-25T00:39:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.233+0000 7ff496200640 1 -- 172.21.15.42:0/1423019533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4901049f0 msgr2=0x7ff49019e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.233+0000 7ff496200640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4901049f0 0x7ff49019e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.233+0000 7ff496200640 1 -- 172.21.15.42:0/1423019533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff490103800 msgr2=0x7ff49019df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.233+0000 7ff496200640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff490103800 0x7ff49019df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.233 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.233+0000 7ff496200640 1 -- 172.21.15.42:0/1423019533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4901a5b90 con 0x7ff4901053b0 2024-04-25T00:39:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.233+0000 7ff494e00640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4901049f0 0x7ff49019e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:11.234 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.233+0000 7ff496200640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4901053b0 0x7ff4901a5520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff48800d680 tx=0x7ff48800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.235+0000 7ff48e000640 1 -- 172.21.15.42:0/1423019533 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488017070 con 0x7ff4901053b0 2024-04-25T00:39:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.236+0000 7ff48e000640 1 -- 172.21.15.42:0/1423019533 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff488009820 con 0x7ff4901053b0 2024-04-25T00:39:11.236 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.236+0000 7ff48e000640 1 -- 172.21.15.42:0/1423019533 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff488004ce0 con 0x7ff4901053b0 2024-04-25T00:39:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.236+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4901a5e80 con 0x7ff4901053b0 2024-04-25T00:39:11.237 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.236+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4901a6370 con 0x7ff4901053b0 2024-04-25T00:39:11.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.242+0000 7ff49b194740 1 Processor -- start 2024-04-25T00:39:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.247+0000 7ff48e000640 1 -- 172.21.15.42:0/1423019533 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff4880099d0 con 0x7ff4901053b0 2024-04-25T00:39:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.247+0000 7ff49b194740 1 -- start start 2024-04-25T00:39:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.247+0000 7ff49b194740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 0x5636fa1d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.248+0000 7ff49b194740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1d8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.248+0000 7ff49b194740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1dae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.248+0000 7ff49b194740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5636fa135ab0 con 0x5636fa1d89b0 2024-04-25T00:39:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.248+0000 7ff49b194740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5636fa0de640 con 0x5636fa1d3840 2024-04-25T00:39:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.248+0000 7ff49b194740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5636fa131a70 con 0x5636fa1d2e50 2024-04-25T00:39:11.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.249+0000 7ff495800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1d8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.249+0000 7ff494e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 0x5636fa1d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.249+0000 7ff495800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1d8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:53998/0 (socket says 172.21.15.42:53998) 2024-04-25T00:39:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.249+0000 7ff495800640 1 -- 172.21.15.42:0/1254068338 learned_addr learned my addr 172.21.15.42:0/1254068338 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.250+0000 7ff494e00640 1 --2- 172.21.15.42:0/1254068338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1dae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.251+0000 7ff495800640 1 -- 172.21.15.42:0/1254068338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 msgr2=0x5636fa1d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:11.251 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.251+0000 7ff495800640 1 --2- 172.21.15.42:0/1254068338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 0x5636fa1d3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.251+0000 7ff495800640 1 -- 172.21.15.42:0/1254068338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 msgr2=0x5636fa1dae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.251+0000 7ff48e000640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4780a2cd0 0x7ff4780a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.252+0000 7ff48e000640 1 -- 172.21.15.42:0/1423019533 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ff4880ce840 con 0x7ff4901053b0 2024-04-25T00:39:11.253 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.253+0000 7ff495800640 1 --2- 172.21.15.42:0/1254068338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1dae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.253+0000 7ff495800640 1 -- 172.21.15.42:0/1254068338 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636f9e4a210 con 0x5636fa1d3840 2024-04-25T00:39:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.253+0000 7ff495800640 1 --2- 172.21.15.42:0/1254068338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1d8340 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff4840304d0 tx=0x7ff484030500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c0b717b3662c7af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.253+0000 7ff494e00640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4780a2cd0 0x7ff4780a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.254+0000 7ff482a00640 1 -- 172.21.15.42:0/1254068338 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484041030 con 0x5636fa1d3840 2024-04-25T00:39:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.254+0000 7ff482a00640 1 -- 172.21.15.42:0/1254068338 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff484003970 con 0x5636fa1d3840 2024-04-25T00:39:11.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.254+0000 7ff482a00640 1 -- 172.21.15.42:0/1254068338 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484041030 con 0x5636fa1d3840 2024-04-25T00:39:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.255+0000 7ff49b194740 1 -- 172.21.15.42:0/1254068338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 msgr2=0x5636fa1d8340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.255+0000 7ff49b194740 1 --2- 172.21.15.42:0/1254068338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1d8340 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff4840304d0 tx=0x7ff484030500 comp rx=0 tx=0).stop 2024-04-25T00:39:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.256+0000 7ff494e00640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4780a2cd0 0x7ff4780a5190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5636fa1d40a0 tx=0x7ff47c0047d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.257+0000 7ff49b194740 1 -- 172.21.15.42:0/1254068338 shutdown_connections 2024-04-25T00:39:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.257+0000 7ff49b194740 1 --2- 172.21.15.42:0/1254068338 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1dae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.257+0000 7ff49b194740 1 --2- 172.21.15.42:0/1254068338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1d8340 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.257+0000 7ff49b194740 1 --2- 172.21.15.42:0/1254068338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 0x5636fa1d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.257+0000 7ff49b194740 1 -- 172.21.15.42:0/1254068338 >> 172.21.15.42:0/1254068338 conn(0x5636fa1ce4c0 msgr2=0x5636fa1ce8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.257+0000 7ff49b194740 1 -- 172.21.15.42:0/1254068338 shutdown_connections 2024-04-25T00:39:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.257+0000 7ff49b194740 1 -- 172.21.15.42:0/1254068338 wait complete. 2024-04-25T00:39:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.258+0000 7ff49b194740 1 -- start start 2024-04-25T00:39:11.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.258+0000 7ff49b194740 1 Processor -- start 2024-04-25T00:39:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.261+0000 7ff49b194740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 0x5636fa1ee070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.261+0000 7ff49b194740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1ee5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.262+0000 7ff495800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 0x5636fa1ee070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.262+0000 7ff495800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 0x5636fa1ee070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48848/0 (socket says 172.21.15.42:48848) 2024-04-25T00:39:11.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.263+0000 7ff494e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1ee5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.263+0000 7ff494e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1ee5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:54000/0 (socket says 172.21.15.42:54000) 2024-04-25T00:39:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.263+0000 7ff495800640 1 -- 172.21.15.42:0/1789872883 learned_addr learned my addr 172.21.15.42:0/1789872883 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.263+0000 7ff49b194740 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.263+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5636fa1ec910 con 0x5636fa1d89b0 2024-04-25T00:39:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.263+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5636fa0f1990 con 0x5636fa1d3840 2024-04-25T00:39:11.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.263+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5636fa1eca90 con 0x5636fa1d2e50 2024-04-25T00:39:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.263+0000 7ff49b194740 10 client.0 _ll_register_callbacks cb 0x5636fa0f3950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.264+0000 7ff495800640 1 -- 172.21.15.42:0/1789872883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 msgr2=0x5636fa1ee5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:11.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.264+0000 7ff495800640 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1ee5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.264+0000 7ff495800640 1 -- 172.21.15.42:0/1789872883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 msgr2=0x5636fa1f55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.264+0000 7ff495800640 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.265+0000 7ff495800640 1 -- 172.21.15.42:0/1789872883 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636f9f16620 con 0x5636fa1d2e50 2024-04-25T00:39:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.266+0000 7ff477400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:39:11.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.266+0000 7ff495800640 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 0x5636fa1ee070 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff484009b30 tx=0x7ff484009960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b31bcd1902b5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.267+0000 7ff477400640 1 -- 172.21.15.42:0/1789872883 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff484041040 con 0x5636fa1d2e50 2024-04-25T00:39:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.268+0000 7ff477400640 1 -- 172.21.15.42:0/1789872883 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ff484030af0 con 0x5636fa1d2e50 2024-04-25T00:39:11.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.269+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff484003970 con 0x5636fa1d2e50 2024-04-25T00:39:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.271+0000 7ff477400640 1 -- 172.21.15.42:0/1789872883 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff484036040 con 0x5636fa1d2e50 2024-04-25T00:39:11.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.271+0000 7ff49b194740 10 client.26719 fetch_fsmap learned FSMap version 45 2024-04-25T00:39:11.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.271+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5636fa1ec980 con 0x5636fa1d2e50 2024-04-25T00:39:11.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.273+0000 7ff477400640 1 -- 172.21.15.42:0/1789872883 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff484003970 con 0x5636fa1d2e50 2024-04-25T00:39:11.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.273+0000 7ff477400640 1 -- 172.21.15.42:0/1789872883 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7ff48403d6a0 con 0x5636fa1d2e50 2024-04-25T00:39:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.275+0000 7ff49b194740 10 client.26719 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:39:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.275+0000 7ff49b194740 10 client.26719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:39:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.275+0000 7ff49b194740 20 client.26719 populate_metadata read hostname 'smithi042' 2024-04-25T00:39:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.275+0000 7ff49b194740 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5636fa1f98e0 0x5636fa1fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.275+0000 7ff49b194740 4 client.26719 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.275+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636fa142ff0 con 0x5636fa1f98e0 2024-04-25T00:39:11.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.275+0000 7ff494e00640 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5636fa1f98e0 0x5636fa1fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.280+0000 7ff477400640 10 client.26719 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:11.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.280+0000 7ff494e00640 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5636fa1f98e0 0x5636fa1fbcd0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfad8f82589ea8bd server_cookie=590353726eab1880 in_seq=0 out_seq=0 2024-04-25T00:39:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.282+0000 7ff477400640 1 -- 172.21.15.42:0/1789872883 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5636fa142ff0 con 0x5636fa1f98e0 2024-04-25T00:39:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.282+0000 7ff477400640 10 client.26719 handle_command_reply: tid=0 2024-04-25T00:39:11.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.283+0000 7ff49b194740 1 client.26719 shutdown 2024-04-25T00:39:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.283+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 msgr2=0x5636fa1ee070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.283+0000 7ff49b194740 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 0x5636fa1ee070 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff484009b30 tx=0x7ff484009960 comp rx=0 tx=0).stop 2024-04-25T00:39:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.286+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 shutdown_connections 2024-04-25T00:39:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.286+0000 7ff49b194740 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5636fa1f98e0 0x5636fa1fbcd0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.286+0000 7ff49b194740 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1f55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.286+0000 7ff49b194740 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d3840 0x5636fa1ee5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.286+0000 7ff49b194740 1 --2- 172.21.15.42:0/1789872883 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d2e50 0x5636fa1ee070 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.286+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 >> 172.21.15.42:0/1789872883 conn(0x5636fa1ce4c0 msgr2=0x5636fa1cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.286+0000 7ff477400640 0 client.26719 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.286+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 shutdown_connections 2024-04-25T00:39:11.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.287+0000 7ff49b194740 1 -- 172.21.15.42:0/1789872883 wait complete. 2024-04-25T00:39:11.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.287+0000 7ff49b194740 20 client.26719 trim_cache size 0 max 16384 2024-04-25T00:39:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.338+0000 7ff49b194740 1 Processor -- start 2024-04-25T00:39:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.338+0000 7ff49b194740 1 -- start start 2024-04-25T00:39:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.339+0000 7ff49b194740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d2e50 0x5636fa1ed690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.339+0000 7ff49b194740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.339+0000 7ff49b194740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1ee110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.339+0000 7ff495800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d2e50 0x5636fa1ed690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.339+0000 7ff495800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d2e50 0x5636fa1ed690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:54010/0 (socket says 172.21.15.42:54010) 2024-04-25T00:39:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.339+0000 7ff49b194740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901a90c0 con 0x5636fa1d89b0 2024-04-25T00:39:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.340+0000 7ff49b194740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4901a6990 con 0x5636fa1d2e50 2024-04-25T00:39:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.340+0000 7ff49b194740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff490115460 con 0x5636fa1d3840 2024-04-25T00:39:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.340+0000 7ff494e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.340+0000 7ff494e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1edbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48854/0 (socket says 172.21.15.42:48854) 2024-04-25T00:39:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.340+0000 7ff494e00640 1 -- 172.21.15.42:0/1140375227 learned_addr learned my addr 172.21.15.42:0/1140375227 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.340+0000 7ff496200640 1 --2- 172.21.15.42:0/1140375227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1ee110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.340+0000 7ff495800640 1 -- 172.21.15.42:0/1140375227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 msgr2=0x5636fa1edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.340+0000 7ff495800640 1 --2- 172.21.15.42:0/1140375227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1edbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.340+0000 7ff495800640 1 -- 172.21.15.42:0/1140375227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 msgr2=0x5636fa1ee110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.340+0000 7ff495800640 1 --2- 172.21.15.42:0/1140375227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1ee110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.341+0000 7ff495800640 1 -- 172.21.15.42:0/1140375227 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636fa1fc040 con 0x5636fa1d2e50 2024-04-25T00:39:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.341+0000 7ff495800640 1 --2- 172.21.15.42:0/1140375227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d2e50 0x5636fa1ed690 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff48403dac0 tx=0x7ff484000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13dc82e894bb26bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.341+0000 7ff482000640 1 -- 172.21.15.42:0/1140375227 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48405f030 con 0x5636fa1d2e50 2024-04-25T00:39:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.342+0000 7ff482000640 1 -- 172.21.15.42:0/1140375227 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff48403d460 con 0x5636fa1d2e50 2024-04-25T00:39:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.343+0000 7ff49b194740 1 -- 172.21.15.42:0/1140375227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d2e50 msgr2=0x5636fa1ed690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.344+0000 7ff482000640 1 -- 172.21.15.42:0/1140375227 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48405f030 con 0x5636fa1d2e50 2024-04-25T00:39:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.344+0000 7ff49b194740 1 --2- 172.21.15.42:0/1140375227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d2e50 0x5636fa1ed690 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff48403dac0 tx=0x7ff484000f40 comp rx=0 tx=0).stop 2024-04-25T00:39:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.344+0000 7ff49b194740 1 -- 172.21.15.42:0/1140375227 shutdown_connections 2024-04-25T00:39:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.344+0000 7ff49b194740 1 --2- 172.21.15.42:0/1140375227 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d89b0 0x5636fa1ee110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.344+0000 7ff49b194740 1 --2- 172.21.15.42:0/1140375227 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.344+0000 7ff49b194740 1 --2- 172.21.15.42:0/1140375227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d2e50 0x5636fa1ed690 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.345 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.345+0000 7ff49b194740 1 -- 172.21.15.42:0/1140375227 >> 172.21.15.42:0/1140375227 conn(0x5636fa1ce4c0 msgr2=0x5636fa1d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.345+0000 7ff49b194740 1 -- 172.21.15.42:0/1140375227 shutdown_connections 2024-04-25T00:39:11.346 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.345+0000 7ff49b194740 1 -- 172.21.15.42:0/1140375227 wait complete. 2024-04-25T00:39:11.348 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.346+0000 7ff49b194740 1 -- start start 2024-04-25T00:39:11.349 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.348+0000 7ff49b194740 1 Processor -- start 2024-04-25T00:39:11.351 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.351+0000 7ff49b194740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d2e50 0x5636fa1f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.351+0000 7ff49b194740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.352 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.352+0000 7ff495800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d2e50 0x5636fa1f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.352+0000 7ff49b194740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d89b0 0x5636fa1f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.352+0000 7ff495800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d2e50 0x5636fa1f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:48218/0 (socket says 172.21.15.42:48218) 2024-04-25T00:39:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.353+0000 7ff494e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.353+0000 7ff494e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1f25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48866/0 (socket says 172.21.15.42:48866) 2024-04-25T00:39:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.354+0000 7ff49b194740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff490115ab0 con 0x5636fa1d2e50 2024-04-25T00:39:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.354+0000 7ff49b194740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff490116f20 con 0x5636fa1d89b0 2024-04-25T00:39:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.354+0000 7ff49b194740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff490116180 con 0x5636fa1d3840 2024-04-25T00:39:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.354+0000 7ff49b194740 10 client.0 _ll_register_callbacks cb 0x7ff47c00c290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:11.356 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.354+0000 7ff496200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d89b0 0x5636fa1f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.356+0000 7ff494e00640 1 -- 172.21.15.42:0/1189175921 learned_addr learned my addr 172.21.15.42:0/1189175921 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:11.357 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.357+0000 7ff494e00640 1 -- 172.21.15.42:0/1189175921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d89b0 msgr2=0x5636fa1f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.357+0000 7ff494e00640 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d89b0 0x5636fa1f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.357+0000 7ff494e00640 1 -- 172.21.15.42:0/1189175921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d2e50 msgr2=0x5636fa1f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.357+0000 7ff494e00640 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d2e50 0x5636fa1f20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.357+0000 7ff494e00640 1 -- 172.21.15.42:0/1189175921 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636fa1f3be0 con 0x5636fa1d3840 2024-04-25T00:39:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.357+0000 7ff495800640 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d2e50 0x5636fa1f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:11.358 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.357+0000 7ff496200640 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d89b0 0x5636fa1f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.358+0000 7ff494e00640 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1f25e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00cbc0 tx=0x7ff47c00c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87781d8f1fdb0ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.358+0000 7ff477400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:39:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.358+0000 7ff477400640 1 -- 172.21.15.42:0/1189175921 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff47c01e040 con 0x5636fa1d3840 2024-04-25T00:39:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.360+0000 7ff477400640 1 -- 172.21.15.42:0/1189175921 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ff47c018cc0 con 0x5636fa1d3840 2024-04-25T00:39:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.360+0000 7ff49b194740 1 -- 172.21.15.42:0/1189175921 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff48403d460 con 0x5636fa1d3840 2024-04-25T00:39:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.360+0000 7ff477400640 1 -- 172.21.15.42:0/1189175921 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7ff47c00f040 con 0x5636fa1d3840 2024-04-25T00:39:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.361+0000 7ff49b194740 10 client.26725 fetch_fsmap learned FSMap version 45 2024-04-25T00:39:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.361+0000 7ff49b194740 1 -- 172.21.15.42:0/1189175921 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5636fa25fe30 con 0x5636fa1d3840 2024-04-25T00:39:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.361+0000 7ff477400640 1 -- 172.21.15.42:0/1189175921 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff48403d460 con 0x5636fa1d3840 2024-04-25T00:39:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.361+0000 7ff477400640 1 -- 172.21.15.42:0/1189175921 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7ff47c031070 con 0x5636fa1d3840 2024-04-25T00:39:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.364+0000 7ff49b194740 10 client.26725 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:39:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.364+0000 7ff49b194740 10 client.26725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:39:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.364+0000 7ff49b194740 20 client.26725 populate_metadata read hostname 'smithi042' 2024-04-25T00:39:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.364+0000 7ff49b194740 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5636fa260710 0x5636fa262b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.364+0000 7ff49b194740 4 client.26725 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:39:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.364+0000 7ff49b194740 1 -- 172.21.15.42:0/1189175921 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5636fa13d340 con 0x5636fa260710 2024-04-25T00:39:11.365 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.364+0000 7ff495800640 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5636fa260710 0x5636fa262b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.365+0000 7ff495800640 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5636fa260710 0x5636fa262b30 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c17065beee913b79 server_cookie=8b6dbd4875972b05 in_seq=0 out_seq=0 2024-04-25T00:39:11.366 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:11.365+0000 7ff477400640 10 client.26725 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:11.787 INFO:tasks.workunit.client.0.smithi042.stdout: 10 265617 33.90 MB/sec execute 301 sec latency 565.004 ms 2024-04-25T00:39:11.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:11 smithi106 ceph-mon[32510]: pgmap v1003: 97 pgs: 97 active+clean; 663 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.5 MiB/s wr, 633 op/s 2024-04-25T00:39:11.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:11 smithi042 ceph-mon[28130]: pgmap v1003: 97 pgs: 97 active+clean; 663 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.5 MiB/s wr, 633 op/s 2024-04-25T00:39:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:11 smithi136 ceph-mon[32923]: pgmap v1003: 97 pgs: 97 active+clean; 663 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 8.5 MiB/s wr, 633 op/s 2024-04-25T00:39:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.370+0000 7ff477400640 1 -- 172.21.15.42:0/1189175921 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5636fa13d340 con 0x5636fa260710 2024-04-25T00:39:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.371+0000 7ff477400640 10 client.26725 handle_command_reply: tid=0 2024-04-25T00:39:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.371+0000 7ff49b194740 1 client.26725 shutdown 2024-04-25T00:39:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.373+0000 7ff49b194740 1 -- 172.21.15.42:0/1189175921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 msgr2=0x5636fa1f25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.373+0000 7ff49b194740 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1f25e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00cbc0 tx=0x7ff47c00c3f0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.373+0000 7ff49b194740 1 -- 172.21.15.42:0/1189175921 shutdown_connections 2024-04-25T00:39:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.373+0000 7ff49b194740 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5636fa260710 0x5636fa262b30 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.373+0000 7ff49b194740 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5636fa1d89b0 0x5636fa1f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.373+0000 7ff477400640 0 client.26725 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.373+0000 7ff49b194740 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5636fa1d3840 0x5636fa1f25e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.373+0000 7ff49b194740 1 --2- 172.21.15.42:0/1189175921 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5636fa1d2e50 0x5636fa1f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.374 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.373+0000 7ff49b194740 1 -- 172.21.15.42:0/1189175921 >> 172.21.15.42:0/1189175921 conn(0x5636fa1ce4c0 msgr2=0x5636fa1d1930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.373+0000 7ff49b194740 1 -- 172.21.15.42:0/1189175921 shutdown_connections 2024-04-25T00:39:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.374+0000 7ff49b194740 1 -- 172.21.15.42:0/1189175921 wait complete. 2024-04-25T00:39:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.374+0000 7ff49b194740 20 client.26725 trim_cache size 0 max 16384 2024-04-25T00:39:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:39:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:39:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:39:12.375 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:39:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.375+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4780a2cd0 msgr2=0x7ff4780a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.375+0000 7ff496c00640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4780a2cd0 0x7ff4780a5190 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5636fa1d40a0 tx=0x7ff47c0047d0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.375+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4901053b0 msgr2=0x7ff4901a5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.375+0000 7ff496c00640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4901053b0 0x7ff4901a5520 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff48800d680 tx=0x7ff48800db40 comp rx=0 tx=0).stop 2024-04-25T00:39:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.375+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 shutdown_connections 2024-04-25T00:39:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.375+0000 7ff496c00640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4780a2cd0 0x7ff4780a5190 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.375+0000 7ff496c00640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4901053b0 0x7ff4901a5520 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.375+0000 7ff496c00640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4901049f0 0x7ff49019e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.375+0000 7ff496c00640 1 --2- 172.21.15.42:0/1423019533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff490103800 0x7ff49019df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:12.376 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.375+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 >> 172.21.15.42:0/1423019533 conn(0x7ff4900fed70 msgr2=0x7ff490079080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.376+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 shutdown_connections 2024-04-25T00:39:12.377 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:12.376+0000 7ff496c00640 1 -- 172.21.15.42:0/1423019533 wait complete. 2024-04-25T00:39:12.787 INFO:tasks.workunit.client.0.smithi042.stdout: 10 265833 33.83 MB/sec execute 302 sec latency 898.121 ms 2024-04-25T00:39:13.789 INFO:tasks.workunit.client.0.smithi042.stdout: 10 266234 33.77 MB/sec execute 303 sec latency 172.957 ms 2024-04-25T00:39:13.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:13 smithi106 ceph-mon[32510]: pgmap v1004: 97 pgs: 97 active+clean; 679 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 750 op/s 2024-04-25T00:39:13.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:13 smithi042 ceph-mon[28130]: pgmap v1004: 97 pgs: 97 active+clean; 679 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 750 op/s 2024-04-25T00:39:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:13 smithi136 ceph-mon[32923]: pgmap v1004: 97 pgs: 97 active+clean; 679 MiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 750 op/s 2024-04-25T00:39:14.789 INFO:tasks.workunit.client.0.smithi042.stdout: 10 266894 33.79 MB/sec execute 304 sec latency 68.821 ms 2024-04-25T00:39:15.789 INFO:tasks.workunit.client.0.smithi042.stdout: 10 267713 33.81 MB/sec execute 305 sec latency 61.015 ms 2024-04-25T00:39:15.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:15 smithi106 ceph-mon[32510]: pgmap v1005: 97 pgs: 97 active+clean; 688 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.1 MiB/s wr, 477 op/s 2024-04-25T00:39:15.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:15 smithi042 ceph-mon[28130]: pgmap v1005: 97 pgs: 97 active+clean; 688 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.1 MiB/s wr, 477 op/s 2024-04-25T00:39:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:15 smithi136 ceph-mon[32923]: pgmap v1005: 97 pgs: 97 active+clean; 688 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 6.1 MiB/s wr, 477 op/s 2024-04-25T00:39:16.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 268375 33.83 MB/sec execute 306 sec latency 51.028 ms 2024-04-25T00:39:17.789 INFO:tasks.workunit.client.0.smithi042.stdout: 10 269051 33.85 MB/sec execute 307 sec latency 51.108 ms 2024-04-25T00:39:17.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:17 smithi042 ceph-mon[28130]: pgmap v1006: 97 pgs: 97 active+clean; 687 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 721 op/s 2024-04-25T00:39:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:17 smithi136 ceph-mon[32923]: pgmap v1006: 97 pgs: 97 active+clean; 687 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 721 op/s 2024-04-25T00:39:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:17 smithi106 ceph-mon[32510]: pgmap v1006: 97 pgs: 97 active+clean; 687 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 11 MiB/s wr, 721 op/s 2024-04-25T00:39:18.789 INFO:tasks.workunit.client.0.smithi042.stdout: 10 269599 33.83 MB/sec execute 308 sec latency 62.506 ms 2024-04-25T00:39:19.789 INFO:tasks.workunit.client.0.smithi042.stdout: 10 270039 33.79 MB/sec execute 309 sec latency 276.867 ms 2024-04-25T00:39:19.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:19 smithi042 ceph-mon[28130]: pgmap v1007: 97 pgs: 97 active+clean; 659 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-04-25T00:39:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:19 smithi136 ceph-mon[32923]: pgmap v1007: 97 pgs: 97 active+clean; 659 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-04-25T00:39:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:19 smithi106 ceph-mon[32510]: pgmap v1007: 97 pgs: 97 active+clean; 659 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 11 MiB/s wr, 723 op/s 2024-04-25T00:39:20.789 INFO:tasks.workunit.client.0.smithi042.stdout: 10 270748 33.79 MB/sec execute 310 sec latency 314.854 ms 2024-04-25T00:39:21.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 271549 33.83 MB/sec execute 311 sec latency 57.828 ms 2024-04-25T00:39:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:21 smithi106 ceph-mon[32510]: pgmap v1008: 97 pgs: 97 active+clean; 661 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 697 op/s 2024-04-25T00:39:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:21 smithi042 ceph-mon[28130]: pgmap v1008: 97 pgs: 97 active+clean; 661 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 697 op/s 2024-04-25T00:39:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:21 smithi136 ceph-mon[32923]: pgmap v1008: 97 pgs: 97 active+clean; 661 MiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 10 MiB/s wr, 697 op/s 2024-04-25T00:39:22.789 INFO:tasks.workunit.client.0.smithi042.stdout: 10 272279 33.87 MB/sec execute 312 sec latency 72.369 ms 2024-04-25T00:39:23.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 272844 33.85 MB/sec execute 313 sec latency 57.155 ms 2024-04-25T00:39:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:23 smithi106 ceph-mon[32510]: pgmap v1009: 97 pgs: 97 active+clean; 669 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 786 op/s 2024-04-25T00:39:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:23 smithi042 ceph-mon[28130]: pgmap v1009: 97 pgs: 97 active+clean; 669 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 786 op/s 2024-04-25T00:39:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:23 smithi136 ceph-mon[32923]: pgmap v1009: 97 pgs: 97 active+clean; 669 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 11 MiB/s wr, 786 op/s 2024-04-25T00:39:24.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 273390 33.84 MB/sec execute 314 sec latency 60.437 ms 2024-04-25T00:39:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.641+0000 7fcbae600640 1 -- 172.21.15.42:0/4268271077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81039d0 msgr2=0x7fcba8103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.641+0000 7fcbae600640 1 --2- 172.21.15.42:0/4268271077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81039d0 0x7fcba8103df0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009900 tx=0x7fcb9802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.641+0000 7fcbae600640 1 -- 172.21.15.42:0/4268271077 shutdown_connections 2024-04-25T00:39:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.641+0000 7fcbae600640 1 --2- 172.21.15.42:0/4268271077 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba8105580 0x7fcba810be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.641+0000 7fcbae600640 1 --2- 172.21.15.42:0/4268271077 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcba8104bc0 0x7fcba8105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.641+0000 7fcbae600640 1 --2- 172.21.15.42:0/4268271077 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81039d0 0x7fcba8103df0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.641+0000 7fcbae600640 1 -- 172.21.15.42:0/4268271077 >> 172.21.15.42:0/4268271077 conn(0x7fcba80fef60 msgr2=0x7fcba8101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.641+0000 7fcbae600640 1 -- 172.21.15.42:0/4268271077 shutdown_connections 2024-04-25T00:39:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.641+0000 7fcbae600640 1 -- 172.21.15.42:0/4268271077 wait complete. 2024-04-25T00:39:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.643+0000 7fcbae600640 1 Processor -- start 2024-04-25T00:39:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.643+0000 7fcbae600640 1 -- start start 2024-04-25T00:39:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.643+0000 7fcbae600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81039d0 0x7fcba8070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.643+0000 7fcbae600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba8104bc0 0x7fcba8070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.643+0000 7fcbae600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcba8105580 0x7fcba81a9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.643+0000 7fcbae600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcba810f6f0 con 0x7fcba8104bc0 2024-04-25T00:39:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.643+0000 7fcbae600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcba810f570 con 0x7fcba81039d0 2024-04-25T00:39:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.643+0000 7fcbae600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcba810f870 con 0x7fcba8105580 2024-04-25T00:39:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.644+0000 7fcba7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcba8105580 0x7fcba81a9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.644+0000 7fcba7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81039d0 0x7fcba8070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.644+0000 7fcba7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81039d0 0x7fcba8070050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:48584/0 (socket says 172.21.15.42:48584) 2024-04-25T00:39:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.644+0000 7fcba7400640 1 -- 172.21.15.42:0/612167303 learned_addr learned my addr 172.21.15.42:0/612167303 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.644+0000 7fcba6a00640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba8104bc0 0x7fcba8070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.645+0000 7fcba7e00640 1 -- 172.21.15.42:0/612167303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81039d0 msgr2=0x7fcba8070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.645+0000 7fcba7e00640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81039d0 0x7fcba8070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.645+0000 7fcba7e00640 1 -- 172.21.15.42:0/612167303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba8104bc0 msgr2=0x7fcba8070590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.645+0000 7fcba7e00640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba8104bc0 0x7fcba8070590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.645+0000 7fcba7e00640 1 -- 172.21.15.42:0/612167303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba8073710 con 0x7fcba8105580 2024-04-25T00:39:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.645+0000 7fcba7e00640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcba8105580 0x7fcba81a9e00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcba000be20 tx=0x7fcba0009ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.646+0000 7fcb9fe00640 1 -- 172.21.15.42:0/612167303 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba0019070 con 0x7fcba8105580 2024-04-25T00:39:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.646+0000 7fcb9fe00640 1 -- 172.21.15.42:0/612167303 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba000cd70 con 0x7fcba8105580 2024-04-25T00:39:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.646+0000 7fcbae600640 1 -- 172.21.15.42:0/612167303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba81aa400 con 0x7fcba8105580 2024-04-25T00:39:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.646+0000 7fcb9fe00640 1 -- 172.21.15.42:0/612167303 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00127f0 con 0x7fcba8105580 2024-04-25T00:39:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.646+0000 7fcbae600640 1 -- 172.21.15.42:0/612167303 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba81aa920 con 0x7fcba8105580 2024-04-25T00:39:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.650+0000 7fcb9fe00640 1 -- 172.21.15.42:0/612167303 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcba001c070 con 0x7fcba8105580 2024-04-25T00:39:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.651+0000 7fcbb145c740 1 Processor -- start 2024-04-25T00:39:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.651+0000 7fcbb145c740 1 -- start start 2024-04-25T00:39:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcb9fe00640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcb8c0a2cb0 0x7fcb8c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336ce690 0x5631336d3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcbb145c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x563133630ab0 con 0x5631336ce690 2024-04-25T00:39:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcbb145c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5631335d9640 con 0x5631336cdca0 2024-04-25T00:39:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcbb145c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56313362ca70 con 0x5631336d3800 2024-04-25T00:39:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcba7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcba7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcba7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336d5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44384/0 (socket says 172.21.15.42:44384) 2024-04-25T00:39:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.652+0000 7fcba7e00640 1 -- 172.21.15.42:0/3353970650 learned_addr learned my addr 172.21.15.42:0/3353970650 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcba6a00640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcb8c0a2cb0 0x7fcb8c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcb9fe00640 1 -- 172.21.15.42:0/612167303 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fcba00ce6f0 con 0x7fcba8105580 2024-04-25T00:39:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcba7e00640 1 -- 172.21.15.42:0/3353970650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 msgr2=0x5631336ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcba7e00640 1 --2- 172.21.15.42:0/3353970650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336ce0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.654 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcba7e00640 1 -- 172.21.15.42:0/3353970650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336ce690 msgr2=0x5631336d3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcba6a00640 1 --2- 172.21.15.42:0/3353970650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336ce690 0x5631336d3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcba7e00640 1 --2- 172.21.15.42:0/3353970650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336ce690 0x5631336d3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcba7e00640 1 -- 172.21.15.42:0/3353970650 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563133345210 con 0x5631336d3800 2024-04-25T00:39:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcba6a00640 1 --2- 172.21.15.42:0/3353970650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336ce690 0x5631336d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcba7400640 1 --2- 172.21.15.42:0/3353970650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.653+0000 7fcba7e00640 1 --2- 172.21.15.42:0/3353970650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336d5c90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcba00cc1c0 tx=0x7fcba0004b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6f8f06cc7000f62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.654+0000 7fcb9cc00640 1 -- 172.21.15.42:0/3353970650 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba009f330 con 0x5631336d3800 2024-04-25T00:39:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.654+0000 7fcb9cc00640 1 -- 172.21.15.42:0/3353970650 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba0014a90 con 0x5631336d3800 2024-04-25T00:39:25.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.654+0000 7fcb9cc00640 1 -- 172.21.15.42:0/3353970650 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00b9bc0 con 0x5631336d3800 2024-04-25T00:39:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.654+0000 7fcba6a00640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcb8c0a2cb0 0x7fcb8c0a5170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5631336cea90 tx=0x7fcb94005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.655+0000 7fcbb145c740 1 -- 172.21.15.42:0/3353970650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 msgr2=0x5631336d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.655+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3353970650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336d5c90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fcba00cc1c0 tx=0x7fcba0004b50 comp rx=0 tx=0).stop 2024-04-25T00:39:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.655+0000 7fcbb145c740 1 -- 172.21.15.42:0/3353970650 shutdown_connections 2024-04-25T00:39:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.655+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3353970650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336d5c90 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.655+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3353970650 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336ce690 0x5631336d3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.655+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3353970650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336ce0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.655+0000 7fcbb145c740 1 -- 172.21.15.42:0/3353970650 >> 172.21.15.42:0/3353970650 conn(0x5631336c9310 msgr2=0x5631336c9710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:25.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.655+0000 7fcbb145c740 1 -- 172.21.15.42:0/3353970650 shutdown_connections 2024-04-25T00:39:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.655+0000 7fcbb145c740 1 -- 172.21.15.42:0/3353970650 wait complete. 2024-04-25T00:39:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.656+0000 7fcbb145c740 1 -- start start 2024-04-25T00:39:25.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.656+0000 7fcbb145c740 1 Processor -- start 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336cdca0 0x5631336d7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336d83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcbb145c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5631336d6750 con 0x5631336cdca0 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcbb145c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5631335ec990 con 0x5631336ce690 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcbb145c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631336d68d0 con 0x5631336d3800 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcbb145c740 10 client.0 _ll_register_callbacks cb 0x5631335ee950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcba7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcba7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336df410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44394/0 (socket says 172.21.15.42:44394) 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcba7e00640 1 -- 172.21.15.42:0/3347220823 learned_addr learned my addr 172.21.15.42:0/3347220823 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.657+0000 7fcba6a00640 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336d83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcba7e00640 1 -- 172.21.15.42:0/3347220823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 msgr2=0x5631336d83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcba7e00640 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336d83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcba7e00640 1 -- 172.21.15.42:0/3347220823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336cdca0 msgr2=0x5631336d7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcba7e00640 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336cdca0 0x5631336d7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcba7e00640 1 -- 172.21.15.42:0/3347220823 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563133411620 con 0x5631336d3800 2024-04-25T00:39:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcba6a00640 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336d83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcba7e00640 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336df410 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcba00cc1f0 tx=0x7fcba00976a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15f2534958ab07e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcb91600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:39:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcb91600640 1 -- 172.21.15.42:0/3347220823 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba00b9dc0 con 0x5631336d3800 2024-04-25T00:39:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcb91600640 1 -- 172.21.15.42:0/3347220823 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fcba00ceba0 con 0x5631336d3800 2024-04-25T00:39:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.658+0000 7fcbb145c740 1 -- 172.21.15.42:0/3347220823 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcba0014a90 con 0x5631336d3800 2024-04-25T00:39:25.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcb91600640 1 -- 172.21.15.42:0/3347220823 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcba00ceda0 con 0x5631336d3800 2024-04-25T00:39:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcbb145c740 10 client.26734 fetch_fsmap learned FSMap version 45 2024-04-25T00:39:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcbb145c740 1 -- 172.21.15.42:0/3347220823 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5631336d67c0 con 0x5631336d3800 2024-04-25T00:39:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcb91600640 1 -- 172.21.15.42:0/3347220823 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcba0014a90 con 0x5631336d3800 2024-04-25T00:39:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcb91600640 1 -- 172.21.15.42:0/3347220823 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fcba00b8030 con 0x5631336d3800 2024-04-25T00:39:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcbb145c740 10 client.26734 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:39:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcbb145c740 10 client.26734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:39:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcbb145c740 20 client.26734 populate_metadata read hostname 'smithi042' 2024-04-25T00:39:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5631336e3680 0x5631336e5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcbb145c740 4 client.26734 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.659+0000 7fcbb145c740 1 -- 172.21.15.42:0/3347220823 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56313363dff0 con 0x5631336e3680 2024-04-25T00:39:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.660+0000 7fcba7400640 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5631336e3680 0x5631336e5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.660+0000 7fcb91600640 10 client.26734 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.662+0000 7fcba7400640 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5631336e3680 0x5631336e5ad0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5820fbff5a7df75a server_cookie=e5e2e68b0f93831f in_seq=0 out_seq=0 2024-04-25T00:39:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.663+0000 7fcb91600640 1 -- 172.21.15.42:0/3347220823 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56313363dff0 con 0x5631336e3680 2024-04-25T00:39:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.663+0000 7fcb91600640 10 client.26734 handle_command_reply: tid=0 2024-04-25T00:39:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.663+0000 7fcbb145c740 1 client.26734 shutdown 2024-04-25T00:39:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.664+0000 7fcbb145c740 1 -- 172.21.15.42:0/3347220823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 msgr2=0x5631336df410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.664+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336df410 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcba00cc1f0 tx=0x7fcba00976a0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.664+0000 7fcbb145c740 1 -- 172.21.15.42:0/3347220823 shutdown_connections 2024-04-25T00:39:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.664+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5631336e3680 0x5631336e5ad0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.664+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336df410 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.664+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336d83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.664+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3347220823 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336cdca0 0x5631336d7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.664+0000 7fcb91600640 0 client.26734 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.664+0000 7fcbb145c740 1 -- 172.21.15.42:0/3347220823 >> 172.21.15.42:0/3347220823 conn(0x5631336c9310 msgr2=0x5631336c8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.665+0000 7fcbb145c740 1 -- 172.21.15.42:0/3347220823 shutdown_connections 2024-04-25T00:39:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.665+0000 7fcbb145c740 1 -- 172.21.15.42:0/3347220823 wait complete. 2024-04-25T00:39:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.665+0000 7fcbb145c740 20 client.26734 trim_cache size 0 max 16384 2024-04-25T00:39:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.694+0000 7fcbb145c740 1 Processor -- start 2024-04-25T00:39:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.694+0000 7fcbb145c740 1 -- start start 2024-04-25T00:39:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.694+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336cdca0 0x5631336cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.694+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336df3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.694+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.694+0000 7fcbb145c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b5f20 con 0x5631336cdca0 2024-04-25T00:39:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.694+0000 7fcbb145c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b37f0 con 0x5631336ce690 2024-04-25T00:39:25.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.694+0000 7fcbb145c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b1640 con 0x5631336d3800 2024-04-25T00:39:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336df3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336df3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:48602/0 (socket says 172.21.15.42:48602) 2024-04-25T00:39:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba6a00640 1 -- 172.21.15.42:0/1857113936 learned_addr learned my addr 172.21.15.42:0/1857113936 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba7e00640 1 --2- 172.21.15.42:0/1857113936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba6a00640 1 -- 172.21.15.42:0/1857113936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 msgr2=0x5631336cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba6a00640 1 --2- 172.21.15.42:0/1857113936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba7400640 1 --2- 172.21.15.42:0/1857113936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336cdca0 0x5631336cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba6a00640 1 -- 172.21.15.42:0/1857113936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336cdca0 msgr2=0x5631336cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba6a00640 1 --2- 172.21.15.42:0/1857113936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336cdca0 0x5631336cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba6a00640 1 -- 172.21.15.42:0/1857113936 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631336c9140 con 0x5631336ce690 2024-04-25T00:39:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.695+0000 7fcba7400640 1 --2- 172.21.15.42:0/1857113936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336cdca0 0x5631336cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.696+0000 7fcba6a00640 1 --2- 172.21.15.42:0/1857113936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336df3d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcba8071560 tx=0x7fcb94013070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15a8a6698fe86edd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.696+0000 7fcb93e00640 1 -- 172.21.15.42:0/1857113936 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9400f710 con 0x5631336ce690 2024-04-25T00:39:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.696+0000 7fcb93e00640 1 -- 172.21.15.42:0/1857113936 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9400fd90 con 0x5631336ce690 2024-04-25T00:39:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.697+0000 7fcb93e00640 1 -- 172.21.15.42:0/1857113936 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb940253e0 con 0x5631336ce690 2024-04-25T00:39:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.697+0000 7fcbb145c740 1 -- 172.21.15.42:0/1857113936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 msgr2=0x5631336df3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.697+0000 7fcbb145c740 1 --2- 172.21.15.42:0/1857113936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336df3d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fcba8071560 tx=0x7fcb94013070 comp rx=0 tx=0).stop 2024-04-25T00:39:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.697+0000 7fcbb145c740 1 -- 172.21.15.42:0/1857113936 shutdown_connections 2024-04-25T00:39:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.697+0000 7fcbb145c740 1 --2- 172.21.15.42:0/1857113936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336d3800 0x5631336cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.697+0000 7fcbb145c740 1 --2- 172.21.15.42:0/1857113936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336ce690 0x5631336df3d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.699 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.697+0000 7fcbb145c740 1 --2- 172.21.15.42:0/1857113936 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336cdca0 0x5631336cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.697+0000 7fcbb145c740 1 -- 172.21.15.42:0/1857113936 >> 172.21.15.42:0/1857113936 conn(0x5631336c9310 msgr2=0x5631336d7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.697+0000 7fcbb145c740 1 -- 172.21.15.42:0/1857113936 shutdown_connections 2024-04-25T00:39:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.698+0000 7fcbb145c740 1 -- 172.21.15.42:0/1857113936 wait complete. 2024-04-25T00:39:25.700 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.698+0000 7fcbb145c740 1 -- start start 2024-04-25T00:39:25.701 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.698+0000 7fcbb145c740 1 Processor -- start 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.699+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336dcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.699+0000 7fcbb145c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336ce690 0x5631336dd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.700+0000 7fcba7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336dcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.700+0000 7fcba7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336dcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:48618/0 (socket says 172.21.15.42:48618) 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.700+0000 7fcba7400640 1 -- 172.21.15.42:0/3596882784 learned_addr learned my addr 172.21.15.42:0/3596882784 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.700+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336d3800 0x5631336de020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.700+0000 7fcba6a00640 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336ce690 0x5631336dd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.700+0000 7fcba7e00640 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336d3800 0x5631336de020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.701+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b1cb0 con 0x5631336d3800 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.701+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b3120 con 0x5631336cdca0 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.701+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcba81b2380 con 0x5631336ce690 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.701+0000 7fcbb145c740 10 client.0 _ll_register_callbacks cb 0x7fcba009aa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:25.703 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.701+0000 7fcba7e00640 1 -- 172.21.15.42:0/3596882784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336ce690 msgr2=0x5631336dd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.702+0000 7fcba7e00640 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336ce690 0x5631336dd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.702+0000 7fcba7e00640 1 -- 172.21.15.42:0/3596882784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 msgr2=0x5631336dcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.702+0000 7fcba7e00640 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336dcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.702+0000 7fcba7e00640 1 -- 172.21.15.42:0/3596882784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631336de610 con 0x5631336d3800 2024-04-25T00:39:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.702+0000 7fcba7400640 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336dcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.702+0000 7fcba6a00640 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336ce690 0x5631336dd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.702+0000 7fcba7e00640 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336d3800 0x5631336de020 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcba00cc220 tx=0x7fcba0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c844e6552630ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:25.704 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.702+0000 7fcb91600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:39:25.705 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.703+0000 7fcb91600640 1 -- 172.21.15.42:0/3596882784 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba009ae00 con 0x5631336d3800 2024-04-25T00:39:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.703+0000 7fcb91600640 1 -- 172.21.15.42:0/3596882784 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fcba0097710 con 0x5631336d3800 2024-04-25T00:39:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.703+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb9400fd90 con 0x5631336d3800 2024-04-25T00:39:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.703+0000 7fcb91600640 1 -- 172.21.15.42:0/3596882784 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcba009f330 con 0x5631336d3800 2024-04-25T00:39:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.703+0000 7fcbb145c740 10 client.16530 fetch_fsmap learned FSMap version 45 2024-04-25T00:39:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.703+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x563133743c20 con 0x5631336d3800 2024-04-25T00:39:25.706 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.704+0000 7fcb91600640 1 -- 172.21.15.42:0/3596882784 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9400fd90 con 0x5631336d3800 2024-04-25T00:39:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.706+0000 7fcb91600640 1 -- 172.21.15.42:0/3596882784 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fcba00b8040 con 0x5631336d3800 2024-04-25T00:39:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.706+0000 7fcbb145c740 10 client.16530 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:39:25.707 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.706+0000 7fcbb145c740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:39:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.706+0000 7fcbb145c740 20 client.16530 populate_metadata read hostname 'smithi042' 2024-04-25T00:39:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.706+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5631337443a0 0x5631337467c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.706+0000 7fcbb145c740 4 client.16530 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:39:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.706+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563133638340 con 0x5631337443a0 2024-04-25T00:39:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.706+0000 7fcba7400640 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5631337443a0 0x5631337467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.707+0000 7fcba7400640 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5631337443a0 0x5631337467c0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c1722e135addfd server_cookie=2e2cb5cdb9f56890 in_seq=0 out_seq=0 2024-04-25T00:39:25.708 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:25.707+0000 7fcb91600640 10 client.16530 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:25.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 273924 33.81 MB/sec execute 315 sec latency 90.931 ms 2024-04-25T00:39:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.044+0000 7fcb91600640 1 -- 172.21.15.42:0/3596882784 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563133638340 con 0x5631337443a0 2024-04-25T00:39:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.044+0000 7fcb91600640 10 client.16530 handle_command_reply: tid=0 2024-04-25T00:39:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.044+0000 7fcbb145c740 1 client.16530 shutdown 2024-04-25T00:39:26.046 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.044+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336d3800 msgr2=0x5631336de020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.044+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336d3800 0x5631336de020 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcba00cc220 tx=0x7fcba0004b50 comp rx=0 tx=0).stop 2024-04-25T00:39:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.045+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 shutdown_connections 2024-04-25T00:39:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.045+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5631337443a0 0x5631337467c0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.045+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5631336d3800 0x5631336de020 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.045+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631336ce690 0x5631336dd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.045+0000 7fcbb145c740 1 --2- 172.21.15.42:0/3596882784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5631336cdca0 0x5631336dcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.045+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 >> 172.21.15.42:0/3596882784 conn(0x5631336c9310 msgr2=0x5631336cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:26.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.045+0000 7fcb91600640 0 client.16530 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.046+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 shutdown_connections 2024-04-25T00:39:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.046+0000 7fcbb145c740 1 -- 172.21.15.42:0/3596882784 wait complete. 2024-04-25T00:39:26.049 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.047+0000 7fcbb145c740 20 client.16530 trim_cache size 0 max 16384 2024-04-25T00:39:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:39:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:39:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:39:26.050 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:39:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 -- 172.21.15.42:0/612167303 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcb8c0a2cb0 msgr2=0x7fcb8c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.051 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcb8c0a2cb0 0x7fcb8c0a5170 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x5631336cea90 tx=0x7fcb94005d60 comp rx=0 tx=0).stop 2024-04-25T00:39:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 -- 172.21.15.42:0/612167303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcba8105580 msgr2=0x7fcba81a9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcba8105580 0x7fcba81a9e00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fcba000be20 tx=0x7fcba0009ed0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 -- 172.21.15.42:0/612167303 shutdown_connections 2024-04-25T00:39:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcb8c0a2cb0 0x7fcb8c0a5170 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcba8105580 0x7fcba81a9e00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.052 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcba8104bc0 0x7fcba8070590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 --2- 172.21.15.42:0/612167303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcba81039d0 0x7fcba8070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 -- 172.21.15.42:0/612167303 >> 172.21.15.42:0/612167303 conn(0x7fcba80fef60 msgr2=0x7fcba8107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 -- 172.21.15.42:0/612167303 shutdown_connections 2024-04-25T00:39:26.053 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:26.048+0000 7fcbae600640 1 -- 172.21.15.42:0/612167303 wait complete. 2024-04-25T00:39:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:25 smithi042 ceph-mon[28130]: pgmap v1010: 97 pgs: 97 active+clean; 669 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.3 MiB/s wr, 603 op/s 2024-04-25T00:39:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:25 smithi136 ceph-mon[32923]: pgmap v1010: 97 pgs: 97 active+clean; 669 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.3 MiB/s wr, 603 op/s 2024-04-25T00:39:26.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:25 smithi106 ceph-mon[32510]: pgmap v1010: 97 pgs: 97 active+clean; 669 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 8.3 MiB/s wr, 603 op/s 2024-04-25T00:39:26.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 274542 33.79 MB/sec execute 316 sec latency 217.937 ms 2024-04-25T00:39:27.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 275240 33.80 MB/sec execute 317 sec latency 58.102 ms 2024-04-25T00:39:28.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:27 smithi042 ceph-mon[28130]: pgmap v1011: 97 pgs: 97 active+clean; 680 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.2 MiB/s wr, 729 op/s 2024-04-25T00:39:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:27 smithi136 ceph-mon[32923]: pgmap v1011: 97 pgs: 97 active+clean; 680 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.2 MiB/s wr, 729 op/s 2024-04-25T00:39:28.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:27 smithi106 ceph-mon[32510]: pgmap v1011: 97 pgs: 97 active+clean; 680 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 8.2 MiB/s wr, 729 op/s 2024-04-25T00:39:28.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 275936 33.84 MB/sec execute 318 sec latency 73.265 ms 2024-04-25T00:39:29.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 276516 33.84 MB/sec execute 319 sec latency 107.251 ms 2024-04-25T00:39:30.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:29 smithi042 ceph-mon[28130]: pgmap v1012: 97 pgs: 97 active+clean; 685 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.1 MiB/s wr, 566 op/s 2024-04-25T00:39:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:29 smithi136 ceph-mon[32923]: pgmap v1012: 97 pgs: 97 active+clean; 685 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.1 MiB/s wr, 566 op/s 2024-04-25T00:39:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:29 smithi106 ceph-mon[32510]: pgmap v1012: 97 pgs: 97 active+clean; 685 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.1 MiB/s wr, 566 op/s 2024-04-25T00:39:30.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 276860 33.79 MB/sec execute 320 sec latency 366.966 ms 2024-04-25T00:39:31.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 276994 33.70 MB/sec execute 321 sec latency 826.107 ms 2024-04-25T00:39:32.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:31 smithi042 ceph-mon[28130]: pgmap v1013: 97 pgs: 97 active+clean; 686 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.2 MiB/s wr, 468 op/s 2024-04-25T00:39:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:31 smithi136 ceph-mon[32923]: pgmap v1013: 97 pgs: 97 active+clean; 686 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.2 MiB/s wr, 468 op/s 2024-04-25T00:39:32.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:31 smithi106 ceph-mon[32510]: pgmap v1013: 97 pgs: 97 active+clean; 686 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.2 MiB/s wr, 468 op/s 2024-04-25T00:39:32.790 INFO:tasks.workunit.client.0.smithi042.stdout: 10 277327 33.65 MB/sec execute 322 sec latency 135.669 ms 2024-04-25T00:39:33.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 278020 33.65 MB/sec execute 323 sec latency 70.460 ms 2024-04-25T00:39:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:33 smithi042 ceph-mon[28130]: pgmap v1014: 97 pgs: 97 active+clean; 747 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.9 MiB/s wr, 664 op/s 2024-04-25T00:39:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:33 smithi136 ceph-mon[32923]: pgmap v1014: 97 pgs: 97 active+clean; 747 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.9 MiB/s wr, 664 op/s 2024-04-25T00:39:34.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:33 smithi106 ceph-mon[32510]: pgmap v1014: 97 pgs: 97 active+clean; 747 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 MiB/s rd, 9.9 MiB/s wr, 664 op/s 2024-04-25T00:39:34.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 278874 33.66 MB/sec execute 324 sec latency 51.025 ms 2024-04-25T00:39:35.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 279802 33.76 MB/sec execute 325 sec latency 46.342 ms 2024-04-25T00:39:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:35 smithi042 ceph-mon[28130]: pgmap v1015: 97 pgs: 97 active+clean; 746 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.9 MiB/s wr, 578 op/s 2024-04-25T00:39:36.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:35 smithi136 ceph-mon[32923]: pgmap v1015: 97 pgs: 97 active+clean; 746 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.9 MiB/s wr, 578 op/s 2024-04-25T00:39:36.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:35 smithi106 ceph-mon[32510]: pgmap v1015: 97 pgs: 97 active+clean; 746 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 9.9 MiB/s wr, 578 op/s 2024-04-25T00:39:36.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 280341 33.75 MB/sec execute 326 sec latency 83.977 ms 2024-04-25T00:39:37.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 280858 33.71 MB/sec execute 327 sec latency 64.340 ms 2024-04-25T00:39:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:37 smithi042 ceph-mon[28130]: pgmap v1016: 97 pgs: 97 active+clean; 696 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 790 op/s 2024-04-25T00:39:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:37 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:39:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:37 smithi136 ceph-mon[32923]: pgmap v1016: 97 pgs: 97 active+clean; 696 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 790 op/s 2024-04-25T00:39:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:37 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:39:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:37 smithi106 ceph-mon[32510]: pgmap v1016: 97 pgs: 97 active+clean; 696 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 790 op/s 2024-04-25T00:39:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:37 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:39:38.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 281213 33.67 MB/sec execute 328 sec latency 430.472 ms 2024-04-25T00:39:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.415+0000 7f41ffe00640 1 -- 172.21.15.42:0/2012963762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4200109350 msgr2=0x7f4200072770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.415+0000 7f41ffe00640 1 --2- 172.21.15.42:0/2012963762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4200109350 0x7f4200072770 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009900 tx=0x7f41f002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.415+0000 7f41ffe00640 1 -- 172.21.15.42:0/2012963762 shutdown_connections 2024-04-25T00:39:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.415+0000 7f41ffe00640 1 --2- 172.21.15.42:0/2012963762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f420010fa80 0x7f4200111e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.415+0000 7f41ffe00640 1 --2- 172.21.15.42:0/2012963762 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4200072cb0 0x7f4200073130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.417 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.415+0000 7f41ffe00640 1 --2- 172.21.15.42:0/2012963762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4200109350 0x7f4200072770 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.415+0000 7f41ffe00640 1 -- 172.21.15.42:0/2012963762 >> 172.21.15.42:0/2012963762 conn(0x7f420006e5e0 msgr2=0x7f4200070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.415+0000 7f41ffe00640 1 -- 172.21.15.42:0/2012963762 shutdown_connections 2024-04-25T00:39:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.415+0000 7f41ffe00640 1 -- 172.21.15.42:0/2012963762 wait complete. 2024-04-25T00:39:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.417+0000 7f41ffe00640 1 Processor -- start 2024-04-25T00:39:39.418 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.417+0000 7f41ffe00640 1 -- start start 2024-04-25T00:39:39.419 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.417+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4200072cb0 0x7f42001a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.418+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4200109350 0x7f42001a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.418+0000 7f41ffe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f420010fa80 0x7f42001adbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.418+0000 7f41ffe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f4200113890 con 0x7f4200072cb0 2024-04-25T00:39:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.418+0000 7f41ffe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f4200113710 con 0x7f4200109350 2024-04-25T00:39:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.418+0000 7f41ffe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4200113a10 con 0x7f420010fa80 2024-04-25T00:39:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.418+0000 7f41fea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4200072cb0 0x7f42001a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.418+0000 7f41f7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4200109350 0x7f42001a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.420 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.418+0000 7f41f7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4200109350 0x7f42001a6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:47700/0 (socket says 172.21.15.42:47700) 2024-04-25T00:39:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.418+0000 7f41fea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4200072cb0 0x7f42001a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:55056/0 (socket says 172.21.15.42:55056) 2024-04-25T00:39:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.418+0000 7f41f7e00640 1 -- 172.21.15.42:0/1918230296 learned_addr learned my addr 172.21.15.42:0/1918230296 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.419+0000 7f41f7e00640 1 -- 172.21.15.42:0/1918230296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f420010fa80 msgr2=0x7f42001adbe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:39:39.421 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.419+0000 7f41f7e00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f420010fa80 0x7f42001adbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.419+0000 7f41f7e00640 1 -- 172.21.15.42:0/1918230296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4200072cb0 msgr2=0x7f42001a6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.419+0000 7f41f7e00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4200072cb0 0x7f42001a6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.419+0000 7f41f7e00640 1 -- 172.21.15.42:0/1918230296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42001ae250 con 0x7f4200109350 2024-04-25T00:39:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.419+0000 7f41fea00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4200072cb0 0x7f42001a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.419+0000 7f41f7e00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4200109350 0x7f42001a6b70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f41e800c970 tx=0x7f41e800ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.420+0000 7f4204a00640 1 -- 172.21.15.42:0/1918230296 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8007c10 con 0x7f4200109350 2024-04-25T00:39:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.420+0000 7f4204a00640 1 -- 172.21.15.42:0/1918230296 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41e8007db0 con 0x7f4200109350 2024-04-25T00:39:39.422 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.420+0000 7f4204a00640 1 -- 172.21.15.42:0/1918230296 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e800f940 con 0x7f4200109350 2024-04-25T00:39:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.421+0000 7f41ffe00640 1 -- 172.21.15.42:0/1918230296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42001ae540 con 0x7f4200109350 2024-04-25T00:39:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.421+0000 7f41ffe00640 1 -- 172.21.15.42:0/1918230296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001aea40 con 0x7f4200109350 2024-04-25T00:39:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.436+0000 7f4208d9e740 1 Processor -- start 2024-04-25T00:39:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.440+0000 7f4204a00640 1 -- 172.21.15.42:0/1918230296 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f41e800fae0 con 0x7f4200109350 2024-04-25T00:39:39.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.441+0000 7f4208d9e740 1 -- start start 2024-04-25T00:39:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.442+0000 7f4204a00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2c00 0x7f41e40a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.443+0000 7f4204a00640 1 -- 172.21.15.42:0/1918230296 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f41e80cf430 con 0x7f4200109350 2024-04-25T00:39:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.443+0000 7f41ff400640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2c00 0x7f41e40a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.444+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.444+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 0x560d658cc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.444+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 0x560d658cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.444+0000 7f4208d9e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560d65829ab0 con 0x560d658c7840 2024-04-25T00:39:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.444+0000 7f4208d9e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560d657d2640 con 0x560d658cc9b0 2024-04-25T00:39:39.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.444+0000 7f4208d9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560d65825a70 con 0x560d658c6e50 2024-04-25T00:39:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.445+0000 7f41fea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.445+0000 7f41fea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:54428/0 (socket says 172.21.15.42:54428) 2024-04-25T00:39:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.445+0000 7f41fea00640 1 -- 172.21.15.42:0/2303629146 learned_addr learned my addr 172.21.15.42:0/2303629146 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.445+0000 7f41f7e00640 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 0x560d658cc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.445+0000 7f41fea00640 1 -- 172.21.15.42:0/2303629146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 msgr2=0x560d658cee40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:39:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.446+0000 7f41ff400640 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 0x560d658cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.446+0000 7f41ff400640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2c00 0x7f41e40a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f41f8005c40 tx=0x7f41f8005bd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.446+0000 7f41fea00640 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 0x560d658cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.446+0000 7f41fea00640 1 -- 172.21.15.42:0/2303629146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 msgr2=0x560d658cc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.446+0000 7f41fea00640 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 0x560d658cc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.446+0000 7f41fea00640 1 -- 172.21.15.42:0/2303629146 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d6553e210 con 0x560d658c6e50 2024-04-25T00:39:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.447+0000 7f41f7e00640 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 0x560d658cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.447+0000 7f41ff400640 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 0x560d658cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:39:39.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.447+0000 7f41fea00640 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c7270 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f41f0031440 tx=0x7f41f00043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eca246977144a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.448+0000 7f41efe00640 1 -- 172.21.15.42:0/2303629146 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0035030 con 0x560d658c6e50 2024-04-25T00:39:39.450 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.449+0000 7f41efe00640 1 -- 172.21.15.42:0/2303629146 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f00046c0 con 0x560d658c6e50 2024-04-25T00:39:39.451 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.449+0000 7f41efe00640 1 -- 172.21.15.42:0/2303629146 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0031910 con 0x560d658c6e50 2024-04-25T00:39:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.454+0000 7f4208d9e740 1 -- 172.21.15.42:0/2303629146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 msgr2=0x560d658c7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.454+0000 7f4208d9e740 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c7270 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f41f0031440 tx=0x7f41f00043b0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.456 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.455+0000 7f4208d9e740 1 -- 172.21.15.42:0/2303629146 shutdown_connections 2024-04-25T00:39:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.455+0000 7f4208d9e740 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 0x560d658cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.455+0000 7f4208d9e740 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 0x560d658cc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.455+0000 7f4208d9e740 1 --2- 172.21.15.42:0/2303629146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c7270 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.455+0000 7f4208d9e740 1 -- 172.21.15.42:0/2303629146 >> 172.21.15.42:0/2303629146 conn(0x560d658c24c0 msgr2=0x560d658c28c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:39.457 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.455+0000 7f4208d9e740 1 -- 172.21.15.42:0/2303629146 shutdown_connections 2024-04-25T00:39:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.456+0000 7f4208d9e740 1 -- 172.21.15.42:0/2303629146 wait complete. 2024-04-25T00:39:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.457+0000 7f4208d9e740 1 -- start start 2024-04-25T00:39:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.457+0000 7f4208d9e740 1 Processor -- start 2024-04-25T00:39:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.462+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.462+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 0x560d658d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.462+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658cc9b0 0x560d658d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.462+0000 7f4208d9e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560d658cf900 con 0x560d658cc9b0 2024-04-25T00:39:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.462+0000 7f4208d9e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560d657e5990 con 0x560d658c6e50 2024-04-25T00:39:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.462+0000 7f4208d9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560d658cfa80 con 0x560d658c7840 2024-04-25T00:39:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.462+0000 7f4208d9e740 10 client.0 _ll_register_callbacks cb 0x560d657e7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.463+0000 7f41fea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.463+0000 7f41fea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:47710/0 (socket says 172.21.15.42:47710) 2024-04-25T00:39:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.463+0000 7f41fea00640 1 -- 172.21.15.42:0/3323375496 learned_addr learned my addr 172.21.15.42:0/3323375496 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.465+0000 7f41fea00640 1 -- 172.21.15.42:0/3323375496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 msgr2=0x560d658d15a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:39:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.465+0000 7f41fea00640 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 0x560d658d15a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.465+0000 7f41fea00640 1 -- 172.21.15.42:0/3323375496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658cc9b0 msgr2=0x560d658d85e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:39:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.465+0000 7f41fea00640 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658cc9b0 0x560d658d85e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.465+0000 7f41fea00640 1 -- 172.21.15.42:0/3323375496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d6560a620 con 0x560d658c6e50 2024-04-25T00:39:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.465+0000 7f41fea00640 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d1060 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006010 tx=0x7f41f00051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8b3e256f0d5c364 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.466+0000 7f41ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:39:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.466+0000 7f41ecc00640 1 -- 172.21.15.42:0/3323375496 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0051060 con 0x560d658c6e50 2024-04-25T00:39:39.471 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.466+0000 7f41ecc00640 1 -- 172.21.15.42:0/3323375496 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f41f00318b0 con 0x560d658c6e50 2024-04-25T00:39:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.470+0000 7f4208d9e740 1 -- 172.21.15.42:0/3323375496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f00046c0 con 0x560d658c6e50 2024-04-25T00:39:39.472 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.471+0000 7f41ecc00640 1 -- 172.21.15.42:0/3323375496 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f41f0035040 con 0x560d658c6e50 2024-04-25T00:39:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.472+0000 7f4208d9e740 10 client.26795 fetch_fsmap learned FSMap version 45 2024-04-25T00:39:39.473 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.472+0000 7f4208d9e740 1 -- 172.21.15.42:0/3323375496 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560d658db530 con 0x560d658c6e50 2024-04-25T00:39:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.472+0000 7f41ecc00640 1 -- 172.21.15.42:0/3323375496 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f00046c0 con 0x560d658c6e50 2024-04-25T00:39:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.472+0000 7f41ecc00640 1 -- 172.21.15.42:0/3323375496 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f41f004b030 con 0x560d658c6e50 2024-04-25T00:39:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.473+0000 7f4208d9e740 10 client.26795 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:39:39.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.473+0000 7f4208d9e740 10 client.26795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:39:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.473+0000 7f4208d9e740 20 client.26795 populate_metadata read hostname 'smithi042' 2024-04-25T00:39:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.473+0000 7f4208d9e740 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560d658dc880 0x560d658dec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.473+0000 7f4208d9e740 4 client.26795 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:39.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.473+0000 7f4208d9e740 1 -- 172.21.15.42:0/3323375496 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d65836ff0 con 0x560d658dc880 2024-04-25T00:39:39.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.474+0000 7f41f7e00640 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560d658dc880 0x560d658dec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.477+0000 7f41f7e00640 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560d658dc880 0x560d658dec70 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ff6ead7378c9ba5 server_cookie=378e24aa4bda9512 in_seq=0 out_seq=0 2024-04-25T00:39:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.477+0000 7f41ecc00640 10 client.26795 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.480+0000 7f41ecc00640 1 -- 172.21.15.42:0/3323375496 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560d65836ff0 con 0x560d658dc880 2024-04-25T00:39:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.480+0000 7f41ecc00640 10 client.26795 handle_command_reply: tid=0 2024-04-25T00:39:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.480+0000 7f4208d9e740 1 client.26795 shutdown 2024-04-25T00:39:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.481+0000 7f4208d9e740 1 -- 172.21.15.42:0/3323375496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 msgr2=0x560d658d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.481+0000 7f4208d9e740 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d1060 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f41f0006010 tx=0x7f41f00051f0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.482+0000 7f4208d9e740 1 -- 172.21.15.42:0/3323375496 shutdown_connections 2024-04-25T00:39:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.482+0000 7f4208d9e740 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560d658dc880 0x560d658dec70 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.482+0000 7f4208d9e740 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658cc9b0 0x560d658d85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.482+0000 7f4208d9e740 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 0x560d658d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.482+0000 7f4208d9e740 1 --2- 172.21.15.42:0/3323375496 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d1060 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.482+0000 7f4208d9e740 1 -- 172.21.15.42:0/3323375496 >> 172.21.15.42:0/3323375496 conn(0x560d658c24c0 msgr2=0x560d658c1620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:39.483 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.482+0000 7f41ecc00640 0 client.26795 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:39.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.485+0000 7f4208d9e740 1 -- 172.21.15.42:0/3323375496 shutdown_connections 2024-04-25T00:39:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.485+0000 7f4208d9e740 1 -- 172.21.15.42:0/3323375496 wait complete. 2024-04-25T00:39:39.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.486+0000 7f4208d9e740 20 client.26795 trim_cache size 0 max 16384 2024-04-25T00:39:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.517+0000 7f4208d9e740 1 Processor -- start 2024-04-25T00:39:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.517+0000 7f4208d9e740 1 -- start start 2024-04-25T00:39:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.517+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.518+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 0x560d658c12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.518+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 0x560d658c1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.518+0000 7f4208d9e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f42001ba170 con 0x560d658c7840 2024-04-25T00:39:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.518+0000 7f4208d9e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b7a40 con 0x560d658cc9b0 2024-04-25T00:39:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.518+0000 7f4208d9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b5890 con 0x560d658c6e50 2024-04-25T00:39:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.518+0000 7f41fea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.518+0000 7f41fea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:54456/0 (socket says 172.21.15.42:54456) 2024-04-25T00:39:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.518+0000 7f41fea00640 1 -- 172.21.15.42:0/2353810723 learned_addr learned my addr 172.21.15.42:0/2353810723 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:39.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.518+0000 7f41f7e00640 1 --2- 172.21.15.42:0/2353810723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 0x560d658c12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.519+0000 7f41ff400640 1 --2- 172.21.15.42:0/2353810723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 0x560d658c1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.520+0000 7f41fea00640 1 -- 172.21.15.42:0/2353810723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 msgr2=0x560d658c1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.520+0000 7f41fea00640 1 --2- 172.21.15.42:0/2353810723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 0x560d658c1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.520+0000 7f41fea00640 1 -- 172.21.15.42:0/2353810723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 msgr2=0x560d658c12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.520+0000 7f41fea00640 1 --2- 172.21.15.42:0/2353810723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 0x560d658c12d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.520+0000 7f41fea00640 1 -- 172.21.15.42:0/2353810723 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d658cfbc0 con 0x560d658c6e50 2024-04-25T00:39:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.520+0000 7f41f7e00640 1 --2- 172.21.15.42:0/2353810723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 0x560d658c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:39:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.521+0000 7f41fea00640 1 --2- 172.21.15.42:0/2353810723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c0d90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f41f003bb50 tx=0x7f41f0005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1deb484d0c02ef3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.521+0000 7f41ef400640 1 -- 172.21.15.42:0/2353810723 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0051040 con 0x560d658c6e50 2024-04-25T00:39:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.521+0000 7f41ef400640 1 -- 172.21.15.42:0/2353810723 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f003b910 con 0x560d658c6e50 2024-04-25T00:39:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.521+0000 7f41ef400640 1 -- 172.21.15.42:0/2353810723 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0031df0 con 0x560d658c6e50 2024-04-25T00:39:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.523+0000 7f4208d9e740 1 -- 172.21.15.42:0/2353810723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 msgr2=0x560d658c0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.523+0000 7f4208d9e740 1 --2- 172.21.15.42:0/2353810723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c0d90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f41f003bb50 tx=0x7f41f0005dc0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.523+0000 7f4208d9e740 1 -- 172.21.15.42:0/2353810723 shutdown_connections 2024-04-25T00:39:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.523+0000 7f4208d9e740 1 --2- 172.21.15.42:0/2353810723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658cc9b0 0x560d658c1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.523+0000 7f4208d9e740 1 --2- 172.21.15.42:0/2353810723 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658c7840 0x560d658c12d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.523+0000 7f4208d9e740 1 --2- 172.21.15.42:0/2353810723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c6e50 0x560d658c0d90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.523+0000 7f4208d9e740 1 -- 172.21.15.42:0/2353810723 >> 172.21.15.42:0/2353810723 conn(0x560d658c24c0 msgr2=0x560d658cc0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.524+0000 7f4208d9e740 1 -- 172.21.15.42:0/2353810723 shutdown_connections 2024-04-25T00:39:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.524+0000 7f4208d9e740 1 -- 172.21.15.42:0/2353810723 wait complete. 2024-04-25T00:39:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.525+0000 7f4208d9e740 1 -- start start 2024-04-25T00:39:39.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.525+0000 7f4208d9e740 1 Processor -- start 2024-04-25T00:39:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.526+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.526+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 0x560d658d5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.526+0000 7f4208d9e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658cc9b0 0x560d658d6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.527+0000 7f4208d9e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b5f00 con 0x560d658cc9b0 2024-04-25T00:39:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.527+0000 7f4208d9e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b7370 con 0x560d658c6e50 2024-04-25T00:39:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.527+0000 7f4208d9e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42001b65d0 con 0x560d658c7840 2024-04-25T00:39:39.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.527+0000 7f4208d9e740 10 client.0 _ll_register_callbacks cb 0x7f41f0004260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.527+0000 7f41f7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 0x560d658d5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.527+0000 7f41f7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 0x560d658d5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:54468/0 (socket says 172.21.15.42:54468) 2024-04-25T00:39:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.527+0000 7f41f7e00640 1 -- 172.21.15.42:0/391885458 learned_addr learned my addr 172.21.15.42:0/391885458 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.528+0000 7f41fea00640 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.528+0000 7f41ff400640 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658cc9b0 0x560d658d6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.528+0000 7f41f7e00640 1 -- 172.21.15.42:0/391885458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 msgr2=0x560d658d5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.528+0000 7f41f7e00640 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.528+0000 7f41f7e00640 1 -- 172.21.15.42:0/391885458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658cc9b0 msgr2=0x560d658d6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.528+0000 7f41f7e00640 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658cc9b0 0x560d658d6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.528+0000 7f41f7e00640 1 -- 172.21.15.42:0/391885458 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d658d6b50 con 0x560d658c7840 2024-04-25T00:39:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.528+0000 7f41fea00640 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.529+0000 7f41f7e00640 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 0x560d658d5550 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f41e80ccf60 tx=0x7f41e80a49f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e5a24c726e17b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.529+0000 7f41ecc00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:39:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.529+0000 7f41ecc00640 1 -- 172.21.15.42:0/391885458 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e8099df0 con 0x560d658c7840 2024-04-25T00:39:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.529+0000 7f41ecc00640 1 -- 172.21.15.42:0/391885458 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f41e809a050 con 0x560d658c7840 2024-04-25T00:39:39.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.529+0000 7f4208d9e740 1 -- 172.21.15.42:0/391885458 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f003b910 con 0x560d658c7840 2024-04-25T00:39:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.532+0000 7f41ecc00640 1 -- 172.21.15.42:0/391885458 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f41e8018b80 con 0x560d658c7840 2024-04-25T00:39:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.533+0000 7f4208d9e740 10 client.26752 fetch_fsmap learned FSMap version 45 2024-04-25T00:39:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.533+0000 7f4208d9e740 1 -- 172.21.15.42:0/391885458 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560d65942c50 con 0x560d658c7840 2024-04-25T00:39:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.534+0000 7f41ecc00640 1 -- 172.21.15.42:0/391885458 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41f003b910 con 0x560d658c7840 2024-04-25T00:39:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.534+0000 7f41ecc00640 1 -- 172.21.15.42:0/391885458 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f41e80a44a0 con 0x560d658c7840 2024-04-25T00:39:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.534+0000 7f4208d9e740 10 client.26752 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:39:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.534+0000 7f4208d9e740 10 client.26752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:39:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.534+0000 7f4208d9e740 20 client.26752 populate_metadata read hostname 'smithi042' 2024-04-25T00:39:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.534+0000 7f4208d9e740 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560d65943530 0x560d65945950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.534+0000 7f4208d9e740 4 client.26752 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:39:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.534+0000 7f4208d9e740 1 -- 172.21.15.42:0/391885458 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d65831340 con 0x560d65943530 2024-04-25T00:39:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.535+0000 7f41fea00640 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560d65943530 0x560d65945950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.536+0000 7f41fea00640 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560d65943530 0x560d65945950 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7762171d343089bb server_cookie=c8b32317aec089a6 in_seq=0 out_seq=0 2024-04-25T00:39:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.536+0000 7f41ecc00640 10 client.26752 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:39.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 281662 33.63 MB/sec execute 329 sec latency 166.449 ms 2024-04-25T00:39:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.814+0000 7f41ecc00640 1 -- 172.21.15.42:0/391885458 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d65831340 con 0x560d65943530 2024-04-25T00:39:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.814+0000 7f41ecc00640 10 client.26752 handle_command_reply: tid=0 2024-04-25T00:39:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.814+0000 7f4208d9e740 1 client.26752 shutdown 2024-04-25T00:39:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f4208d9e740 1 -- 172.21.15.42:0/391885458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 msgr2=0x560d658d5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f4208d9e740 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 0x560d658d5550 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f41e80ccf60 tx=0x7f41e80a49f0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f4208d9e740 1 -- 172.21.15.42:0/391885458 shutdown_connections 2024-04-25T00:39:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f4208d9e740 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560d65943530 0x560d65945950 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f4208d9e740 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560d658cc9b0 0x560d658d6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f4208d9e740 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560d658c7840 0x560d658d5550 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f4208d9e740 1 --2- 172.21.15.42:0/391885458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560d658c6e50 0x560d658d5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f4208d9e740 1 -- 172.21.15.42:0/391885458 >> 172.21.15.42:0/391885458 conn(0x560d658c24c0 msgr2=0x560d658c51a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f41ecc00640 0 client.26752 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f4208d9e740 1 -- 172.21.15.42:0/391885458 shutdown_connections 2024-04-25T00:39:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.815+0000 7f4208d9e740 1 -- 172.21.15.42:0/391885458 wait complete. 2024-04-25T00:39:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.816+0000 7f4208d9e740 20 client.26752 trim_cache size 0 max 16384 2024-04-25T00:39:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:39:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:39:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:39:39.818 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:39:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.817+0000 7f41ffe00640 1 -- 172.21.15.42:0/1918230296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2c00 msgr2=0x7f41e40a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.818+0000 7f41ffe00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2c00 0x7f41e40a50c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f41f8005c40 tx=0x7f41f8005bd0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.818+0000 7f41ffe00640 1 -- 172.21.15.42:0/1918230296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4200109350 msgr2=0x7f42001a6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:39.819 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.818+0000 7f41ffe00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4200109350 0x7f42001a6b70 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f41e800c970 tx=0x7f41e800ce30 comp rx=0 tx=0).stop 2024-04-25T00:39:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.821+0000 7f41ffe00640 1 -- 172.21.15.42:0/1918230296 shutdown_connections 2024-04-25T00:39:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.821+0000 7f41ffe00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f41e40a2c00 0x7f41e40a50c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.821+0000 7f41ffe00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f420010fa80 0x7f42001adbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.821+0000 7f41ffe00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4200109350 0x7f42001a6b70 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.821+0000 7f41ffe00640 1 --2- 172.21.15.42:0/1918230296 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f4200072cb0 0x7f42001a6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.821+0000 7f41ffe00640 1 -- 172.21.15.42:0/1918230296 >> 172.21.15.42:0/1918230296 conn(0x7f420006e5e0 msgr2=0x7f420010ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.821+0000 7f41ffe00640 1 -- 172.21.15.42:0/1918230296 shutdown_connections 2024-04-25T00:39:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:39.821+0000 7f41ffe00640 1 -- 172.21.15.42:0/1918230296 wait complete. 2024-04-25T00:39:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:39 smithi136 ceph-mon[32923]: pgmap v1017: 97 pgs: 97 active+clean; 668 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 12 MiB/s wr, 732 op/s 2024-04-25T00:39:40.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:39 smithi106 ceph-mon[32510]: pgmap v1017: 97 pgs: 97 active+clean; 668 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 12 MiB/s wr, 732 op/s 2024-04-25T00:39:40.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:39 smithi042 ceph-mon[28130]: pgmap v1017: 97 pgs: 97 active+clean; 668 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 12 MiB/s wr, 732 op/s 2024-04-25T00:39:40.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 282302 33.61 MB/sec execute 330 sec latency 154.848 ms 2024-04-25T00:39:41.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:41 smithi136 ceph-mon[32923]: pgmap v1018: 97 pgs: 97 active+clean; 656 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 700 op/s 2024-04-25T00:39:41.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:41 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:41 smithi106 ceph-mon[32510]: pgmap v1018: 97 pgs: 97 active+clean; 656 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 700 op/s 2024-04-25T00:39:41.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.589 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:41 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.628 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:41 smithi042 ceph-mon[28130]: pgmap v1018: 97 pgs: 97 active+clean; 656 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 13 MiB/s wr, 700 op/s 2024-04-25T00:39:41.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.629 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:41 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:41.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 283035 33.66 MB/sec execute 331 sec latency 112.764 ms 2024-04-25T00:39:42.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 283624 33.67 MB/sec execute 332 sec latency 81.825 ms 2024-04-25T00:39:43.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:43 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.514 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:43 smithi136 ceph-mon[32923]: pgmap v1019: 97 pgs: 97 active+clean; 640 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 837 op/s 2024-04-25T00:39:43.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 284071 33.64 MB/sec execute 333 sec latency 79.482 ms 2024-04-25T00:39:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:43 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:43 smithi106 ceph-mon[32510]: pgmap v1019: 97 pgs: 97 active+clean; 640 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 837 op/s 2024-04-25T00:39:43.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:43 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:43.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:43 smithi042 ceph-mon[28130]: pgmap v1019: 97 pgs: 97 active+clean; 640 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 15 MiB/s wr, 837 op/s 2024-04-25T00:39:44.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 284529 33.62 MB/sec execute 334 sec latency 104.282 ms 2024-04-25T00:39:45.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:45 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:45.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:45 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:45.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:45 smithi106 ceph-mon[32510]: pgmap v1020: 97 pgs: 97 active+clean; 639 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.1 MiB/s wr, 655 op/s 2024-04-25T00:39:45.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:45.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:45 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:45.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:45 smithi042 ceph-mon[28130]: pgmap v1020: 97 pgs: 97 active+clean; 639 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.1 MiB/s wr, 655 op/s 2024-04-25T00:39:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:45 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:45 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:45.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:45 smithi136 ceph-mon[32923]: pgmap v1020: 97 pgs: 97 active+clean; 639 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.1 MiB/s wr, 655 op/s 2024-04-25T00:39:45.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 285170 33.58 MB/sec execute 335 sec latency 84.230 ms 2024-04-25T00:39:46.792 INFO:tasks.workunit.client.0.smithi042.stdout: 10 285917 33.60 MB/sec execute 336 sec latency 70.992 ms 2024-04-25T00:39:47.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 286678 33.66 MB/sec execute 337 sec latency 89.553 ms 2024-04-25T00:39:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:47 smithi106 ceph-mon[32510]: pgmap v1021: 97 pgs: 97 active+clean; 646 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.1 MiB/s wr, 751 op/s 2024-04-25T00:39:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:39:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:39:47.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:47.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:47 smithi042 ceph-mon[28130]: pgmap v1021: 97 pgs: 97 active+clean; 646 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.1 MiB/s wr, 751 op/s 2024-04-25T00:39:47.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:39:47.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:39:47.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:47 smithi136 ceph-mon[32923]: pgmap v1021: 97 pgs: 97 active+clean; 646 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 9.1 MiB/s wr, 751 op/s 2024-04-25T00:39:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:39:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:39:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:39:48.791 INFO:tasks.workunit.client.0.smithi042.stdout: 10 287364 33.65 MB/sec execute 338 sec latency 40.428 ms 2024-04-25T00:39:49.792 INFO:tasks.workunit.client.0.smithi042.stdout: 10 287967 33.65 MB/sec execute 339 sec latency 61.975 ms 2024-04-25T00:39:49.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:49 smithi106 ceph-mon[32510]: pgmap v1022: 97 pgs: 97 active+clean; 648 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.8 MiB/s wr, 573 op/s 2024-04-25T00:39:49.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:49 smithi042 ceph-mon[28130]: pgmap v1022: 97 pgs: 97 active+clean; 648 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.8 MiB/s wr, 573 op/s 2024-04-25T00:39:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:49 smithi136 ceph-mon[32923]: pgmap v1022: 97 pgs: 97 active+clean; 648 MiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 6.8 MiB/s wr, 573 op/s 2024-04-25T00:39:50.792 INFO:tasks.workunit.client.0.smithi042.stdout: 10 288676 33.67 MB/sec execute 340 sec latency 60.139 ms 2024-04-25T00:39:51.792 INFO:tasks.workunit.client.0.smithi042.stdout: 10 289395 33.66 MB/sec execute 341 sec latency 43.894 ms 2024-04-25T00:39:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:51 smithi106 ceph-mon[32510]: pgmap v1023: 97 pgs: 97 active+clean; 655 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.5 MiB/s wr, 541 op/s 2024-04-25T00:39:51.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:51 smithi042 ceph-mon[28130]: pgmap v1023: 97 pgs: 97 active+clean; 655 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.5 MiB/s wr, 541 op/s 2024-04-25T00:39:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:51 smithi136 ceph-mon[32923]: pgmap v1023: 97 pgs: 97 active+clean; 655 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 6.5 MiB/s wr, 541 op/s 2024-04-25T00:39:52.792 INFO:tasks.workunit.client.0.smithi042.stdout: 10 290238 33.71 MB/sec execute 342 sec latency 48.936 ms 2024-04-25T00:39:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.023+0000 7f77b1800640 1 -- 172.21.15.42:0/895767940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77ac1039d0 msgr2=0x7f77ac103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.023+0000 7f77b1800640 1 --2- 172.21.15.42:0/895767940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77ac1039d0 0x7f77ac103df0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f779c009900 tx=0x7f779c02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.023+0000 7f77b1800640 1 -- 172.21.15.42:0/895767940 shutdown_connections 2024-04-25T00:39:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.023+0000 7f77b1800640 1 --2- 172.21.15.42:0/895767940 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77ac105580 0x7f77ac10be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.023+0000 7f77b1800640 1 --2- 172.21.15.42:0/895767940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac104bc0 0x7f77ac105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.023+0000 7f77b1800640 1 --2- 172.21.15.42:0/895767940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77ac1039d0 0x7f77ac103df0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.023+0000 7f77b1800640 1 -- 172.21.15.42:0/895767940 >> 172.21.15.42:0/895767940 conn(0x7f77ac0fef60 msgr2=0x7f77ac101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.023+0000 7f77b1800640 1 -- 172.21.15.42:0/895767940 shutdown_connections 2024-04-25T00:39:53.025 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.024+0000 7f77b1800640 1 -- 172.21.15.42:0/895767940 wait complete. 2024-04-25T00:39:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.025+0000 7f77b1800640 1 Processor -- start 2024-04-25T00:39:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.025+0000 7f77b1800640 1 -- start start 2024-04-25T00:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.026+0000 7f77b1800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1039d0 0x7f77ac199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.026+0000 7f77b1800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77ac104bc0 0x7f77ac19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.026+0000 7f77b1800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77ac105580 0x7f77ac1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.026+0000 7f77b1800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac10f670 con 0x7f77ac104bc0 2024-04-25T00:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.026+0000 7f77b1800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac10f4f0 con 0x7f77ac1039d0 2024-04-25T00:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.026+0000 7f77b1800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac10f7f0 con 0x7f77ac105580 2024-04-25T00:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.026+0000 7f77aaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1039d0 0x7f77ac199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.026+0000 7f77aaa00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1039d0 0x7f77ac199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:32944/0 (socket says 172.21.15.42:32944) 2024-04-25T00:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.026+0000 7f77aaa00640 1 -- 172.21.15.42:0/3016077544 learned_addr learned my addr 172.21.15.42:0/3016077544 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.026+0000 7f77ab400640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77ac105580 0x7f77ac1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.027+0000 7f77aa000640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77ac104bc0 0x7f77ac19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.027+0000 7f77aaa00640 1 -- 172.21.15.42:0/3016077544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77ac105580 msgr2=0x7f77ac1a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.027+0000 7f77aaa00640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77ac105580 0x7f77ac1a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.027+0000 7f77aaa00640 1 -- 172.21.15.42:0/3016077544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77ac104bc0 msgr2=0x7f77ac19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.027+0000 7f77aaa00640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77ac104bc0 0x7f77ac19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.027+0000 7f77aaa00640 1 -- 172.21.15.42:0/3016077544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac1a17e0 con 0x7f77ac1039d0 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.027+0000 7f77aaa00640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1039d0 0x7f77ac199bc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f779c031440 tx=0x7f779c032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.028+0000 7f77a3400640 1 -- 172.21.15.42:0/3016077544 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c03f030 con 0x7f77ac1039d0 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.028+0000 7f77a3400640 1 -- 172.21.15.42:0/3016077544 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f779c003ce0 con 0x7f77ac1039d0 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.028+0000 7f77a3400640 1 -- 172.21.15.42:0/3016077544 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c035070 con 0x7f77ac1039d0 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.028+0000 7f77b1800640 1 -- 172.21.15.42:0/3016077544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ac1a1a70 con 0x7f77ac1039d0 2024-04-25T00:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.028+0000 7f77b1800640 1 -- 172.21.15.42:0/3016077544 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ac1a1f10 con 0x7f77ac1039d0 2024-04-25T00:39:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.033+0000 7f77b4683740 1 Processor -- start 2024-04-25T00:39:53.034 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.033+0000 7f77a3400640 1 -- 172.21.15.42:0/3016077544 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f779c042030 con 0x7f77ac1039d0 2024-04-25T00:39:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.033+0000 7f77b4683740 1 -- start start 2024-04-25T00:39:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.033+0000 7f77b4683740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e5220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.035 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.033+0000 7f77b4683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e527190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.033+0000 7f77b4683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 0x56278e529c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.033+0000 7f77b4683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56278e484ab0 con 0x56278e527800 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.033+0000 7f77b4683740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56278e42d640 con 0x56278e521ca0 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77b4683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56278e480a70 con 0x56278e522690 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77ab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e5220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77ab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e5220c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:32956/0 (socket says 172.21.15.42:32956) 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77ab400640 1 -- 172.21.15.42:0/171079614 learned_addr learned my addr 172.21.15.42:0/171079614 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77aaa00640 1 --2- 172.21.15.42:0/171079614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e527190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77aa000640 1 --2- 172.21.15.42:0/171079614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 0x56278e529c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77ab400640 1 -- 172.21.15.42:0/171079614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 msgr2=0x56278e527190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77ab400640 1 --2- 172.21.15.42:0/171079614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e527190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.036 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77ab400640 1 -- 172.21.15.42:0/171079614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 msgr2=0x56278e529c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77ab400640 1 --2- 172.21.15.42:0/171079614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 0x56278e529c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77ab400640 1 -- 172.21.15.42:0/171079614 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278e199210 con 0x56278e521ca0 2024-04-25T00:39:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.034+0000 7f77aa000640 1 --2- 172.21.15.42:0/171079614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 0x56278e529c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.035+0000 7f77ab400640 1 --2- 172.21.15.42:0/171079614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e5220c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f77a4007f40 tx=0x7f77a400f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ee6534a0b3645c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.035+0000 7f7797e00640 1 -- 172.21.15.42:0/171079614 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4010900 con 0x56278e521ca0 2024-04-25T00:39:53.038 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.035+0000 7f7797e00640 1 -- 172.21.15.42:0/171079614 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a40183f0 con 0x56278e521ca0 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.035+0000 7f7797e00640 1 -- 172.21.15.42:0/171079614 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4017600 con 0x56278e521ca0 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.036+0000 7f77a3400640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f77900a2cb0 0x7f77900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.036+0000 7f77a3400640 1 -- 172.21.15.42:0/3016077544 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f779c0ef280 con 0x7f77ac1039d0 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.036+0000 7f77b4683740 1 -- 172.21.15.42:0/171079614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 msgr2=0x56278e5220c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.036+0000 7f77b4683740 1 --2- 172.21.15.42:0/171079614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e5220c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f77a4007f40 tx=0x7f77a400f850 comp rx=0 tx=0).stop 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.037+0000 7f77aaa00640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f77900a2cb0 0x7f77900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.037+0000 7f77b4683740 1 -- 172.21.15.42:0/171079614 shutdown_connections 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.037+0000 7f77b4683740 1 --2- 172.21.15.42:0/171079614 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 0x56278e529c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.037+0000 7f77b4683740 1 --2- 172.21.15.42:0/171079614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e527190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.037+0000 7f77b4683740 1 --2- 172.21.15.42:0/171079614 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e5220c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.037+0000 7f77b4683740 1 -- 172.21.15.42:0/171079614 >> 172.21.15.42:0/171079614 conn(0x56278e51d310 msgr2=0x56278e51d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.039 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.037+0000 7f77b4683740 1 -- 172.21.15.42:0/171079614 shutdown_connections 2024-04-25T00:39:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.037+0000 7f77aaa00640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f77900a2cb0 0x7f77900a5170 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ed120 tx=0x7f779c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.038+0000 7f77b4683740 1 -- 172.21.15.42:0/171079614 wait complete. 2024-04-25T00:39:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.038+0000 7f77b4683740 1 -- start start 2024-04-25T00:39:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.038+0000 7f77b4683740 1 Processor -- start 2024-04-25T00:39:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.039+0000 7f77b4683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e521ca0 0x56278e53cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.039+0000 7f77b4683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e53d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.039+0000 7f77b4683740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e527800 0x56278e544440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.039+0000 7f77b4683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56278e53b7c0 con 0x56278e521ca0 2024-04-25T00:39:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.039+0000 7f77b4683740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x56278e440990 con 0x56278e527800 2024-04-25T00:39:53.041 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.039+0000 7f77b4683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56278e53b940 con 0x56278e522690 2024-04-25T00:39:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.039+0000 7f77b4683740 10 client.0 _ll_register_callbacks cb 0x56278e442950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77aa000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e53d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77ab400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e521ca0 0x56278e53cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77aa000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e53d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59458/0 (socket says 172.21.15.42:59458) 2024-04-25T00:39:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77aa000640 1 -- 172.21.15.42:0/122458155 learned_addr learned my addr 172.21.15.42:0/122458155 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:53.042 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77ab400640 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e527800 0x56278e544440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77aa000640 1 -- 172.21.15.42:0/122458155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e527800 msgr2=0x56278e544440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77aa000640 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e527800 0x56278e544440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77aa000640 1 -- 172.21.15.42:0/122458155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e521ca0 msgr2=0x56278e53cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77aa000640 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e521ca0 0x56278e53cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77aa000640 1 -- 172.21.15.42:0/122458155 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278e265620 con 0x56278e522690 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77ab400640 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e521ca0 0x56278e53cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f77aa000640 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e53d400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f779800beb0 tx=0x7f77980117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46c6c525ac587c89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.040+0000 7f7794c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.041+0000 7f7794c00640 1 -- 172.21.15.42:0/122458155 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7798004230 con 0x56278e522690 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.041+0000 7f7794c00640 1 -- 172.21.15.42:0/122458155 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f77980043d0 con 0x56278e522690 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.041+0000 7f77b4683740 1 -- 172.21.15.42:0/122458155 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77a40183f0 con 0x56278e522690 2024-04-25T00:39:53.044 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.041+0000 7f7794c00640 1 -- 172.21.15.42:0/122458155 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f7798022070 con 0x56278e522690 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.041+0000 7f77b4683740 10 client.26764 fetch_fsmap learned FSMap version 45 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.041+0000 7f77b4683740 1 -- 172.21.15.42:0/122458155 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56278e5474f0 con 0x56278e522690 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.042+0000 7f7794c00640 1 -- 172.21.15.42:0/122458155 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a40183f0 con 0x56278e522690 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.042+0000 7f7794c00640 1 -- 172.21.15.42:0/122458155 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f779801b880 con 0x56278e522690 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.042+0000 7f77b4683740 10 client.26764 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.042+0000 7f77b4683740 10 client.26764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.042+0000 7f77b4683740 20 client.26764 populate_metadata read hostname 'smithi042' 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.042+0000 7f77b4683740 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56278e548bc0 0x56278e54afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.042+0000 7f77b4683740 4 client.26764 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.042+0000 7f77b4683740 1 -- 172.21.15.42:0/122458155 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56278e491ff0 con 0x56278e548bc0 2024-04-25T00:39:53.045 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.042+0000 7f77ab400640 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56278e548bc0 0x56278e54afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.043+0000 7f77ab400640 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56278e548bc0 0x56278e54afb0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80888f6c08e0089b server_cookie=1c75c29f7ca23f73 in_seq=0 out_seq=0 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.043+0000 7f7794c00640 10 client.26764 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.044+0000 7f7794c00640 1 -- 172.21.15.42:0/122458155 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56278e491ff0 con 0x56278e548bc0 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.044+0000 7f7794c00640 10 client.26764 handle_command_reply: tid=0 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.044+0000 7f77b4683740 1 client.26764 shutdown 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.044+0000 7f77b4683740 1 -- 172.21.15.42:0/122458155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 msgr2=0x56278e53d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.044+0000 7f77b4683740 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e53d400 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f779800beb0 tx=0x7f77980117b0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.044+0000 7f77b4683740 1 -- 172.21.15.42:0/122458155 shutdown_connections 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.045+0000 7f77b4683740 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56278e548bc0 0x56278e54afb0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.045+0000 7f77b4683740 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e527800 0x56278e544440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.047 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.045+0000 7f77b4683740 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e53d400 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.045+0000 7f77b4683740 1 --2- 172.21.15.42:0/122458155 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e521ca0 0x56278e53cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.045+0000 7f77b4683740 1 -- 172.21.15.42:0/122458155 >> 172.21.15.42:0/122458155 conn(0x56278e51d310 msgr2=0x56278e51c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.045+0000 7f7794c00640 0 client.26764 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.045+0000 7f77b4683740 1 -- 172.21.15.42:0/122458155 shutdown_connections 2024-04-25T00:39:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.045+0000 7f77b4683740 1 -- 172.21.15.42:0/122458155 wait complete. 2024-04-25T00:39:53.048 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.045+0000 7f77b4683740 20 client.26764 trim_cache size 0 max 16384 2024-04-25T00:39:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.071+0000 7f77b4683740 1 Processor -- start 2024-04-25T00:39:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77b4683740 1 -- start start 2024-04-25T00:39:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77b4683740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e51bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77b4683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e544400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77b4683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 0x56278e51c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77b4683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1a4dc0 con 0x56278e527800 2024-04-25T00:39:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77b4683740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1a2690 con 0x56278e521ca0 2024-04-25T00:39:53.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77b4683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac071d70 con 0x56278e522690 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77ab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e51bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77ab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e51bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:32984/0 (socket says 172.21.15.42:32984) 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77ab400640 1 -- 172.21.15.42:0/232209768 learned_addr learned my addr 172.21.15.42:0/232209768 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77aa000640 1 --2- 172.21.15.42:0/232209768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e544400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.072+0000 7f77ab400640 1 --2- 172.21.15.42:0/232209768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 0x56278e51c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.073+0000 7f77ab400640 1 -- 172.21.15.42:0/232209768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 msgr2=0x56278e544400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.073+0000 7f77ab400640 1 --2- 172.21.15.42:0/232209768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e544400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.073+0000 7f77ab400640 1 -- 172.21.15.42:0/232209768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 msgr2=0x56278e51c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.073+0000 7f77ab400640 1 --2- 172.21.15.42:0/232209768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 0x56278e51c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.073+0000 7f77ab400640 1 -- 172.21.15.42:0/232209768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56278e523360 con 0x56278e521ca0 2024-04-25T00:39:53.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.073+0000 7f77ab400640 1 --2- 172.21.15.42:0/232209768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e51bbf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f77a4007fd0 tx=0x7f77a4018c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af887ad49451002b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.073+0000 7f7797400640 1 -- 172.21.15.42:0/232209768 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4032030 con 0x56278e521ca0 2024-04-25T00:39:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.073+0000 7f7797400640 1 -- 172.21.15.42:0/232209768 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a4018d30 con 0x56278e521ca0 2024-04-25T00:39:53.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.073+0000 7f7797400640 1 -- 172.21.15.42:0/232209768 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4010bc0 con 0x56278e521ca0 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.074+0000 7f77b4683740 1 -- 172.21.15.42:0/232209768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 msgr2=0x56278e51bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.074+0000 7f77b4683740 1 --2- 172.21.15.42:0/232209768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e51bbf0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f77a4007fd0 tx=0x7f77a4018c00 comp rx=0 tx=0).stop 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.074+0000 7f77b4683740 1 -- 172.21.15.42:0/232209768 shutdown_connections 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.074+0000 7f77b4683740 1 --2- 172.21.15.42:0/232209768 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e527800 0x56278e51c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.074+0000 7f77b4683740 1 --2- 172.21.15.42:0/232209768 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e522690 0x56278e544400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.074+0000 7f77b4683740 1 --2- 172.21.15.42:0/232209768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e51bbf0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.074+0000 7f77b4683740 1 -- 172.21.15.42:0/232209768 >> 172.21.15.42:0/232209768 conn(0x56278e51d310 msgr2=0x56278e523870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.075+0000 7f77b4683740 1 -- 172.21.15.42:0/232209768 shutdown_connections 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.075+0000 7f77b4683740 1 -- 172.21.15.42:0/232209768 wait complete. 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.075+0000 7f77b4683740 1 -- start start 2024-04-25T00:39:53.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.075+0000 7f77b4683740 1 Processor -- start 2024-04-25T00:39:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77b4683740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e541930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77b4683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e522690 0x56278e541e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77b4683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e527800 0x56278e542e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77b4683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac0723e0 con 0x56278e522690 2024-04-25T00:39:53.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77b4683740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac1a20e0 con 0x56278e521ca0 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77b4683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac072ab0 con 0x56278e527800 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77b4683740 10 client.0 _ll_register_callbacks cb 0x7f779800b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77ab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e541930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77ab400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e541930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:32998/0 (socket says 172.21.15.42:32998) 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77ab400640 1 -- 172.21.15.42:0/1693177705 learned_addr learned my addr 172.21.15.42:0/1693177705 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77ab400640 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e527800 0x56278e542e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.076+0000 7f77aa000640 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e522690 0x56278e541e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.077+0000 7f77ab400640 1 -- 172.21.15.42:0/1693177705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e527800 msgr2=0x56278e542e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.077+0000 7f77ab400640 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e527800 0x56278e542e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.077+0000 7f77ab400640 1 -- 172.21.15.42:0/1693177705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e522690 msgr2=0x56278e541e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.077+0000 7f77ab400640 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e522690 0x56278e541e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.077+0000 7f77ab400640 1 -- 172.21.15.42:0/1693177705 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56278e527550 con 0x56278e521ca0 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.077+0000 7f77ab400640 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e541930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f77a4007cd0 tx=0x7f77a400f850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c465b844d2b5ab3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:39:53.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.077+0000 7f7794c00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:39:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.077+0000 7f7794c00640 1 -- 172.21.15.42:0/1693177705 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a4032040 con 0x56278e521ca0 2024-04-25T00:39:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f7794c00640 1 -- 172.21.15.42:0/1693177705 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f77a4010940 con 0x56278e521ca0 2024-04-25T00:39:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f77b4683740 1 -- 172.21.15.42:0/1693177705 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77a4018d30 con 0x56278e521ca0 2024-04-25T00:39:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f7794c00640 1 -- 172.21.15.42:0/1693177705 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f77a4010b40 con 0x56278e521ca0 2024-04-25T00:39:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f77b4683740 10 client.26819 fetch_fsmap learned FSMap version 45 2024-04-25T00:39:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f77b4683740 1 -- 172.21.15.42:0/1693177705 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x56278e5445b0 con 0x56278e521ca0 2024-04-25T00:39:53.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f7794c00640 1 -- 172.21.15.42:0/1693177705 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a4018d30 con 0x56278e521ca0 2024-04-25T00:39:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f7794c00640 1 -- 172.21.15.42:0/1693177705 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f77a4026440 con 0x56278e521ca0 2024-04-25T00:39:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f77b4683740 10 client.26819 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:39:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f77b4683740 10 client.26819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:39:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f77b4683740 20 client.26819 populate_metadata read hostname 'smithi042' 2024-04-25T00:39:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.078+0000 7f77b4683740 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56278e5af470 0x56278e5b1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:39:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.079+0000 7f77b4683740 4 client.26819 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:39:53.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.079+0000 7f77b4683740 1 -- 172.21.15.42:0/1693177705 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56278e48c340 con 0x56278e5af470 2024-04-25T00:39:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.079+0000 7f77aa000640 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56278e5af470 0x56278e5b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:39:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.079+0000 7f77aa000640 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56278e5af470 0x56278e5b1860 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787b24a9d75f269 server_cookie=f476a25b5f638dd8 in_seq=0 out_seq=0 2024-04-25T00:39:53.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.079+0000 7f7794c00640 10 client.26819 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.403+0000 7f7794c00640 1 -- 172.21.15.42:0/1693177705 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56278e48c340 con 0x56278e5af470 2024-04-25T00:39:53.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.403+0000 7f7794c00640 10 client.26819 handle_command_reply: tid=0 2024-04-25T00:39:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.403+0000 7f77b4683740 1 client.26819 shutdown 2024-04-25T00:39:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f77b4683740 1 -- 172.21.15.42:0/1693177705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 msgr2=0x56278e541930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f77b4683740 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e541930 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f77a4007cd0 tx=0x7f77a400f850 comp rx=0 tx=0).stop 2024-04-25T00:39:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f77b4683740 1 -- 172.21.15.42:0/1693177705 shutdown_connections 2024-04-25T00:39:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f77b4683740 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x56278e5af470 0x56278e5b1860 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f77b4683740 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56278e527800 0x56278e542e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f77b4683740 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x56278e522690 0x56278e541e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f77b4683740 1 --2- 172.21.15.42:0/1693177705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56278e521ca0 0x56278e541930 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f77b4683740 1 -- 172.21.15.42:0/1693177705 >> 172.21.15.42:0/1693177705 conn(0x56278e51d310 msgr2=0x56278e51bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f7794c00640 0 client.26819 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:39:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f77b4683740 1 -- 172.21.15.42:0/1693177705 shutdown_connections 2024-04-25T00:39:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.404+0000 7f77b4683740 1 -- 172.21.15.42:0/1693177705 wait complete. 2024-04-25T00:39:53.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.405+0000 7f77b4683740 20 client.26819 trim_cache size 0 max 16384 2024-04-25T00:39:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:39:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:39:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:39:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:39:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.406+0000 7f77b1800640 1 -- 172.21.15.42:0/3016077544 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f77900a2cb0 msgr2=0x7f77900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.406+0000 7f77b1800640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f77900a2cb0 0x7f77900a5170 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ed120 tx=0x7f779c009260 comp rx=0 tx=0).stop 2024-04-25T00:39:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.406+0000 7f77b1800640 1 -- 172.21.15.42:0/3016077544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1039d0 msgr2=0x7f77ac199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:39:53.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.406+0000 7f77b1800640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1039d0 0x7f77ac199bc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f779c031440 tx=0x7f779c032e20 comp rx=0 tx=0).stop 2024-04-25T00:39:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.408+0000 7f77b1800640 1 -- 172.21.15.42:0/3016077544 shutdown_connections 2024-04-25T00:39:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.408+0000 7f77b1800640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f77900a2cb0 0x7f77900a5170 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.408+0000 7f77b1800640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f77ac105580 0x7f77ac1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.408+0000 7f77b1800640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f77ac104bc0 0x7f77ac19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.408+0000 7f77b1800640 1 --2- 172.21.15.42:0/3016077544 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f77ac1039d0 0x7f77ac199bc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:39:53.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.408+0000 7f77b1800640 1 -- 172.21.15.42:0/3016077544 >> 172.21.15.42:0/3016077544 conn(0x7f77ac0fef60 msgr2=0x7f77ac107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:39:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.408+0000 7f77b1800640 1 -- 172.21.15.42:0/3016077544 shutdown_connections 2024-04-25T00:39:53.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:39:53.409+0000 7f77b1800640 1 -- 172.21.15.42:0/3016077544 wait complete. 2024-04-25T00:39:53.792 INFO:tasks.workunit.client.0.smithi042.stdout: 10 290818 33.72 MB/sec execute 343 sec latency 243.764 ms 2024-04-25T00:39:53.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:53 smithi042 ceph-mon[28130]: pgmap v1024: 97 pgs: 97 active+clean; 659 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.0 MiB/s wr, 520 op/s 2024-04-25T00:39:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:53 smithi136 ceph-mon[32923]: pgmap v1024: 97 pgs: 97 active+clean; 659 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.0 MiB/s wr, 520 op/s 2024-04-25T00:39:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:53 smithi106 ceph-mon[32510]: pgmap v1024: 97 pgs: 97 active+clean; 659 MiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 6.0 MiB/s wr, 520 op/s 2024-04-25T00:39:54.792 INFO:tasks.workunit.client.0.smithi042.stdout: 10 291342 33.70 MB/sec execute 344 sec latency 88.279 ms 2024-04-25T00:39:55.793 INFO:tasks.workunit.client.0.smithi042.stdout: 10 291981 33.69 MB/sec execute 345 sec latency 56.341 ms 2024-04-25T00:39:55.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:55 smithi042 ceph-mon[28130]: pgmap v1025: 97 pgs: 97 active+clean; 667 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.0 MiB/s wr, 473 op/s 2024-04-25T00:39:55.889 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:55 smithi106 ceph-mon[32510]: pgmap v1025: 97 pgs: 97 active+clean; 667 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.0 MiB/s wr, 473 op/s 2024-04-25T00:39:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:55 smithi136 ceph-mon[32923]: pgmap v1025: 97 pgs: 97 active+clean; 667 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 4.0 MiB/s wr, 473 op/s 2024-04-25T00:39:56.793 INFO:tasks.workunit.client.0.smithi042.stdout: 10 292702 33.69 MB/sec execute 346 sec latency 58.068 ms 2024-04-25T00:39:57.793 INFO:tasks.workunit.client.0.smithi042.stdout: 10 293488 33.72 MB/sec execute 347 sec latency 70.281 ms 2024-04-25T00:39:57.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:57 smithi042 ceph-mon[28130]: pgmap v1026: 97 pgs: 97 active+clean; 669 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.0 MiB/s wr, 546 op/s 2024-04-25T00:39:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:57 smithi136 ceph-mon[32923]: pgmap v1026: 97 pgs: 97 active+clean; 669 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.0 MiB/s wr, 546 op/s 2024-04-25T00:39:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:57 smithi106 ceph-mon[32510]: pgmap v1026: 97 pgs: 97 active+clean; 669 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.0 MiB/s wr, 546 op/s 2024-04-25T00:39:58.793 INFO:tasks.workunit.client.0.smithi042.stdout: 10 294223 33.76 MB/sec execute 348 sec latency 48.807 ms 2024-04-25T00:39:59.794 INFO:tasks.workunit.client.0.smithi042.stdout: 10 294736 33.75 MB/sec execute 349 sec latency 77.905 ms 2024-04-25T00:39:59.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:39:59 smithi106 ceph-mon[32510]: pgmap v1027: 97 pgs: 97 active+clean; 671 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.6 MiB/s wr, 520 op/s 2024-04-25T00:39:59.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:39:59 smithi042 ceph-mon[28130]: pgmap v1027: 97 pgs: 97 active+clean; 671 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.6 MiB/s wr, 520 op/s 2024-04-25T00:40:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:39:59 smithi136 ceph-mon[32923]: pgmap v1027: 97 pgs: 97 active+clean; 671 MiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.6 MiB/s wr, 520 op/s 2024-04-25T00:40:00.793 INFO:tasks.workunit.client.0.smithi042.stdout: 10 295268 33.73 MB/sec execute 350 sec latency 76.756 ms 2024-04-25T00:40:00.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:00 smithi042 ceph-mon[28130]: overall HEALTH_OK 2024-04-25T00:40:01.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:00 smithi136 ceph-mon[32923]: overall HEALTH_OK 2024-04-25T00:40:01.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:00 smithi106 ceph-mon[32510]: overall HEALTH_OK 2024-04-25T00:40:01.794 INFO:tasks.workunit.client.0.smithi042.stdout: 10 295890 33.70 MB/sec execute 351 sec latency 64.827 ms 2024-04-25T00:40:01.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:01 smithi042 ceph-mon[28130]: pgmap v1028: 97 pgs: 97 active+clean; 676 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.7 MiB/s wr, 534 op/s 2024-04-25T00:40:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:01 smithi136 ceph-mon[32923]: pgmap v1028: 97 pgs: 97 active+clean; 676 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.7 MiB/s wr, 534 op/s 2024-04-25T00:40:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:01 smithi106 ceph-mon[32510]: pgmap v1028: 97 pgs: 97 active+clean; 676 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.7 MiB/s wr, 534 op/s 2024-04-25T00:40:02.794 INFO:tasks.workunit.client.0.smithi042.stdout: 10 296500 33.69 MB/sec execute 352 sec latency 151.427 ms 2024-04-25T00:40:03.614 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:03 smithi042 ceph-mon[28130]: pgmap v1029: 97 pgs: 97 active+clean; 685 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 576 op/s 2024-04-25T00:40:03.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 297168 33.72 MB/sec execute 353 sec latency 330.958 ms 2024-04-25T00:40:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:03 smithi136 ceph-mon[32923]: pgmap v1029: 97 pgs: 97 active+clean; 685 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 576 op/s 2024-04-25T00:40:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:03 smithi106 ceph-mon[32510]: pgmap v1029: 97 pgs: 97 active+clean; 685 MiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.1 MiB/s wr, 576 op/s 2024-04-25T00:40:04.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 297282 33.65 MB/sec execute 354 sec latency 880.058 ms 2024-04-25T00:40:05.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:05 smithi136 ceph-mon[32923]: pgmap v1030: 97 pgs: 97 active+clean; 688 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.9 MiB/s wr, 573 op/s 2024-04-25T00:40:05.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 297336 33.56 MB/sec execute 355 sec latency 1395.785 ms 2024-04-25T00:40:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:05 smithi106 ceph-mon[32510]: pgmap v1030: 97 pgs: 97 active+clean; 688 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.9 MiB/s wr, 573 op/s 2024-04-25T00:40:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:05 smithi042 ceph-mon[28130]: pgmap v1030: 97 pgs: 97 active+clean; 688 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.9 MiB/s wr, 573 op/s 2024-04-25T00:40:06.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 297748 33.54 MB/sec execute 356 sec latency 172.292 ms 2024-04-25T00:40:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.175+0000 7fedef000640 1 -- 172.21.15.42:0/842823437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fede8105570 msgr2=0x7fede810be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.175+0000 7fedef000640 1 --2- 172.21.15.42:0/842823437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fede8105570 0x7fede810be00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fede0009ca0 tx=0x7fede002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.177 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.176+0000 7fedef000640 1 -- 172.21.15.42:0/842823437 shutdown_connections 2024-04-25T00:40:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.176+0000 7fedef000640 1 --2- 172.21.15.42:0/842823437 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fede8105570 0x7fede810be00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.176+0000 7fedef000640 1 --2- 172.21.15.42:0/842823437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede8104bb0 0x7fede8105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.176+0000 7fedef000640 1 --2- 172.21.15.42:0/842823437 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede81039c0 0x7fede8103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.178 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.176+0000 7fedef000640 1 -- 172.21.15.42:0/842823437 >> 172.21.15.42:0/842823437 conn(0x7fede80fef30 msgr2=0x7fede8101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.176+0000 7fedef000640 1 -- 172.21.15.42:0/842823437 shutdown_connections 2024-04-25T00:40:07.179 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.177+0000 7fedef000640 1 -- 172.21.15.42:0/842823437 wait complete. 2024-04-25T00:40:07.180 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.179+0000 7fedef000640 1 Processor -- start 2024-04-25T00:40:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.180+0000 7fedef000640 1 -- start start 2024-04-25T00:40:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.181+0000 7fedef000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede81039c0 0x7fede8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.181+0000 7fedef000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fede8104bb0 0x7fede819a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.181+0000 7fedef000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede8105570 0x7fede81a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.181+0000 7fedef000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fede810f670 con 0x7fede81039c0 2024-04-25T00:40:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.181+0000 7fedef000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fede810f4f0 con 0x7fede8105570 2024-04-25T00:40:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.181+0000 7fedef000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fede810f7f0 con 0x7fede8104bb0 2024-04-25T00:40:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7feded000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede8105570 0x7fede81a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7feded000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede8105570 0x7fede81a1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:50832/0 (socket says 172.21.15.42:50832) 2024-04-25T00:40:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7feded000640 1 -- 172.21.15.42:0/3008827153 learned_addr learned my addr 172.21.15.42:0/3008827153 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7fede7e00640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede81039c0 0x7fede8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7feded000640 1 -- 172.21.15.42:0/3008827153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fede8104bb0 msgr2=0x7fede819a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7fede7400640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fede8104bb0 0x7fede819a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7feded000640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fede8104bb0 0x7fede819a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7feded000640 1 -- 172.21.15.42:0/3008827153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede81039c0 msgr2=0x7fede8199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7feded000640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede81039c0 0x7fede8199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7feded000640 1 -- 172.21.15.42:0/3008827153 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fede81a1700 con 0x7fede8105570 2024-04-25T00:40:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.182+0000 7fede7400640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fede8104bb0 0x7fede819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.183+0000 7feded000640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede8105570 0x7fede81a1090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fede0030930 tx=0x7fede00317c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.183+0000 7fede4c00640 1 -- 172.21.15.42:0/3008827153 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede0030c00 con 0x7fede8105570 2024-04-25T00:40:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.183+0000 7fede4c00640 1 -- 172.21.15.42:0/3008827153 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fede0003e50 con 0x7fede8105570 2024-04-25T00:40:07.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.183+0000 7fede4c00640 1 -- 172.21.15.42:0/3008827153 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede003b410 con 0x7fede8105570 2024-04-25T00:40:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.183+0000 7fedef000640 1 -- 172.21.15.42:0/3008827153 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fede81a1990 con 0x7fede8105570 2024-04-25T00:40:07.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.183+0000 7fedef000640 1 -- 172.21.15.42:0/3008827153 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fede81a1e90 con 0x7fede8105570 2024-04-25T00:40:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.197+0000 7fedf1e3c740 1 Processor -- start 2024-04-25T00:40:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.198+0000 7fede4c00640 1 -- 172.21.15.42:0/3008827153 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fede0004040 con 0x7fede8105570 2024-04-25T00:40:07.202 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.200+0000 7fedf1e3c740 1 -- start start 2024-04-25T00:40:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fede4c00640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fedcc0a2d30 0x7fedcc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 0x55a54ba308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 0x55a54ba359c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 0x55a54ba384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fedf1e3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a54b98aab0 con 0x55a54ba304d0 2024-04-25T00:40:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fedf1e3c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a54b933640 con 0x55a54ba36030 2024-04-25T00:40:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fedf1e3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a54b986a70 con 0x55a54ba30ec0 2024-04-25T00:40:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fede4c00640 1 -- 172.21.15.42:0/3008827153 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fede00f0690 con 0x7fede8105570 2024-04-25T00:40:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fede7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 0x55a54ba359c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fede7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 0x55a54ba359c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57456/0 (socket says 172.21.15.42:57456) 2024-04-25T00:40:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.201+0000 7fede7400640 1 -- 172.21.15.42:0/3987638265 learned_addr learned my addr 172.21.15.42:0/3987638265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.202+0000 7fede7400640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fedcc0a2d30 0x7fedcc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.202+0000 7feded000640 1 --2- 172.21.15.42:0/3987638265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 0x55a54ba384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.202+0000 7fede7400640 1 -- 172.21.15.42:0/3987638265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 msgr2=0x55a54ba384c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.202+0000 7fede7400640 1 --2- 172.21.15.42:0/3987638265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 0x55a54ba384c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.202+0000 7fede7400640 1 -- 172.21.15.42:0/3987638265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 msgr2=0x55a54ba308f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.202+0000 7fede7400640 1 --2- 172.21.15.42:0/3987638265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 0x55a54ba308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.202+0000 7fede7400640 1 -- 172.21.15.42:0/3987638265 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54b69f210 con 0x55a54ba30ec0 2024-04-25T00:40:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.203+0000 7fede7400640 1 --2- 172.21.15.42:0/3987638265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 0x55a54ba359c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fedd4012b30 tx=0x7fedd400f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac80b2493871a23c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.203+0000 7fede7400640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fedcc0a2d30 0x7fedcc0a51f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fedd400f860 tx=0x7fedd400c000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.203+0000 7feddd600640 1 -- 172.21.15.42:0/3987638265 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd4022070 con 0x55a54ba30ec0 2024-04-25T00:40:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.203+0000 7feddd600640 1 -- 172.21.15.42:0/3987638265 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd400d970 con 0x55a54ba30ec0 2024-04-25T00:40:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.203+0000 7feddd600640 1 -- 172.21.15.42:0/3987638265 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd401edd0 con 0x55a54ba30ec0 2024-04-25T00:40:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.205+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3987638265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 msgr2=0x55a54ba359c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.205+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3987638265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 0x55a54ba359c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fedd4012b30 tx=0x7fedd400f360 comp rx=0 tx=0).stop 2024-04-25T00:40:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.206+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3987638265 shutdown_connections 2024-04-25T00:40:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.206+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3987638265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 0x55a54ba384c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.206+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3987638265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 0x55a54ba359c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.206+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3987638265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 0x55a54ba308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.206+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3987638265 >> 172.21.15.42:0/3987638265 conn(0x55a54ba2bb40 msgr2=0x55a54ba2bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.207+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3987638265 shutdown_connections 2024-04-25T00:40:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.207+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3987638265 wait complete. 2024-04-25T00:40:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.208+0000 7fedf1e3c740 1 -- start start 2024-04-25T00:40:07.213 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.208+0000 7fedf1e3c740 1 Processor -- start 2024-04-25T00:40:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.212+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba304d0 0x55a54ba3a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.214 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.213+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba30ec0 0x55a54ba3acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.213+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 0x55a54ba41cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.213+0000 7fedf1e3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a54ba38f80 con 0x55a54ba30ec0 2024-04-25T00:40:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.213+0000 7fedf1e3c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a54b946990 con 0x55a54ba36030 2024-04-25T00:40:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.213+0000 7fedf1e3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a54ba39100 con 0x55a54ba304d0 2024-04-25T00:40:07.215 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.213+0000 7fede7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba304d0 0x55a54ba3a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.213+0000 7fede7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba304d0 0x55a54ba3a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57470/0 (socket says 172.21.15.42:57470) 2024-04-25T00:40:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.213+0000 7feded000640 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 0x55a54ba41cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.216 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.213+0000 7fede7e00640 1 -- 172.21.15.42:0/1622965766 learned_addr learned my addr 172.21.15.42:0/1622965766 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.214+0000 7feded000640 1 -- 172.21.15.42:0/1622965766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba304d0 msgr2=0x55a54ba3a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.214+0000 7feded000640 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba304d0 0x55a54ba3a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.214+0000 7feded000640 1 -- 172.21.15.42:0/1622965766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba30ec0 msgr2=0x55a54ba3acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.214+0000 7feded000640 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba30ec0 0x55a54ba3acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.214+0000 7feded000640 1 -- 172.21.15.42:0/1622965766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54b76b620 con 0x55a54ba36030 2024-04-25T00:40:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.214+0000 7feded000640 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 0x55a54ba41cf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fede00ee160 tx=0x7fede00095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ae37245c93a4e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.214+0000 7fedf1e3c740 10 client.0 _ll_register_callbacks cb 0x55a54b948950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.215+0000 7fedf1e3c740 1 -- 172.21.15.42:0/1622965766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a54b997ff0 con 0x55a54ba36030 2024-04-25T00:40:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.214+0000 7fedd2000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:40:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.215+0000 7fedd2000640 1 -- 172.21.15.42:0/1622965766 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fede00bd2c0 con 0x55a54ba36030 2024-04-25T00:40:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.215+0000 7fedd2000640 1 -- 172.21.15.42:0/1622965766 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fede00c20d0 con 0x55a54ba36030 2024-04-25T00:40:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.215+0000 7fedd2000640 1 -- 172.21.15.42:0/1622965766 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fede00bd2c0 con 0x55a54ba36030 2024-04-25T00:40:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.216+0000 7fedf1e3c740 10 client.26828 fetch_fsmap learned FSMap version 45 2024-04-25T00:40:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.216+0000 7fedf1e3c740 1 -- 172.21.15.42:0/1622965766 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({fsmap=45}) -- 0x55a54ba390a0 con 0x55a54ba36030 2024-04-25T00:40:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.216+0000 7fedd2000640 1 -- 172.21.15.42:0/1622965766 <== mon.2 v2:172.21.15.106:3300/0 4 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fede00b9930 con 0x55a54ba36030 2024-04-25T00:40:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.216+0000 7fedf1e3c740 10 client.26828 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:40:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.216+0000 7fedf1e3c740 10 client.26828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:40:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.216+0000 7fedf1e3c740 20 client.26828 populate_metadata read hostname 'smithi042' 2024-04-25T00:40:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.216+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a54ba46190 0x55a54ba48620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.216+0000 7fedf1e3c740 4 client.26828 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:40:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.216+0000 7fedf1e3c740 1 -- 172.21.15.42:0/1622965766 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a54b992340 con 0x55a54ba46190 2024-04-25T00:40:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.217+0000 7fede7e00640 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a54ba46190 0x55a54ba48620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.218+0000 7fede7e00640 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a54ba46190 0x55a54ba48620 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c30df6ea876c1a8 server_cookie=d9b795b660b11a11 in_seq=0 out_seq=0 2024-04-25T00:40:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.218+0000 7fedd2000640 10 client.26828 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.219+0000 7fedd2000640 1 -- 172.21.15.42:0/1622965766 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a54b992340 con 0x55a54ba46190 2024-04-25T00:40:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.219+0000 7fedd2000640 10 client.26828 handle_command_reply: tid=0 2024-04-25T00:40:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.220+0000 7fedf1e3c740 1 client.26828 shutdown 2024-04-25T00:40:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.220+0000 7fedf1e3c740 1 -- 172.21.15.42:0/1622965766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 msgr2=0x55a54ba41cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.220+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 0x55a54ba41cf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fede00ee160 tx=0x7fede00095c0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.221+0000 7fedf1e3c740 1 -- 172.21.15.42:0/1622965766 shutdown_connections 2024-04-25T00:40:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.221+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a54ba46190 0x55a54ba48620 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.221+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba36030 0x55a54ba41cf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.221+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba30ec0 0x55a54ba3acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.221+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/1622965766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba304d0 0x55a54ba3a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.221+0000 7fedf1e3c740 1 -- 172.21.15.42:0/1622965766 >> 172.21.15.42:0/1622965766 conn(0x55a54ba2bb40 msgr2=0x55a54ba22400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.221+0000 7fedd2000640 0 client.26828 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:07.225 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.222+0000 7fedf1e3c740 1 -- 172.21.15.42:0/1622965766 shutdown_connections 2024-04-25T00:40:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.223+0000 7fedf1e3c740 1 -- 172.21.15.42:0/1622965766 wait complete. 2024-04-25T00:40:07.226 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.223+0000 7fedf1e3c740 20 client.26828 trim_cache size 0 max 16384 2024-04-25T00:40:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.266+0000 7fedf1e3c740 1 Processor -- start 2024-04-25T00:40:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.266+0000 7fedf1e3c740 1 -- start start 2024-04-25T00:40:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba2bb40 0x55a54ba21b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 0x55a54ba220b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba30ec0 0x55a54ba225f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fedf1e3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fede81a90f0 con 0x55a54ba304d0 2024-04-25T00:40:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fedf1e3c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fede81a69c0 con 0x55a54ba30ec0 2024-04-25T00:40:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fedf1e3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fede81a47b0 con 0x55a54ba2bb40 2024-04-25T00:40:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fede7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba2bb40 0x55a54ba21b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fede7e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba2bb40 0x55a54ba21b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:57476/0 (socket says 172.21.15.42:57476) 2024-04-25T00:40:07.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fede7e00640 1 -- 172.21.15.42:0/3731034365 learned_addr learned my addr 172.21.15.42:0/3731034365 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fede7e00640 1 -- 172.21.15.42:0/3731034365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba30ec0 msgr2=0x55a54ba225f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:40:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fede7e00640 1 --2- 172.21.15.42:0/3731034365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba30ec0 0x55a54ba225f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fede7e00640 1 -- 172.21.15.42:0/3731034365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 msgr2=0x55a54ba220b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fede7e00640 1 --2- 172.21.15.42:0/3731034365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 0x55a54ba220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.267+0000 7fede7e00640 1 -- 172.21.15.42:0/3731034365 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54ba48ca0 con 0x55a54ba2bb40 2024-04-25T00:40:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.268+0000 7fede7e00640 1 --2- 172.21.15.42:0/3731034365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba2bb40 0x55a54ba21b70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a54ba48ea0 tx=0x7fedd8005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d93cb56f5c67f00d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.268+0000 7feddcc00640 1 -- 172.21.15.42:0/3731034365 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8013d70 con 0x55a54ba2bb40 2024-04-25T00:40:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.268+0000 7feddcc00640 1 -- 172.21.15.42:0/3731034365 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd801b890 con 0x55a54ba2bb40 2024-04-25T00:40:07.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.268+0000 7feddcc00640 1 -- 172.21.15.42:0/3731034365 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd8004ad0 con 0x55a54ba2bb40 2024-04-25T00:40:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.269+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3731034365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba2bb40 msgr2=0x55a54ba21b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.269+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3731034365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba2bb40 0x55a54ba21b70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x55a54ba48ea0 tx=0x7fedd8005c90 comp rx=0 tx=0).stop 2024-04-25T00:40:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.270+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3731034365 shutdown_connections 2024-04-25T00:40:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.270+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3731034365 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba30ec0 0x55a54ba225f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.270+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3731034365 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 0x55a54ba220b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.270+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3731034365 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba2bb40 0x55a54ba21b70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.270+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3731034365 >> 172.21.15.42:0/3731034365 conn(0x55a54ba35b00 msgr2=0x55a54ba32140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:07.271 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.270+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3731034365 shutdown_connections 2024-04-25T00:40:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.270+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3731034365 wait complete. 2024-04-25T00:40:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.270+0000 7fedf1e3c740 1 -- start start 2024-04-25T00:40:07.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.271+0000 7fedf1e3c740 1 Processor -- start 2024-04-25T00:40:07.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.273+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba2bb40 0x55a54ba40a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.273+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 0x55a54ba40f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.273+0000 7fede7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba2bb40 0x55a54ba40a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.273+0000 7fede7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba2bb40 0x55a54ba40a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:50872/0 (socket says 172.21.15.42:50872) 2024-04-25T00:40:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.273+0000 7fedf1e3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 0x55a54ba3f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.274+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fede81a4e80 con 0x55a54ba304d0 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.274+0000 7fede7e00640 1 -- 172.21.15.42:0/3144227268 learned_addr learned my addr 172.21.15.42:0/3144227268 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.274+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fede81a62f0 con 0x55a54ba2bb40 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.274+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fede81a5550 con 0x55a54ba30ec0 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.274+0000 7fedf1e3c740 10 client.0 _ll_register_callbacks cb 0x7fede00dd0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.274+0000 7fede7e00640 1 -- 172.21.15.42:0/3144227268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 msgr2=0x55a54ba3f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.274+0000 7fede7e00640 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 0x55a54ba3f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.274+0000 7fede7e00640 1 -- 172.21.15.42:0/3144227268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 msgr2=0x55a54ba40f40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.275+0000 7fede7e00640 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 0x55a54ba40f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.275+0000 7fede7e00640 1 -- 172.21.15.42:0/3144227268 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54ba3f670 con 0x55a54ba2bb40 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.276+0000 7fede7e00640 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba2bb40 0x55a54ba40a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fedd800b530 tx=0x7fedd8014c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=583ee1da2fc179c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.276+0000 7fedd2000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:40:07.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.276+0000 7fedd2000640 1 -- 172.21.15.42:0/3144227268 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedd801b5b0 con 0x55a54ba2bb40 2024-04-25T00:40:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.276+0000 7fedd2000640 1 -- 172.21.15.42:0/3144227268 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fedd8005020 con 0x55a54ba2bb40 2024-04-25T00:40:07.279 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.276+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedd801b890 con 0x55a54ba2bb40 2024-04-25T00:40:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.277+0000 7fedd2000640 1 -- 172.21.15.42:0/3144227268 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fedd8005220 con 0x55a54ba2bb40 2024-04-25T00:40:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.277+0000 7fedf1e3c740 10 client.26831 fetch_fsmap learned FSMap version 45 2024-04-25T00:40:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.277+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a54baad090 con 0x55a54ba2bb40 2024-04-25T00:40:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.278+0000 7fedd2000640 1 -- 172.21.15.42:0/3144227268 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fedd801b890 con 0x55a54ba2bb40 2024-04-25T00:40:07.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.278+0000 7fedd2000640 1 -- 172.21.15.42:0/3144227268 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fedd8024790 con 0x55a54ba2bb40 2024-04-25T00:40:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.279+0000 7fedf1e3c740 10 client.26831 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:40:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.279+0000 7fedf1e3c740 10 client.26831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:40:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.279+0000 7fedf1e3c740 20 client.26831 populate_metadata read hostname 'smithi042' 2024-04-25T00:40:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.279+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a54ba36030 0x55a54baafa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.279+0000 7fedf1e3c740 4 client.26831 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:40:07.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.279+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a54b9107a0 con 0x55a54ba36030 2024-04-25T00:40:07.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.281+0000 7fede7400640 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a54ba36030 0x55a54baafa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.281+0000 7fede7400640 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a54ba36030 0x55a54baafa50 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845edefacf89fde server_cookie=62c280b21415f700 in_seq=0 out_seq=0 2024-04-25T00:40:07.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.281+0000 7fedd2000640 10 client.26831 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:07.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.671+0000 7fedd2000640 1 -- 172.21.15.42:0/3144227268 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a54b9107a0 con 0x55a54ba36030 2024-04-25T00:40:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.671+0000 7fedd2000640 10 client.26831 handle_command_reply: tid=0 2024-04-25T00:40:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.671+0000 7fedf1e3c740 1 client.26831 shutdown 2024-04-25T00:40:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.672+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba2bb40 msgr2=0x55a54ba40a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.672+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba2bb40 0x55a54ba40a00 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fedd800b530 tx=0x7fedd8014c70 comp rx=0 tx=0).stop 2024-04-25T00:40:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.672+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 shutdown_connections 2024-04-25T00:40:07.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.672+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a54ba36030 0x55a54baafa50 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.672+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a54ba30ec0 0x55a54ba3f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.672+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a54ba304d0 0x55a54ba40f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.672+0000 7fedf1e3c740 1 --2- 172.21.15.42:0/3144227268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a54ba2bb40 0x55a54ba40a00 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.672+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 >> 172.21.15.42:0/3144227268 conn(0x55a54ba35b00 msgr2=0x55a54ba2e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:07.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.673+0000 7fedd2000640 0 client.26831 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.673+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 shutdown_connections 2024-04-25T00:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.673+0000 7fedf1e3c740 1 -- 172.21.15.42:0/3144227268 wait complete. 2024-04-25T00:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.673+0000 7fedf1e3c740 20 client.26831 trim_cache size 0 max 16384 2024-04-25T00:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:40:07.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.674+0000 7fedef000640 1 -- 172.21.15.42:0/3008827153 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fedcc0a2d30 msgr2=0x7fedcc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.674+0000 7fedef000640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fedcc0a2d30 0x7fedcc0a51f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fedd400f860 tx=0x7fedd400c000 comp rx=0 tx=0).stop 2024-04-25T00:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.674+0000 7fedef000640 1 -- 172.21.15.42:0/3008827153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede8105570 msgr2=0x7fede81a1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.674+0000 7fedef000640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede8105570 0x7fede81a1090 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fede0030930 tx=0x7fede00317c0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.675+0000 7fedef000640 1 -- 172.21.15.42:0/3008827153 shutdown_connections 2024-04-25T00:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.675+0000 7fedef000640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fedcc0a2d30 0x7fedcc0a51f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.675+0000 7fedef000640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fede8105570 0x7fede81a1090 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.675+0000 7fedef000640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fede8104bb0 0x7fede819a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.675+0000 7fedef000640 1 --2- 172.21.15.42:0/3008827153 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fede81039c0 0x7fede8199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.675+0000 7fedef000640 1 -- 172.21.15.42:0/3008827153 >> 172.21.15.42:0/3008827153 conn(0x7fede80fef30 msgr2=0x7fede8107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.675+0000 7fedef000640 1 -- 172.21.15.42:0/3008827153 shutdown_connections 2024-04-25T00:40:07.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:07.675+0000 7fedef000640 1 -- 172.21.15.42:0/3008827153 wait complete. 2024-04-25T00:40:07.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:07 smithi136 ceph-mon[32923]: pgmap v1031: 97 pgs: 97 active+clean; 755 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 634 op/s 2024-04-25T00:40:07.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 297914 33.46 MB/sec execute 357 sec latency 225.964 ms 2024-04-25T00:40:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:07 smithi106 ceph-mon[32510]: pgmap v1031: 97 pgs: 97 active+clean; 755 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 634 op/s 2024-04-25T00:40:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:07 smithi042 ceph-mon[28130]: pgmap v1031: 97 pgs: 97 active+clean; 755 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 10 MiB/s wr, 634 op/s 2024-04-25T00:40:08.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 298270 33.44 MB/sec execute 358 sec latency 193.059 ms 2024-04-25T00:40:09.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 298601 33.37 MB/sec execute 359 sec latency 131.163 ms 2024-04-25T00:40:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:09 smithi136 ceph-mon[32923]: pgmap v1032: 97 pgs: 97 active+clean; 774 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 698 op/s 2024-04-25T00:40:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:09 smithi106 ceph-mon[32510]: pgmap v1032: 97 pgs: 97 active+clean; 774 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 698 op/s 2024-04-25T00:40:10.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:09 smithi042 ceph-mon[28130]: pgmap v1032: 97 pgs: 97 active+clean; 774 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 15 MiB/s wr, 698 op/s 2024-04-25T00:40:10.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 299136 33.36 MB/sec execute 360 sec latency 78.382 ms 2024-04-25T00:40:11.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 299724 33.33 MB/sec execute 361 sec latency 104.520 ms 2024-04-25T00:40:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:11 smithi136 ceph-mon[32923]: pgmap v1033: 97 pgs: 97 active+clean; 745 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 669 op/s 2024-04-25T00:40:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:11 smithi106 ceph-mon[32510]: pgmap v1033: 97 pgs: 97 active+clean; 745 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 669 op/s 2024-04-25T00:40:12.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:11 smithi042 ceph-mon[28130]: pgmap v1033: 97 pgs: 97 active+clean; 745 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 15 MiB/s wr, 669 op/s 2024-04-25T00:40:12.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 300352 33.34 MB/sec execute 362 sec latency 71.666 ms 2024-04-25T00:40:13.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 301127 33.39 MB/sec execute 363 sec latency 81.271 ms 2024-04-25T00:40:13.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:13 smithi042 ceph-mon[28130]: pgmap v1034: 97 pgs: 97 active+clean; 684 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 840 op/s 2024-04-25T00:40:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:13 smithi136 ceph-mon[32923]: pgmap v1034: 97 pgs: 97 active+clean; 684 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 840 op/s 2024-04-25T00:40:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:13 smithi106 ceph-mon[32510]: pgmap v1034: 97 pgs: 97 active+clean; 684 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 16 MiB/s wr, 840 op/s 2024-04-25T00:40:14.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 301647 33.38 MB/sec execute 364 sec latency 57.060 ms 2024-04-25T00:40:15.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 302143 33.35 MB/sec execute 365 sec latency 128.950 ms 2024-04-25T00:40:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:15 smithi136 ceph-mon[32923]: pgmap v1035: 97 pgs: 97 active+clean; 681 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 727 op/s 2024-04-25T00:40:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:15 smithi106 ceph-mon[32510]: pgmap v1035: 97 pgs: 97 active+clean; 681 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 727 op/s 2024-04-25T00:40:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:15 smithi042 ceph-mon[28130]: pgmap v1035: 97 pgs: 97 active+clean; 681 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 15 MiB/s wr, 727 op/s 2024-04-25T00:40:16.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 302771 33.35 MB/sec execute 366 sec latency 67.043 ms 2024-04-25T00:40:17.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 303555 33.35 MB/sec execute 367 sec latency 90.245 ms 2024-04-25T00:40:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:17 smithi136 ceph-mon[32923]: pgmap v1036: 97 pgs: 97 active+clean; 665 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 784 op/s 2024-04-25T00:40:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:17 smithi106 ceph-mon[32510]: pgmap v1036: 97 pgs: 97 active+clean; 665 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 784 op/s 2024-04-25T00:40:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:17 smithi042 ceph-mon[28130]: pgmap v1036: 97 pgs: 97 active+clean; 665 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 16 MiB/s wr, 784 op/s 2024-04-25T00:40:18.795 INFO:tasks.workunit.client.0.smithi042.stdout: 10 304347 33.39 MB/sec execute 368 sec latency 69.641 ms 2024-04-25T00:40:19.796 INFO:tasks.workunit.client.0.smithi042.stdout: 10 304931 33.40 MB/sec execute 369 sec latency 161.498 ms 2024-04-25T00:40:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:19 smithi136 ceph-mon[32923]: pgmap v1037: 97 pgs: 97 active+clean; 651 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.8 MiB/s wr, 688 op/s 2024-04-25T00:40:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:19 smithi106 ceph-mon[32510]: pgmap v1037: 97 pgs: 97 active+clean; 651 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.8 MiB/s wr, 688 op/s 2024-04-25T00:40:20.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:19 smithi042 ceph-mon[28130]: pgmap v1037: 97 pgs: 97 active+clean; 651 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 9.8 MiB/s wr, 688 op/s 2024-04-25T00:40:20.796 INFO:tasks.workunit.client.0.smithi042.stdout: 10 305426 33.40 MB/sec execute 370 sec latency 57.211 ms 2024-04-25T00:40:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.874+0000 7f17ab000640 1 -- 172.21.15.42:0/3420260108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a4102040 msgr2=0x7f17a4104430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.874+0000 7f17ab000640 1 --2- 172.21.15.42:0/3420260108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a4102040 0x7f17a4104430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f1798009900 tx=0x7f179802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.874+0000 7f17ab000640 1 -- 172.21.15.42:0/3420260108 shutdown_connections 2024-04-25T00:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.874+0000 7f17ab000640 1 --2- 172.21.15.42:0/3420260108 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a41072a0 0x7f17a41096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.874+0000 7f17ab000640 1 --2- 172.21.15.42:0/3420260108 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17a4104970 0x7f17a4106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.874+0000 7f17ab000640 1 --2- 172.21.15.42:0/3420260108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a4102040 0x7f17a4104430 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.874+0000 7f17ab000640 1 -- 172.21.15.42:0/3420260108 >> 172.21.15.42:0/3420260108 conn(0x7f17a40fba30 msgr2=0x7f17a40fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.874+0000 7f17ab000640 1 -- 172.21.15.42:0/3420260108 shutdown_connections 2024-04-25T00:40:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.874+0000 7f17ab000640 1 -- 172.21.15.42:0/3420260108 wait complete. 2024-04-25T00:40:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.875+0000 7f17ab000640 1 Processor -- start 2024-04-25T00:40:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.876+0000 7f17ab000640 1 -- start start 2024-04-25T00:40:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.876+0000 7f17ab000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17a4102040 0x7f17a4196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.876+0000 7f17ab000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a4104970 0x7f17a4196840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.876+0000 7f17ab000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a41072a0 0x7f17a4192e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.876+0000 7f17ab000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17a410d2b0 con 0x7f17a4102040 2024-04-25T00:40:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.876+0000 7f17ab000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17a410d130 con 0x7f17a4104970 2024-04-25T00:40:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.876+0000 7f17ab000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f17a410d430 con 0x7f17a41072a0 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.876+0000 7f17a9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17a4102040 0x7f17a4196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.877+0000 7f17aa600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a41072a0 0x7f17a4192e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.877+0000 7f17aa600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a41072a0 0x7f17a4192e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48040/0 (socket says 172.21.15.42:48040) 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.877+0000 7f17aa600640 1 -- 172.21.15.42:0/2974154262 learned_addr learned my addr 172.21.15.42:0/2974154262 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.877+0000 7f17a9c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17a4102040 0x7f17a4196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:46896/0 (socket says 172.21.15.42:46896) 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.877+0000 7f17a9200640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a4104970 0x7f17a4196840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.877+0000 7f17aa600640 1 -- 172.21.15.42:0/2974154262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a4104970 msgr2=0x7f17a4196840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.877+0000 7f17aa600640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a4104970 0x7f17a4196840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.877+0000 7f17aa600640 1 -- 172.21.15.42:0/2974154262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17a4102040 msgr2=0x7f17a4196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.877+0000 7f17aa600640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17a4102040 0x7f17a4196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.877+0000 7f17aa600640 1 -- 172.21.15.42:0/2974154262 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a4193730 con 0x7f17a41072a0 2024-04-25T00:40:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.878+0000 7f17aa600640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a41072a0 0x7f17a4192e70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f179c00eb10 tx=0x7f179c00c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.878+0000 7f17a2000640 1 -- 172.21.15.42:0/2974154262 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c004b20 con 0x7f17a41072a0 2024-04-25T00:40:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.878+0000 7f17a2000640 1 -- 172.21.15.42:0/2974154262 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f179c004cc0 con 0x7f17a41072a0 2024-04-25T00:40:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.878+0000 7f17a2000640 1 -- 172.21.15.42:0/2974154262 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c010760 con 0x7f17a41072a0 2024-04-25T00:40:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.878+0000 7f17ab000640 1 -- 172.21.15.42:0/2974154262 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a4199760 con 0x7f17a41072a0 2024-04-25T00:40:20.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.879+0000 7f17ab000640 1 -- 172.21.15.42:0/2974154262 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a4199ce0 con 0x7f17a41072a0 2024-04-25T00:40:20.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.887+0000 7f17af5bc740 1 Processor -- start 2024-04-25T00:40:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.888+0000 7f17a2000640 1 -- 172.21.15.42:0/2974154262 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f179c005000 con 0x7f17a41072a0 2024-04-25T00:40:20.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.889+0000 7f17af5bc740 1 -- start start 2024-04-25T00:40:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.889+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.890+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0ea2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.890+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0ea4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.890+0000 7f17af5bc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5580d0dffab0 con 0x5580d0e9d840 2024-04-25T00:40:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.890+0000 7f17af5bc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5580d0da8640 con 0x5580d0ea29b0 2024-04-25T00:40:20.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.890+0000 7f17af5bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5580d0dfba70 con 0x5580d0e9ce50 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.890+0000 7f17a9200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.890+0000 7f17a9200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48052/0 (socket says 172.21.15.42:48052) 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.890+0000 7f17a9200640 1 -- 172.21.15.42:0/2645181232 learned_addr learned my addr 172.21.15.42:0/2645181232 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.890+0000 7f17a9c00640 1 --2- 172.21.15.42:0/2645181232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0ea2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.890+0000 7f17a9200640 1 --2- 172.21.15.42:0/2645181232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0ea4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.891+0000 7f17a9c00640 1 -- 172.21.15.42:0/2645181232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 msgr2=0x5580d0e9d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.891+0000 7f17a9c00640 1 --2- 172.21.15.42:0/2645181232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.891+0000 7f17a9c00640 1 -- 172.21.15.42:0/2645181232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 msgr2=0x5580d0ea4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.891+0000 7f17a9c00640 1 --2- 172.21.15.42:0/2645181232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0ea4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.891+0000 7f17a9c00640 1 -- 172.21.15.42:0/2645181232 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580d0b14210 con 0x5580d0e9d840 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.891+0000 7f17a9200640 1 --2- 172.21.15.42:0/2645181232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:40:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.891+0000 7f17a2000640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f178c0a2cd0 0x7f178c0a5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.892+0000 7f17a2000640 1 -- 172.21.15.42:0/2974154262 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f179c0cf570 con 0x7f17a41072a0 2024-04-25T00:40:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.893+0000 7f17a9c00640 1 --2- 172.21.15.42:0/2645181232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0ea2340 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1798031440 tx=0x7f1798005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecf286509901cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.893+0000 7f17a9200640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f178c0a2cd0 0x7f178c0a5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.893+0000 7f1796a00640 1 -- 172.21.15.42:0/2645181232 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798040070 con 0x5580d0e9d840 2024-04-25T00:40:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.893+0000 7f1796a00640 1 -- 172.21.15.42:0/2645181232 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17980318b0 con 0x5580d0e9d840 2024-04-25T00:40:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.893+0000 7f1796a00640 1 -- 172.21.15.42:0/2645181232 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798035040 con 0x5580d0e9d840 2024-04-25T00:40:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.894+0000 7f17af5bc740 1 -- 172.21.15.42:0/2645181232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 msgr2=0x5580d0ea2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.894+0000 7f17af5bc740 1 --2- 172.21.15.42:0/2645181232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0ea2340 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1798031440 tx=0x7f1798005fd0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.895+0000 7f17a9200640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f178c0a2cd0 0x7f178c0a5190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1790007fd0 tx=0x7f17900046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.896+0000 7f17af5bc740 1 -- 172.21.15.42:0/2645181232 shutdown_connections 2024-04-25T00:40:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.896+0000 7f17af5bc740 1 --2- 172.21.15.42:0/2645181232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0ea4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.896+0000 7f17af5bc740 1 --2- 172.21.15.42:0/2645181232 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0ea2340 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.896+0000 7f17af5bc740 1 --2- 172.21.15.42:0/2645181232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.896+0000 7f17af5bc740 1 -- 172.21.15.42:0/2645181232 >> 172.21.15.42:0/2645181232 conn(0x5580d0e984c0 msgr2=0x5580d0e988c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:20.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.896+0000 7f17af5bc740 1 -- 172.21.15.42:0/2645181232 shutdown_connections 2024-04-25T00:40:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.896+0000 7f17af5bc740 1 -- 172.21.15.42:0/2645181232 wait complete. 2024-04-25T00:40:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.896+0000 7f17af5bc740 1 -- start start 2024-04-25T00:40:20.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.897+0000 7f17af5bc740 1 Processor -- start 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.897+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0ea7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.897+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0ea75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.897+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0eae5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.897+0000 7f17af5bc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5580d0ea5900 con 0x5580d0e9d840 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.897+0000 7f17af5bc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5580d0dbb990 con 0x5580d0ea29b0 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.897+0000 7f17af5bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5580d0ea5a80 con 0x5580d0e9ce50 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.897+0000 7f17af5bc740 10 client.0 _ll_register_callbacks cb 0x5580d0dbd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.898+0000 7f17a9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0ea7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.898+0000 7f17a9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0ea7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48058/0 (socket says 172.21.15.42:48058) 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.898+0000 7f17a9c00640 1 -- 172.21.15.42:0/1151014737 learned_addr learned my addr 172.21.15.42:0/1151014737 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.898+0000 7f17aa600640 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0eae5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.898+0000 7f17a9200640 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0ea75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.898+0000 7f17a9c00640 1 -- 172.21.15.42:0/1151014737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 msgr2=0x5580d0eae5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.898+0000 7f17a9c00640 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0eae5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.898+0000 7f17a9c00640 1 -- 172.21.15.42:0/1151014737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 msgr2=0x5580d0ea75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.898+0000 7f17a9c00640 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0ea75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.898+0000 7f17a9c00640 1 -- 172.21.15.42:0/1151014737 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d0be0620 con 0x5580d0e9ce50 2024-04-25T00:40:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.899+0000 7f17a9c00640 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0ea7070 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f179802a910 tx=0x7f17980056f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4efda95ca2fea1dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.899+0000 7f178b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:40:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.899+0000 7f178b400640 1 -- 172.21.15.42:0/1151014737 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798035040 con 0x5580d0e9ce50 2024-04-25T00:40:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.899+0000 7f178b400640 1 -- 172.21.15.42:0/1151014737 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f1798004250 con 0x5580d0e9ce50 2024-04-25T00:40:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.900+0000 7f17af5bc740 1 -- 172.21.15.42:0/1151014737 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17980318b0 con 0x5580d0e9ce50 2024-04-25T00:40:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.900+0000 7f178b400640 1 -- 172.21.15.42:0/1151014737 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1798040040 con 0x5580d0e9ce50 2024-04-25T00:40:20.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.900+0000 7f17af5bc740 10 client.26794 fetch_fsmap learned FSMap version 45 2024-04-25T00:40:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.900+0000 7f17af5bc740 1 -- 172.21.15.42:0/1151014737 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5580d0ea5900 con 0x5580d0e9ce50 2024-04-25T00:40:20.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.901+0000 7f178b400640 1 -- 172.21.15.42:0/1151014737 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17980318b0 con 0x5580d0e9ce50 2024-04-25T00:40:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.901+0000 7f178b400640 1 -- 172.21.15.42:0/1151014737 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f1798004a00 con 0x5580d0e9ce50 2024-04-25T00:40:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.901+0000 7f17af5bc740 10 client.26794 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:40:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.901+0000 7f17af5bc740 10 client.26794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:40:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.901+0000 7f17af5bc740 20 client.26794 populate_metadata read hostname 'smithi042' 2024-04-25T00:40:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.901+0000 7f17af5bc740 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5580d0eb27f0 0x5580d0eb4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.901+0000 7f17af5bc740 4 client.26794 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:40:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.901+0000 7f17af5bc740 1 -- 172.21.15.42:0/1151014737 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580d0e0cff0 con 0x5580d0eb27f0 2024-04-25T00:40:20.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.902+0000 7f17a9200640 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5580d0eb27f0 0x5580d0eb4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.903+0000 7f17a9200640 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5580d0eb27f0 0x5580d0eb4c00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c781169aba0b2af server_cookie=b9f0938f5b0fe1e6 in_seq=0 out_seq=0 2024-04-25T00:40:20.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.903+0000 7f178b400640 10 client.26794 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.906+0000 7f178b400640 1 -- 172.21.15.42:0/1151014737 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5580d0e0cff0 con 0x5580d0eb27f0 2024-04-25T00:40:20.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.906+0000 7f178b400640 10 client.26794 handle_command_reply: tid=0 2024-04-25T00:40:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.906+0000 7f17af5bc740 1 client.26794 shutdown 2024-04-25T00:40:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.906+0000 7f17af5bc740 1 -- 172.21.15.42:0/1151014737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 msgr2=0x5580d0ea7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.906+0000 7f17af5bc740 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0ea7070 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f179802a910 tx=0x7f17980056f0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.907+0000 7f17af5bc740 1 -- 172.21.15.42:0/1151014737 shutdown_connections 2024-04-25T00:40:20.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.907+0000 7f17af5bc740 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5580d0eb27f0 0x5580d0eb4c00 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.907+0000 7f17af5bc740 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0eae5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.907+0000 7f17af5bc740 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0ea75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.907+0000 7f17af5bc740 1 --2- 172.21.15.42:0/1151014737 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0ea7070 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.907+0000 7f17af5bc740 1 -- 172.21.15.42:0/1151014737 >> 172.21.15.42:0/1151014737 conn(0x5580d0e984c0 msgr2=0x5580d0e97620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.907+0000 7f178b400640 0 client.26794 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.908+0000 7f17af5bc740 1 -- 172.21.15.42:0/1151014737 shutdown_connections 2024-04-25T00:40:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.908+0000 7f17af5bc740 1 -- 172.21.15.42:0/1151014737 wait complete. 2024-04-25T00:40:20.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.909+0000 7f17af5bc740 20 client.26794 trim_cache size 0 max 16384 2024-04-25T00:40:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.939+0000 7f17af5bc740 1 Processor -- start 2024-04-25T00:40:20.940 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.939+0000 7f17af5bc740 1 -- start start 2024-04-25T00:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.939+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.939+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0e974a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.941 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.939+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0e979e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.939+0000 7f17af5bc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17a406e390 con 0x5580d0e9d840 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.939+0000 7f17af5bc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17a406bc60 con 0x5580d0ea29b0 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.939+0000 7f17af5bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41a08b0 con 0x5580d0e9ce50 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17a9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17a9c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48074/0 (socket says 172.21.15.42:48074) 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17a9c00640 1 -- 172.21.15.42:0/115712453 learned_addr learned my addr 172.21.15.42:0/115712453 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17aa600640 1 --2- 172.21.15.42:0/115712453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0e979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17a9200640 1 --2- 172.21.15.42:0/115712453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0e974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17a9c00640 1 -- 172.21.15.42:0/115712453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 msgr2=0x5580d0e979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17a9c00640 1 --2- 172.21.15.42:0/115712453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0e979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17a9c00640 1 -- 172.21.15.42:0/115712453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 msgr2=0x5580d0e974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17a9c00640 1 --2- 172.21.15.42:0/115712453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0e974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17a9c00640 1 -- 172.21.15.42:0/115712453 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580d0e92730 con 0x5580d0e9ce50 2024-04-25T00:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.940+0000 7f17aa600640 1 --2- 172.21.15.42:0/115712453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0e979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.941+0000 7f17a9c00640 1 --2- 172.21.15.42:0/115712453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9bc30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f17980314e0 tx=0x7f1798005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97e982c3676c3e61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.941+0000 7f1796000640 1 -- 172.21.15.42:0/115712453 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17980043d0 con 0x5580d0e9ce50 2024-04-25T00:40:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.941+0000 7f1796000640 1 -- 172.21.15.42:0/115712453 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1798031ba0 con 0x5580d0e9ce50 2024-04-25T00:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.941+0000 7f1796000640 1 -- 172.21.15.42:0/115712453 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798004b60 con 0x5580d0e9ce50 2024-04-25T00:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.942+0000 7f17af5bc740 1 -- 172.21.15.42:0/115712453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 msgr2=0x5580d0e9bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.942+0000 7f17af5bc740 1 --2- 172.21.15.42:0/115712453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9bc30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f17980314e0 tx=0x7f1798005fd0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.942+0000 7f17af5bc740 1 -- 172.21.15.42:0/115712453 shutdown_connections 2024-04-25T00:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.942+0000 7f17af5bc740 1 --2- 172.21.15.42:0/115712453 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0ea29b0 0x5580d0e979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.942+0000 7f17af5bc740 1 --2- 172.21.15.42:0/115712453 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9d840 0x5580d0e974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.942+0000 7f17af5bc740 1 --2- 172.21.15.42:0/115712453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0e9ce50 0x5580d0e9bc30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.942+0000 7f17af5bc740 1 -- 172.21.15.42:0/115712453 >> 172.21.15.42:0/115712453 conn(0x5580d0e984c0 msgr2=0x5580d0e96d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.942+0000 7f17af5bc740 1 -- 172.21.15.42:0/115712453 shutdown_connections 2024-04-25T00:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.942+0000 7f17af5bc740 1 -- 172.21.15.42:0/115712453 wait complete. 2024-04-25T00:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.943+0000 7f17af5bc740 1 -- start start 2024-04-25T00:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.943+0000 7f17af5bc740 1 Processor -- start 2024-04-25T00:40:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9ce50 0x5580d0eab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0e9d840 0x5580d0eabd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17af5bc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0ea29b0 0x5580d0eacda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17af5bc740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41a0f20 con 0x5580d0e9ce50 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17af5bc740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17a406b590 con 0x5580d0e9d840 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17af5bc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f17a41a15f0 con 0x5580d0ea29b0 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17af5bc740 10 client.0 _ll_register_callbacks cb 0x7f179000b230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17aa600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0ea29b0 0x5580d0eacda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17a9200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0e9d840 0x5580d0eabd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17aa600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0ea29b0 0x5580d0eacda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48078/0 (socket says 172.21.15.42:48078) 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17a9200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0e9d840 0x5580d0eabd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46320/0 (socket says 172.21.15.42:46320) 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17aa600640 1 -- 172.21.15.42:0/2132497190 learned_addr learned my addr 172.21.15.42:0/2132497190 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.944+0000 7f17a9c00640 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9ce50 0x5580d0eab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.945+0000 7f17aa600640 1 -- 172.21.15.42:0/2132497190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0e9d840 msgr2=0x5580d0eabd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.945+0000 7f17aa600640 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0e9d840 0x5580d0eabd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.945+0000 7f17aa600640 1 -- 172.21.15.42:0/2132497190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9ce50 msgr2=0x5580d0eab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.945+0000 7f17aa600640 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9ce50 0x5580d0eab850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.945+0000 7f17aa600640 1 -- 172.21.15.42:0/2132497190 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580d0ead390 con 0x5580d0ea29b0 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.945+0000 7f178b400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.945+0000 7f17aa600640 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0ea29b0 0x5580d0eacda0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f179c0cd410 tx=0x7f179c09b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99d8dcef71c0d515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.945+0000 7f178b400640 1 -- 172.21.15.42:0/2132497190 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f179c0b9070 con 0x5580d0ea29b0 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.945+0000 7f178b400640 1 -- 172.21.15.42:0/2132497190 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f179c0a47e0 con 0x5580d0ea29b0 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.946+0000 7f17af5bc740 1 -- 172.21.15.42:0/2132497190 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1798031ba0 con 0x5580d0ea29b0 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.946+0000 7f178b400640 1 -- 172.21.15.42:0/2132497190 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f179c0a4a70 con 0x5580d0ea29b0 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.946+0000 7f17af5bc740 10 client.26800 fetch_fsmap learned FSMap version 45 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.946+0000 7f17af5bc740 1 -- 172.21.15.42:0/2132497190 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5580d0f18d80 con 0x5580d0ea29b0 2024-04-25T00:40:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.946+0000 7f178b400640 1 -- 172.21.15.42:0/2132497190 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1798031ba0 con 0x5580d0ea29b0 2024-04-25T00:40:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.947+0000 7f178b400640 1 -- 172.21.15.42:0/2132497190 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f179c0cf1d0 con 0x5580d0ea29b0 2024-04-25T00:40:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.947+0000 7f17af5bc740 10 client.26800 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:40:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.947+0000 7f17af5bc740 10 client.26800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:40:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.947+0000 7f17af5bc740 20 client.26800 populate_metadata read hostname 'smithi042' 2024-04-25T00:40:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.947+0000 7f17af5bc740 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5580d0f195f0 0x5580d0f1ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.947+0000 7f17af5bc740 4 client.26800 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:40:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.947+0000 7f17af5bc740 1 -- 172.21.15.42:0/2132497190 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580d0e07340 con 0x5580d0f195f0 2024-04-25T00:40:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.947+0000 7f17a9c00640 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5580d0f195f0 0x5580d0f1ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.948+0000 7f17a9c00640 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5580d0f195f0 0x5580d0f1ba10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3db8d8ab6e4f26af server_cookie=b5f4e873ebc1555d in_seq=0 out_seq=0 2024-04-25T00:40:20.950 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:20.948+0000 7f178b400640 10 client.26800 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.286+0000 7f178b400640 1 -- 172.21.15.42:0/2132497190 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580d0e07340 con 0x5580d0f195f0 2024-04-25T00:40:21.288 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.287+0000 7f178b400640 10 client.26800 handle_command_reply: tid=0 2024-04-25T00:40:21.289 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.287+0000 7f17af5bc740 1 client.26800 shutdown 2024-04-25T00:40:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.289+0000 7f17af5bc740 1 -- 172.21.15.42:0/2132497190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0ea29b0 msgr2=0x5580d0eacda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.290 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.289+0000 7f17af5bc740 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0ea29b0 0x5580d0eacda0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f179c0cd410 tx=0x7f179c09b820 comp rx=0 tx=0).stop 2024-04-25T00:40:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.289+0000 7f17af5bc740 1 -- 172.21.15.42:0/2132497190 shutdown_connections 2024-04-25T00:40:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.289+0000 7f17af5bc740 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5580d0f195f0 0x5580d0f1ba10 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.289+0000 7f178b400640 0 client.26800 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.289+0000 7f17af5bc740 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5580d0ea29b0 0x5580d0eacda0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.289+0000 7f17af5bc740 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5580d0e9d840 0x5580d0eabd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.289+0000 7f17af5bc740 1 --2- 172.21.15.42:0/2132497190 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5580d0e9ce50 0x5580d0eab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.289+0000 7f17af5bc740 1 -- 172.21.15.42:0/2132497190 >> 172.21.15.42:0/2132497190 conn(0x5580d0e984c0 msgr2=0x5580d0e9bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.289+0000 7f17af5bc740 1 -- 172.21.15.42:0/2132497190 shutdown_connections 2024-04-25T00:40:21.291 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.290+0000 7f17af5bc740 1 -- 172.21.15.42:0/2132497190 wait complete. 2024-04-25T00:40:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.290+0000 7f17af5bc740 20 client.26800 trim_cache size 0 max 16384 2024-04-25T00:40:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:40:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:40:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:40:21.292 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:40:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.291+0000 7f17ab000640 1 -- 172.21.15.42:0/2974154262 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f178c0a2cd0 msgr2=0x7f178c0a5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.291+0000 7f17ab000640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f178c0a2cd0 0x7f178c0a5190 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1790007fd0 tx=0x7f17900046c0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.291+0000 7f17ab000640 1 -- 172.21.15.42:0/2974154262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a41072a0 msgr2=0x7f17a4192e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.291+0000 7f17ab000640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a41072a0 0x7f17a4192e70 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f179c00eb10 tx=0x7f179c00c2d0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.293 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.291+0000 7f17ab000640 1 -- 172.21.15.42:0/2974154262 shutdown_connections 2024-04-25T00:40:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.291+0000 7f17ab000640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f178c0a2cd0 0x7f178c0a5190 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.291+0000 7f17ab000640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17a41072a0 0x7f17a4192e70 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.291+0000 7f17ab000640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17a4104970 0x7f17a4196840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.291+0000 7f17ab000640 1 --2- 172.21.15.42:0/2974154262 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f17a4102040 0x7f17a4196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.291+0000 7f17ab000640 1 -- 172.21.15.42:0/2974154262 >> 172.21.15.42:0/2974154262 conn(0x7f17a40fba30 msgr2=0x7f17a4101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.292+0000 7f17ab000640 1 -- 172.21.15.42:0/2974154262 shutdown_connections 2024-04-25T00:40:21.294 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:21.292+0000 7f17ab000640 1 -- 172.21.15.42:0/2974154262 wait complete. 2024-04-25T00:40:21.796 INFO:tasks.workunit.client.0.smithi042.stdout: 10 305809 33.35 MB/sec execute 371 sec latency 171.027 ms 2024-04-25T00:40:22.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:21 smithi042 ceph-mon[28130]: pgmap v1038: 97 pgs: 97 active+clean; 655 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.7 MiB/s wr, 550 op/s 2024-04-25T00:40:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:21 smithi136 ceph-mon[32923]: pgmap v1038: 97 pgs: 97 active+clean; 655 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.7 MiB/s wr, 550 op/s 2024-04-25T00:40:22.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:21 smithi106 ceph-mon[32510]: pgmap v1038: 97 pgs: 97 active+clean; 655 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.7 MiB/s wr, 550 op/s 2024-04-25T00:40:22.796 INFO:tasks.workunit.client.0.smithi042.stdout: 10 306508 33.34 MB/sec execute 372 sec latency 47.543 ms 2024-04-25T00:40:23.796 INFO:tasks.workunit.client.0.smithi042.stdout: 10 307367 33.36 MB/sec execute 373 sec latency 67.159 ms 2024-04-25T00:40:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:23 smithi042 ceph-mon[28130]: pgmap v1039: 97 pgs: 97 active+clean; 654 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.9 MiB/s wr, 596 op/s 2024-04-25T00:40:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:23 smithi136 ceph-mon[32923]: pgmap v1039: 97 pgs: 97 active+clean; 654 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.9 MiB/s wr, 596 op/s 2024-04-25T00:40:24.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:23 smithi106 ceph-mon[32510]: pgmap v1039: 97 pgs: 97 active+clean; 654 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.9 MiB/s wr, 596 op/s 2024-04-25T00:40:24.796 INFO:tasks.workunit.client.0.smithi042.stdout: 10 308161 33.42 MB/sec execute 374 sec latency 72.703 ms 2024-04-25T00:40:25.796 INFO:tasks.workunit.client.0.smithi042.stdout: 10 308755 33.43 MB/sec execute 375 sec latency 50.484 ms 2024-04-25T00:40:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:25 smithi042 ceph-mon[28130]: pgmap v1040: 97 pgs: 97 active+clean; 656 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2024-04-25T00:40:26.140 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:25 smithi106 ceph-mon[32510]: pgmap v1040: 97 pgs: 97 active+clean; 656 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2024-04-25T00:40:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:25 smithi136 ceph-mon[32923]: pgmap v1040: 97 pgs: 97 active+clean; 656 MiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.3 MiB/s wr, 423 op/s 2024-04-25T00:40:26.796 INFO:tasks.workunit.client.0.smithi042.stdout: 10 309249 33.40 MB/sec execute 376 sec latency 95.094 ms 2024-04-25T00:40:27.797 INFO:tasks.workunit.client.0.smithi042.stdout: 10 309851 33.39 MB/sec execute 377 sec latency 66.887 ms 2024-04-25T00:40:28.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:27 smithi042 ceph-mon[28130]: pgmap v1041: 97 pgs: 97 active+clean; 654 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.6 MiB/s wr, 508 op/s 2024-04-25T00:40:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:27 smithi136 ceph-mon[32923]: pgmap v1041: 97 pgs: 97 active+clean; 654 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.6 MiB/s wr, 508 op/s 2024-04-25T00:40:28.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:27 smithi106 ceph-mon[32510]: pgmap v1041: 97 pgs: 97 active+clean; 654 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.6 MiB/s wr, 508 op/s 2024-04-25T00:40:28.797 INFO:tasks.workunit.client.0.smithi042.stdout: 10 310606 33.39 MB/sec execute 378 sec latency 64.921 ms 2024-04-25T00:40:29.797 INFO:tasks.workunit.client.0.smithi042.stdout: 10 311319 33.41 MB/sec execute 379 sec latency 67.638 ms 2024-04-25T00:40:30.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:29 smithi042 ceph-mon[28130]: pgmap v1042: 97 pgs: 97 active+clean; 665 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.0 MiB/s wr, 541 op/s 2024-04-25T00:40:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:29 smithi136 ceph-mon[32923]: pgmap v1042: 97 pgs: 97 active+clean; 665 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.0 MiB/s wr, 541 op/s 2024-04-25T00:40:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:29 smithi106 ceph-mon[32510]: pgmap v1042: 97 pgs: 97 active+clean; 665 MiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 4.0 MiB/s wr, 541 op/s 2024-04-25T00:40:30.798 INFO:tasks.workunit.client.0.smithi042.stdout: 10 311914 33.41 MB/sec execute 380 sec latency 95.726 ms 2024-04-25T00:40:31.797 INFO:tasks.workunit.client.0.smithi042.stdout: 10 312578 33.43 MB/sec execute 381 sec latency 56.329 ms 2024-04-25T00:40:32.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:31 smithi042 ceph-mon[28130]: pgmap v1043: 97 pgs: 97 active+clean; 669 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.6 MiB/s wr, 506 op/s 2024-04-25T00:40:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:31 smithi136 ceph-mon[32923]: pgmap v1043: 97 pgs: 97 active+clean; 669 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.6 MiB/s wr, 506 op/s 2024-04-25T00:40:32.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:31 smithi106 ceph-mon[32510]: pgmap v1043: 97 pgs: 97 active+clean; 669 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 3.6 MiB/s wr, 506 op/s 2024-04-25T00:40:32.798 INFO:tasks.workunit.client.0.smithi042.stdout: 10 313264 33.44 MB/sec execute 382 sec latency 47.368 ms 2024-04-25T00:40:33.798 INFO:tasks.workunit.client.0.smithi042.stdout: 10 313961 33.45 MB/sec execute 383 sec latency 47.996 ms 2024-04-25T00:40:34.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:33 smithi042 ceph-mon[28130]: pgmap v1044: 97 pgs: 97 active+clean; 675 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.9 MiB/s wr, 533 op/s 2024-04-25T00:40:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:33 smithi136 ceph-mon[32923]: pgmap v1044: 97 pgs: 97 active+clean; 675 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.9 MiB/s wr, 533 op/s 2024-04-25T00:40:34.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:33 smithi106 ceph-mon[32510]: pgmap v1044: 97 pgs: 97 active+clean; 675 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.9 MiB/s wr, 533 op/s 2024-04-25T00:40:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.472+0000 7f065d200640 1 -- 172.21.15.42:0/1609574817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06580ffe30 msgr2=0x7f0658102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.472+0000 7f065d200640 1 --2- 172.21.15.42:0/1609574817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06580ffe30 0x7f0658102260 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0648009900 tx=0x7f064802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.473+0000 7f065d200640 1 -- 172.21.15.42:0/1609574817 shutdown_connections 2024-04-25T00:40:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.473+0000 7f065d200640 1 --2- 172.21.15.42:0/1609574817 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0658105150 0x7f06581055d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.473+0000 7f065d200640 1 --2- 172.21.15.42:0/1609574817 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06581027a0 0x7f0658104c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.473+0000 7f065d200640 1 --2- 172.21.15.42:0/1609574817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f06580ffe30 0x7f0658102260 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.473+0000 7f065d200640 1 -- 172.21.15.42:0/1609574817 >> 172.21.15.42:0/1609574817 conn(0x7f06580fba30 msgr2=0x7f06580fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.473+0000 7f065d200640 1 -- 172.21.15.42:0/1609574817 shutdown_connections 2024-04-25T00:40:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.474+0000 7f065d200640 1 -- 172.21.15.42:0/1609574817 wait complete. 2024-04-25T00:40:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f065d200640 1 Processor -- start 2024-04-25T00:40:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f065d200640 1 -- start start 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f065d200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06580ffe30 0x7f065819dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f065d200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06581027a0 0x7f065819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f065d200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0658105150 0x7f06581a5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f065d200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0658113810 con 0x7f06581027a0 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f065d200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0658113690 con 0x7f06580ffe30 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f065d200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0658113990 con 0x7f0658105150 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f0657400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06580ffe30 0x7f065819dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f0657400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06580ffe30 0x7f065819dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39234/0 (socket says 172.21.15.42:39234) 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.475+0000 7f0657400640 1 -- 172.21.15.42:0/3618694632 learned_addr learned my addr 172.21.15.42:0/3618694632 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.476+0000 7f0656a00640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06581027a0 0x7f065819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.476+0000 7f0657e00640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0658105150 0x7f06581a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.476+0000 7f0657400640 1 -- 172.21.15.42:0/3618694632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0658105150 msgr2=0x7f06581a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.476+0000 7f0657400640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0658105150 0x7f06581a5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.476+0000 7f0657400640 1 -- 172.21.15.42:0/3618694632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06581027a0 msgr2=0x7f065819e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.476+0000 7f0657400640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06581027a0 0x7f065819e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.476+0000 7f0657400640 1 -- 172.21.15.42:0/3618694632 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06581a5ac0 con 0x7f06580ffe30 2024-04-25T00:40:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.476+0000 7f0657400640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06580ffe30 0x7f065819dea0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0648009230 tx=0x7f06480042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.477+0000 7f064fe00640 1 -- 172.21.15.42:0/3618694632 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064803f030 con 0x7f06580ffe30 2024-04-25T00:40:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.477+0000 7f064fe00640 1 -- 172.21.15.42:0/3618694632 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0648003ce0 con 0x7f06580ffe30 2024-04-25T00:40:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.477+0000 7f064fe00640 1 -- 172.21.15.42:0/3618694632 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648035070 con 0x7f06580ffe30 2024-04-25T00:40:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.477+0000 7f065d200640 1 -- 172.21.15.42:0/3618694632 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06581a5d50 con 0x7f06580ffe30 2024-04-25T00:40:34.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.477+0000 7f065d200640 1 -- 172.21.15.42:0/3618694632 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06581a6190 con 0x7f06580ffe30 2024-04-25T00:40:34.482 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.481+0000 7f064fe00640 1 -- 172.21.15.42:0/3618694632 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0648042030 con 0x7f06580ffe30 2024-04-25T00:40:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.482+0000 7f0661570740 1 Processor -- start 2024-04-25T00:40:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.482+0000 7f064fe00640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f063c0a2c00 0x7f063c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.482+0000 7f064fe00640 1 -- 172.21.15.42:0/3618694632 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f06480ef440 con 0x7f06580ffe30 2024-04-25T00:40:34.484 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.483+0000 7f0661570740 1 -- start start 2024-04-25T00:40:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.483+0000 7f0661570740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e011e50 0x55cf7e012270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.483+0000 7f0657e00640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f063c0a2c00 0x7f063c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.483+0000 7f0661570740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e017340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.483+0000 7f0661570740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e0179b0 0x55cf7e019e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.483+0000 7f0661570740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cf7df74ab0 con 0x55cf7e011e50 2024-04-25T00:40:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.483+0000 7f0661570740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cf7df1d640 con 0x55cf7e012840 2024-04-25T00:40:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.483+0000 7f0661570740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cf7df70a70 con 0x55cf7e0179b0 2024-04-25T00:40:34.485 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.483+0000 7f0657400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e011e50 0x55cf7e012270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.484+0000 7f0656a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e017340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.484+0000 7f0656a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e017340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39248/0 (socket says 172.21.15.42:39248) 2024-04-25T00:40:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.484+0000 7f0656a00640 1 -- 172.21.15.42:0/505061263 learned_addr learned my addr 172.21.15.42:0/505061263 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.484+0000 7f0657e00640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f063c0a2c00 0x7f063c0a50c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0650005fd0 tx=0x7f0650009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.484+0000 7f0657e00640 1 --2- 172.21.15.42:0/505061263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e0179b0 0x55cf7e019e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.484+0000 7f0656a00640 1 -- 172.21.15.42:0/505061263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e0179b0 msgr2=0x55cf7e019e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.484+0000 7f0656a00640 1 --2- 172.21.15.42:0/505061263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e0179b0 0x55cf7e019e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.485+0000 7f0656a00640 1 -- 172.21.15.42:0/505061263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e011e50 msgr2=0x55cf7e012270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.485+0000 7f0656a00640 1 --2- 172.21.15.42:0/505061263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e011e50 0x55cf7e012270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.485+0000 7f0656a00640 1 -- 172.21.15.42:0/505061263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7dc89210 con 0x55cf7e012840 2024-04-25T00:40:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.485+0000 7f0656a00640 1 --2- 172.21.15.42:0/505061263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e017340 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0644009c90 tx=0x7f0644011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb5f4bdd0d6165fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.485+0000 7f064cc00640 1 -- 172.21.15.42:0/505061263 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064401e070 con 0x55cf7e012840 2024-04-25T00:40:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.485+0000 7f064cc00640 1 -- 172.21.15.42:0/505061263 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0644019b10 con 0x55cf7e012840 2024-04-25T00:40:34.487 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.485+0000 7f064cc00640 1 -- 172.21.15.42:0/505061263 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064400ea30 con 0x55cf7e012840 2024-04-25T00:40:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.486+0000 7f0661570740 1 -- 172.21.15.42:0/505061263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 msgr2=0x55cf7e017340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.486+0000 7f0661570740 1 --2- 172.21.15.42:0/505061263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e017340 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f0644009c90 tx=0x7f0644011810 comp rx=0 tx=0).stop 2024-04-25T00:40:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.486+0000 7f0661570740 1 -- 172.21.15.42:0/505061263 shutdown_connections 2024-04-25T00:40:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.486+0000 7f0661570740 1 --2- 172.21.15.42:0/505061263 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e0179b0 0x55cf7e019e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.486+0000 7f0661570740 1 --2- 172.21.15.42:0/505061263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e017340 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.486+0000 7f0661570740 1 --2- 172.21.15.42:0/505061263 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e011e50 0x55cf7e012270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.486+0000 7f0661570740 1 -- 172.21.15.42:0/505061263 >> 172.21.15.42:0/505061263 conn(0x55cf7e00d4c0 msgr2=0x55cf7e00d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.486+0000 7f0661570740 1 -- 172.21.15.42:0/505061263 shutdown_connections 2024-04-25T00:40:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.487+0000 7f0661570740 1 -- 172.21.15.42:0/505061263 wait complete. 2024-04-25T00:40:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.487+0000 7f0661570740 1 -- start start 2024-04-25T00:40:34.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.487+0000 7f0661570740 1 Processor -- start 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0661570740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0661570740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e01c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0661570740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e0179b0 0x55cf7e0235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0661570740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55cf7e01a960 con 0x55cf7e0179b0 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0661570740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55cf7df30990 con 0x55cf7e012840 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0661570740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cf7e01aae0 con 0x55cf7e011e50 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0661570740 10 client.0 _ll_register_callbacks cb 0x55cf7df32950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0657400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0657400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:43396/0 (socket says 172.21.15.42:43396) 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0657400640 1 -- 172.21.15.42:0/600530491 learned_addr learned my addr 172.21.15.42:0/600530491 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:34.490 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.488+0000 7f0656a00640 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e01c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0657400640 1 -- 172.21.15.42:0/600530491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 msgr2=0x55cf7e01c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0657400640 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e01c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0657400640 1 -- 172.21.15.42:0/600530491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e0179b0 msgr2=0x55cf7e0235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0657400640 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e0179b0 0x55cf7e0235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0657400640 1 -- 172.21.15.42:0/600530491 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7dd55620 con 0x55cf7e011e50 2024-04-25T00:40:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0656a00640 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e01c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0657400640 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01c060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f06480ecf10 tx=0x7f06480045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c4b255d21d94c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0641600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0641600640 1 -- 172.21.15.42:0/600530491 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648049040 con 0x55cf7e011e50 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0641600640 1 -- 172.21.15.42:0/600530491 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f0648003800 con 0x55cf7e011e50 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.489+0000 7f0661570740 1 -- 172.21.15.42:0/600530491 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0644019b10 con 0x55cf7e011e50 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0641600640 1 -- 172.21.15.42:0/600530491 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f06480b8390 con 0x55cf7e011e50 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0661570740 10 client.26812 fetch_fsmap learned FSMap version 45 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0661570740 1 -- 172.21.15.42:0/600530491 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cf7e01aa30 con 0x55cf7e011e50 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0641600640 1 -- 172.21.15.42:0/600530491 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0644019b10 con 0x55cf7e011e50 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0641600640 1 -- 172.21.15.42:0/600530491 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f06480c4450 con 0x55cf7e011e50 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0661570740 10 client.26812 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0661570740 10 client.26812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0661570740 20 client.26812 populate_metadata read hostname 'smithi042' 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0661570740 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cf7e027d70 0x55cf7e02a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0661570740 4 client.26812 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:40:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.490+0000 7f0661570740 1 -- 172.21.15.42:0/600530491 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf7df81ff0 con 0x55cf7e027d70 2024-04-25T00:40:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.491+0000 7f0656a00640 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cf7e027d70 0x55cf7e02a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.491+0000 7f0641600640 10 client.26812 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.491+0000 7f0656a00640 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cf7e027d70 0x55cf7e02a1e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=603eb51de8d01dfe server_cookie=c27e16bf3e19a833 in_seq=0 out_seq=0 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.492+0000 7f0641600640 1 -- 172.21.15.42:0/600530491 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf7df81ff0 con 0x55cf7e027d70 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.492+0000 7f0641600640 10 client.26812 handle_command_reply: tid=0 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.492+0000 7f0661570740 1 client.26812 shutdown 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.492+0000 7f0661570740 1 -- 172.21.15.42:0/600530491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 msgr2=0x55cf7e01c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0661570740 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01c060 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f06480ecf10 tx=0x7f06480045a0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0661570740 1 -- 172.21.15.42:0/600530491 shutdown_connections 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0661570740 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cf7e027d70 0x55cf7e02a1e0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0661570740 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e0179b0 0x55cf7e0235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0641600640 0 client.26812 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0661570740 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e01c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0661570740 1 --2- 172.21.15.42:0/600530491 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01c060 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0661570740 1 -- 172.21.15.42:0/600530491 >> 172.21.15.42:0/600530491 conn(0x55cf7e00d4c0 msgr2=0x55cf7e00c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0661570740 1 -- 172.21.15.42:0/600530491 shutdown_connections 2024-04-25T00:40:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0661570740 1 -- 172.21.15.42:0/600530491 wait complete. 2024-04-25T00:40:34.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.493+0000 7f0661570740 20 client.26812 trim_cache size 0 max 16384 2024-04-25T00:40:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.517+0000 7f0661570740 1 Processor -- start 2024-04-25T00:40:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.517+0000 7f0661570740 1 -- start start 2024-04-25T00:40:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.517+0000 7f0661570740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.517+0000 7f0661570740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e01c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.517+0000 7f0661570740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e0179b0 0x55cf7e0103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0661570740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f06581b18a0 con 0x55cf7e0179b0 2024-04-25T00:40:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0661570740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06581af170 con 0x55cf7e012840 2024-04-25T00:40:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0661570740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06581acfc0 con 0x55cf7e011e50 2024-04-25T00:40:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0657400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0657400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:43412/0 (socket says 172.21.15.42:43412) 2024-04-25T00:40:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0657400640 1 -- 172.21.15.42:0/1350307356 learned_addr learned my addr 172.21.15.42:0/1350307356 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0656a00640 1 --2- 172.21.15.42:0/1350307356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e01c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0657e00640 1 --2- 172.21.15.42:0/1350307356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e0179b0 0x55cf7e0103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0657400640 1 -- 172.21.15.42:0/1350307356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 msgr2=0x55cf7e01c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0657400640 1 --2- 172.21.15.42:0/1350307356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e01c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0657400640 1 -- 172.21.15.42:0/1350307356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e0179b0 msgr2=0x55cf7e0103d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0657400640 1 --2- 172.21.15.42:0/1350307356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e0179b0 0x55cf7e0103d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0657400640 1 -- 172.21.15.42:0/1350307356 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7e00d2f0 con 0x55cf7e011e50 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.518+0000 7f0657e00640 1 --2- 172.21.15.42:0/1350307356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e0179b0 0x55cf7e0103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0657400640 1 --2- 172.21.15.42:0/1350307356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01bc70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f06480b86f0 tx=0x7f06480ba230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d6f3ddd4ed1908e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0643e00640 1 -- 172.21.15.42:0/1350307356 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648049040 con 0x55cf7e011e50 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0643e00640 1 -- 172.21.15.42:0/1350307356 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0648031c00 con 0x55cf7e011e50 2024-04-25T00:40:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0643e00640 1 -- 172.21.15.42:0/1350307356 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06480c4450 con 0x55cf7e011e50 2024-04-25T00:40:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0661570740 1 -- 172.21.15.42:0/1350307356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 msgr2=0x55cf7e01bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0661570740 1 --2- 172.21.15.42:0/1350307356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01bc70 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f06480b86f0 tx=0x7f06480ba230 comp rx=0 tx=0).stop 2024-04-25T00:40:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0661570740 1 -- 172.21.15.42:0/1350307356 shutdown_connections 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0661570740 1 --2- 172.21.15.42:0/1350307356 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e0179b0 0x55cf7e0103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0661570740 1 --2- 172.21.15.42:0/1350307356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e012840 0x55cf7e01c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0661570740 1 --2- 172.21.15.42:0/1350307356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e011e50 0x55cf7e01bc70 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0661570740 1 -- 172.21.15.42:0/1350307356 >> 172.21.15.42:0/1350307356 conn(0x55cf7e00d4c0 msgr2=0x55cf7e017090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0661570740 1 -- 172.21.15.42:0/1350307356 shutdown_connections 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.520+0000 7f0661570740 1 -- 172.21.15.42:0/1350307356 wait complete. 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.521+0000 7f0661570740 1 -- start start 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.521+0000 7f0661570740 1 Processor -- start 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.521+0000 7f0661570740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e011e50 0x55cf7e020020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.521+0000 7f0661570740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e012840 0x55cf7e020560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.521+0000 7f0661570740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e0179b0 0x55cf7e021570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.521+0000 7f0661570740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f06581ad630 con 0x55cf7e011e50 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.521+0000 7f0661570740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f06581aeaa0 con 0x55cf7e0179b0 2024-04-25T00:40:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.521+0000 7f0661570740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f06581add00 con 0x55cf7e012840 2024-04-25T00:40:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.521+0000 7f0661570740 10 client.0 _ll_register_callbacks cb 0x7f06480bbaf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0657e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e0179b0 0x55cf7e021570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0657e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e0179b0 0x55cf7e021570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39280/0 (socket says 172.21.15.42:39280) 2024-04-25T00:40:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0657e00640 1 -- 172.21.15.42:0/828452231 learned_addr learned my addr 172.21.15.42:0/828452231 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0656a00640 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e012840 0x55cf7e020560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0657e00640 1 -- 172.21.15.42:0/828452231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e012840 msgr2=0x55cf7e020560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0657e00640 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e012840 0x55cf7e020560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0657e00640 1 -- 172.21.15.42:0/828452231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e011e50 msgr2=0x55cf7e020020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0657e00640 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e011e50 0x55cf7e020020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0657e00640 1 -- 172.21.15.42:0/828452231 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7e021b60 con 0x55cf7e0179b0 2024-04-25T00:40:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0657e00640 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e0179b0 0x55cf7e021570 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0650020ca0 tx=0x7f0650004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd131efbf064b7fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.522+0000 7f0641600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:40:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.523+0000 7f0641600640 1 -- 172.21.15.42:0/828452231 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0650020280 con 0x55cf7e0179b0 2024-04-25T00:40:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.523+0000 7f0641600640 1 -- 172.21.15.42:0/828452231 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f0650003970 con 0x55cf7e0179b0 2024-04-25T00:40:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.523+0000 7f0661570740 1 -- 172.21.15.42:0/828452231 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0648031c00 con 0x55cf7e0179b0 2024-04-25T00:40:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.523+0000 7f0641600640 1 -- 172.21.15.42:0/828452231 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0650003b70 con 0x55cf7e0179b0 2024-04-25T00:40:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.523+0000 7f0661570740 10 client.26867 fetch_fsmap learned FSMap version 45 2024-04-25T00:40:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.523+0000 7f0661570740 1 -- 172.21.15.42:0/828452231 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55cf7e022e50 con 0x55cf7e0179b0 2024-04-25T00:40:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.524+0000 7f0641600640 1 -- 172.21.15.42:0/828452231 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0648031c00 con 0x55cf7e0179b0 2024-04-25T00:40:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.524+0000 7f0641600640 1 -- 172.21.15.42:0/828452231 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f0650027020 con 0x55cf7e0179b0 2024-04-25T00:40:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.525+0000 7f0661570740 10 client.26867 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:40:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.525+0000 7f0661570740 10 client.26867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:40:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.525+0000 7f0661570740 20 client.26867 populate_metadata read hostname 'smithi042' 2024-04-25T00:40:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.525+0000 7f0661570740 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cf7e027d70 0x55cf7e090b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.525+0000 7f0661570740 4 client.26867 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:40:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.525+0000 7f0661570740 1 -- 172.21.15.42:0/828452231 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf7df7c340 con 0x55cf7e027d70 2024-04-25T00:40:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.525+0000 7f0657400640 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cf7e027d70 0x55cf7e090b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.526+0000 7f0657400640 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cf7e027d70 0x55cf7e090b70 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6624a873c8cd7ce7 server_cookie=2d3ea41ea217be06 in_seq=0 out_seq=0 2024-04-25T00:40:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.526+0000 7f0641600640 10 client.26867 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:34.798 INFO:tasks.workunit.client.0.smithi042.stdout: 10 314481 33.43 MB/sec execute 384 sec latency 193.737 ms 2024-04-25T00:40:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.887+0000 7f0641600640 1 -- 172.21.15.42:0/828452231 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf7df7c340 con 0x55cf7e027d70 2024-04-25T00:40:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.887+0000 7f0641600640 10 client.26867 handle_command_reply: tid=0 2024-04-25T00:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.887+0000 7f0661570740 1 client.26867 shutdown 2024-04-25T00:40:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.888+0000 7f0661570740 1 -- 172.21.15.42:0/828452231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e0179b0 msgr2=0x55cf7e021570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.888+0000 7f0661570740 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e0179b0 0x55cf7e021570 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f0650020ca0 tx=0x7f0650004a40 comp rx=0 tx=0).stop 2024-04-25T00:40:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.889+0000 7f0661570740 1 -- 172.21.15.42:0/828452231 shutdown_connections 2024-04-25T00:40:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.889+0000 7f0661570740 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55cf7e027d70 0x55cf7e090b70 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.889+0000 7f0661570740 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55cf7e0179b0 0x55cf7e021570 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.889+0000 7f0661570740 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7e012840 0x55cf7e020560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.889+0000 7f0661570740 1 --2- 172.21.15.42:0/828452231 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55cf7e011e50 0x55cf7e020020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.889+0000 7f0661570740 1 -- 172.21.15.42:0/828452231 >> 172.21.15.42:0/828452231 conn(0x55cf7e00d4c0 msgr2=0x55cf7e0103d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.889+0000 7f0641600640 0 client.26867 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.889+0000 7f0661570740 1 -- 172.21.15.42:0/828452231 shutdown_connections 2024-04-25T00:40:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.889+0000 7f0661570740 1 -- 172.21.15.42:0/828452231 wait complete. 2024-04-25T00:40:34.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.889+0000 7f0661570740 20 client.26867 trim_cache size 0 max 16384 2024-04-25T00:40:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:40:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:40:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:40:34.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:40:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.892+0000 7f065d200640 1 -- 172.21.15.42:0/3618694632 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f063c0a2c00 msgr2=0x7f063c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.892+0000 7f065d200640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f063c0a2c00 0x7f063c0a50c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f0650005fd0 tx=0x7f0650009500 comp rx=0 tx=0).stop 2024-04-25T00:40:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.892+0000 7f065d200640 1 -- 172.21.15.42:0/3618694632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06580ffe30 msgr2=0x7f065819dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:34.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.892+0000 7f065d200640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06580ffe30 0x7f065819dea0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f0648009230 tx=0x7f06480042f0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.893+0000 7f065d200640 1 -- 172.21.15.42:0/3618694632 shutdown_connections 2024-04-25T00:40:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.893+0000 7f065d200640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f063c0a2c00 0x7f063c0a50c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.893+0000 7f065d200640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0658105150 0x7f06581a5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.893+0000 7f065d200640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f06581027a0 0x7f065819e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.893+0000 7f065d200640 1 --2- 172.21.15.42:0/3618694632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06580ffe30 0x7f065819dea0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.893+0000 7f065d200640 1 -- 172.21.15.42:0/3618694632 >> 172.21.15.42:0/3618694632 conn(0x7f06580fba30 msgr2=0x7f0658103130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.893+0000 7f065d200640 1 -- 172.21.15.42:0/3618694632 shutdown_connections 2024-04-25T00:40:34.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:34.894+0000 7f065d200640 1 -- 172.21.15.42:0/3618694632 wait complete. 2024-04-25T00:40:35.798 INFO:tasks.workunit.client.0.smithi042.stdout: 10 315263 33.46 MB/sec execute 385 sec latency 229.794 ms 2024-04-25T00:40:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:35 smithi042 ceph-mon[28130]: pgmap v1045: 97 pgs: 97 active+clean; 677 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 436 op/s 2024-04-25T00:40:36.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:35 smithi136 ceph-mon[32923]: pgmap v1045: 97 pgs: 97 active+clean; 677 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 436 op/s 2024-04-25T00:40:36.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:35 smithi106 ceph-mon[32510]: pgmap v1045: 97 pgs: 97 active+clean; 677 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.3 MiB/s wr, 436 op/s 2024-04-25T00:40:36.798 INFO:tasks.workunit.client.0.smithi042.stdout: 10 315943 33.46 MB/sec execute 386 sec latency 56.635 ms 2024-04-25T00:40:37.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 316578 33.48 MB/sec execute 387 sec latency 63.323 ms 2024-04-25T00:40:38.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:37 smithi042 ceph-mon[28130]: pgmap v1046: 97 pgs: 97 active+clean; 691 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 609 op/s 2024-04-25T00:40:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:37 smithi136 ceph-mon[32923]: pgmap v1046: 97 pgs: 97 active+clean; 691 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 609 op/s 2024-04-25T00:40:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:37 smithi106 ceph-mon[32510]: pgmap v1046: 97 pgs: 97 active+clean; 691 MiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 4.5 MiB/s wr, 609 op/s 2024-04-25T00:40:38.799 INFO:tasks.workunit.client.0.smithi042.stdout: 10 316970 33.44 MB/sec execute 388 sec latency 424.895 ms 2024-04-25T00:40:39.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 317255 33.38 MB/sec execute 389 sec latency 585.323 ms 2024-04-25T00:40:40.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:39 smithi042 ceph-mon[28130]: pgmap v1047: 97 pgs: 97 active+clean; 688 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-04-25T00:40:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:39 smithi136 ceph-mon[32923]: pgmap v1047: 97 pgs: 97 active+clean; 688 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-04-25T00:40:40.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:39 smithi106 ceph-mon[32510]: pgmap v1047: 97 pgs: 97 active+clean; 688 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.1 MiB/s wr, 570 op/s 2024-04-25T00:40:40.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 317337 33.30 MB/sec execute 390 sec latency 879.722 ms 2024-04-25T00:40:41.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 317739 33.27 MB/sec execute 391 sec latency 114.299 ms 2024-04-25T00:40:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:41 smithi136 ceph-mon[32923]: pgmap v1048: 97 pgs: 97 active+clean; 710 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.5 MiB/s wr, 520 op/s 2024-04-25T00:40:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:41 smithi106 ceph-mon[32510]: pgmap v1048: 97 pgs: 97 active+clean; 710 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.5 MiB/s wr, 520 op/s 2024-04-25T00:40:42.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:41 smithi042 ceph-mon[28130]: pgmap v1048: 97 pgs: 97 active+clean; 710 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 5.5 MiB/s wr, 520 op/s 2024-04-25T00:40:42.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 318313 33.26 MB/sec execute 392 sec latency 200.850 ms 2024-04-25T00:40:43.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 319037 33.29 MB/sec execute 393 sec latency 59.898 ms 2024-04-25T00:40:44.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:43 smithi136 ceph-mon[32923]: pgmap v1049: 97 pgs: 97 active+clean; 725 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.9 MiB/s wr, 673 op/s 2024-04-25T00:40:44.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:43 smithi106 ceph-mon[32510]: pgmap v1049: 97 pgs: 97 active+clean; 725 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.9 MiB/s wr, 673 op/s 2024-04-25T00:40:44.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:43 smithi042 ceph-mon[28130]: pgmap v1049: 97 pgs: 97 active+clean; 725 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 9.9 MiB/s wr, 673 op/s 2024-04-25T00:40:44.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 319684 33.30 MB/sec execute 394 sec latency 55.544 ms 2024-04-25T00:40:45.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 319799 33.23 MB/sec execute 395 sec latency 719.182 ms 2024-04-25T00:40:46.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:45 smithi136 ceph-mon[32923]: pgmap v1050: 97 pgs: 97 active+clean; 727 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 665 op/s 2024-04-25T00:40:46.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:45 smithi106 ceph-mon[32510]: pgmap v1050: 97 pgs: 97 active+clean; 727 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 665 op/s 2024-04-25T00:40:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:45 smithi042 ceph-mon[28130]: pgmap v1050: 97 pgs: 97 active+clean; 727 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 11 MiB/s wr, 665 op/s 2024-04-25T00:40:46.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 320197 33.21 MB/sec execute 396 sec latency 120.599 ms 2024-04-25T00:40:47.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 320718 33.18 MB/sec execute 397 sec latency 184.380 ms 2024-04-25T00:40:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.090+0000 7efc32600640 1 -- 172.21.15.42:0/2148855221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc2c101660 msgr2=0x7efc2c101a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.090+0000 7efc32600640 1 --2- 172.21.15.42:0/2148855221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc2c101660 0x7efc2c101a80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efc24009ca0 tx=0x7efc2402a8e0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.090+0000 7efc32600640 1 -- 172.21.15.42:0/2148855221 shutdown_connections 2024-04-25T00:40:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.090+0000 7efc32600640 1 --2- 172.21.15.42:0/2148855221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc2c103210 0x7efc2c109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.090+0000 7efc32600640 1 --2- 172.21.15.42:0/2148855221 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc2c102850 0x7efc2c102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.090+0000 7efc32600640 1 --2- 172.21.15.42:0/2148855221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc2c101660 0x7efc2c101a80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.090+0000 7efc32600640 1 -- 172.21.15.42:0/2148855221 >> 172.21.15.42:0/2148855221 conn(0x7efc2c0fcbd0 msgr2=0x7efc2c0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.090+0000 7efc32600640 1 -- 172.21.15.42:0/2148855221 shutdown_connections 2024-04-25T00:40:48.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.090+0000 7efc32600640 1 -- 172.21.15.42:0/2148855221 wait complete. 2024-04-25T00:40:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.091+0000 7efc32600640 1 Processor -- start 2024-04-25T00:40:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc32600640 1 -- start start 2024-04-25T00:40:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc32600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc2c101660 0x7efc2c0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.092 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc32600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc2c102850 0x7efc2c073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc32600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc2c103210 0x7efc2c074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc32600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c10d240 con 0x7efc2c103210 2024-04-25T00:40:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc32600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c10d0c0 con 0x7efc2c101660 2024-04-25T00:40:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc32600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c10d3c0 con 0x7efc2c102850 2024-04-25T00:40:48.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc2be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc2c102850 0x7efc2c073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc31200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc2c101660 0x7efc2c0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc2be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc2c102850 0x7efc2c073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45902/0 (socket says 172.21.15.42:45902) 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.092+0000 7efc2be00640 1 -- 172.21.15.42:0/697818539 learned_addr learned my addr 172.21.15.42:0/697818539 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.093+0000 7efc31c00640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc2c103210 0x7efc2c074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.093+0000 7efc31200640 1 -- 172.21.15.42:0/697818539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc2c102850 msgr2=0x7efc2c073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.093+0000 7efc31200640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc2c102850 0x7efc2c073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.093+0000 7efc31200640 1 -- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc2c103210 msgr2=0x7efc2c074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.093+0000 7efc31200640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc2c103210 0x7efc2c074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.093+0000 7efc31200640 1 -- 172.21.15.42:0/697818539 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2c074900 con 0x7efc2c101660 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.093+0000 7efc31c00640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc2c103210 0x7efc2c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:40:48.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.093+0000 7efc31200640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc2c101660 0x7efc2c0754f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efc2402ae00 tx=0x7efc24003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.094+0000 7efc29600640 1 -- 172.21.15.42:0/697818539 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24004240 con 0x7efc2c101660 2024-04-25T00:40:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.094+0000 7efc29600640 1 -- 172.21.15.42:0/697818539 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc24032ea0 con 0x7efc2c101660 2024-04-25T00:40:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.094+0000 7efc32600640 1 -- 172.21.15.42:0/697818539 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2c1a1a30 con 0x7efc2c101660 2024-04-25T00:40:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.094+0000 7efc29600640 1 -- 172.21.15.42:0/697818539 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc24030e10 con 0x7efc2c101660 2024-04-25T00:40:48.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.094+0000 7efc32600640 1 -- 172.21.15.42:0/697818539 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc2c1a1fa0 con 0x7efc2c101660 2024-04-25T00:40:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.098+0000 7efc36a21740 1 Processor -- start 2024-04-25T00:40:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.098+0000 7efc36a21740 1 -- start start 2024-04-25T00:40:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.098+0000 7efc36a21740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9bd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.098+0000 7efc36a21740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 0x55e61a9c2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.099 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.098+0000 7efc36a21740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.098+0000 7efc36a21740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e61a91fab0 con 0x55e61a9bce50 2024-04-25T00:40:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.098+0000 7efc36a21740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e61a8c8640 con 0x55e61a9c29b0 2024-04-25T00:40:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.098+0000 7efc36a21740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e61a91ba70 con 0x55e61a9bd840 2024-04-25T00:40:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc29600640 1 -- 172.21.15.42:0/697818539 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc240305e0 con 0x7efc2c101660 2024-04-25T00:40:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc31c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9bd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc2be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc31c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9bd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:59998/0 (socket says 172.21.15.42:59998) 2024-04-25T00:40:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc31c00640 1 -- 172.21.15.42:0/3878953195 learned_addr learned my addr 172.21.15.42:0/3878953195 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:48.100 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc2be00640 1 --2- 172.21.15.42:0/3878953195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46122/0 (socket says 172.21.15.42:46122) 2024-04-25T00:40:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc31200640 1 --2- 172.21.15.42:0/3878953195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 0x55e61a9c2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc31c00640 1 -- 172.21.15.42:0/3878953195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 msgr2=0x55e61a9c2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc31c00640 1 --2- 172.21.15.42:0/3878953195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 0x55e61a9c2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.101 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc31c00640 1 -- 172.21.15.42:0/3878953195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 msgr2=0x55e61a9c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc31c00640 1 --2- 172.21.15.42:0/3878953195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.099+0000 7efc31c00640 1 -- 172.21.15.42:0/3878953195 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61a634210 con 0x55e61a9bce50 2024-04-25T00:40:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.100+0000 7efc31c00640 1 --2- 172.21.15.42:0/3878953195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9bd270 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00bcd0 tx=0x7efc1c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39c8e56281a7fff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.100+0000 7efc22000640 1 -- 172.21.15.42:0/3878953195 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c018be0 con 0x55e61a9bce50 2024-04-25T00:40:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.100+0000 7efc22000640 1 -- 172.21.15.42:0/3878953195 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c018d80 con 0x55e61a9bce50 2024-04-25T00:40:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.100+0000 7efc22000640 1 -- 172.21.15.42:0/3878953195 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c021480 con 0x55e61a9bce50 2024-04-25T00:40:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.101+0000 7efc29600640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc100a2c00 0x7efc100a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.102 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.101+0000 7efc29600640 1 -- 172.21.15.42:0/697818539 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7efc240f0500 con 0x7efc2c101660 2024-04-25T00:40:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.101+0000 7efc31200640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc100a2c00 0x7efc100a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.101+0000 7efc36a21740 1 -- 172.21.15.42:0/3878953195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 msgr2=0x55e61a9bd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.101+0000 7efc36a21740 1 --2- 172.21.15.42:0/3878953195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9bd270 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efc1c00bcd0 tx=0x7efc1c00f730 comp rx=0 tx=0).stop 2024-04-25T00:40:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.101+0000 7efc36a21740 1 -- 172.21.15.42:0/3878953195 shutdown_connections 2024-04-25T00:40:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.101+0000 7efc36a21740 1 --2- 172.21.15.42:0/3878953195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.103 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.101+0000 7efc36a21740 1 --2- 172.21.15.42:0/3878953195 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 0x55e61a9c2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.101+0000 7efc36a21740 1 --2- 172.21.15.42:0/3878953195 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9bd270 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.101+0000 7efc36a21740 1 -- 172.21.15.42:0/3878953195 >> 172.21.15.42:0/3878953195 conn(0x55e61a9b84c0 msgr2=0x55e61a9b88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.102+0000 7efc36a21740 1 -- 172.21.15.42:0/3878953195 shutdown_connections 2024-04-25T00:40:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.102+0000 7efc36a21740 1 -- 172.21.15.42:0/3878953195 wait complete. 2024-04-25T00:40:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.102+0000 7efc31200640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc100a2c00 0x7efc100a50c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efc24009ca0 tx=0x7efc240095c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.102+0000 7efc36a21740 1 -- start start 2024-04-25T00:40:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.102+0000 7efc36a21740 1 Processor -- start 2024-04-25T00:40:48.104 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc36a21740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc36a21740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9bd840 0x55e61a9c7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc36a21740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9c29b0 0x55e61a9ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc36a21740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e61a9c5960 con 0x55e61a9c29b0 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc36a21740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e61a8db990 con 0x55e61a9bd840 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc36a21740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e61a9c5ae0 con 0x55e61a9bce50 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc36a21740 10 client.0 _ll_register_callbacks cb 0x55e61a8dd950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc31c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc31c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45926/0 (socket says 172.21.15.42:45926) 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc31c00640 1 -- 172.21.15.42:0/2194484916 learned_addr learned my addr 172.21.15.42:0/2194484916 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.103+0000 7efc2be00640 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9bd840 0x55e61a9c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.104+0000 7efc31c00640 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9c29b0 0x55e61a9ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.104+0000 7efc31c00640 1 -- 172.21.15.42:0/2194484916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9bd840 msgr2=0x55e61a9c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.104+0000 7efc31c00640 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9bd840 0x55e61a9c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.104+0000 7efc31c00640 1 -- 172.21.15.42:0/2194484916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9c29b0 msgr2=0x55e61a9ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.104+0000 7efc31c00640 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9c29b0 0x55e61a9ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.104+0000 7efc31c00640 1 -- 172.21.15.42:0/2194484916 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61a700620 con 0x55e61a9bce50 2024-04-25T00:40:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.104+0000 7efc31c00640 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9c70f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0035f0 tx=0x7efc1c018f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=587f6de84a152015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.104+0000 7efc16a00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:40:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.104+0000 7efc16a00640 1 -- 172.21.15.42:0/2194484916 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c033060 con 0x55e61a9bce50 2024-04-25T00:40:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.104+0000 7efc36a21740 1 -- 172.21.15.42:0/2194484916 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc1c018d80 con 0x55e61a9bce50 2024-04-25T00:40:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.105+0000 7efc16a00640 1 -- 172.21.15.42:0/2194484916 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7efc1c00be40 con 0x55e61a9bce50 2024-04-25T00:40:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.105+0000 7efc16a00640 1 -- 172.21.15.42:0/2194484916 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efc1c003cd0 con 0x55e61a9bce50 2024-04-25T00:40:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.105+0000 7efc36a21740 10 client.26830 fetch_fsmap learned FSMap version 45 2024-04-25T00:40:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.105+0000 7efc36a21740 1 -- 172.21.15.42:0/2194484916 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e61a9d1650 con 0x55e61a9bce50 2024-04-25T00:40:48.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.105+0000 7efc16a00640 1 -- 172.21.15.42:0/2194484916 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c018d80 con 0x55e61a9bce50 2024-04-25T00:40:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.106+0000 7efc16a00640 1 -- 172.21.15.42:0/2194484916 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7efc1c021ac0 con 0x55e61a9bce50 2024-04-25T00:40:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.106+0000 7efc36a21740 10 client.26830 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:40:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.106+0000 7efc36a21740 10 client.26830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:40:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.106+0000 7efc36a21740 20 client.26830 populate_metadata read hostname 'smithi042' 2024-04-25T00:40:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.106+0000 7efc36a21740 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e61a9d2a50 0x55e61a9d4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.106+0000 7efc36a21740 4 client.26830 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:40:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.106+0000 7efc36a21740 1 -- 172.21.15.42:0/2194484916 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e61a92cff0 con 0x55e61a9d2a50 2024-04-25T00:40:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.106+0000 7efc2be00640 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e61a9d2a50 0x55e61a9d4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.107+0000 7efc2be00640 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e61a9d2a50 0x55e61a9d4e40 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=318c21aa07bc6f43 server_cookie=a1443683059f7d7a in_seq=0 out_seq=0 2024-04-25T00:40:48.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.107+0000 7efc16a00640 10 client.26830 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.109+0000 7efc16a00640 1 -- 172.21.15.42:0/2194484916 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e61a92cff0 con 0x55e61a9d2a50 2024-04-25T00:40:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc16a00640 10 client.26830 handle_command_reply: tid=0 2024-04-25T00:40:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 client.26830 shutdown 2024-04-25T00:40:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 -- 172.21.15.42:0/2194484916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 msgr2=0x55e61a9c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9c70f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0035f0 tx=0x7efc1c018f50 comp rx=0 tx=0).stop 2024-04-25T00:40:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 -- 172.21.15.42:0/2194484916 shutdown_connections 2024-04-25T00:40:48.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e61a9d2a50 0x55e61a9d4e40 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9c29b0 0x55e61a9ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9bd840 0x55e61a9c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 --2- 172.21.15.42:0/2194484916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9c70f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc16a00640 0 client.26830 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 -- 172.21.15.42:0/2194484916 >> 172.21.15.42:0/2194484916 conn(0x55e61a9b84c0 msgr2=0x55e61a9b7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 -- 172.21.15.42:0/2194484916 shutdown_connections 2024-04-25T00:40:48.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 1 -- 172.21.15.42:0/2194484916 wait complete. 2024-04-25T00:40:48.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.110+0000 7efc36a21740 20 client.26830 trim_cache size 0 max 16384 2024-04-25T00:40:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.133+0000 7efc36a21740 1 Processor -- start 2024-04-25T00:40:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.134+0000 7efc36a21740 1 -- start start 2024-04-25T00:40:48.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.134+0000 7efc36a21740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9b3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.134+0000 7efc36a21740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bd840 0x55e61a9ce600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.134+0000 7efc36a21740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.134+0000 7efc36a21740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1a90b0 con 0x55e61a9bd840 2024-04-25T00:40:48.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.134+0000 7efc36a21740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1a6980 con 0x55e61a9c29b0 2024-04-25T00:40:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.134+0000 7efc36a21740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1a47d0 con 0x55e61a9bce50 2024-04-25T00:40:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.135+0000 7efc31c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.135+0000 7efc2be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bd840 0x55e61a9ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.135+0000 7efc31c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45932/0 (socket says 172.21.15.42:45932) 2024-04-25T00:40:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.135+0000 7efc31c00640 1 -- 172.21.15.42:0/3687408044 learned_addr learned my addr 172.21.15.42:0/3687408044 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.135+0000 7efc31c00640 1 --2- 172.21.15.42:0/3687408044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.135+0000 7efc31c00640 1 -- 172.21.15.42:0/3687408044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 msgr2=0x55e61a9c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.135+0000 7efc31c00640 1 --2- 172.21.15.42:0/3687408044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.135+0000 7efc31c00640 1 -- 172.21.15.42:0/3687408044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bd840 msgr2=0x55e61a9ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.136+0000 7efc31c00640 1 --2- 172.21.15.42:0/3687408044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bd840 0x55e61a9ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.136+0000 7efc31c00640 1 -- 172.21.15.42:0/3687408044 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61a9d5470 con 0x55e61a9bce50 2024-04-25T00:40:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.136+0000 7efc2be00640 1 --2- 172.21.15.42:0/3687408044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bd840 0x55e61a9ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:40:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.136+0000 7efc31c00640 1 --2- 172.21.15.42:0/3687408044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9b3110 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efc1c039760 tx=0x7efc1c00b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34072d7a36598d94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.136+0000 7efc21600640 1 -- 172.21.15.42:0/3687408044 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c03e040 con 0x55e61a9bce50 2024-04-25T00:40:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.136+0000 7efc21600640 1 -- 172.21.15.42:0/3687408044 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c00bdc0 con 0x55e61a9bce50 2024-04-25T00:40:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.136+0000 7efc21600640 1 -- 172.21.15.42:0/3687408044 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c033040 con 0x55e61a9bce50 2024-04-25T00:40:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.137+0000 7efc36a21740 1 -- 172.21.15.42:0/3687408044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 msgr2=0x55e61a9b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.138 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.137+0000 7efc36a21740 1 --2- 172.21.15.42:0/3687408044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9b3110 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7efc1c039760 tx=0x7efc1c00b9f0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.137+0000 7efc36a21740 1 -- 172.21.15.42:0/3687408044 shutdown_connections 2024-04-25T00:40:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.137+0000 7efc36a21740 1 --2- 172.21.15.42:0/3687408044 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.138+0000 7efc36a21740 1 --2- 172.21.15.42:0/3687408044 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bd840 0x55e61a9ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.138+0000 7efc36a21740 1 --2- 172.21.15.42:0/3687408044 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bce50 0x55e61a9b3110 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.138+0000 7efc36a21740 1 -- 172.21.15.42:0/3687408044 >> 172.21.15.42:0/3687408044 conn(0x55e61a9b84c0 msgr2=0x55e61a9b88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.138+0000 7efc36a21740 1 -- 172.21.15.42:0/3687408044 shutdown_connections 2024-04-25T00:40:48.139 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.138+0000 7efc36a21740 1 -- 172.21.15.42:0/3687408044 wait complete. 2024-04-25T00:40:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.138+0000 7efc36a21740 1 -- start start 2024-04-25T00:40:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.139+0000 7efc36a21740 1 Processor -- start 2024-04-25T00:40:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.140+0000 7efc36a21740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.140+0000 7efc36a21740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 0x55e61a9cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.140+0000 7efc36a21740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.140+0000 7efc36a21740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1a4e40 con 0x55e61a9bce50 2024-04-25T00:40:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.140+0000 7efc36a21740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1a62b0 con 0x55e61a9c29b0 2024-04-25T00:40:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.140+0000 7efc36a21740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efc2c1a5510 con 0x55e61a9bd840 2024-04-25T00:40:48.140 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.140+0000 7efc36a21740 10 client.0 _ll_register_callbacks cb 0x7efc1c00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:40:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.140+0000 7efc31c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.141+0000 7efc31c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60018/0 (socket says 172.21.15.42:60018) 2024-04-25T00:40:48.141 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.141+0000 7efc31c00640 1 -- 172.21.15.42:0/1485314758 learned_addr learned my addr 172.21.15.42:0/1485314758 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:40:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.141+0000 7efc31c00640 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.141+0000 7efc2be00640 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 0x55e61a9cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.141+0000 7efc31c00640 1 -- 172.21.15.42:0/1485314758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 msgr2=0x55e61a9cbf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.141+0000 7efc31c00640 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 0x55e61a9cbf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.141+0000 7efc31c00640 1 -- 172.21.15.42:0/1485314758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 msgr2=0x55e61a9ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:40:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.141+0000 7efc31c00640 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.141+0000 7efc31c00640 1 -- 172.21.15.42:0/1485314758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61a9b82f0 con 0x55e61a9bce50 2024-04-25T00:40:48.142 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.141+0000 7efc2be00640 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 0x55e61a9cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:40:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.142+0000 7efc31c00640 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9cb9e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0059f0 tx=0x7efc1c003940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a00eabd5c9b89e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:40:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.142+0000 7efc16a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:40:48.143 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.142+0000 7efc16a00640 1 -- 172.21.15.42:0/1485314758 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc1c04d060 con 0x55e61a9bce50 2024-04-25T00:40:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.143+0000 7efc16a00640 1 -- 172.21.15.42:0/1485314758 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7efc1c004070 con 0x55e61a9bce50 2024-04-25T00:40:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.143+0000 7efc36a21740 1 -- 172.21.15.42:0/1485314758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc1c00bdc0 con 0x55e61a9bce50 2024-04-25T00:40:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.143+0000 7efc16a00640 1 -- 172.21.15.42:0/1485314758 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7efc1c03e070 con 0x55e61a9bce50 2024-04-25T00:40:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.143+0000 7efc36a21740 10 client.16602 fetch_fsmap learned FSMap version 45 2024-04-25T00:40:48.144 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.143+0000 7efc36a21740 1 -- 172.21.15.42:0/1485314758 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e61a9ce660 con 0x55e61a9bce50 2024-04-25T00:40:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.144+0000 7efc16a00640 1 -- 172.21.15.42:0/1485314758 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc1c00bdc0 con 0x55e61a9bce50 2024-04-25T00:40:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.144+0000 7efc16a00640 1 -- 172.21.15.42:0/1485314758 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7efc1c004680 con 0x55e61a9bce50 2024-04-25T00:40:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.144+0000 7efc36a21740 10 client.16602 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:40:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.144+0000 7efc36a21740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:40:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.144+0000 7efc36a21740 20 client.16602 populate_metadata read hostname 'smithi042' 2024-04-25T00:40:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.144+0000 7efc36a21740 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e61aa392b0 0x55e61aa3b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:40:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.144+0000 7efc36a21740 4 client.16602 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:40:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.144+0000 7efc36a21740 1 -- 172.21.15.42:0/1485314758 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e61a927340 con 0x55e61aa392b0 2024-04-25T00:40:48.145 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.145+0000 7efc2be00640 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e61aa392b0 0x55e61aa3b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:40:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.145+0000 7efc2be00640 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e61aa392b0 0x55e61aa3b6d0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53013a349f250c6a server_cookie=9bd07e9b3b1096d2 in_seq=0 out_seq=0 2024-04-25T00:40:48.146 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.145+0000 7efc16a00640 10 client.16602 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:47 smithi136 ceph-mon[32923]: pgmap v1051: 97 pgs: 97 active+clean; 725 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 843 op/s 2024-04-25T00:40:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:47 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:40:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:47 smithi106 ceph-mon[32510]: pgmap v1051: 97 pgs: 97 active+clean; 725 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 843 op/s 2024-04-25T00:40:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:47 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:40:48.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:47 smithi042 ceph-mon[28130]: pgmap v1051: 97 pgs: 97 active+clean; 725 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 15 MiB/s wr, 843 op/s 2024-04-25T00:40:48.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:47 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:40:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.407+0000 7efc16a00640 1 -- 172.21.15.42:0/1485314758 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e61a927340 con 0x55e61aa392b0 2024-04-25T00:40:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.407+0000 7efc16a00640 10 client.16602 handle_command_reply: tid=0 2024-04-25T00:40:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.407+0000 7efc36a21740 1 client.16602 shutdown 2024-04-25T00:40:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.407+0000 7efc36a21740 1 -- 172.21.15.42:0/1485314758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 msgr2=0x55e61a9cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.407+0000 7efc36a21740 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9cb9e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7efc1c0059f0 tx=0x7efc1c003940 comp rx=0 tx=0).stop 2024-04-25T00:40:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.408+0000 7efc36a21740 1 -- 172.21.15.42:0/1485314758 shutdown_connections 2024-04-25T00:40:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.408+0000 7efc36a21740 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e61aa392b0 0x55e61aa3b6d0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.408+0000 7efc36a21740 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e61a9c29b0 0x55e61a9ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.408+0000 7efc36a21740 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e61a9bd840 0x55e61a9cbf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.408+0000 7efc36a21740 1 --2- 172.21.15.42:0/1485314758 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e61a9bce50 0x55e61a9cb9e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.408+0000 7efc36a21740 1 -- 172.21.15.42:0/1485314758 >> 172.21.15.42:0/1485314758 conn(0x55e61a9b84c0 msgr2=0x55e61a9b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.408+0000 7efc16a00640 0 client.16602 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:40:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.408+0000 7efc36a21740 1 -- 172.21.15.42:0/1485314758 shutdown_connections 2024-04-25T00:40:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.408+0000 7efc36a21740 1 -- 172.21.15.42:0/1485314758 wait complete. 2024-04-25T00:40:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.408+0000 7efc36a21740 20 client.16602 trim_cache size 0 max 16384 2024-04-25T00:40:48.409 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:40:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:40:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:40:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:40:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.409+0000 7efc32600640 1 -- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc100a2c00 msgr2=0x7efc100a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.409+0000 7efc32600640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc100a2c00 0x7efc100a50c0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7efc24009ca0 tx=0x7efc240095c0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.409+0000 7efc32600640 1 -- 172.21.15.42:0/697818539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc2c101660 msgr2=0x7efc2c0754f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:40:48.410 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.409+0000 7efc32600640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc2c101660 0x7efc2c0754f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7efc2402ae00 tx=0x7efc24003c20 comp rx=0 tx=0).stop 2024-04-25T00:40:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.412+0000 7efc32600640 1 -- 172.21.15.42:0/697818539 shutdown_connections 2024-04-25T00:40:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.412+0000 7efc32600640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7efc100a2c00 0x7efc100a50c0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.412+0000 7efc32600640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7efc2c103210 0x7efc2c074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.412+0000 7efc32600640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efc2c102850 0x7efc2c073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.412+0000 7efc32600640 1 --2- 172.21.15.42:0/697818539 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efc2c101660 0x7efc2c0754f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:40:48.413 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.412+0000 7efc32600640 1 -- 172.21.15.42:0/697818539 >> 172.21.15.42:0/697818539 conn(0x7efc2c0fcbd0 msgr2=0x7efc2c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:40:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.414+0000 7efc32600640 1 -- 172.21.15.42:0/697818539 shutdown_connections 2024-04-25T00:40:48.414 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:40:48.414+0000 7efc32600640 1 -- 172.21.15.42:0/697818539 wait complete. 2024-04-25T00:40:48.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 320977 33.12 MB/sec execute 398 sec latency 335.986 ms 2024-04-25T00:40:49.765 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:49 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:49 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:49 smithi136 ceph-mon[32923]: pgmap v1052: 97 pgs: 97 active+clean; 721 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 701 op/s 2024-04-25T00:40:49.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:49 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.766 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:49 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 321497 33.12 MB/sec execute 399 sec latency 471.263 ms 2024-04-25T00:40:49.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:49 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:49 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:49 smithi106 ceph-mon[32510]: pgmap v1052: 97 pgs: 97 active+clean; 721 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 701 op/s 2024-04-25T00:40:49.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:49 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:49 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:49 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:49 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:49 smithi042 ceph-mon[28130]: pgmap v1052: 97 pgs: 97 active+clean; 721 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 15 MiB/s wr, 701 op/s 2024-04-25T00:40:49.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:49 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:49.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:49 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:50.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:40:50.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:40:50.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:50.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 322287 33.12 MB/sec execute 400 sec latency 96.277 ms 2024-04-25T00:40:50.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:40:50.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:40:50.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:50.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:40:50.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:40:50.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:40:51.763 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:51 smithi136 ceph-mon[32923]: pgmap v1053: 97 pgs: 97 active+clean; 699 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 804 op/s 2024-04-25T00:40:51.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 323053 33.18 MB/sec execute 401 sec latency 53.938 ms 2024-04-25T00:40:51.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:51 smithi106 ceph-mon[32510]: pgmap v1053: 97 pgs: 97 active+clean; 699 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 804 op/s 2024-04-25T00:40:51.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:51 smithi042 ceph-mon[28130]: pgmap v1053: 97 pgs: 97 active+clean; 699 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 17 MiB/s wr, 804 op/s 2024-04-25T00:40:52.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 323561 33.17 MB/sec execute 402 sec latency 57.216 ms 2024-04-25T00:40:53.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 324065 33.14 MB/sec execute 403 sec latency 57.956 ms 2024-04-25T00:40:53.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:53 smithi042 ceph-mon[28130]: pgmap v1054: 97 pgs: 97 active+clean; 662 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 957 op/s 2024-04-25T00:40:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:53 smithi136 ceph-mon[32923]: pgmap v1054: 97 pgs: 97 active+clean; 662 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 957 op/s 2024-04-25T00:40:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:53 smithi106 ceph-mon[32510]: pgmap v1054: 97 pgs: 97 active+clean; 662 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 17 MiB/s wr, 957 op/s 2024-04-25T00:40:54.800 INFO:tasks.workunit.client.0.smithi042.stdout: 10 324710 33.13 MB/sec execute 404 sec latency 70.564 ms 2024-04-25T00:40:55.801 INFO:tasks.workunit.client.0.smithi042.stdout: 10 325556 33.16 MB/sec execute 405 sec latency 75.905 ms 2024-04-25T00:40:55.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:55 smithi042 ceph-mon[28130]: pgmap v1055: 97 pgs: 97 active+clean; 661 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 13 MiB/s wr, 790 op/s 2024-04-25T00:40:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:55 smithi136 ceph-mon[32923]: pgmap v1055: 97 pgs: 97 active+clean; 661 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 13 MiB/s wr, 790 op/s 2024-04-25T00:40:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:55 smithi106 ceph-mon[32510]: pgmap v1055: 97 pgs: 97 active+clean; 661 MiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 13 MiB/s wr, 790 op/s 2024-04-25T00:40:56.801 INFO:tasks.workunit.client.0.smithi042.stdout: 10 326189 33.17 MB/sec execute 406 sec latency 89.945 ms 2024-04-25T00:40:57.801 INFO:tasks.workunit.client.0.smithi042.stdout: 10 326805 33.18 MB/sec execute 407 sec latency 81.682 ms 2024-04-25T00:40:57.879 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:57 smithi042 ceph-mon[28130]: pgmap v1056: 97 pgs: 97 active+clean; 667 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 835 op/s 2024-04-25T00:40:57.889 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:57 smithi106 ceph-mon[32510]: pgmap v1056: 97 pgs: 97 active+clean; 667 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 835 op/s 2024-04-25T00:40:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:57 smithi136 ceph-mon[32923]: pgmap v1056: 97 pgs: 97 active+clean; 667 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 12 MiB/s wr, 835 op/s 2024-04-25T00:40:58.803 INFO:tasks.workunit.client.0.smithi042.stdout: 10 327374 33.17 MB/sec execute 408 sec latency 58.449 ms 2024-04-25T00:40:59.803 INFO:tasks.workunit.client.0.smithi042.stdout: 10 328020 33.16 MB/sec execute 409 sec latency 61.685 ms 2024-04-25T00:40:59.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:40:59 smithi042 ceph-mon[28130]: pgmap v1057: 97 pgs: 97 active+clean; 670 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 682 op/s 2024-04-25T00:41:00.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:40:59 smithi136 ceph-mon[32923]: pgmap v1057: 97 pgs: 97 active+clean; 670 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 682 op/s 2024-04-25T00:41:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:40:59 smithi106 ceph-mon[32510]: pgmap v1057: 97 pgs: 97 active+clean; 670 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 8.5 MiB/s wr, 682 op/s 2024-04-25T00:41:00.803 INFO:tasks.workunit.client.0.smithi042.stdout: 10 328648 33.19 MB/sec execute 410 sec latency 55.840 ms 2024-04-25T00:41:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.656+0000 7f94e9e00640 1 -- 172.21.15.42:0/2548641565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94e41039d0 msgr2=0x7f94e4103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.656+0000 7f94e9e00640 1 --2- 172.21.15.42:0/2548641565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94e41039d0 0x7f94e4103df0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009be0 tx=0x7f94dc02a6c0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.657+0000 7f94e9e00640 1 -- 172.21.15.42:0/2548641565 shutdown_connections 2024-04-25T00:41:01.657 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.657+0000 7f94e9e00640 1 --2- 172.21.15.42:0/2548641565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94e4105580 0x7f94e410be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.657+0000 7f94e9e00640 1 --2- 172.21.15.42:0/2548641565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e4104bc0 0x7f94e4105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.657+0000 7f94e9e00640 1 --2- 172.21.15.42:0/2548641565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94e41039d0 0x7f94e4103df0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.657+0000 7f94e9e00640 1 -- 172.21.15.42:0/2548641565 >> 172.21.15.42:0/2548641565 conn(0x7f94e40fef60 msgr2=0x7f94e4101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.658+0000 7f94e9e00640 1 -- 172.21.15.42:0/2548641565 shutdown_connections 2024-04-25T00:41:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.658+0000 7f94e9e00640 1 -- 172.21.15.42:0/2548641565 wait complete. 2024-04-25T00:41:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.659+0000 7f94e9e00640 1 Processor -- start 2024-04-25T00:41:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.659+0000 7f94e9e00640 1 -- start start 2024-04-25T00:41:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.659+0000 7f94e9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e41039d0 0x7f94e4110b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.659+0000 7f94e9e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94e4104bc0 0x7f94e4111060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.659+0000 7f94e9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94e4105580 0x7f94e4114060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.659+0000 7f94e9e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f94e40779c0 con 0x7f94e4104bc0 2024-04-25T00:41:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.659+0000 7f94e9e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4077840 con 0x7f94e4105580 2024-04-25T00:41:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.659+0000 7f94e9e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94e4077b40 con 0x7f94e41039d0 2024-04-25T00:41:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.660+0000 7f94e2000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94e4104bc0 0x7f94e4111060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.660+0000 7f94e2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e41039d0 0x7f94e4110b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.660+0000 7f94e2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e41039d0 0x7f94e4110b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45974/0 (socket says 172.21.15.42:45974) 2024-04-25T00:41:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.660+0000 7f94e2a00640 1 -- 172.21.15.42:0/3371163169 learned_addr learned my addr 172.21.15.42:0/3371163169 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.660+0000 7f94e3400640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94e4105580 0x7f94e4114060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.660+0000 7f94e3400640 1 -- 172.21.15.42:0/3371163169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e41039d0 msgr2=0x7f94e4110b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.660+0000 7f94e3400640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e41039d0 0x7f94e4110b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.660+0000 7f94e3400640 1 -- 172.21.15.42:0/3371163169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94e4104bc0 msgr2=0x7f94e4111060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.660+0000 7f94e3400640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94e4104bc0 0x7f94e4111060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.660+0000 7f94e3400640 1 -- 172.21.15.42:0/3371163169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e41146d0 con 0x7f94e4105580 2024-04-25T00:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.661+0000 7f94e3400640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94e4105580 0x7f94e4114060 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f94d400c970 tx=0x7f94d400cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.661+0000 7f94e8a00640 1 -- 172.21.15.42:0/3371163169 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4007bb0 con 0x7f94e4105580 2024-04-25T00:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.661+0000 7f94e8a00640 1 -- 172.21.15.42:0/3371163169 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94d400ae40 con 0x7f94e4105580 2024-04-25T00:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.661+0000 7f94e8a00640 1 -- 172.21.15.42:0/3371163169 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4018660 con 0x7f94e4105580 2024-04-25T00:41:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.661+0000 7f94e9e00640 1 -- 172.21.15.42:0/3371163169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e41149c0 con 0x7f94e4105580 2024-04-25T00:41:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.661+0000 7f94e9e00640 1 -- 172.21.15.42:0/3371163169 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e4114ec0 con 0x7f94e4105580 2024-04-25T00:41:01.670 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.668+0000 7f94e8a00640 1 -- 172.21.15.42:0/3371163169 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f94d400f450 con 0x7f94e4105580 2024-04-25T00:41:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.670+0000 7f94ecd77740 1 Processor -- start 2024-04-25T00:41:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.671+0000 7f94ecd77740 1 -- start start 2024-04-25T00:41:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.671+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde120c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.671 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.671+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde12690 0x55e0cde17190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.671+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde17800 0x55e0cde19c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.671+0000 7f94ecd77740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdd74ab0 con 0x55e0cde17800 2024-04-25T00:41:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.671+0000 7f94ecd77740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdd1d640 con 0x55e0cde11ca0 2024-04-25T00:41:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.671+0000 7f94ecd77740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdd70a70 con 0x55e0cde12690 2024-04-25T00:41:01.672 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.671+0000 7f94e8a00640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94c80a2d80 0x7f94c80a5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.672+0000 7f94e8a00640 1 -- 172.21.15.42:0/3371163169 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f94d40ceee0 con 0x7f94e4105580 2024-04-25T00:41:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.672+0000 7f94e2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde12690 0x55e0cde17190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.672+0000 7f94e2a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde12690 0x55e0cde17190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:45982/0 (socket says 172.21.15.42:45982) 2024-04-25T00:41:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.672+0000 7f94e2a00640 1 -- 172.21.15.42:0/867189997 learned_addr learned my addr 172.21.15.42:0/867189997 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.672+0000 7f94e3400640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94c80a2d80 0x7f94c80a5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.673 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.672+0000 7f94e2000640 1 --2- 172.21.15.42:0/867189997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde17800 0x55e0cde19c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.673+0000 7f94e2000640 1 --2- 172.21.15.42:0/867189997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde120c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.673+0000 7f94e2a00640 1 -- 172.21.15.42:0/867189997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 msgr2=0x55e0cde120c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.673+0000 7f94e2a00640 1 --2- 172.21.15.42:0/867189997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde120c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.673+0000 7f94e2a00640 1 -- 172.21.15.42:0/867189997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde17800 msgr2=0x55e0cde19c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.673+0000 7f94e2a00640 1 --2- 172.21.15.42:0/867189997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde17800 0x55e0cde19c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.673+0000 7f94e2a00640 1 -- 172.21.15.42:0/867189997 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0cda89210 con 0x55e0cde12690 2024-04-25T00:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.673+0000 7f94e2000640 1 --2- 172.21.15.42:0/867189997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.673+0000 7f94e3400640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94c80a2d80 0x7f94c80a5240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f94d4097b70 tx=0x7f94d401cf90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.673+0000 7f94e2a00640 1 --2- 172.21.15.42:0/867189997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde12690 0x55e0cde17190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f94dc030ba0 tx=0x7f94dc030bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2fa9d4672ec8cef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.674 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.674+0000 7f94cfe00640 1 -- 172.21.15.42:0/867189997 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc041030 con 0x55e0cde12690 2024-04-25T00:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.674+0000 7f94cfe00640 1 -- 172.21.15.42:0/867189997 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc030db0 con 0x55e0cde12690 2024-04-25T00:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.674+0000 7f94cfe00640 1 -- 172.21.15.42:0/867189997 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc036040 con 0x55e0cde12690 2024-04-25T00:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.675+0000 7f94ecd77740 1 -- 172.21.15.42:0/867189997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde12690 msgr2=0x55e0cde17190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.675+0000 7f94ecd77740 1 --2- 172.21.15.42:0/867189997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde12690 0x55e0cde17190 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f94dc030ba0 tx=0x7f94dc030bd0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.675 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.675+0000 7f94ecd77740 1 -- 172.21.15.42:0/867189997 shutdown_connections 2024-04-25T00:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.675+0000 7f94ecd77740 1 --2- 172.21.15.42:0/867189997 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde17800 0x55e0cde19c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.675+0000 7f94ecd77740 1 --2- 172.21.15.42:0/867189997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde12690 0x55e0cde17190 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.675+0000 7f94ecd77740 1 --2- 172.21.15.42:0/867189997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde120c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.675+0000 7f94ecd77740 1 -- 172.21.15.42:0/867189997 >> 172.21.15.42:0/867189997 conn(0x55e0cde0d310 msgr2=0x55e0cde0d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.675+0000 7f94ecd77740 1 -- 172.21.15.42:0/867189997 shutdown_connections 2024-04-25T00:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.675+0000 7f94ecd77740 1 -- 172.21.15.42:0/867189997 wait complete. 2024-04-25T00:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.676+0000 7f94ecd77740 1 -- start start 2024-04-25T00:41:01.676 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.676+0000 7f94ecd77740 1 Processor -- start 2024-04-25T00:41:01.677 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde11ca0 0x55e0cde1beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 0x55e0cde1c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde17800 0x55e0cde23430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94ecd77740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55e0cde1a750 con 0x55e0cde12690 2024-04-25T00:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94ecd77740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55e0cdd30990 con 0x55e0cde17800 2024-04-25T00:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94ecd77740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e0cde1a8d0 con 0x55e0cde11ca0 2024-04-25T00:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94ecd77740 10 client.0 _ll_register_callbacks cb 0x55e0cdd32950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94e2000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde11ca0 0x55e0cde1beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94e2000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde17800 0x55e0cde23430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94e2000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde17800 0x55e0cde23430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49156/0 (socket says 172.21.15.42:49156) 2024-04-25T00:41:01.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.677+0000 7f94e2000640 1 -- 172.21.15.42:0/3240596607 learned_addr learned my addr 172.21.15.42:0/3240596607 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.678+0000 7f94e2a00640 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 0x55e0cde1c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.678+0000 7f94e2000640 1 -- 172.21.15.42:0/3240596607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde11ca0 msgr2=0x55e0cde1beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.678+0000 7f94e2000640 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde11ca0 0x55e0cde1beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.678+0000 7f94e2000640 1 -- 172.21.15.42:0/3240596607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 msgr2=0x55e0cde1c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.678+0000 7f94e2000640 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 0x55e0cde1c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.678+0000 7f94e2000640 1 -- 172.21.15.42:0/3240596607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0cdb55620 con 0x55e0cde17800 2024-04-25T00:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.678+0000 7f94e2000640 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde17800 0x55e0cde23430 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f94d0013e40 tx=0x7f94d00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=876064d77579c333 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.678+0000 7f94ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.679+0000 7f94ccc00640 1 -- 172.21.15.42:0/3240596607 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0013170 con 0x55e0cde17800 2024-04-25T00:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.679+0000 7f94ccc00640 1 -- 172.21.15.42:0/3240596607 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f94d000ec40 con 0x55e0cde17800 2024-04-25T00:41:01.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.679+0000 7f94ecd77740 1 -- 172.21.15.42:0/3240596607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc030db0 con 0x55e0cde17800 2024-04-25T00:41:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.679+0000 7f94ccc00640 1 -- 172.21.15.42:0/3240596607 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f94d0029050 con 0x55e0cde17800 2024-04-25T00:41:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.679+0000 7f94ecd77740 10 client.26894 fetch_fsmap learned FSMap version 45 2024-04-25T00:41:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.679+0000 7f94ecd77740 1 -- 172.21.15.42:0/3240596607 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e0cde263f0 con 0x55e0cde17800 2024-04-25T00:41:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.679+0000 7f94ccc00640 1 -- 172.21.15.42:0/3240596607 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc030db0 con 0x55e0cde17800 2024-04-25T00:41:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.680+0000 7f94ccc00640 1 -- 172.21.15.42:0/3240596607 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f94d00047a0 con 0x55e0cde17800 2024-04-25T00:41:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.680+0000 7f94ecd77740 10 client.26894 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:41:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.680+0000 7f94ecd77740 10 client.26894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:41:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.680+0000 7f94ecd77740 20 client.26894 populate_metadata read hostname 'smithi042' 2024-04-25T00:41:01.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.680+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e0cde27a10 0x55e0cde29e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.680+0000 7f94ecd77740 4 client.26894 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.680+0000 7f94ecd77740 1 -- 172.21.15.42:0/3240596607 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0cdd81ff0 con 0x55e0cde27a10 2024-04-25T00:41:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.680+0000 7f94e2a00640 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e0cde27a10 0x55e0cde29e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.682+0000 7f94e2a00640 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e0cde27a10 0x55e0cde29e60 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc06b09946516d86 server_cookie=2331e3c4b3fb540 in_seq=0 out_seq=0 2024-04-25T00:41:01.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.683+0000 7f94ccc00640 10 client.26894 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.684+0000 7f94ccc00640 1 -- 172.21.15.42:0/3240596607 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0cdd81ff0 con 0x55e0cde27a10 2024-04-25T00:41:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.684+0000 7f94ccc00640 10 client.26894 handle_command_reply: tid=0 2024-04-25T00:41:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.684+0000 7f94ecd77740 1 client.26894 shutdown 2024-04-25T00:41:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.685+0000 7f94ecd77740 1 -- 172.21.15.42:0/3240596607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde17800 msgr2=0x55e0cde23430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.685+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde17800 0x55e0cde23430 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f94d0013e40 tx=0x7f94d00046c0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.685+0000 7f94ecd77740 1 -- 172.21.15.42:0/3240596607 shutdown_connections 2024-04-25T00:41:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.685+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e0cde27a10 0x55e0cde29e60 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.685+0000 7f94ccc00640 0 client.26894 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.685+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde17800 0x55e0cde23430 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.685+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 0x55e0cde1c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.685+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3240596607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde11ca0 0x55e0cde1beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.685+0000 7f94ecd77740 1 -- 172.21.15.42:0/3240596607 >> 172.21.15.42:0/3240596607 conn(0x55e0cde0d310 msgr2=0x55e0cde0c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.686+0000 7f94ecd77740 1 -- 172.21.15.42:0/3240596607 shutdown_connections 2024-04-25T00:41:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.686+0000 7f94ecd77740 1 -- 172.21.15.42:0/3240596607 wait complete. 2024-04-25T00:41:01.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.686+0000 7f94ecd77740 20 client.26894 trim_cache size 0 max 16384 2024-04-25T00:41:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.708+0000 7f94ecd77740 1 Processor -- start 2024-04-25T00:41:01.710 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94ecd77740 1 -- start start 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde11ca0 0x55e0cde07fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde12690 0x55e0cde233f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde17800 0x55e0cde10220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94ecd77740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41ba200 con 0x55e0cde11ca0 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94ecd77740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41b7ad0 con 0x55e0cde12690 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94ecd77740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41b5920 con 0x55e0cde17800 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94e2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde12690 0x55e0cde233f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94e2a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde12690 0x55e0cde233f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49166/0 (socket says 172.21.15.42:49166) 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94e2a00640 1 -- 172.21.15.42:0/2296842215 learned_addr learned my addr 172.21.15.42:0/2296842215 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.709+0000 7f94e2000640 1 --2- 172.21.15.42:0/2296842215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde11ca0 0x55e0cde07fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.710+0000 7f94e2000640 1 --2- 172.21.15.42:0/2296842215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde17800 0x55e0cde10220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.710+0000 7f94e2a00640 1 -- 172.21.15.42:0/2296842215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde17800 msgr2=0x55e0cde10220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.710+0000 7f94e2a00640 1 --2- 172.21.15.42:0/2296842215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde17800 0x55e0cde10220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.711 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.710+0000 7f94e2a00640 1 -- 172.21.15.42:0/2296842215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde11ca0 msgr2=0x55e0cde07fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.710+0000 7f94e2a00640 1 --2- 172.21.15.42:0/2296842215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde11ca0 0x55e0cde07fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.710+0000 7f94e2a00640 1 -- 172.21.15.42:0/2296842215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0cde0b890 con 0x55e0cde12690 2024-04-25T00:41:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.710+0000 7f94e2000640 1 --2- 172.21.15.42:0/2296842215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde11ca0 0x55e0cde07fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:01.712 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.710+0000 7f94e2a00640 1 --2- 172.21.15.42:0/2296842215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde12690 0x55e0cde233f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f94dc02acb0 tx=0x7f94dc005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2b4b0fdfee49d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.710+0000 7f94cf400640 1 -- 172.21.15.42:0/2296842215 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc041040 con 0x55e0cde12690 2024-04-25T00:41:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.711+0000 7f94cf400640 1 -- 172.21.15.42:0/2296842215 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc003970 con 0x55e0cde12690 2024-04-25T00:41:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.711+0000 7f94cf400640 1 -- 172.21.15.42:0/2296842215 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc036040 con 0x55e0cde12690 2024-04-25T00:41:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.711+0000 7f94ecd77740 1 -- 172.21.15.42:0/2296842215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde12690 msgr2=0x55e0cde233f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.711+0000 7f94ecd77740 1 --2- 172.21.15.42:0/2296842215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde12690 0x55e0cde233f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f94dc02acb0 tx=0x7f94dc005fd0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.712+0000 7f94ecd77740 1 -- 172.21.15.42:0/2296842215 shutdown_connections 2024-04-25T00:41:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.712+0000 7f94ecd77740 1 --2- 172.21.15.42:0/2296842215 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde17800 0x55e0cde10220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.712+0000 7f94ecd77740 1 --2- 172.21.15.42:0/2296842215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde12690 0x55e0cde233f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.712+0000 7f94ecd77740 1 --2- 172.21.15.42:0/2296842215 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde11ca0 0x55e0cde07fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.713 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.712+0000 7f94ecd77740 1 -- 172.21.15.42:0/2296842215 >> 172.21.15.42:0/2296842215 conn(0x55e0cde0d310 msgr2=0x55e0cde16f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.712+0000 7f94ecd77740 1 -- 172.21.15.42:0/2296842215 shutdown_connections 2024-04-25T00:41:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.712+0000 7f94ecd77740 1 -- 172.21.15.42:0/2296842215 wait complete. 2024-04-25T00:41:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.712+0000 7f94ecd77740 1 -- start start 2024-04-25T00:41:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.712+0000 7f94ecd77740 1 Processor -- start 2024-04-25T00:41:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.713+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde21260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.713+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 0x55e0cde217a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.713+0000 7f94ecd77740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde17800 0x55e0cde227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.713+0000 7f94ecd77740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41b5f90 con 0x55e0cde12690 2024-04-25T00:41:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.713+0000 7f94ecd77740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41b7400 con 0x55e0cde11ca0 2024-04-25T00:41:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.713+0000 7f94ecd77740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94e41b6660 con 0x55e0cde17800 2024-04-25T00:41:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.713+0000 7f94ecd77740 10 client.0 _ll_register_callbacks cb 0x7f94d0009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.713+0000 7f94e2000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde21260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.714+0000 7f94e2000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde21260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49168/0 (socket says 172.21.15.42:49168) 2024-04-25T00:41:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.714+0000 7f94e2000640 1 -- 172.21.15.42:0/3499137381 learned_addr learned my addr 172.21.15.42:0/3499137381 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.714+0000 7f94e2000640 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde17800 0x55e0cde227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.714+0000 7f94e2a00640 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 0x55e0cde217a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.714+0000 7f94e2000640 1 -- 172.21.15.42:0/3499137381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde17800 msgr2=0x55e0cde227b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.714+0000 7f94e2000640 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde17800 0x55e0cde227b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.715+0000 7f94e2000640 1 -- 172.21.15.42:0/3499137381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 msgr2=0x55e0cde217a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.715+0000 7f94e2000640 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 0x55e0cde217a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.715+0000 7f94e2000640 1 -- 172.21.15.42:0/3499137381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0cde22da0 con 0x55e0cde11ca0 2024-04-25T00:41:01.716 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.715+0000 7f94e2a00640 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 0x55e0cde217a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.715+0000 7f94e2000640 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde21260 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f94d0004c90 tx=0x7f94d0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a95910c18776b2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.715+0000 7f94ccc00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:41:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.715+0000 7f94ccc00640 1 -- 172.21.15.42:0/3499137381 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d0013170 con 0x55e0cde11ca0 2024-04-25T00:41:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.715+0000 7f94ccc00640 1 -- 172.21.15.42:0/3499137381 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f94d001c040 con 0x55e0cde11ca0 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.715+0000 7f94ecd77740 1 -- 172.21.15.42:0/3499137381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc003970 con 0x55e0cde11ca0 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ccc00640 1 -- 172.21.15.42:0/3499137381 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f94d0004730 con 0x55e0cde11ca0 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ecd77740 10 client.26900 fetch_fsmap learned FSMap version 45 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ecd77740 1 -- 172.21.15.42:0/3499137381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e0cde8e7e0 con 0x55e0cde11ca0 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ccc00640 1 -- 172.21.15.42:0/3499137381 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f94dc003970 con 0x55e0cde11ca0 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ccc00640 1 -- 172.21.15.42:0/3499137381 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f94d003a020 con 0x55e0cde11ca0 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ecd77740 10 client.26900 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ecd77740 10 client.26900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ecd77740 20 client.26900 populate_metadata read hostname 'smithi042' 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e0cde8f0d0 0x55e0cde914f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ecd77740 4 client.26900 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.716+0000 7f94ecd77740 1 -- 172.21.15.42:0/3499137381 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0cdd7c340 con 0x55e0cde8f0d0 2024-04-25T00:41:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.717+0000 7f94e2a00640 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e0cde8f0d0 0x55e0cde914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.717+0000 7f94e2a00640 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e0cde8f0d0 0x55e0cde914f0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beca7d72c4bced23 server_cookie=45531cff91de6e10 in_seq=0 out_seq=0 2024-04-25T00:41:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:01.717+0000 7f94ccc00640 10 client.26900 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:01.803 INFO:tasks.workunit.client.0.smithi042.stdout: 10 329226 33.16 MB/sec execute 411 sec latency 74.326 ms 2024-04-25T00:41:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:01 smithi042 ceph-mon[28130]: pgmap v1058: 97 pgs: 97 active+clean; 665 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.7 MiB/s wr, 675 op/s 2024-04-25T00:41:02.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:01 smithi136 ceph-mon[32923]: pgmap v1058: 97 pgs: 97 active+clean; 665 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.7 MiB/s wr, 675 op/s 2024-04-25T00:41:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.073+0000 7f94ccc00640 1 -- 172.21.15.42:0/3499137381 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e0cdd7c340 con 0x55e0cde8f0d0 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.073+0000 7f94ccc00640 10 client.26900 handle_command_reply: tid=0 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.073+0000 7f94ecd77740 1 client.26900 shutdown 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 1 -- 172.21.15.42:0/3499137381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 msgr2=0x55e0cde21260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde21260 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f94d0004c90 tx=0x7f94d0009260 comp rx=0 tx=0).stop 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 1 -- 172.21.15.42:0/3499137381 shutdown_connections 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55e0cde8f0d0 0x55e0cde914f0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0cde17800 0x55e0cde227b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ccc00640 0 client.26900 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55e0cde12690 0x55e0cde217a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 1 --2- 172.21.15.42:0/3499137381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55e0cde11ca0 0x55e0cde21260 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 1 -- 172.21.15.42:0/3499137381 >> 172.21.15.42:0/3499137381 conn(0x55e0cde0d310 msgr2=0x55e0cde07fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 1 -- 172.21.15.42:0/3499137381 shutdown_connections 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 1 -- 172.21.15.42:0/3499137381 wait complete. 2024-04-25T00:41:02.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.074+0000 7f94ecd77740 20 client.26900 trim_cache size 0 max 16384 2024-04-25T00:41:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:41:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:41:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:41:02.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 -- 172.21.15.42:0/3371163169 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94c80a2d80 msgr2=0x7f94c80a5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94c80a2d80 0x7f94c80a5240 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f94d4097b70 tx=0x7f94d401cf90 comp rx=0 tx=0).stop 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 -- 172.21.15.42:0/3371163169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94e4105580 msgr2=0x7f94e4114060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94e4105580 0x7f94e4114060 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f94d400c970 tx=0x7f94d400cd70 comp rx=0 tx=0).stop 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 -- 172.21.15.42:0/3371163169 shutdown_connections 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94c80a2d80 0x7f94c80a5240 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94e4105580 0x7f94e4114060 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f94e4104bc0 0x7f94e4111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 --2- 172.21.15.42:0/3371163169 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e41039d0 0x7f94e4110b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 -- 172.21.15.42:0/3371163169 >> 172.21.15.42:0/3371163169 conn(0x7f94e40fef60 msgr2=0x7f94e40ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.075+0000 7f94e9e00640 1 -- 172.21.15.42:0/3371163169 shutdown_connections 2024-04-25T00:41:02.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:02.076+0000 7f94e9e00640 1 -- 172.21.15.42:0/3371163169 wait complete. 2024-04-25T00:41:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:01 smithi106 ceph-mon[32510]: pgmap v1058: 97 pgs: 97 active+clean; 665 MiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 7.7 MiB/s wr, 675 op/s 2024-04-25T00:41:02.803 INFO:tasks.workunit.client.0.smithi042.stdout: 10 329616 33.14 MB/sec execute 412 sec latency 254.154 ms 2024-04-25T00:41:03.803 INFO:tasks.workunit.client.0.smithi042.stdout: 10 329668 33.06 MB/sec execute 413 sec latency 1008.309 ms 2024-04-25T00:41:04.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:03 smithi136 ceph-mon[32923]: pgmap v1059: 97 pgs: 97 active+clean; 668 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.8 MiB/s wr, 618 op/s 2024-04-25T00:41:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:03 smithi106 ceph-mon[32510]: pgmap v1059: 97 pgs: 97 active+clean; 668 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.8 MiB/s wr, 618 op/s 2024-04-25T00:41:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:03 smithi042 ceph-mon[28130]: pgmap v1059: 97 pgs: 97 active+clean; 668 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 5.8 MiB/s wr, 618 op/s 2024-04-25T00:41:04.804 INFO:tasks.workunit.client.0.smithi042.stdout: 10 330185 33.05 MB/sec execute 414 sec latency 612.662 ms 2024-04-25T00:41:05.804 INFO:tasks.workunit.client.0.smithi042.stdout: 10 330871 33.04 MB/sec execute 415 sec latency 119.382 ms 2024-04-25T00:41:06.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:05 smithi136 ceph-mon[32923]: pgmap v1060: 97 pgs: 97 active+clean; 663 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-04-25T00:41:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:05 smithi106 ceph-mon[32510]: pgmap v1060: 97 pgs: 97 active+clean; 663 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-04-25T00:41:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:05 smithi042 ceph-mon[28130]: pgmap v1060: 97 pgs: 97 active+clean; 663 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.9 MiB/s wr, 429 op/s 2024-04-25T00:41:06.804 INFO:tasks.workunit.client.0.smithi042.stdout: 10 331554 33.09 MB/sec execute 416 sec latency 72.111 ms 2024-04-25T00:41:07.804 INFO:tasks.workunit.client.0.smithi042.stdout: 10 332065 33.07 MB/sec execute 417 sec latency 73.652 ms 2024-04-25T00:41:08.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:07 smithi136 ceph-mon[32923]: pgmap v1061: 97 pgs: 97 active+clean; 665 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.3 MiB/s wr, 704 op/s 2024-04-25T00:41:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:07 smithi106 ceph-mon[32510]: pgmap v1061: 97 pgs: 97 active+clean; 665 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.3 MiB/s wr, 704 op/s 2024-04-25T00:41:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:07 smithi042 ceph-mon[28130]: pgmap v1061: 97 pgs: 97 active+clean; 665 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.3 MiB/s wr, 704 op/s 2024-04-25T00:41:08.804 INFO:tasks.workunit.client.0.smithi042.stdout: 10 332526 33.04 MB/sec execute 418 sec latency 121.414 ms 2024-04-25T00:41:09.804 INFO:tasks.workunit.client.0.smithi042.stdout: 10 333262 33.03 MB/sec execute 419 sec latency 74.569 ms 2024-04-25T00:41:10.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:09 smithi136 ceph-mon[32923]: pgmap v1062: 97 pgs: 97 active+clean; 664 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.6 MiB/s wr, 645 op/s 2024-04-25T00:41:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:09 smithi106 ceph-mon[32510]: pgmap v1062: 97 pgs: 97 active+clean; 664 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.6 MiB/s wr, 645 op/s 2024-04-25T00:41:10.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:09 smithi042 ceph-mon[28130]: pgmap v1062: 97 pgs: 97 active+clean; 664 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 9.6 MiB/s wr, 645 op/s 2024-04-25T00:41:10.804 INFO:tasks.workunit.client.0.smithi042.stdout: 10 334111 33.07 MB/sec execute 420 sec latency 44.061 ms 2024-04-25T00:41:11.804 INFO:tasks.workunit.client.0.smithi042.stdout: 10 334846 33.11 MB/sec execute 421 sec latency 54.004 ms 2024-04-25T00:41:12.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:11 smithi136 ceph-mon[32923]: pgmap v1063: 97 pgs: 97 active+clean; 659 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.9 MiB/s wr, 685 op/s 2024-04-25T00:41:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:11 smithi106 ceph-mon[32510]: pgmap v1063: 97 pgs: 97 active+clean; 659 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.9 MiB/s wr, 685 op/s 2024-04-25T00:41:12.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:11 smithi042 ceph-mon[28130]: pgmap v1063: 97 pgs: 97 active+clean; 659 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 9.9 MiB/s wr, 685 op/s 2024-04-25T00:41:12.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 335367 33.10 MB/sec execute 422 sec latency 73.871 ms 2024-04-25T00:41:13.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 335909 33.07 MB/sec execute 423 sec latency 74.140 ms 2024-04-25T00:41:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:13 smithi136 ceph-mon[32923]: pgmap v1064: 97 pgs: 97 active+clean; 666 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 732 op/s 2024-04-25T00:41:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:13 smithi106 ceph-mon[32510]: pgmap v1064: 97 pgs: 97 active+clean; 666 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 732 op/s 2024-04-25T00:41:14.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:13 smithi042 ceph-mon[28130]: pgmap v1064: 97 pgs: 97 active+clean; 666 MiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 10 MiB/s wr, 732 op/s 2024-04-25T00:41:14.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 336625 33.09 MB/sec execute 424 sec latency 66.695 ms 2024-04-25T00:41:15.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.245+0000 7f1bd4a00640 1 -- 172.21.15.42:0/1667727565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd01017a0 msgr2=0x7f1bd0101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.245+0000 7f1bd4a00640 1 --2- 172.21.15.42:0/1667727565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd01017a0 0x7f1bd0101bc0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1bc0009ca0 tx=0x7f1bc002a8c0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.245+0000 7f1bd4a00640 1 -- 172.21.15.42:0/1667727565 shutdown_connections 2024-04-25T00:41:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.245+0000 7f1bd4a00640 1 --2- 172.21.15.42:0/1667727565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd0103350 0x7f1bd0109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.245+0000 7f1bd4a00640 1 --2- 172.21.15.42:0/1667727565 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd0102990 0x7f1bd0102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.245+0000 7f1bd4a00640 1 --2- 172.21.15.42:0/1667727565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd01017a0 0x7f1bd0101bc0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.245+0000 7f1bd4a00640 1 -- 172.21.15.42:0/1667727565 >> 172.21.15.42:0/1667727565 conn(0x7f1bd00fcd30 msgr2=0x7f1bd00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.245+0000 7f1bd4a00640 1 -- 172.21.15.42:0/1667727565 shutdown_connections 2024-04-25T00:41:15.246 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.245+0000 7f1bd4a00640 1 -- 172.21.15.42:0/1667727565 wait complete. 2024-04-25T00:41:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.246+0000 7f1bd4a00640 1 Processor -- start 2024-04-25T00:41:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.247+0000 7f1bd4a00640 1 -- start start 2024-04-25T00:41:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.247+0000 7f1bd4a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd01017a0 0x7f1bd0199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.247 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.247+0000 7f1bd4a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd0102990 0x7f1bd019a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.247+0000 7f1bd4a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd0103350 0x7f1bd01a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.247+0000 7f1bd4a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd010d470 con 0x7f1bd01017a0 2024-04-25T00:41:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.247+0000 7f1bd4a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd010d2f0 con 0x7f1bd0102990 2024-04-25T00:41:15.248 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.247+0000 7f1bd4a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd010d5f0 con 0x7f1bd0103350 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.248+0000 7f1bcde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd01017a0 0x7f1bd0199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.248+0000 7f1bcd400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd0102990 0x7f1bd019a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.248+0000 7f1bcde00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd01017a0 0x7f1bd0199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:42332/0 (socket says 172.21.15.42:42332) 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.248+0000 7f1bce800640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd0103350 0x7f1bd01a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.248+0000 7f1bcde00640 1 -- 172.21.15.42:0/3857151143 learned_addr learned my addr 172.21.15.42:0/3857151143 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.248+0000 7f1bcd400640 1 -- 172.21.15.42:0/3857151143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd0103350 msgr2=0x7f1bd01a11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.248+0000 7f1bcd400640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd0103350 0x7f1bd01a11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.248+0000 7f1bcd400640 1 -- 172.21.15.42:0/3857151143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd01017a0 msgr2=0x7f1bd0199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.248+0000 7f1bcd400640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd01017a0 0x7f1bd0199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.248+0000 7f1bcd400640 1 -- 172.21.15.42:0/3857151143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bd01a1870 con 0x7f1bd0102990 2024-04-25T00:41:15.249 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.249+0000 7f1bcd400640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd0102990 0x7f1bd019a1a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e9f0 tx=0x7f1bbc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.249+0000 7f1bc6a00640 1 -- 172.21.15.42:0/3857151143 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc00ccc0 con 0x7f1bd0102990 2024-04-25T00:41:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.249+0000 7f1bd4a00640 1 -- 172.21.15.42:0/3857151143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bd01a1b60 con 0x7f1bd0102990 2024-04-25T00:41:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.249+0000 7f1bc6a00640 1 -- 172.21.15.42:0/3857151143 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc010910 con 0x7f1bd0102990 2024-04-25T00:41:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.249+0000 7f1bc6a00640 1 -- 172.21.15.42:0/3857151143 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc0189c0 con 0x7f1bd0102990 2024-04-25T00:41:15.250 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.249+0000 7f1bd4a00640 1 -- 172.21.15.42:0/3857151143 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bd01a2100 con 0x7f1bd0102990 2024-04-25T00:41:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.253+0000 7f1bd79a0740 1 Processor -- start 2024-04-25T00:41:15.254 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.254+0000 7f1bd79a0740 1 -- start start 2024-04-25T00:41:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.254+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf14b0 0x5579e4bf18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.254+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 0x5579e4bf69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.254+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4bf94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.255 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.254+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5579e4b4bab0 con 0x5579e4bf1ea0 2024-04-25T00:41:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.254+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5579e4af4640 con 0x5579e4bf14b0 2024-04-25T00:41:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.254+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5579e4b47a70 con 0x5579e4bf7010 2024-04-25T00:41:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.255+0000 7f1bce800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf14b0 0x5579e4bf18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.255+0000 7f1bce800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf14b0 0x5579e4bf18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:41134/0 (socket says 172.21.15.42:41134) 2024-04-25T00:41:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.255+0000 7f1bce800640 1 -- 172.21.15.42:0/461530481 learned_addr learned my addr 172.21.15.42:0/461530481 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.255+0000 7f1bc6a00640 1 -- 172.21.15.42:0/3857151143 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1bbc010ac0 con 0x7f1bd0102990 2024-04-25T00:41:15.256 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.255+0000 7f1bcd400640 1 --2- 172.21.15.42:0/461530481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4bf94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.256+0000 7f1bcd400640 1 -- 172.21.15.42:0/461530481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf14b0 msgr2=0x5579e4bf18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.256+0000 7f1bcde00640 1 --2- 172.21.15.42:0/461530481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 0x5579e4bf69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.257+0000 7f1bcd400640 1 --2- 172.21.15.42:0/461530481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf14b0 0x5579e4bf18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.257+0000 7f1bcd400640 1 -- 172.21.15.42:0/461530481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 msgr2=0x5579e4bf69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.257 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.257+0000 7f1bcd400640 1 --2- 172.21.15.42:0/461530481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 0x5579e4bf69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.257+0000 7f1bcd400640 1 -- 172.21.15.42:0/461530481 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e4860210 con 0x5579e4bf7010 2024-04-25T00:41:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.257+0000 7f1bcde00640 1 --2- 172.21.15.42:0/461530481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 0x5579e4bf69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.257+0000 7f1bc6a00640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1bb40a2cb0 0x7f1bb40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.257+0000 7f1bcd400640 1 --2- 172.21.15.42:0/461530481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4bf94a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0cced0 tx=0x7f1bbc0d79f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e014a5335694b9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:15.258 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.258+0000 7f1bc6a00640 1 -- 172.21.15.42:0/3857151143 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f1bbc0cf3d0 con 0x7f1bd0102990 2024-04-25T00:41:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.258+0000 7f1bbb000640 1 -- 172.21.15.42:0/461530481 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc097f20 con 0x5579e4bf7010 2024-04-25T00:41:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.258+0000 7f1bbb000640 1 -- 172.21.15.42:0/461530481 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc09bd10 con 0x5579e4bf7010 2024-04-25T00:41:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.258+0000 7f1bbb000640 1 -- 172.21.15.42:0/461530481 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc099e50 con 0x5579e4bf7010 2024-04-25T00:41:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.258+0000 7f1bce800640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1bb40a2cb0 0x7f1bb40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.259+0000 7f1bd79a0740 1 -- 172.21.15.42:0/461530481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 msgr2=0x5579e4bf94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.259+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/461530481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4bf94a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc0cced0 tx=0x7f1bbc0d79f0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.259+0000 7f1bce800640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1bb40a2cb0 0x7f1bb40a5170 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5579e4bf2700 tx=0x7f1bc8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.259+0000 7f1bd79a0740 1 -- 172.21.15.42:0/461530481 shutdown_connections 2024-04-25T00:41:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.259+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/461530481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4bf94a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.259+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/461530481 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 0x5579e4bf69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.259+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/461530481 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf14b0 0x5579e4bf18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.259+0000 7f1bd79a0740 1 -- 172.21.15.42:0/461530481 >> 172.21.15.42:0/461530481 conn(0x5579e4be4310 msgr2=0x5579e4be4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:15.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.260+0000 7f1bd79a0740 1 -- 172.21.15.42:0/461530481 shutdown_connections 2024-04-25T00:41:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.260+0000 7f1bd79a0740 1 -- 172.21.15.42:0/461530481 wait complete. 2024-04-25T00:41:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.261+0000 7f1bd79a0740 1 -- start start 2024-04-25T00:41:15.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.261+0000 7f1bd79a0740 1 Processor -- start 2024-04-25T00:41:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf14b0 0x5579e4bfb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 0x5579e4bfbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 0x5579e4c02ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5579e4bf9fc0 con 0x5579e4bf1ea0 2024-04-25T00:41:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5579e4b07990 con 0x5579e4bf7010 2024-04-25T00:41:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5579e4bfa140 con 0x5579e4bf14b0 2024-04-25T00:41:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bd79a0740 10 client.0 _ll_register_callbacks cb 0x5579e4b09950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bcde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf14b0 0x5579e4bfb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bcde00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf14b0 0x5579e4bfb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48882/0 (socket says 172.21.15.42:48882) 2024-04-25T00:41:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bcde00640 1 -- 172.21.15.42:0/2192166449 learned_addr learned my addr 172.21.15.42:0/2192166449 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.262+0000 7f1bce800640 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 0x5579e4c02ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.263+0000 7f1bcd400640 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 0x5579e4bfbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.263+0000 7f1bcde00640 1 -- 172.21.15.42:0/2192166449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 msgr2=0x5579e4c02ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.263+0000 7f1bcde00640 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 0x5579e4c02ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.263+0000 7f1bcde00640 1 -- 172.21.15.42:0/2192166449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 msgr2=0x5579e4bfbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.263+0000 7f1bcde00640 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 0x5579e4bfbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.263+0000 7f1bcde00640 1 -- 172.21.15.42:0/2192166449 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e492c620 con 0x5579e4bf14b0 2024-04-25T00:41:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.263+0000 7f1bce800640 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 0x5579e4c02ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.263+0000 7f1bcde00640 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf14b0 0x5579e4bfb720 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1bc002aeb0 tx=0x7f1bc0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=105c8c5916926f29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:15.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.263+0000 7f1baf400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.264+0000 7f1baf400640 1 -- 172.21.15.42:0/2192166449 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc0003d80 con 0x5579e4bf14b0 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.264+0000 7f1baf400640 1 -- 172.21.15.42:0/2192166449 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f1bc0004040 con 0x5579e4bf14b0 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.264+0000 7f1bd79a0740 1 -- 172.21.15.42:0/2192166449 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bbc09bd10 con 0x5579e4bf14b0 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.264+0000 7f1baf400640 1 -- 172.21.15.42:0/2192166449 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1bc0004240 con 0x5579e4bf14b0 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.264+0000 7f1bd79a0740 10 client.26863 fetch_fsmap learned FSMap version 45 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.264+0000 7f1bd79a0740 1 -- 172.21.15.42:0/2192166449 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5579e4c05bf0 con 0x5579e4bf14b0 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.264+0000 7f1baf400640 1 -- 172.21.15.42:0/2192166449 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc09bd10 con 0x5579e4bf14b0 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.264+0000 7f1baf400640 1 -- 172.21.15.42:0/2192166449 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f1bc002fc60 con 0x5579e4bf14b0 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.264+0000 7f1bd79a0740 10 client.26863 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.264+0000 7f1bd79a0740 10 client.26863 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.265+0000 7f1bd79a0740 20 client.26863 populate_metadata read hostname 'smithi042' 2024-04-25T00:41:15.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.265+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5579e4c07330 0x5579e4c09760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.265+0000 7f1bd79a0740 4 client.26863 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.265+0000 7f1bd79a0740 1 -- 172.21.15.42:0/2192166449 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579e4b58ff0 con 0x5579e4c07330 2024-04-25T00:41:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.265+0000 7f1bcd400640 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5579e4c07330 0x5579e4c09760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.265+0000 7f1bcd400640 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5579e4c07330 0x5579e4c09760 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8789fa8d491aea54 server_cookie=7581b638d7bf9841 in_seq=0 out_seq=0 2024-04-25T00:41:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.266+0000 7f1baf400640 10 client.26863 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:15.267 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.267+0000 7f1baf400640 1 -- 172.21.15.42:0/2192166449 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579e4b58ff0 con 0x5579e4c07330 2024-04-25T00:41:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.267+0000 7f1baf400640 10 client.26863 handle_command_reply: tid=0 2024-04-25T00:41:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.267+0000 7f1bd79a0740 1 client.26863 shutdown 2024-04-25T00:41:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.267+0000 7f1bd79a0740 1 -- 172.21.15.42:0/2192166449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf14b0 msgr2=0x5579e4bfb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.268 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.267+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf14b0 0x5579e4bfb720 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1bc002aeb0 tx=0x7f1bc0005f90 comp rx=0 tx=0).stop 2024-04-25T00:41:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.268+0000 7f1bd79a0740 1 -- 172.21.15.42:0/2192166449 shutdown_connections 2024-04-25T00:41:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.268+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5579e4c07330 0x5579e4c09760 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.268+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 0x5579e4c02ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.268+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf1ea0 0x5579e4bfbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.268+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/2192166449 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf14b0 0x5579e4bfb720 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.268+0000 7f1bd79a0740 1 -- 172.21.15.42:0/2192166449 >> 172.21.15.42:0/2192166449 conn(0x5579e4be4310 msgr2=0x5579e4be3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.268+0000 7f1baf400640 0 client.26863 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.268+0000 7f1bd79a0740 1 -- 172.21.15.42:0/2192166449 shutdown_connections 2024-04-25T00:41:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.268+0000 7f1bd79a0740 1 -- 172.21.15.42:0/2192166449 wait complete. 2024-04-25T00:41:15.269 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.268+0000 7f1bd79a0740 20 client.26863 trim_cache size 0 max 16384 2024-04-25T00:41:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.300+0000 7f1bd79a0740 1 Processor -- start 2024-04-25T00:41:15.301 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bd79a0740 1 -- start start 2024-04-25T00:41:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf14b0 0x5579e4be3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf1ea0 0x5579e4be3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 0x5579e4c02bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01a9270 con 0x5579e4bf14b0 2024-04-25T00:41:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01a6b40 con 0x5579e4bf7010 2024-04-25T00:41:15.302 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01a49b0 con 0x5579e4bf1ea0 2024-04-25T00:41:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bcd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf1ea0 0x5579e4be3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bcd400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf1ea0 0x5579e4be3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48888/0 (socket says 172.21.15.42:48888) 2024-04-25T00:41:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bcd400640 1 -- 172.21.15.42:0/3200988091 learned_addr learned my addr 172.21.15.42:0/3200988091 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.301+0000 7f1bce800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 0x5579e4c02bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.303 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.302+0000 7f1bcde00640 1 --2- 172.21.15.42:0/3200988091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf14b0 0x5579e4be3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.302+0000 7f1bcd400640 1 -- 172.21.15.42:0/3200988091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 msgr2=0x5579e4c02bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.302+0000 7f1bcd400640 1 --2- 172.21.15.42:0/3200988091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 0x5579e4c02bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.302+0000 7f1bcd400640 1 -- 172.21.15.42:0/3200988091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf14b0 msgr2=0x5579e4be3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.302+0000 7f1bcd400640 1 --2- 172.21.15.42:0/3200988091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf14b0 0x5579e4be3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.302+0000 7f1bcd400640 1 -- 172.21.15.42:0/3200988091 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579e4bfa270 con 0x5579e4bf1ea0 2024-04-25T00:41:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.302+0000 7f1bcde00640 1 --2- 172.21.15.42:0/3200988091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf14b0 0x5579e4be3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.302+0000 7f1bcd400640 1 --2- 172.21.15.42:0/3200988091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf1ea0 0x5579e4be3840 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00c6a0 tx=0x7f1bbc09cf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e3b3ed32e47db4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.302+0000 7f1bba600640 1 -- 172.21.15.42:0/3200988091 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc09b670 con 0x5579e4bf1ea0 2024-04-25T00:41:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.303+0000 7f1bba600640 1 -- 172.21.15.42:0/3200988091 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc0b1030 con 0x5579e4bf1ea0 2024-04-25T00:41:15.304 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.303+0000 7f1bba600640 1 -- 172.21.15.42:0/3200988091 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bbc09b850 con 0x5579e4bf1ea0 2024-04-25T00:41:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.303+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3200988091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf1ea0 msgr2=0x5579e4be3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.303+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/3200988091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf1ea0 0x5579e4be3840 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00c6a0 tx=0x7f1bbc09cf90 comp rx=0 tx=0).stop 2024-04-25T00:41:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.304+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3200988091 shutdown_connections 2024-04-25T00:41:15.305 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.304+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/3200988091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf7010 0x5579e4c02bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.304+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/3200988091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf1ea0 0x5579e4be3840 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.304+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/3200988091 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf14b0 0x5579e4be3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.304+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3200988091 >> 172.21.15.42:0/3200988091 conn(0x5579e4be4310 msgr2=0x5579e4be2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.304+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3200988091 shutdown_connections 2024-04-25T00:41:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.304+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3200988091 wait complete. 2024-04-25T00:41:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.304+0000 7f1bd79a0740 1 -- start start 2024-04-25T00:41:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.305+0000 7f1bd79a0740 1 Processor -- start 2024-04-25T00:41:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.306+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf14b0 0x5579e4c00540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.306 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.306+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf1ea0 0x5579e4c00a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.306+0000 7f1bd79a0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4c01a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.306+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01a5000 con 0x5579e4bf14b0 2024-04-25T00:41:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.306+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01a6470 con 0x5579e4bf1ea0 2024-04-25T00:41:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.306+0000 7f1bd79a0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1bd01a56d0 con 0x5579e4bf7010 2024-04-25T00:41:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.306+0000 7f1bd79a0740 10 client.0 _ll_register_callbacks cb 0x7f1bc8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.307+0000 7f1bcd400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf1ea0 0x5579e4c00a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.307+0000 7f1bce800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4c01a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.307 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.307+0000 7f1bce800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4c01a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:48902/0 (socket says 172.21.15.42:48902) 2024-04-25T00:41:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.307+0000 7f1bce800640 1 -- 172.21.15.42:0/3287498526 learned_addr learned my addr 172.21.15.42:0/3287498526 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.308+0000 7f1bce800640 1 -- 172.21.15.42:0/3287498526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf1ea0 msgr2=0x5579e4c00a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.308+0000 7f1bce800640 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf1ea0 0x5579e4c00a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.308+0000 7f1bce800640 1 -- 172.21.15.42:0/3287498526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf14b0 msgr2=0x5579e4c00540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.308+0000 7f1bce800640 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf14b0 0x5579e4c00540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.308 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.308+0000 7f1bce800640 1 -- 172.21.15.42:0/3287498526 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579e4c02080 con 0x5579e4bf7010 2024-04-25T00:41:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.308+0000 7f1bce800640 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4c01a90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80045f0 tx=0x7f1bc80046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57a7b6c8a905119d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.308+0000 7f1baf400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:41:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.309+0000 7f1baf400640 1 -- 172.21.15.42:0/3287498526 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bc802e030 con 0x5579e4bf7010 2024-04-25T00:41:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.309+0000 7f1baf400640 1 -- 172.21.15.42:0/3287498526 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f1bc8018030 con 0x5579e4bf7010 2024-04-25T00:41:15.309 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.309+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3287498526 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bbc0b1030 con 0x5579e4bf7010 2024-04-25T00:41:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.310+0000 7f1baf400640 1 -- 172.21.15.42:0/3287498526 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f1bc8005020 con 0x5579e4bf7010 2024-04-25T00:41:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.310+0000 7f1bd79a0740 10 client.26869 fetch_fsmap learned FSMap version 45 2024-04-25T00:41:15.310 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.310+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3287498526 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5579e4c6da70 con 0x5579e4bf7010 2024-04-25T00:41:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.310+0000 7f1baf400640 1 -- 172.21.15.42:0/3287498526 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1bbc0b1030 con 0x5579e4bf7010 2024-04-25T00:41:15.311 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.311+0000 7f1baf400640 1 -- 172.21.15.42:0/3287498526 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f1bc80035f0 con 0x5579e4bf7010 2024-04-25T00:41:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.311+0000 7f1bd79a0740 10 client.26869 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:41:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.311+0000 7f1bd79a0740 10 client.26869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:41:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.311+0000 7f1bd79a0740 20 client.26869 populate_metadata read hostname 'smithi042' 2024-04-25T00:41:15.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.312+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5579e4c6e4d0 0x5579e4c708f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.312+0000 7f1bd79a0740 4 client.26869 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:41:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.312+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3287498526 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579e4b53340 con 0x5579e4c6e4d0 2024-04-25T00:41:15.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.313+0000 7f1bcde00640 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5579e4c6e4d0 0x5579e4c708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.314+0000 7f1bcde00640 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5579e4c6e4d0 0x5579e4c708f0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d167d87aabda346 server_cookie=aeb5d9378ebaf0f1 in_seq=0 out_seq=0 2024-04-25T00:41:15.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.314+0000 7f1baf400640 10 client.26869 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.645+0000 7f1baf400640 1 -- 172.21.15.42:0/3287498526 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579e4b53340 con 0x5579e4c6e4d0 2024-04-25T00:41:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.645+0000 7f1baf400640 10 client.26869 handle_command_reply: tid=0 2024-04-25T00:41:15.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.646+0000 7f1bd79a0740 1 client.26869 shutdown 2024-04-25T00:41:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.646+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3287498526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 msgr2=0x5579e4c01a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.646+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4c01a90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1bc80045f0 tx=0x7f1bc80046c0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.646+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3287498526 shutdown_connections 2024-04-25T00:41:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.646+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5579e4c6e4d0 0x5579e4c708f0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.646+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5579e4bf7010 0x5579e4c01a90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.646+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5579e4bf1ea0 0x5579e4c00a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.646+0000 7f1bd79a0740 1 --2- 172.21.15.42:0/3287498526 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5579e4bf14b0 0x5579e4c00540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.646+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3287498526 >> 172.21.15.42:0/3287498526 conn(0x5579e4be4310 msgr2=0x5579e4be2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.647+0000 7f1baf400640 0 client.26869 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.647+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3287498526 shutdown_connections 2024-04-25T00:41:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.647+0000 7f1bd79a0740 1 -- 172.21.15.42:0/3287498526 wait complete. 2024-04-25T00:41:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.647+0000 7f1bd79a0740 20 client.26869 trim_cache size 0 max 16384 2024-04-25T00:41:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:41:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:41:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:41:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:41:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.649+0000 7f1bd4a00640 1 -- 172.21.15.42:0/3857151143 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1bb40a2cb0 msgr2=0x7f1bb40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.649+0000 7f1bd4a00640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1bb40a2cb0 0x7f1bb40a5170 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x5579e4bf2700 tx=0x7f1bc8005f90 comp rx=0 tx=0).stop 2024-04-25T00:41:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.649+0000 7f1bd4a00640 1 -- 172.21.15.42:0/3857151143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd0102990 msgr2=0x7f1bd019a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.649+0000 7f1bd4a00640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd0102990 0x7f1bd019a1a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1bbc00e9f0 tx=0x7f1bbc00eeb0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.649+0000 7f1bd4a00640 1 -- 172.21.15.42:0/3857151143 shutdown_connections 2024-04-25T00:41:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.649+0000 7f1bd4a00640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1bb40a2cb0 0x7f1bb40a5170 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.649+0000 7f1bd4a00640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1bd0103350 0x7f1bd01a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.649+0000 7f1bd4a00640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bd0102990 0x7f1bd019a1a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.649+0000 7f1bd4a00640 1 --2- 172.21.15.42:0/3857151143 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1bd01017a0 0x7f1bd0199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.649+0000 7f1bd4a00640 1 -- 172.21.15.42:0/3857151143 >> 172.21.15.42:0/3857151143 conn(0x7f1bd00fcd30 msgr2=0x7f1bd0073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.650+0000 7f1bd4a00640 1 -- 172.21.15.42:0/3857151143 shutdown_connections 2024-04-25T00:41:15.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:15.650+0000 7f1bd4a00640 1 -- 172.21.15.42:0/3857151143 wait complete. 2024-04-25T00:41:15.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 337358 33.09 MB/sec execute 425 sec latency 230.965 ms 2024-04-25T00:41:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:15 smithi136 ceph-mon[32923]: pgmap v1065: 97 pgs: 97 active+clean; 670 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.8 MiB/s wr, 679 op/s 2024-04-25T00:41:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:15 smithi106 ceph-mon[32510]: pgmap v1065: 97 pgs: 97 active+clean; 670 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.8 MiB/s wr, 679 op/s 2024-04-25T00:41:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:15 smithi042 ceph-mon[28130]: pgmap v1065: 97 pgs: 97 active+clean; 670 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 9.8 MiB/s wr, 679 op/s 2024-04-25T00:41:16.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 338064 33.12 MB/sec execute 426 sec latency 52.471 ms 2024-04-25T00:41:17.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 338651 33.13 MB/sec execute 427 sec latency 69.006 ms 2024-04-25T00:41:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:17 smithi136 ceph-mon[32923]: pgmap v1066: 97 pgs: 97 active+clean; 670 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.8 MiB/s wr, 751 op/s 2024-04-25T00:41:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:17 smithi106 ceph-mon[32510]: pgmap v1066: 97 pgs: 97 active+clean; 670 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.8 MiB/s wr, 751 op/s 2024-04-25T00:41:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:17 smithi042 ceph-mon[28130]: pgmap v1066: 97 pgs: 97 active+clean; 670 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 9.8 MiB/s wr, 751 op/s 2024-04-25T00:41:18.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 339190 33.10 MB/sec execute 428 sec latency 101.665 ms 2024-04-25T00:41:19.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 339852 33.10 MB/sec execute 429 sec latency 61.945 ms 2024-04-25T00:41:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:19 smithi136 ceph-mon[32923]: pgmap v1067: 97 pgs: 97 active+clean; 673 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.5 MiB/s wr, 507 op/s 2024-04-25T00:41:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:19 smithi106 ceph-mon[32510]: pgmap v1067: 97 pgs: 97 active+clean; 673 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.5 MiB/s wr, 507 op/s 2024-04-25T00:41:20.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:19 smithi042 ceph-mon[28130]: pgmap v1067: 97 pgs: 97 active+clean; 673 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 4.5 MiB/s wr, 507 op/s 2024-04-25T00:41:20.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 340744 33.13 MB/sec execute 430 sec latency 69.044 ms 2024-04-25T00:41:21.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 341468 33.16 MB/sec execute 431 sec latency 47.362 ms 2024-04-25T00:41:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:21 smithi136 ceph-mon[32923]: pgmap v1068: 97 pgs: 97 active+clean; 676 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.0 MiB/s wr, 569 op/s 2024-04-25T00:41:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:21 smithi106 ceph-mon[32510]: pgmap v1068: 97 pgs: 97 active+clean; 676 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.0 MiB/s wr, 569 op/s 2024-04-25T00:41:22.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:21 smithi042 ceph-mon[28130]: pgmap v1068: 97 pgs: 97 active+clean; 676 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.0 MiB/s wr, 569 op/s 2024-04-25T00:41:22.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 341987 33.16 MB/sec execute 432 sec latency 83.086 ms 2024-04-25T00:41:23.805 INFO:tasks.workunit.client.0.smithi042.stdout: 10 342594 33.15 MB/sec execute 433 sec latency 63.986 ms 2024-04-25T00:41:24.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:23 smithi136 ceph-mon[32923]: pgmap v1069: 97 pgs: 97 active+clean; 676 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.1 MiB/s wr, 577 op/s 2024-04-25T00:41:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:23 smithi106 ceph-mon[32510]: pgmap v1069: 97 pgs: 97 active+clean; 676 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.1 MiB/s wr, 577 op/s 2024-04-25T00:41:24.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:23 smithi042 ceph-mon[28130]: pgmap v1069: 97 pgs: 97 active+clean; 676 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.2 MiB/s rd, 4.1 MiB/s wr, 577 op/s 2024-04-25T00:41:24.806 INFO:tasks.workunit.client.0.smithi042.stdout: 10 343137 33.13 MB/sec execute 434 sec latency 78.398 ms 2024-04-25T00:41:25.806 INFO:tasks.workunit.client.0.smithi042.stdout: 10 343848 33.14 MB/sec execute 435 sec latency 80.688 ms 2024-04-25T00:41:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:25 smithi136 ceph-mon[32923]: pgmap v1070: 97 pgs: 97 active+clean; 681 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 509 op/s 2024-04-25T00:41:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:25 smithi106 ceph-mon[32510]: pgmap v1070: 97 pgs: 97 active+clean; 681 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 509 op/s 2024-04-25T00:41:26.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:25 smithi042 ceph-mon[28130]: pgmap v1070: 97 pgs: 97 active+clean; 681 MiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 509 op/s 2024-04-25T00:41:26.806 INFO:tasks.workunit.client.0.smithi042.stdout: 10 344668 33.17 MB/sec execute 436 sec latency 57.359 ms 2024-04-25T00:41:27.806 INFO:tasks.workunit.client.0.smithi042.stdout: 10 345340 33.19 MB/sec execute 437 sec latency 49.044 ms 2024-04-25T00:41:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:27 smithi136 ceph-mon[32923]: pgmap v1071: 97 pgs: 97 active+clean; 672 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.0 MiB/s wr, 541 op/s 2024-04-25T00:41:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:27 smithi106 ceph-mon[32510]: pgmap v1071: 97 pgs: 97 active+clean; 672 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.0 MiB/s wr, 541 op/s 2024-04-25T00:41:28.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:27 smithi042 ceph-mon[28130]: pgmap v1071: 97 pgs: 97 active+clean; 672 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 4.0 MiB/s wr, 541 op/s 2024-04-25T00:41:28.806 INFO:tasks.workunit.client.0.smithi042.stdout: 10 345907 33.19 MB/sec execute 438 sec latency 92.044 ms 2024-04-25T00:41:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.864+0000 7fa318e00640 1 -- 172.21.15.42:0/3150507982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3141039d0 msgr2=0x7fa314103df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.864+0000 7fa318e00640 1 --2- 172.21.15.42:0/3150507982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3141039d0 0x7fa314103df0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fa304009900 tx=0x7fa30402a400 comp rx=0 tx=0).stop 2024-04-25T00:41:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.864+0000 7fa318e00640 1 -- 172.21.15.42:0/3150507982 shutdown_connections 2024-04-25T00:41:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.864+0000 7fa318e00640 1 --2- 172.21.15.42:0/3150507982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa314105580 0x7fa31410be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.864+0000 7fa318e00640 1 --2- 172.21.15.42:0/3150507982 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa314104bc0 0x7fa314105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.864+0000 7fa318e00640 1 --2- 172.21.15.42:0/3150507982 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3141039d0 0x7fa314103df0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.864+0000 7fa318e00640 1 -- 172.21.15.42:0/3150507982 >> 172.21.15.42:0/3150507982 conn(0x7fa3140fef60 msgr2=0x7fa314101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.864+0000 7fa318e00640 1 -- 172.21.15.42:0/3150507982 shutdown_connections 2024-04-25T00:41:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.864+0000 7fa318e00640 1 -- 172.21.15.42:0/3150507982 wait complete. 2024-04-25T00:41:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.865+0000 7fa318e00640 1 Processor -- start 2024-04-25T00:41:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.865+0000 7fa318e00640 1 -- start start 2024-04-25T00:41:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.866+0000 7fa318e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3141039d0 0x7fa314199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.866+0000 7fa318e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa314104bc0 0x7fa31419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.866+0000 7fa318e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa314105580 0x7fa3141a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.866+0000 7fa318e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa31410f670 con 0x7fa314105580 2024-04-25T00:41:28.868 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.866+0000 7fa318e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa31410f4f0 con 0x7fa314104bc0 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.866+0000 7fa318e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa31410f7f0 con 0x7fa3141039d0 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.866+0000 7fa311e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3141039d0 0x7fa314199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.866+0000 7fa311400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa314104bc0 0x7fa31419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.866+0000 7fa311400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa314104bc0 0x7fa31419a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44878/0 (socket says 172.21.15.42:44878) 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.866+0000 7fa311400640 1 -- 172.21.15.42:0/1085500167 learned_addr learned my addr 172.21.15.42:0/1085500167 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.867+0000 7fa312800640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa314105580 0x7fa3141a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.867+0000 7fa311400640 1 -- 172.21.15.42:0/1085500167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3141039d0 msgr2=0x7fa314199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.867+0000 7fa311400640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3141039d0 0x7fa314199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.867+0000 7fa311400640 1 -- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa314105580 msgr2=0x7fa3141a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.867+0000 7fa311400640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa314105580 0x7fa3141a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.867+0000 7fa311400640 1 -- 172.21.15.42:0/1085500167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3141a17b0 con 0x7fa314104bc0 2024-04-25T00:41:28.869 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.867+0000 7fa312800640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa314105580 0x7fa3141a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.867+0000 7fa311400640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa314104bc0 0x7fa31419a0d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa30000c890 tx=0x7fa30000cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.868+0000 7fa30aa00640 1 -- 172.21.15.42:0/1085500167 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa300004220 con 0x7fa314104bc0 2024-04-25T00:41:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.868+0000 7fa30aa00640 1 -- 172.21.15.42:0/1085500167 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3000043c0 con 0x7fa314104bc0 2024-04-25T00:41:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.868+0000 7fa30aa00640 1 -- 172.21.15.42:0/1085500167 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3000049e0 con 0x7fa314104bc0 2024-04-25T00:41:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.868+0000 7fa318e00640 1 -- 172.21.15.42:0/1085500167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3141a1aa0 con 0x7fa314104bc0 2024-04-25T00:41:28.870 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.868+0000 7fa318e00640 1 -- 172.21.15.42:0/1085500167 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3141a1fa0 con 0x7fa314104bc0 2024-04-25T00:41:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.872+0000 7fa31bc28740 1 Processor -- start 2024-04-25T00:41:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.872+0000 7fa31bc28740 1 -- start start 2024-04-25T00:41:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.872+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976b3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.872+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976b8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.872+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.872+0000 7fa31bc28740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x56199761aab0 con 0x5619976b3040 2024-04-25T00:41:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.872+0000 7fa31bc28740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5619975c3640 con 0x5619976b3a30 2024-04-25T00:41:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.872+0000 7fa31bc28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561997616a70 con 0x5619976b8ba0 2024-04-25T00:41:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.872+0000 7fa30aa00640 1 -- 172.21.15.42:0/1085500167 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa3000079b0 con 0x7fa314104bc0 2024-04-25T00:41:28.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.873+0000 7fa312800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.873+0000 7fa312800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976b3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:49558/0 (socket says 172.21.15.42:49558) 2024-04-25T00:41:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.873+0000 7fa312800640 1 -- 172.21.15.42:0/3133279920 learned_addr learned my addr 172.21.15.42:0/3133279920 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.873+0000 7fa311400640 1 --2- 172.21.15.42:0/3133279920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.873+0000 7fa311e00640 1 --2- 172.21.15.42:0/3133279920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976b8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.874+0000 7fa312800640 1 -- 172.21.15.42:0/3133279920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 msgr2=0x5619976d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.874+0000 7fa312800640 1 --2- 172.21.15.42:0/3133279920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.874+0000 7fa312800640 1 -- 172.21.15.42:0/3133279920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 msgr2=0x5619976b8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:28.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.874+0000 7fa312800640 1 --2- 172.21.15.42:0/3133279920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976b8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.874+0000 7fa312800640 1 -- 172.21.15.42:0/3133279920 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56199732f210 con 0x5619976b3040 2024-04-25T00:41:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.874+0000 7fa311400640 1 --2- 172.21.15.42:0/3133279920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.874+0000 7fa30aa00640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa2f80a2cb0 0x7fa2f80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.874+0000 7fa30aa00640 1 -- 172.21.15.42:0/1085500167 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fa3000cf090 con 0x7fa314104bc0 2024-04-25T00:41:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.875+0000 7fa312800640 1 --2- 172.21.15.42:0/3133279920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976b3460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa30c008e60 tx=0x7fa30c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef758327da192953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:28.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.875+0000 7fa311e00640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa2f80a2cb0 0x7fa2f80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.876+0000 7fa2ff400640 1 -- 172.21.15.42:0/3133279920 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00f830 con 0x5619976b3040 2024-04-25T00:41:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.876+0000 7fa2ff400640 1 -- 172.21.15.42:0/3133279920 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c018480 con 0x5619976b3040 2024-04-25T00:41:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.876+0000 7fa2ff400640 1 -- 172.21.15.42:0/3133279920 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00e610 con 0x5619976b3040 2024-04-25T00:41:28.879 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.877+0000 7fa31bc28740 1 -- 172.21.15.42:0/3133279920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 msgr2=0x5619976b3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.877+0000 7fa311e00640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa2f80a2cb0 0x7fa2f80a5170 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa30402a9f0 tx=0x7fa3040315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.877+0000 7fa31bc28740 1 --2- 172.21.15.42:0/3133279920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976b3460 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa30c008e60 tx=0x7fa30c00d710 comp rx=0 tx=0).stop 2024-04-25T00:41:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.877+0000 7fa31bc28740 1 -- 172.21.15.42:0/3133279920 shutdown_connections 2024-04-25T00:41:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.877+0000 7fa31bc28740 1 --2- 172.21.15.42:0/3133279920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.877+0000 7fa31bc28740 1 --2- 172.21.15.42:0/3133279920 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976b8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.880 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.877+0000 7fa31bc28740 1 --2- 172.21.15.42:0/3133279920 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976b3460 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.877+0000 7fa31bc28740 1 -- 172.21.15.42:0/3133279920 >> 172.21.15.42:0/3133279920 conn(0x5619976bbb20 msgr2=0x5619976bbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.877+0000 7fa31bc28740 1 -- 172.21.15.42:0/3133279920 shutdown_connections 2024-04-25T00:41:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.877+0000 7fa31bc28740 1 -- 172.21.15.42:0/3133279920 wait complete. 2024-04-25T00:41:28.881 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.878+0000 7fa31bc28740 1 -- start start 2024-04-25T00:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.880+0000 7fa31bc28740 1 Processor -- start 2024-04-25T00:41:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976d2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976d34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976da4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa31bc28740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x5619976d17e0 con 0x5619976b3040 2024-04-25T00:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa31bc28740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x5619975d6990 con 0x5619976b3a30 2024-04-25T00:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa31bc28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5619976d1960 con 0x5619976b8ba0 2024-04-25T00:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa31bc28740 10 client.0 _ll_register_callbacks cb 0x5619975d8950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa311400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976d34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa311400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976d34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44906/0 (socket says 172.21.15.42:44906) 2024-04-25T00:41:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa311400640 1 -- 172.21.15.42:0/2554626545 learned_addr learned my addr 172.21.15.42:0/2554626545 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa311e00640 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976d2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.881+0000 7fa312800640 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976da4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa311400640 1 -- 172.21.15.42:0/2554626545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 msgr2=0x5619976da4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:28.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa311400640 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976da4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.897 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa311400640 1 -- 172.21.15.42:0/2554626545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 msgr2=0x5619976d2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa311400640 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976d2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa311400640 1 -- 172.21.15.42:0/2554626545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619973fb620 con 0x5619976b3a30 2024-04-25T00:41:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa311e00640 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:28.898 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa311400640 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976d34a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa300016000 tx=0x7fa30009b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db45a46f854a0823 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa2f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:41:28.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2554626545 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30001d040 con 0x5619976b3a30 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa31bc28740 1 -- 172.21.15.42:0/2554626545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa30c018480 con 0x5619976b3a30 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.882+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2554626545 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fa3000d10d0 con 0x5619976b3a30 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2554626545 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa30009b8c0 con 0x5619976b3a30 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa31bc28740 10 client.26930 fetch_fsmap learned FSMap version 45 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa31bc28740 1 -- 172.21.15.42:0/2554626545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5619976ee590 con 0x5619976b3a30 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2554626545 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c018480 con 0x5619976b3a30 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2554626545 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fa3000860c0 con 0x5619976b3a30 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa31bc28740 10 client.26930 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa31bc28740 10 client.26930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa31bc28740 20 client.26930 populate_metadata read hostname 'smithi042' 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619976efb40 0x5619976f1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa31bc28740 4 client.26930 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.883+0000 7fa31bc28740 1 -- 172.21.15.42:0/2554626545 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561997627ff0 con 0x5619976efb40 2024-04-25T00:41:28.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.884+0000 7fa311e00640 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619976efb40 0x5619976f1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.884+0000 7fa311e00640 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619976efb40 0x5619976f1f90 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8d921d80b1921d4 server_cookie=63a4b44315dc12d2 in_seq=0 out_seq=0 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.884+0000 7fa2f3e00640 10 client.26930 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.885+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2554626545 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561997627ff0 con 0x5619976efb40 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.885+0000 7fa2f3e00640 10 client.26930 handle_command_reply: tid=0 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.885+0000 7fa31bc28740 1 client.26930 shutdown 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 1 -- 172.21.15.42:0/2554626545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 msgr2=0x5619976d34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976d34a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa300016000 tx=0x7fa30009b750 comp rx=0 tx=0).stop 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 1 -- 172.21.15.42:0/2554626545 shutdown_connections 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619976efb40 0x5619976f1f90 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976da4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa2f3e00640 0 client.26930 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976d34a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2554626545 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976d2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 1 -- 172.21.15.42:0/2554626545 >> 172.21.15.42:0/2554626545 conn(0x5619976bbb20 msgr2=0x5619976b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 1 -- 172.21.15.42:0/2554626545 shutdown_connections 2024-04-25T00:41:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 1 -- 172.21.15.42:0/2554626545 wait complete. 2024-04-25T00:41:28.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.886+0000 7fa31bc28740 20 client.26930 trim_cache size 0 max 16384 2024-04-25T00:41:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.911+0000 7fa31bc28740 1 Processor -- start 2024-04-25T00:41:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.911+0000 7fa31bc28740 1 -- start start 2024-04-25T00:41:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b3a30 0x5619976da4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b8ba0 0x5619976f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa31bc28740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa314072000 con 0x5619976b3040 2024-04-25T00:41:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa31bc28740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa31406f8d0 con 0x5619976b8ba0 2024-04-25T00:41:28.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa31bc28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa31406d720 con 0x5619976b3a30 2024-04-25T00:41:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa312800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b8ba0 0x5619976f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa312800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b8ba0 0x5619976f6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44918/0 (socket says 172.21.15.42:44918) 2024-04-25T00:41:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa312800640 1 -- 172.21.15.42:0/619827610 learned_addr learned my addr 172.21.15.42:0/619827610 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa311400640 1 --2- 172.21.15.42:0/619827610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b3a30 0x5619976da4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.912+0000 7fa311e00640 1 --2- 172.21.15.42:0/619827610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.913+0000 7fa312800640 1 -- 172.21.15.42:0/619827610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b3a30 msgr2=0x5619976da4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.913+0000 7fa312800640 1 --2- 172.21.15.42:0/619827610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b3a30 0x5619976da4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.913+0000 7fa312800640 1 -- 172.21.15.42:0/619827610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 msgr2=0x5619976f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:28.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.913+0000 7fa312800640 1 --2- 172.21.15.42:0/619827610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.913+0000 7fa312800640 1 -- 172.21.15.42:0/619827610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619976d3810 con 0x5619976b8ba0 2024-04-25T00:41:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.913+0000 7fa311e00640 1 --2- 172.21.15.42:0/619827610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.913+0000 7fa312800640 1 --2- 172.21.15.42:0/619827610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b8ba0 0x5619976f6dd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x5619976b4290 tx=0x7fa30c018c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cae728dff0bfd15e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.914+0000 7fa2fea00640 1 -- 172.21.15.42:0/619827610 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c028070 con 0x5619976b8ba0 2024-04-25T00:41:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.914+0000 7fa2fea00640 1 -- 172.21.15.42:0/619827610 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c018480 con 0x5619976b8ba0 2024-04-25T00:41:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.914+0000 7fa2fea00640 1 -- 172.21.15.42:0/619827610 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00f850 con 0x5619976b8ba0 2024-04-25T00:41:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.914+0000 7fa31bc28740 1 -- 172.21.15.42:0/619827610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b8ba0 msgr2=0x5619976f6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.915+0000 7fa31bc28740 1 --2- 172.21.15.42:0/619827610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b8ba0 0x5619976f6dd0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x5619976b4290 tx=0x7fa30c018c40 comp rx=0 tx=0).stop 2024-04-25T00:41:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.915+0000 7fa31bc28740 1 -- 172.21.15.42:0/619827610 shutdown_connections 2024-04-25T00:41:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.915+0000 7fa31bc28740 1 --2- 172.21.15.42:0/619827610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b8ba0 0x5619976f6dd0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.915+0000 7fa31bc28740 1 --2- 172.21.15.42:0/619827610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b3a30 0x5619976da4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.915+0000 7fa31bc28740 1 --2- 172.21.15.42:0/619827610 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.915+0000 7fa31bc28740 1 -- 172.21.15.42:0/619827610 >> 172.21.15.42:0/619827610 conn(0x5619976bbb20 msgr2=0x5619976bf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.915+0000 7fa31bc28740 1 -- 172.21.15.42:0/619827610 shutdown_connections 2024-04-25T00:41:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.915+0000 7fa31bc28740 1 -- 172.21.15.42:0/619827610 wait complete. 2024-04-25T00:41:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.915+0000 7fa31bc28740 1 -- start start 2024-04-25T00:41:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.915+0000 7fa31bc28740 1 Processor -- start 2024-04-25T00:41:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.916+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976d9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.916+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976d8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.916+0000 7fa31bc28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.916+0000 7fa31bc28740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fa31406dd90 con 0x5619976b3040 2024-04-25T00:41:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.916+0000 7fa31bc28740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fa31406f200 con 0x5619976b3a30 2024-04-25T00:41:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.916+0000 7fa31bc28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa31406e460 con 0x5619976b8ba0 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.916+0000 7fa31bc28740 10 client.0 _ll_register_callbacks cb 0x7fa300097b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.916+0000 7fa311400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976d8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.917+0000 7fa312800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.917+0000 7fa311e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976d9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.917+0000 7fa312800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:52734/0 (socket says 172.21.15.42:52734) 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.917+0000 7fa312800640 1 -- 172.21.15.42:0/2579445035 learned_addr learned my addr 172.21.15.42:0/2579445035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.917+0000 7fa312800640 1 -- 172.21.15.42:0/2579445035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 msgr2=0x5619976d8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.917+0000 7fa312800640 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976d8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.917+0000 7fa312800640 1 -- 172.21.15.42:0/2579445035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 msgr2=0x5619976d9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.917+0000 7fa312800640 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976d9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.917+0000 7fa312800640 1 -- 172.21.15.42:0/2579445035 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619976d9080 con 0x5619976b8ba0 2024-04-25T00:41:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.918+0000 7fa312800640 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d8b40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0085c0 tx=0x7fa30c004c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be6201112ee31409 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.918+0000 7fa2f3e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:41:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.918+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2579445035 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30c028040 con 0x5619976b8ba0 2024-04-25T00:41:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.918+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2579445035 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fa30c00f630 con 0x5619976b8ba0 2024-04-25T00:41:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.918+0000 7fa31bc28740 1 -- 172.21.15.42:0/2579445035 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa30c018480 con 0x5619976b8ba0 2024-04-25T00:41:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.918+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2579445035 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fa30c004e20 con 0x5619976b8ba0 2024-04-25T00:41:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.918+0000 7fa31bc28740 10 client.26884 fetch_fsmap learned FSMap version 45 2024-04-25T00:41:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.918+0000 7fa31bc28740 1 -- 172.21.15.42:0/2579445035 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5619976daa20 con 0x5619976b8ba0 2024-04-25T00:41:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.919+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2579445035 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa30c018480 con 0x5619976b8ba0 2024-04-25T00:41:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.919+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2579445035 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fa30c02b070 con 0x5619976b8ba0 2024-04-25T00:41:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.920+0000 7fa31bc28740 10 client.26884 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:41:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.920+0000 7fa31bc28740 10 client.26884 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:41:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.920+0000 7fa31bc28740 20 client.26884 populate_metadata read hostname 'smithi042' 2024-04-25T00:41:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.920+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619976db0e0 0x5619976dd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.920+0000 7fa31bc28740 4 client.26884 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:41:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.920+0000 7fa31bc28740 1 -- 172.21.15.42:0/2579445035 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561997622340 con 0x5619976db0e0 2024-04-25T00:41:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.920+0000 7fa311e00640 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619976db0e0 0x5619976dd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.920+0000 7fa311e00640 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619976db0e0 0x5619976dd500 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66b30de7cfe12d01 server_cookie=1a0233b8251ab92a in_seq=0 out_seq=0 2024-04-25T00:41:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:28.920+0000 7fa2f3e00640 10 client.26884 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.241+0000 7fa2f3e00640 1 -- 172.21.15.42:0/2579445035 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561997622340 con 0x5619976db0e0 2024-04-25T00:41:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.241+0000 7fa2f3e00640 10 client.26884 handle_command_reply: tid=0 2024-04-25T00:41:29.242 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.241+0000 7fa31bc28740 1 client.26884 shutdown 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 1 -- 172.21.15.42:0/2579445035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 msgr2=0x5619976d8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d8b40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0085c0 tx=0x7fa30c004c30 comp rx=0 tx=0).stop 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 1 -- 172.21.15.42:0/2579445035 shutdown_connections 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x5619976db0e0 0x5619976dd500 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619976b8ba0 0x5619976d8b40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5619976b3a30 0x5619976d8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa2f3e00640 0 client.26884 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 1 --2- 172.21.15.42:0/2579445035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x5619976b3040 0x5619976d9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 1 -- 172.21.15.42:0/2579445035 >> 172.21.15.42:0/2579445035 conn(0x5619976bbb20 msgr2=0x5619976b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 1 -- 172.21.15.42:0/2579445035 shutdown_connections 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 1 -- 172.21.15.42:0/2579445035 wait complete. 2024-04-25T00:41:29.243 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.242+0000 7fa31bc28740 20 client.26884 trim_cache size 0 max 16384 2024-04-25T00:41:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:41:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:41:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:41:29.244 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 -- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa2f80a2cb0 msgr2=0x7fa2f80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa2f80a2cb0 0x7fa2f80a5170 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa30402a9f0 tx=0x7fa3040315b0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 -- 172.21.15.42:0/1085500167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa314104bc0 msgr2=0x7fa31419a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa314104bc0 0x7fa31419a0d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa30000c890 tx=0x7fa30000cd50 comp rx=0 tx=0).stop 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 -- 172.21.15.42:0/1085500167 shutdown_connections 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fa2f80a2cb0 0x7fa2f80a5170 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fa314105580 0x7fa3141a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa314104bc0 0x7fa31419a0d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 --2- 172.21.15.42:0/1085500167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3141039d0 0x7fa314199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 -- 172.21.15.42:0/1085500167 >> 172.21.15.42:0/1085500167 conn(0x7fa3140fef60 msgr2=0x7fa314107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.243+0000 7fa318e00640 1 -- 172.21.15.42:0/1085500167 shutdown_connections 2024-04-25T00:41:29.245 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:29.244+0000 7fa318e00640 1 -- 172.21.15.42:0/1085500167 wait complete. 2024-04-25T00:41:29.806 INFO:tasks.workunit.client.0.smithi042.stdout: 10 346291 33.15 MB/sec execute 439 sec latency 294.993 ms 2024-04-25T00:41:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:29 smithi136 ceph-mon[32923]: pgmap v1072: 97 pgs: 97 active+clean; 675 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.5 MiB/s wr, 456 op/s 2024-04-25T00:41:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:29 smithi106 ceph-mon[32510]: pgmap v1072: 97 pgs: 97 active+clean; 675 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.5 MiB/s wr, 456 op/s 2024-04-25T00:41:30.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:29 smithi042 ceph-mon[28130]: pgmap v1072: 97 pgs: 97 active+clean; 675 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.5 MiB/s wr, 456 op/s 2024-04-25T00:41:30.806 INFO:tasks.workunit.client.0.smithi042.stdout: 10 346313 33.08 MB/sec execute 440 sec latency 1061.319 ms 2024-04-25T00:41:31.806 INFO:tasks.workunit.client.0.smithi042.stdout: 10 346439 33.02 MB/sec execute 441 sec latency 1537.418 ms 2024-04-25T00:41:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:31 smithi136 ceph-mon[32923]: pgmap v1073: 97 pgs: 97 active+clean; 694 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.0 MiB/s wr, 517 op/s 2024-04-25T00:41:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:31 smithi106 ceph-mon[32510]: pgmap v1073: 97 pgs: 97 active+clean; 694 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.0 MiB/s wr, 517 op/s 2024-04-25T00:41:32.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:31 smithi042 ceph-mon[28130]: pgmap v1073: 97 pgs: 97 active+clean; 694 MiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 5.0 MiB/s wr, 517 op/s 2024-04-25T00:41:32.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 346690 32.96 MB/sec execute 442 sec latency 162.086 ms 2024-04-25T00:41:33.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 347064 32.93 MB/sec execute 443 sec latency 193.777 ms 2024-04-25T00:41:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:33 smithi136 ceph-mon[32923]: pgmap v1074: 97 pgs: 97 active+clean; 765 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 675 op/s 2024-04-25T00:41:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:33 smithi106 ceph-mon[32510]: pgmap v1074: 97 pgs: 97 active+clean; 765 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 675 op/s 2024-04-25T00:41:34.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:33 smithi042 ceph-mon[28130]: pgmap v1074: 97 pgs: 97 active+clean; 765 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 675 op/s 2024-04-25T00:41:34.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 347824 32.94 MB/sec execute 444 sec latency 135.676 ms 2024-04-25T00:41:35.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 348556 32.97 MB/sec execute 445 sec latency 68.497 ms 2024-04-25T00:41:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:35 smithi136 ceph-mon[32923]: pgmap v1075: 97 pgs: 97 active+clean; 768 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 710 op/s 2024-04-25T00:41:36.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:35 smithi106 ceph-mon[32510]: pgmap v1075: 97 pgs: 97 active+clean; 768 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 710 op/s 2024-04-25T00:41:36.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:35 smithi042 ceph-mon[28130]: pgmap v1075: 97 pgs: 97 active+clean; 768 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 14 MiB/s wr, 710 op/s 2024-04-25T00:41:36.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 349278 33.01 MB/sec execute 446 sec latency 42.224 ms 2024-04-25T00:41:37.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 349685 32.99 MB/sec execute 447 sec latency 157.619 ms 2024-04-25T00:41:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:37 smithi136 ceph-mon[32923]: pgmap v1076: 97 pgs: 97 active+clean; 683 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 15 MiB/s wr, 855 op/s 2024-04-25T00:41:38.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:37 smithi106 ceph-mon[32510]: pgmap v1076: 97 pgs: 97 active+clean; 683 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 15 MiB/s wr, 855 op/s 2024-04-25T00:41:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:37 smithi042 ceph-mon[28130]: pgmap v1076: 97 pgs: 97 active+clean; 683 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 MiB/s rd, 15 MiB/s wr, 855 op/s 2024-04-25T00:41:38.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 350110 32.94 MB/sec execute 448 sec latency 359.436 ms 2024-04-25T00:41:39.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 350746 32.93 MB/sec execute 449 sec latency 62.598 ms 2024-04-25T00:41:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:39 smithi136 ceph-mon[32923]: pgmap v1077: 97 pgs: 97 active+clean; 669 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 809 op/s 2024-04-25T00:41:40.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:39 smithi106 ceph-mon[32510]: pgmap v1077: 97 pgs: 97 active+clean; 669 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 809 op/s 2024-04-25T00:41:40.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:39 smithi042 ceph-mon[28130]: pgmap v1077: 97 pgs: 97 active+clean; 669 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 15 MiB/s wr, 809 op/s 2024-04-25T00:41:40.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 351645 32.97 MB/sec execute 450 sec latency 203.571 ms 2024-04-25T00:41:41.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 352299 32.98 MB/sec execute 451 sec latency 45.954 ms 2024-04-25T00:41:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:41 smithi136 ceph-mon[32923]: pgmap v1078: 97 pgs: 97 active+clean; 673 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 856 op/s 2024-04-25T00:41:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:41 smithi106 ceph-mon[32510]: pgmap v1078: 97 pgs: 97 active+clean; 673 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 856 op/s 2024-04-25T00:41:42.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:41 smithi042 ceph-mon[28130]: pgmap v1078: 97 pgs: 97 active+clean; 673 MiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 15 MiB/s wr, 856 op/s 2024-04-25T00:41:42.547 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.547+0000 7f3544e00640 1 -- 172.21.15.42:0/3747060399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3540108fc0 msgr2=0x7f3540109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.547+0000 7f3544e00640 1 --2- 172.21.15.42:0/3747060399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3540108fc0 0x7f3540109440 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f352c009900 tx=0x7f352c02a420 comp rx=0 tx=0).stop 2024-04-25T00:41:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.547+0000 7f3544e00640 1 -- 172.21.15.42:0/3747060399 shutdown_connections 2024-04-25T00:41:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.547+0000 7f3544e00640 1 --2- 172.21.15.42:0/3747060399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3540109980 0x7f3540110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.547+0000 7f3544e00640 1 --2- 172.21.15.42:0/3747060399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3540108fc0 0x7f3540109440 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.547+0000 7f3544e00640 1 --2- 172.21.15.42:0/3747060399 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3540107dd0 0x7f35401081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.547+0000 7f3544e00640 1 -- 172.21.15.42:0/3747060399 >> 172.21.15.42:0/3747060399 conn(0x7f3540076d90 msgr2=0x7f35400791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.547+0000 7f3544e00640 1 -- 172.21.15.42:0/3747060399 shutdown_connections 2024-04-25T00:41:42.548 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.548+0000 7f3544e00640 1 -- 172.21.15.42:0/3747060399 wait complete. 2024-04-25T00:41:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.549+0000 7f3544e00640 1 Processor -- start 2024-04-25T00:41:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.550+0000 7f3544e00640 1 -- start start 2024-04-25T00:41:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.550+0000 7f3544e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3540107dd0 0x7f354019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.550+0000 7f3544e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3540109980 0x7f354019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.550+0000 7f3544e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35401a3790 0x7f35401a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.550+0000 7f3544e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3540113a80 con 0x7f3540109980 2024-04-25T00:41:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.550+0000 7f3544e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f3540113900 con 0x7f35401a3790 2024-04-25T00:41:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.550+0000 7f3544e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3540113c00 con 0x7f3540107dd0 2024-04-25T00:41:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.551+0000 7f353e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35401a3790 0x7f35401a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.551+0000 7f353e800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35401a3790 0x7f35401a5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49856/0 (socket says 172.21.15.42:49856) 2024-04-25T00:41:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.551+0000 7f353e800640 1 -- 172.21.15.42:0/3825543497 learned_addr learned my addr 172.21.15.42:0/3825543497 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.551+0000 7f353d400640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3540109980 0x7f354019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.551+0000 7f353de00640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3540107dd0 0x7f354019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.552+0000 7f353d400640 1 -- 172.21.15.42:0/3825543497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3540107dd0 msgr2=0x7f354019e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.552+0000 7f353d400640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3540107dd0 0x7f354019e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.552 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.552+0000 7f353d400640 1 -- 172.21.15.42:0/3825543497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35401a3790 msgr2=0x7f35401a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.552+0000 7f353d400640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35401a3790 0x7f35401a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.552+0000 7f353d400640 1 -- 172.21.15.42:0/3825543497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354007a0c0 con 0x7f3540109980 2024-04-25T00:41:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.552+0000 7f353e800640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35401a3790 0x7f35401a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:42.553 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.553+0000 7f353d400640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3540109980 0x7f354019e730 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f352c008000 tx=0x7f352c005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.554+0000 7f3536a00640 1 -- 172.21.15.42:0/3825543497 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c03f020 con 0x7f3540109980 2024-04-25T00:41:42.554 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.554+0000 7f3536a00640 1 -- 172.21.15.42:0/3825543497 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f352c005600 con 0x7f3540109980 2024-04-25T00:41:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.554+0000 7f3536a00640 1 -- 172.21.15.42:0/3825543497 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c035070 con 0x7f3540109980 2024-04-25T00:41:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.554+0000 7f3544e00640 1 -- 172.21.15.42:0/3825543497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354007a350 con 0x7f3540109980 2024-04-25T00:41:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.555+0000 7f3544e00640 1 -- 172.21.15.42:0/3825543497 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f354007a7c0 con 0x7f3540109980 2024-04-25T00:41:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.560+0000 7f3547c3c740 1 Processor -- start 2024-04-25T00:41:42.560 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.560+0000 7f3547c3c740 1 -- start start 2024-04-25T00:41:42.561 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.561+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac0029cd0 0x557ac002a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.561+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac002f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.561+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac002f830 0x557ac0031cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.561+0000 7f3547c3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557abff8cab0 con 0x557ac0029cd0 2024-04-25T00:41:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.561+0000 7f3547c3c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557abff35640 con 0x557ac002f830 2024-04-25T00:41:42.562 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.561+0000 7f3547c3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557abff88a70 con 0x557ac002a6c0 2024-04-25T00:41:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.562+0000 7f3536a00640 1 -- 172.21.15.42:0/3825543497 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f352c042020 con 0x7f3540109980 2024-04-25T00:41:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.562+0000 7f353e800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac0029cd0 0x557ac002a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.562+0000 7f353de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac002f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.562+0000 7f353de00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac002f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41074/0 (socket says 172.21.15.42:41074) 2024-04-25T00:41:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.563+0000 7f353de00640 1 -- 172.21.15.42:0/378355703 learned_addr learned my addr 172.21.15.42:0/378355703 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.563+0000 7f353d400640 1 --2- 172.21.15.42:0/378355703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac002f830 0x557ac0031cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.563 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.563+0000 7f353de00640 1 -- 172.21.15.42:0/378355703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac002f830 msgr2=0x557ac0031cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.563+0000 7f353de00640 1 --2- 172.21.15.42:0/378355703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac002f830 0x557ac0031cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.563+0000 7f353de00640 1 -- 172.21.15.42:0/378355703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac0029cd0 msgr2=0x557ac002a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.563+0000 7f353de00640 1 --2- 172.21.15.42:0/378355703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac0029cd0 0x557ac002a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.563+0000 7f353de00640 1 -- 172.21.15.42:0/378355703 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557abfca1210 con 0x557ac002a6c0 2024-04-25T00:41:42.564 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.564+0000 7f353d400640 1 --2- 172.21.15.42:0/378355703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac002f830 0x557ac0031cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.564+0000 7f353de00640 1 --2- 172.21.15.42:0/378355703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac002f1c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3530011470 tx=0x7f3530011930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=636b4e7f26315bef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.564+0000 7f352b400640 1 -- 172.21.15.42:0/378355703 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353001e070 con 0x557ac002a6c0 2024-04-25T00:41:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.565+0000 7f352b400640 1 -- 172.21.15.42:0/378355703 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3530003e50 con 0x557ac002a6c0 2024-04-25T00:41:42.565 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.565+0000 7f352b400640 1 -- 172.21.15.42:0/378355703 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353000f860 con 0x557ac002a6c0 2024-04-25T00:41:42.569 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.569+0000 7f3547c3c740 1 -- 172.21.15.42:0/378355703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 msgr2=0x557ac002f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.569+0000 7f3547c3c740 1 --2- 172.21.15.42:0/378355703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac002f1c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f3530011470 tx=0x7f3530011930 comp rx=0 tx=0).stop 2024-04-25T00:41:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.570+0000 7f3536a00640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35240a2da0 0x7f35240a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.570 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.570+0000 7f3536a00640 1 -- 172.21.15.42:0/3825543497 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f352c0ef6f0 con 0x7f3540109980 2024-04-25T00:41:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.571+0000 7f3547c3c740 1 -- 172.21.15.42:0/378355703 shutdown_connections 2024-04-25T00:41:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.571+0000 7f353e800640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35240a2da0 0x7f35240a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.571+0000 7f3547c3c740 1 --2- 172.21.15.42:0/378355703 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac002f830 0x557ac0031cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.571+0000 7f3547c3c740 1 --2- 172.21.15.42:0/378355703 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac002f1c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.571 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.571+0000 7f3547c3c740 1 --2- 172.21.15.42:0/378355703 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac0029cd0 0x557ac002a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.572 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.571+0000 7f3547c3c740 1 -- 172.21.15.42:0/378355703 >> 172.21.15.42:0/378355703 conn(0x557ac0025340 msgr2=0x557ac0025740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.573 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.573+0000 7f3547c3c740 1 -- 172.21.15.42:0/378355703 shutdown_connections 2024-04-25T00:41:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.574+0000 7f3547c3c740 1 -- 172.21.15.42:0/378355703 wait complete. 2024-04-25T00:41:42.577 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.575+0000 7f353e800640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35240a2da0 0x7f35240a5260 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x557ac002af20 tx=0x7f353800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.575+0000 7f3547c3c740 1 -- start start 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.575+0000 7f3547c3c740 1 Processor -- start 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac0044fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 0x557ac0045500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 0x557ac004c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f3547c3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557ac00437f0 con 0x557ac002a6c0 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f3547c3c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557abff48990 con 0x557ac0029cd0 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f3547c3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557ac0043970 con 0x557ac002f830 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f3547c3c740 10 client.0 _ll_register_callbacks cb 0x557abff4a950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f353de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac0044fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f353de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac0044fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49870/0 (socket says 172.21.15.42:49870) 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f353e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 0x557ac004c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.578 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f353de00640 1 -- 172.21.15.42:0/2869062573 learned_addr learned my addr 172.21.15.42:0/2869062573 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.576+0000 7f353d400640 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 0x557ac0045500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.577+0000 7f353de00640 1 -- 172.21.15.42:0/2869062573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 msgr2=0x557ac004c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.577+0000 7f353de00640 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 0x557ac004c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.577+0000 7f353de00640 1 -- 172.21.15.42:0/2869062573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 msgr2=0x557ac0045500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.577+0000 7f353de00640 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 0x557ac0045500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.577+0000 7f353de00640 1 -- 172.21.15.42:0/2869062573 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557abfd6d620 con 0x557ac0029cd0 2024-04-25T00:41:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.577+0000 7f353d400640 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 0x557ac0045500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:41:42.579 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.577+0000 7f353de00640 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac0044fc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3530011440 tx=0x7f3530005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=637d684b6ea034f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.577+0000 7f351fe00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:41:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.578+0000 7f351fe00640 1 -- 172.21.15.42:0/2869062573 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353001e040 con 0x557ac0029cd0 2024-04-25T00:41:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.578+0000 7f351fe00640 1 -- 172.21.15.42:0/2869062573 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f3530004000 con 0x557ac0029cd0 2024-04-25T00:41:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.578+0000 7f3547c3c740 1 -- 172.21.15.42:0/2869062573 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3530003e50 con 0x557ac0029cd0 2024-04-25T00:41:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.578+0000 7f351fe00640 1 -- 172.21.15.42:0/2869062573 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f3530003ca0 con 0x557ac0029cd0 2024-04-25T00:41:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.578+0000 7f3547c3c740 10 client.26945 fetch_fsmap learned FSMap version 45 2024-04-25T00:41:42.580 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.578+0000 7f3547c3c740 1 -- 172.21.15.42:0/2869062573 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557ac004f210 con 0x557ac0029cd0 2024-04-25T00:41:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.580+0000 7f351fe00640 1 -- 172.21.15.42:0/2869062573 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3530003e50 con 0x557ac0029cd0 2024-04-25T00:41:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.580+0000 7f351fe00640 1 -- 172.21.15.42:0/2869062573 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f3530021030 con 0x557ac0029cd0 2024-04-25T00:41:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.580+0000 7f3547c3c740 10 client.26945 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:41:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.580+0000 7f3547c3c740 10 client.26945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:41:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.581+0000 7f3547c3c740 20 client.26945 populate_metadata read hostname 'smithi042' 2024-04-25T00:41:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.581+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ac0050630 0x557ac0052a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.581+0000 7f3547c3c740 4 client.26945 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:42.581 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.581+0000 7f3547c3c740 1 -- 172.21.15.42:0/2869062573 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557abff99ff0 con 0x557ac0050630 2024-04-25T00:41:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.582+0000 7f353d400640 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ac0050630 0x557ac0052a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.582+0000 7f353d400640 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ac0050630 0x557ac0052a60 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=333648db14f9d822 server_cookie=33841cdf7ef76f3f in_seq=0 out_seq=0 2024-04-25T00:41:42.582 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.582+0000 7f351fe00640 10 client.26945 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.584+0000 7f351fe00640 1 -- 172.21.15.42:0/2869062573 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557abff99ff0 con 0x557ac0050630 2024-04-25T00:41:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.584+0000 7f351fe00640 10 client.26945 handle_command_reply: tid=0 2024-04-25T00:41:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.584+0000 7f3547c3c740 1 client.26945 shutdown 2024-04-25T00:41:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f3547c3c740 1 -- 172.21.15.42:0/2869062573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 msgr2=0x557ac0044fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.585 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac0044fc0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3530011440 tx=0x7f3530005fd0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f3547c3c740 1 -- 172.21.15.42:0/2869062573 shutdown_connections 2024-04-25T00:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ac0050630 0x557ac0052a60 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 0x557ac004c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 0x557ac0045500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2869062573 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac0044fc0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f3547c3c740 1 -- 172.21.15.42:0/2869062573 >> 172.21.15.42:0/2869062573 conn(0x557ac0025340 msgr2=0x557ac00244a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f351fe00640 0 client.26945 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f3547c3c740 1 -- 172.21.15.42:0/2869062573 shutdown_connections 2024-04-25T00:41:42.586 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.585+0000 7f3547c3c740 1 -- 172.21.15.42:0/2869062573 wait complete. 2024-04-25T00:41:42.587 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.586+0000 7f3547c3c740 20 client.26945 trim_cache size 0 max 16384 2024-04-25T00:41:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.613+0000 7f3547c3c740 1 Processor -- start 2024-04-25T00:41:42.613 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.613+0000 7f3547c3c740 1 -- start start 2024-04-25T00:41:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.613+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac00289b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.613+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 0x557ac004c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.613+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 0x557ac00445e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.613+0000 7f3547c3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f354006e770 con 0x557ac002a6c0 2024-04-25T00:41:42.614 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.613+0000 7f3547c3c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f354006c040 con 0x557ac0029cd0 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.613+0000 7f3547c3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f35400814c0 con 0x557ac002f830 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 0x557ac00445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 0x557ac00445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41098/0 (socket says 172.21.15.42:41098) 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353e800640 1 -- 172.21.15.42:0/1197689209 learned_addr learned my addr 172.21.15.42:0/1197689209 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353de00640 1 --2- 172.21.15.42:0/1197689209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac00289b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353d400640 1 --2- 172.21.15.42:0/1197689209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 0x557ac004c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353e800640 1 -- 172.21.15.42:0/1197689209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 msgr2=0x557ac00289b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353e800640 1 --2- 172.21.15.42:0/1197689209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac00289b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353e800640 1 -- 172.21.15.42:0/1197689209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 msgr2=0x557ac004c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353e800640 1 --2- 172.21.15.42:0/1197689209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 0x557ac004c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.615 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353e800640 1 -- 172.21.15.42:0/1197689209 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ac00436d0 con 0x557ac002f830 2024-04-25T00:41:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.614+0000 7f353e800640 1 --2- 172.21.15.42:0/1197689209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 0x557ac00445e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3538010840 tx=0x7f3538012f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da0824f33146b906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.615+0000 7f352aa00640 1 -- 172.21.15.42:0/1197689209 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353802d070 con 0x557ac002f830 2024-04-25T00:41:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.615+0000 7f352aa00640 1 -- 172.21.15.42:0/1197689209 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f353801cbb0 con 0x557ac002f830 2024-04-25T00:41:42.616 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.615+0000 7f352aa00640 1 -- 172.21.15.42:0/1197689209 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3538028dc0 con 0x557ac002f830 2024-04-25T00:41:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.616+0000 7f3547c3c740 1 -- 172.21.15.42:0/1197689209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 msgr2=0x557ac00445e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.616+0000 7f3547c3c740 1 --2- 172.21.15.42:0/1197689209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 0x557ac00445e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3538010840 tx=0x7f3538012f90 comp rx=0 tx=0).stop 2024-04-25T00:41:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.616+0000 7f3547c3c740 1 -- 172.21.15.42:0/1197689209 shutdown_connections 2024-04-25T00:41:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.616+0000 7f3547c3c740 1 --2- 172.21.15.42:0/1197689209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002f830 0x557ac00445e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.616+0000 7f3547c3c740 1 --2- 172.21.15.42:0/1197689209 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002a6c0 0x557ac004c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.616+0000 7f3547c3c740 1 --2- 172.21.15.42:0/1197689209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac00289b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.616+0000 7f3547c3c740 1 -- 172.21.15.42:0/1197689209 >> 172.21.15.42:0/1197689209 conn(0x557ac0025340 msgr2=0x557ac00282a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.616+0000 7f3547c3c740 1 -- 172.21.15.42:0/1197689209 shutdown_connections 2024-04-25T00:41:42.617 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.616+0000 7f3547c3c740 1 -- 172.21.15.42:0/1197689209 wait complete. 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.617+0000 7f3547c3c740 1 -- start start 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.617+0000 7f3547c3c740 1 Processor -- start 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.617+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac00494a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.617+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac00499e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.617+0000 7f3547c3c740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002f830 0x557ac004a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.617+0000 7f3547c3c740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f3540081b30 con 0x557ac002f830 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.617+0000 7f3547c3c740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f354006b970 con 0x557ac0029cd0 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.617+0000 7f3547c3c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3540082200 con 0x557ac002a6c0 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.617+0000 7f3547c3c740 10 client.0 _ll_register_callbacks cb 0x7f353001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac00499e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.620 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac00494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353d400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac00499e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41112/0 (socket says 172.21.15.42:41112) 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353d400640 1 -- 172.21.15.42:0/2362030730 learned_addr learned my addr 172.21.15.42:0/2362030730 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac00494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49890/0 (socket says 172.21.15.42:49890) 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353e800640 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002f830 0x557ac004a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353d400640 1 -- 172.21.15.42:0/2362030730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 msgr2=0x557ac00494a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353d400640 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac00494a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353d400640 1 -- 172.21.15.42:0/2362030730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002f830 msgr2=0x557ac004a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353d400640 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002f830 0x557ac004a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353d400640 1 -- 172.21.15.42:0/2362030730 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ac004afe0 con 0x557ac002a6c0 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353de00640 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac00494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f353d400640 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac00499e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f352c0ed1c0 tx=0x7f352c009620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cd2e1e19e8d0515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.618+0000 7f351fe00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:41:42.621 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f351fe00640 1 -- 172.21.15.42:0/2362030730 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f352c0bc620 con 0x557ac002a6c0 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f351fe00640 1 -- 172.21.15.42:0/2362030730 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f352c0ed2d0 con 0x557ac002a6c0 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f3547c3c740 1 -- 172.21.15.42:0/2362030730 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f353801cbb0 con 0x557ac002a6c0 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f351fe00640 1 -- 172.21.15.42:0/2362030730 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f352c0bbf20 con 0x557ac002a6c0 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f3547c3c740 10 client.26902 fetch_fsmap learned FSMap version 45 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f3547c3c740 1 -- 172.21.15.42:0/2362030730 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557ac004c360 con 0x557ac002a6c0 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f351fe00640 1 -- 172.21.15.42:0/2362030730 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f353801cbb0 con 0x557ac002a6c0 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f351fe00640 1 -- 172.21.15.42:0/2362030730 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f352c0bf030 con 0x557ac002a6c0 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f3547c3c740 10 client.26902 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f3547c3c740 10 client.26902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f3547c3c740 20 client.26902 populate_metadata read hostname 'smithi042' 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ac00b6f70 0x557ac00b9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f3547c3c740 4 client.26902 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.619+0000 7f3547c3c740 1 -- 172.21.15.42:0/2362030730 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557abff94340 con 0x557ac00b6f70 2024-04-25T00:41:42.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.620+0000 7f353de00640 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ac00b6f70 0x557ac00b9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.620+0000 7f353de00640 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ac00b6f70 0x557ac00b9390 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6653e44a824694b server_cookie=a9cbd261e195b1b0 in_seq=0 out_seq=0 2024-04-25T00:41:42.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.620+0000 7f351fe00640 10 client.26902 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:42.809 INFO:tasks.workunit.client.0.smithi042.stdout: 10 352844 32.99 MB/sec execute 452 sec latency 100.125 ms 2024-04-25T00:41:42.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.915+0000 7f351fe00640 1 -- 172.21.15.42:0/2362030730 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557abff94340 con 0x557ac00b6f70 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.915+0000 7f351fe00640 10 client.26902 handle_command_reply: tid=0 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.915+0000 7f3547c3c740 1 client.26902 shutdown 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 1 -- 172.21.15.42:0/2362030730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 msgr2=0x557ac00499e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac00499e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f352c0ed1c0 tx=0x7f352c009620 comp rx=0 tx=0).stop 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 1 -- 172.21.15.42:0/2362030730 shutdown_connections 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ac00b6f70 0x557ac00b9390 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ac002f830 0x557ac004a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ac002a6c0 0x557ac00499e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 1 --2- 172.21.15.42:0/2362030730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ac0029cd0 0x557ac00494a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f351fe00640 0 client.26902 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 1 -- 172.21.15.42:0/2362030730 >> 172.21.15.42:0/2362030730 conn(0x557ac0025340 msgr2=0x557ac0023c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 1 -- 172.21.15.42:0/2362030730 shutdown_connections 2024-04-25T00:41:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 1 -- 172.21.15.42:0/2362030730 wait complete. 2024-04-25T00:41:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.916+0000 7f3547c3c740 20 client.26902 trim_cache size 0 max 16384 2024-04-25T00:41:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:41:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:41:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:41:42.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.917+0000 7f3544e00640 1 -- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35240a2da0 msgr2=0x7f35240a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.917+0000 7f3544e00640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35240a2da0 0x7f35240a5260 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x557ac002af20 tx=0x7f353800b390 comp rx=0 tx=0).stop 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.917+0000 7f3544e00640 1 -- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3540109980 msgr2=0x7f354019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.917+0000 7f3544e00640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3540109980 0x7f354019e730 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f352c008000 tx=0x7f352c005000 comp rx=0 tx=0).stop 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.917+0000 7f3544e00640 1 -- 172.21.15.42:0/3825543497 shutdown_connections 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.917+0000 7f3544e00640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f35240a2da0 0x7f35240a5260 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.917+0000 7f3544e00640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35401a3790 0x7f35401a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.917+0000 7f3544e00640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f3540109980 0x7f354019e730 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.917+0000 7f3544e00640 1 --2- 172.21.15.42:0/3825543497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3540107dd0 0x7f354019e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.917+0000 7f3544e00640 1 -- 172.21.15.42:0/3825543497 >> 172.21.15.42:0/3825543497 conn(0x7f3540076d90 msgr2=0x7f354010c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.918+0000 7f3544e00640 1 -- 172.21.15.42:0/3825543497 shutdown_connections 2024-04-25T00:41:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:42.918+0000 7f3544e00640 1 -- 172.21.15.42:0/3825543497 wait complete. 2024-04-25T00:41:43.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 353118 32.96 MB/sec execute 453 sec latency 480.587 ms 2024-04-25T00:41:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:43 smithi136 ceph-mon[32923]: pgmap v1079: 97 pgs: 97 active+clean; 682 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 14 MiB/s wr, 843 op/s 2024-04-25T00:41:44.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:43 smithi106 ceph-mon[32510]: pgmap v1079: 97 pgs: 97 active+clean; 682 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 14 MiB/s wr, 843 op/s 2024-04-25T00:41:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:43 smithi042 ceph-mon[28130]: pgmap v1079: 97 pgs: 97 active+clean; 682 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 14 MiB/s wr, 843 op/s 2024-04-25T00:41:44.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 353357 32.91 MB/sec execute 454 sec latency 754.477 ms 2024-04-25T00:41:45.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 353844 32.86 MB/sec execute 455 sec latency 71.975 ms 2024-04-25T00:41:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:45 smithi136 ceph-mon[32923]: pgmap v1080: 97 pgs: 97 active+clean; 679 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 621 op/s 2024-04-25T00:41:46.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:45 smithi106 ceph-mon[32510]: pgmap v1080: 97 pgs: 97 active+clean; 679 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 621 op/s 2024-04-25T00:41:46.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:45 smithi042 ceph-mon[28130]: pgmap v1080: 97 pgs: 97 active+clean; 679 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.5 MiB/s wr, 621 op/s 2024-04-25T00:41:46.807 INFO:tasks.workunit.client.0.smithi042.stdout: 10 354423 32.87 MB/sec execute 456 sec latency 100.619 ms 2024-04-25T00:41:47.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 355276 32.90 MB/sec execute 457 sec latency 80.053 ms 2024-04-25T00:41:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:47 smithi136 ceph-mon[32923]: pgmap v1081: 97 pgs: 97 active+clean; 670 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.7 MiB/s wr, 743 op/s 2024-04-25T00:41:48.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:47 smithi106 ceph-mon[32510]: pgmap v1081: 97 pgs: 97 active+clean; 670 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.7 MiB/s wr, 743 op/s 2024-04-25T00:41:48.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:47 smithi042 ceph-mon[28130]: pgmap v1081: 97 pgs: 97 active+clean; 670 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 8.7 MiB/s wr, 743 op/s 2024-04-25T00:41:48.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 355957 32.91 MB/sec execute 458 sec latency 64.680 ms 2024-04-25T00:41:49.809 INFO:tasks.workunit.client.0.smithi042.stdout: 10 356538 32.92 MB/sec execute 459 sec latency 135.796 ms 2024-04-25T00:41:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:49 smithi136 ceph-mon[32923]: pgmap v1082: 97 pgs: 97 active+clean; 670 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.6 MiB/s wr, 658 op/s 2024-04-25T00:41:50.094 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:49 smithi106 ceph-mon[32510]: pgmap v1082: 97 pgs: 97 active+clean; 670 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.6 MiB/s wr, 658 op/s 2024-04-25T00:41:50.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:49 smithi042 ceph-mon[28130]: pgmap v1082: 97 pgs: 97 active+clean; 670 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 8.6 MiB/s wr, 658 op/s 2024-04-25T00:41:50.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 356790 32.88 MB/sec execute 460 sec latency 363.309 ms 2024-04-25T00:41:51.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:50 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:41:51.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:50 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:41:51.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:50 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:41:51.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 357361 32.86 MB/sec execute 461 sec latency 91.101 ms 2024-04-25T00:41:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:51 smithi136 ceph-mon[32923]: pgmap v1083: 97 pgs: 97 active+clean; 667 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.0 MiB/s wr, 668 op/s 2024-04-25T00:41:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:51 smithi106 ceph-mon[32510]: pgmap v1083: 97 pgs: 97 active+clean; 667 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.0 MiB/s wr, 668 op/s 2024-04-25T00:41:52.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:51 smithi042 ceph-mon[28130]: pgmap v1083: 97 pgs: 97 active+clean; 667 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 9.0 MiB/s wr, 668 op/s 2024-04-25T00:41:52.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 357983 32.85 MB/sec execute 462 sec latency 75.606 ms 2024-04-25T00:41:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:41:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:41:53.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:41:53.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:41:53.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:41:53.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:41:53.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:41:53.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:41:53.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:41:53.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 358455 32.85 MB/sec execute 463 sec latency 206.069 ms 2024-04-25T00:41:54.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:53 smithi136 ceph-mon[32923]: pgmap v1084: 97 pgs: 97 active+clean; 679 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 685 op/s 2024-04-25T00:41:54.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:53 smithi106 ceph-mon[32510]: pgmap v1084: 97 pgs: 97 active+clean; 679 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 685 op/s 2024-04-25T00:41:54.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:53 smithi042 ceph-mon[28130]: pgmap v1084: 97 pgs: 97 active+clean; 679 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 11 MiB/s wr, 685 op/s 2024-04-25T00:41:54.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 359181 32.85 MB/sec execute 464 sec latency 62.878 ms 2024-04-25T00:41:55.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 359795 32.87 MB/sec execute 465 sec latency 74.922 ms 2024-04-25T00:41:56.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:55 smithi136 ceph-mon[32923]: pgmap v1085: 97 pgs: 97 active+clean; 689 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 683 op/s 2024-04-25T00:41:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:55 smithi106 ceph-mon[32510]: pgmap v1085: 97 pgs: 97 active+clean; 689 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 683 op/s 2024-04-25T00:41:56.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.105+0000 7f21ac800640 1 -- 172.21.15.42:0/877762800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a41039c0 msgr2=0x7f21a4103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.105+0000 7f21ac800640 1 --2- 172.21.15.42:0/877762800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a41039c0 0x7f21a4103de0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f2198009900 tx=0x7f219802a410 comp rx=0 tx=0).stop 2024-04-25T00:41:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.106+0000 7f21ac800640 1 -- 172.21.15.42:0/877762800 shutdown_connections 2024-04-25T00:41:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.106+0000 7f21ac800640 1 --2- 172.21.15.42:0/877762800 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21a4105570 0x7f21a410be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.106+0000 7f21ac800640 1 --2- 172.21.15.42:0/877762800 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21a4104bb0 0x7f21a4105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.106+0000 7f21ac800640 1 --2- 172.21.15.42:0/877762800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a41039c0 0x7f21a4103de0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.106+0000 7f21ac800640 1 -- 172.21.15.42:0/877762800 >> 172.21.15.42:0/877762800 conn(0x7f21a40fef30 msgr2=0x7f21a4101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.107+0000 7f21ac800640 1 -- 172.21.15.42:0/877762800 shutdown_connections 2024-04-25T00:41:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.109+0000 7f21ac800640 1 -- 172.21.15.42:0/877762800 wait complete. 2024-04-25T00:41:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.112+0000 7f21ac800640 1 Processor -- start 2024-04-25T00:41:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.112+0000 7f21ac800640 1 -- start start 2024-04-25T00:41:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.113+0000 7f21ac800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21a41039c0 0x7f21a4070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.113+0000 7f21ac800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a4104bb0 0x7f21a4070590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.113+0000 7f21ac800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21a4105570 0x7f21a41ae560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.113+0000 7f21ac800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f21a410f660 con 0x7f21a41039c0 2024-04-25T00:41:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.113+0000 7f21ac800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21a410f4e0 con 0x7f21a4104bb0 2024-04-25T00:41:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.113+0000 7f21ac800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21a410f7e0 con 0x7f21a4105570 2024-04-25T00:41:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.113+0000 7f21a9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a4104bb0 0x7f21a4070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.114+0000 7f21a9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a4104bb0 0x7f21a4070590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49582/0 (socket says 172.21.15.42:49582) 2024-04-25T00:41:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.114+0000 7f21a9400640 1 -- 172.21.15.42:0/3182453358 learned_addr learned my addr 172.21.15.42:0/3182453358 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.114+0000 7f21aa800640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21a4105570 0x7f21a41ae560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.114+0000 7f21a9e00640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21a41039c0 0x7f21a4070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.114+0000 7f21a9400640 1 -- 172.21.15.42:0/3182453358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21a4105570 msgr2=0x7f21a41ae560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.114+0000 7f21a9400640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21a4105570 0x7f21a41ae560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.114+0000 7f21a9400640 1 -- 172.21.15.42:0/3182453358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21a41039c0 msgr2=0x7f21a4070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.114+0000 7f21a9400640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21a41039c0 0x7f21a4070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.114+0000 7f21a9400640 1 -- 172.21.15.42:0/3182453358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a4073710 con 0x7f21a4104bb0 2024-04-25T00:41:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.115+0000 7f21a9400640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a4104bb0 0x7f21a4070590 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f219400d950 tx=0x7f219400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.115+0000 7f219ea00640 1 -- 172.21.15.42:0/3182453358 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194017070 con 0x7f21a4104bb0 2024-04-25T00:41:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.115+0000 7f219ea00640 1 -- 172.21.15.42:0/3182453358 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2194009c30 con 0x7f21a4104bb0 2024-04-25T00:41:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.115+0000 7f219ea00640 1 -- 172.21.15.42:0/3182453358 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194017070 con 0x7f21a4104bb0 2024-04-25T00:41:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.115+0000 7f21ac800640 1 -- 172.21.15.42:0/3182453358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a4079c60 con 0x7f21a4104bb0 2024-04-25T00:41:56.117 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.116+0000 7f21ac800640 1 -- 172.21.15.42:0/3182453358 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a41a5f30 con 0x7f21a4104bb0 2024-04-25T00:41:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.120+0000 7f219ea00640 1 -- 172.21.15.42:0/3182453358 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f21940109d0 con 0x7f21a4104bb0 2024-04-25T00:41:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.121+0000 7f21af683740 1 Processor -- start 2024-04-25T00:41:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.121+0000 7f21af683740 1 -- start start 2024-04-25T00:41:56.121 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.121+0000 7f21af683740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21af683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 0x55caee1d7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21af683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1d9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21af683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55caee134ab0 con 0x55caee1d7800 2024-04-25T00:41:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21af683740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55caee0dd640 con 0x55caee1d1ca0 2024-04-25T00:41:56.122 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21af683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55caee130a70 con 0x55caee1d2690 2024-04-25T00:41:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21aa800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21aa800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49590/0 (socket says 172.21.15.42:49590) 2024-04-25T00:41:56.123 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21aa800640 1 -- 172.21.15.42:0/372279478 learned_addr learned my addr 172.21.15.42:0/372279478 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21aa800640 1 -- 172.21.15.42:0/372279478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 msgr2=0x55caee1d7190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:41:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21aa800640 1 --2- 172.21.15.42:0/372279478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 0x55caee1d7190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21aa800640 1 -- 172.21.15.42:0/372279478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 msgr2=0x55caee1d9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21aa800640 1 --2- 172.21.15.42:0/372279478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1d9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.122+0000 7f21aa800640 1 -- 172.21.15.42:0/372279478 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caede49210 con 0x55caee1d1ca0 2024-04-25T00:41:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.123+0000 7f21aa800640 1 --2- 172.21.15.42:0/372279478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1d20c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009fc0 tx=0x7f21a0010860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b1adbded4476b63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.124 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.123+0000 7f2193000640 1 -- 172.21.15.42:0/372279478 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0003b70 con 0x55caee1d1ca0 2024-04-25T00:41:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.124+0000 7f2193000640 1 -- 172.21.15.42:0/372279478 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a0003d10 con 0x55caee1d1ca0 2024-04-25T00:41:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.124+0000 7f2193000640 1 -- 172.21.15.42:0/372279478 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0011aa0 con 0x55caee1d1ca0 2024-04-25T00:41:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.124+0000 7f219ea00640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f218c0a2ec0 0x7f218c0a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.124+0000 7f21a9e00640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f218c0a2ec0 0x7f218c0a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.125+0000 7f21af683740 1 -- 172.21.15.42:0/372279478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 msgr2=0x55caee1d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.125+0000 7f21af683740 1 --2- 172.21.15.42:0/372279478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1d20c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f21a0009fc0 tx=0x7f21a0010860 comp rx=0 tx=0).stop 2024-04-25T00:41:56.125 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.125+0000 7f219ea00640 1 -- 172.21.15.42:0/3182453358 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f21940ce2c0 con 0x7f21a4104bb0 2024-04-25T00:41:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.125+0000 7f21af683740 1 -- 172.21.15.42:0/372279478 shutdown_connections 2024-04-25T00:41:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.125+0000 7f21af683740 1 --2- 172.21.15.42:0/372279478 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1d9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.125+0000 7f21af683740 1 --2- 172.21.15.42:0/372279478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 0x55caee1d7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.125+0000 7f21af683740 1 --2- 172.21.15.42:0/372279478 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1d20c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.125+0000 7f21af683740 1 -- 172.21.15.42:0/372279478 >> 172.21.15.42:0/372279478 conn(0x55caee1cd310 msgr2=0x55caee1cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:55 smithi042 ceph-mon[28130]: pgmap v1085: 97 pgs: 97 active+clean; 689 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 12 MiB/s wr, 683 op/s 2024-04-25T00:41:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.126+0000 7f21af683740 1 -- 172.21.15.42:0/372279478 shutdown_connections 2024-04-25T00:41:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.126+0000 7f21a9e00640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f218c0a2ec0 0x7f218c0a5380 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f219802aa00 tx=0x7f2198031400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.126+0000 7f21af683740 1 -- 172.21.15.42:0/372279478 wait complete. 2024-04-25T00:41:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.126+0000 7f21af683740 1 -- start start 2024-04-25T00:41:56.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.127+0000 7f21af683740 1 Processor -- start 2024-04-25T00:41:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21af683740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1dbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21af683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d2690 0x55caee1dc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21af683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d7800 0x55caee1e3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21af683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55caee1da800 con 0x55caee1d2690 2024-04-25T00:41:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21af683740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55caee0f0990 con 0x55caee1d1ca0 2024-04-25T00:41:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21af683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55caee1da980 con 0x55caee1d7800 2024-04-25T00:41:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21af683740 10 client.0 _ll_register_callbacks cb 0x55caee0f2950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21aa800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d7800 0x55caee1e3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21a9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1dbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21a9e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1dbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:49596/0 (socket says 172.21.15.42:49596) 2024-04-25T00:41:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.128+0000 7f21a9e00640 1 -- 172.21.15.42:0/2882597954 learned_addr learned my addr 172.21.15.42:0/2882597954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.129+0000 7f21aa800640 1 -- 172.21.15.42:0/2882597954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 msgr2=0x55caee1dbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.129+0000 7f21a9400640 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d2690 0x55caee1dc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.129+0000 7f21aa800640 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1dbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.129+0000 7f21aa800640 1 -- 172.21.15.42:0/2882597954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d2690 msgr2=0x55caee1dc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.129+0000 7f21aa800640 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d2690 0x55caee1dc450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.129+0000 7f21aa800640 1 -- 172.21.15.42:0/2882597954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caedf15620 con 0x55caee1d7800 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.129+0000 7f21a9400640 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d2690 0x55caee1dc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.129+0000 7f21aa800640 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d7800 0x55caee1e3490 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f21a0010d60 tx=0x7f21a0004c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c069915b23e8eb29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.129+0000 7f2187400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.130+0000 7f2187400640 1 -- 172.21.15.42:0/2882597954 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a0011800 con 0x55caee1d7800 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.130+0000 7f2187400640 1 -- 172.21.15.42:0/2882597954 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f21a00119a0 con 0x55caee1d7800 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.130+0000 7f21af683740 1 -- 172.21.15.42:0/2882597954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21a0003d10 con 0x55caee1d7800 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.130+0000 7f2187400640 1 -- 172.21.15.42:0/2882597954 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f21a002d050 con 0x55caee1d7800 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.130+0000 7f21af683740 10 client.26911 fetch_fsmap learned FSMap version 45 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.130+0000 7f21af683740 1 -- 172.21.15.42:0/2882597954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55caee1e6470 con 0x55caee1d7800 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.130+0000 7f2187400640 1 -- 172.21.15.42:0/2882597954 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f21a0003d10 con 0x55caee1d7800 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.131+0000 7f2187400640 1 -- 172.21.15.42:0/2882597954 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f21a0005200 con 0x55caee1d7800 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.131+0000 7f21af683740 10 client.26911 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.131+0000 7f21af683740 10 client.26911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:41:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.131+0000 7f21af683740 20 client.26911 populate_metadata read hostname 'smithi042' 2024-04-25T00:41:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.131+0000 7f21af683740 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55caee1e7960 0x55caee1e9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.131+0000 7f21af683740 4 client.26911 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:41:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.131+0000 7f21af683740 1 -- 172.21.15.42:0/2882597954 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caee141ff0 con 0x55caee1e7960 2024-04-25T00:41:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.131+0000 7f21a9e00640 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55caee1e7960 0x55caee1e9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.132+0000 7f21a9e00640 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55caee1e7960 0x55caee1e9d50 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97988fbee3ca3d server_cookie=865864ef91361114 in_seq=0 out_seq=0 2024-04-25T00:41:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.132+0000 7f2187400640 10 client.26911 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.134+0000 7f2187400640 1 -- 172.21.15.42:0/2882597954 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55caee141ff0 con 0x55caee1e7960 2024-04-25T00:41:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.134+0000 7f2187400640 10 client.26911 handle_command_reply: tid=0 2024-04-25T00:41:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.134+0000 7f21af683740 1 client.26911 shutdown 2024-04-25T00:41:56.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.134+0000 7f21af683740 1 -- 172.21.15.42:0/2882597954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d7800 msgr2=0x55caee1e3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.134+0000 7f21af683740 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d7800 0x55caee1e3490 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f21a0010d60 tx=0x7f21a0004c70 comp rx=0 tx=0).stop 2024-04-25T00:41:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.135+0000 7f21af683740 1 -- 172.21.15.42:0/2882597954 shutdown_connections 2024-04-25T00:41:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.135+0000 7f21af683740 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55caee1e7960 0x55caee1e9d50 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.135+0000 7f21af683740 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d7800 0x55caee1e3490 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.135+0000 7f21af683740 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d2690 0x55caee1dc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.135+0000 7f21af683740 1 --2- 172.21.15.42:0/2882597954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1dbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.135+0000 7f21af683740 1 -- 172.21.15.42:0/2882597954 >> 172.21.15.42:0/2882597954 conn(0x55caee1cd310 msgr2=0x55caee1cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.135+0000 7f2187400640 0 client.26911 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.135+0000 7f21af683740 1 -- 172.21.15.42:0/2882597954 shutdown_connections 2024-04-25T00:41:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.135+0000 7f21af683740 1 -- 172.21.15.42:0/2882597954 wait complete. 2024-04-25T00:41:56.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.135+0000 7f21af683740 20 client.26911 trim_cache size 0 max 16384 2024-04-25T00:41:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.162+0000 7f21af683740 1 Processor -- start 2024-04-25T00:41:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.162+0000 7f21af683740 1 -- start start 2024-04-25T00:41:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.162+0000 7f21af683740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.162+0000 7f21af683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 0x55caee1e3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21af683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1ee630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21af683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f21a41c2ac0 con 0x55caee1d7800 2024-04-25T00:41:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21af683740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21a41c0390 con 0x55caee1d1ca0 2024-04-25T00:41:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21af683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21a41acc90 con 0x55caee1d2690 2024-04-25T00:41:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21aa800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1ee630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21a9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 0x55caee1e3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21a9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 0x55caee1e3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41524/0 (socket says 172.21.15.42:41524) 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21a9400640 1 -- 172.21.15.42:0/1125491035 learned_addr learned my addr 172.21.15.42:0/1125491035 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21a9e00640 1 --2- 172.21.15.42:0/1125491035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21a9400640 1 -- 172.21.15.42:0/1125491035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 msgr2=0x55caee1ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21a9400640 1 --2- 172.21.15.42:0/1125491035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21a9400640 1 -- 172.21.15.42:0/1125491035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 msgr2=0x55caee1ee630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21a9400640 1 --2- 172.21.15.42:0/1125491035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1ee630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.163+0000 7f21a9400640 1 -- 172.21.15.42:0/1125491035 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caee1d18e0 con 0x55caee1d2690 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.164+0000 7f21a9e00640 1 --2- 172.21.15.42:0/1125491035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.164+0000 7f21aa800640 1 --2- 172.21.15.42:0/1125491035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1ee630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.164+0000 7f21a9400640 1 --2- 172.21.15.42:0/1125491035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 0x55caee1e3450 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f21940a3230 tx=0x7f219409a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d384c90217420d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.164+0000 7f2192600640 1 -- 172.21.15.42:0/1125491035 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194020040 con 0x55caee1d2690 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.164+0000 7f2192600640 1 -- 172.21.15.42:0/1125491035 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f219409eea0 con 0x55caee1d2690 2024-04-25T00:41:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.164+0000 7f2192600640 1 -- 172.21.15.42:0/1125491035 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219409a1a0 con 0x55caee1d2690 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.165+0000 7f21af683740 1 -- 172.21.15.42:0/1125491035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 msgr2=0x55caee1e3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.165+0000 7f21af683740 1 --2- 172.21.15.42:0/1125491035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 0x55caee1e3450 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f21940a3230 tx=0x7f219409a4d0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.165+0000 7f21af683740 1 -- 172.21.15.42:0/1125491035 shutdown_connections 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.165+0000 7f21af683740 1 --2- 172.21.15.42:0/1125491035 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1ee630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.165+0000 7f21af683740 1 --2- 172.21.15.42:0/1125491035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d2690 0x55caee1e3450 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.165+0000 7f21af683740 1 --2- 172.21.15.42:0/1125491035 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1d1ca0 0x55caee1ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.165+0000 7f21af683740 1 -- 172.21.15.42:0/1125491035 >> 172.21.15.42:0/1125491035 conn(0x55caee1cd310 msgr2=0x55caee1d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.165+0000 7f21af683740 1 -- 172.21.15.42:0/1125491035 shutdown_connections 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.166+0000 7f21af683740 1 -- 172.21.15.42:0/1125491035 wait complete. 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.166+0000 7f21af683740 1 -- start start 2024-04-25T00:41:56.167 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21af683740 1 Processor -- start 2024-04-25T00:41:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21af683740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d1ca0 0x55caee1e1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21af683740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1e1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21af683740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1e28e0 0x55caee1e2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21af683740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f21a41ad300 con 0x55caee1d7800 2024-04-25T00:41:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21af683740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f21a41bfcc0 con 0x55caee1e28e0 2024-04-25T00:41:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21af683740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21a41ad9d0 con 0x55caee1d1ca0 2024-04-25T00:41:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21af683740 10 client.0 _ll_register_callbacks cb 0x7f21a0009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:41:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21a9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d1ca0 0x55caee1e1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21a9e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d1ca0 0x55caee1e1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:41526/0 (socket says 172.21.15.42:41526) 2024-04-25T00:41:56.168 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.167+0000 7f21a9e00640 1 -- 172.21.15.42:0/3998572579 learned_addr learned my addr 172.21.15.42:0/3998572579 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:41:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.168+0000 7f21a9400640 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1e1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.168+0000 7f21aa800640 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1e28e0 0x55caee1e2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.169 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.168+0000 7f21a9e00640 1 -- 172.21.15.42:0/3998572579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1e28e0 msgr2=0x55caee1e2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.168+0000 7f21a9e00640 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1e28e0 0x55caee1e2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.168+0000 7f21a9e00640 1 -- 172.21.15.42:0/3998572579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 msgr2=0x55caee1e1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:41:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.168+0000 7f21a9e00640 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1e1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.168+0000 7f21a9e00640 1 -- 172.21.15.42:0/3998572579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caee1cd140 con 0x55caee1d1ca0 2024-04-25T00:41:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.169+0000 7f21a9e00640 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d1ca0 0x55caee1e1400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f21980314e0 tx=0x7f219802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dc658333a7849fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:41:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.169+0000 7f2187400640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:41:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.169+0000 7f2187400640 1 -- 172.21.15.42:0/3998572579 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2198035050 con 0x55caee1d1ca0 2024-04-25T00:41:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.169+0000 7f2187400640 1 -- 172.21.15.42:0/3998572579 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f2198005340 con 0x55caee1d1ca0 2024-04-25T00:41:56.170 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.169+0000 7f21af683740 1 -- 172.21.15.42:0/3998572579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f219409eea0 con 0x55caee1d1ca0 2024-04-25T00:41:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.170+0000 7f2187400640 1 -- 172.21.15.42:0/3998572579 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f2198005540 con 0x55caee1d1ca0 2024-04-25T00:41:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.170+0000 7f21af683740 10 client.26917 fetch_fsmap learned FSMap version 45 2024-04-25T00:41:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.170+0000 7f21af683740 1 -- 172.21.15.42:0/3998572579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55caee1ea3b0 con 0x55caee1d1ca0 2024-04-25T00:41:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.171+0000 7f2187400640 1 -- 172.21.15.42:0/3998572579 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f219409eea0 con 0x55caee1d1ca0 2024-04-25T00:41:56.171 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.171+0000 7f2187400640 1 -- 172.21.15.42:0/3998572579 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f2198004a70 con 0x55caee1d1ca0 2024-04-25T00:41:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.171+0000 7f21af683740 10 client.26917 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:41:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.171+0000 7f21af683740 10 client.26917 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:41:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.171+0000 7f21af683740 20 client.26917 populate_metadata read hostname 'smithi042' 2024-04-25T00:41:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.171+0000 7f21af683740 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55caee24f1b0 0x55caee2515d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:41:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.171+0000 7f21af683740 4 client.26917 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:41:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.171+0000 7f21af683740 1 -- 172.21.15.42:0/3998572579 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55caee13c340 con 0x55caee24f1b0 2024-04-25T00:41:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.172+0000 7f21a9400640 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55caee24f1b0 0x55caee2515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:41:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.172+0000 7f21a9400640 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55caee24f1b0 0x55caee2515d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f8e91359386043a server_cookie=b3f8e977656e68d7 in_seq=0 out_seq=0 2024-04-25T00:41:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.172+0000 7f2187400640 10 client.26917 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.395+0000 7f2187400640 1 -- 172.21.15.42:0/3998572579 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55caee13c340 con 0x55caee24f1b0 2024-04-25T00:41:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.395+0000 7f2187400640 10 client.26917 handle_command_reply: tid=0 2024-04-25T00:41:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.396+0000 7f21af683740 1 client.26917 shutdown 2024-04-25T00:41:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.396+0000 7f21af683740 1 -- 172.21.15.42:0/3998572579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d1ca0 msgr2=0x55caee1e1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.396+0000 7f21af683740 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d1ca0 0x55caee1e1400 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f21980314e0 tx=0x7f219802a410 comp rx=0 tx=0).stop 2024-04-25T00:41:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.396+0000 7f21af683740 1 -- 172.21.15.42:0/3998572579 shutdown_connections 2024-04-25T00:41:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.396+0000 7f21af683740 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55caee24f1b0 0x55caee2515d0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.397+0000 7f2187400640 0 client.26917 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:41:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.397+0000 7f21af683740 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55caee1e28e0 0x55caee1e2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.397+0000 7f21af683740 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55caee1d7800 0x55caee1e1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.397+0000 7f21af683740 1 --2- 172.21.15.42:0/3998572579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55caee1d1ca0 0x55caee1e1400 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.397+0000 7f21af683740 1 -- 172.21.15.42:0/3998572579 >> 172.21.15.42:0/3998572579 conn(0x55caee1cd310 msgr2=0x55caee1cbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.397+0000 7f21af683740 1 -- 172.21.15.42:0/3998572579 shutdown_connections 2024-04-25T00:41:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.397+0000 7f21af683740 1 -- 172.21.15.42:0/3998572579 wait complete. 2024-04-25T00:41:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.397+0000 7f21af683740 20 client.26917 trim_cache size 0 max 16384 2024-04-25T00:41:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:41:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:41:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:41:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.399+0000 7f21ac800640 1 -- 172.21.15.42:0/3182453358 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f218c0a2ec0 msgr2=0x7f218c0a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.399+0000 7f21ac800640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f218c0a2ec0 0x7f218c0a5380 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f219802aa00 tx=0x7f2198031400 comp rx=0 tx=0).stop 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.399+0000 7f21ac800640 1 -- 172.21.15.42:0/3182453358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a4104bb0 msgr2=0x7f21a4070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.399+0000 7f21ac800640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a4104bb0 0x7f21a4070590 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f219400d950 tx=0x7f219400de10 comp rx=0 tx=0).stop 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.399+0000 7f21ac800640 1 -- 172.21.15.42:0/3182453358 shutdown_connections 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.399+0000 7f21ac800640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f218c0a2ec0 0x7f218c0a5380 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.399+0000 7f21ac800640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f21a4105570 0x7f21a41ae560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.399+0000 7f21ac800640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21a4104bb0 0x7f21a4070590 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.399+0000 7f21ac800640 1 --2- 172.21.15.42:0/3182453358 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f21a41039c0 0x7f21a4070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.399+0000 7f21ac800640 1 -- 172.21.15.42:0/3182453358 >> 172.21.15.42:0/3182453358 conn(0x7f21a40fef30 msgr2=0x7f21a4107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.400+0000 7f21ac800640 1 -- 172.21.15.42:0/3182453358 shutdown_connections 2024-04-25T00:41:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:41:56.400+0000 7f21ac800640 1 -- 172.21.15.42:0/3182453358 wait complete. 2024-04-25T00:41:56.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 360409 32.87 MB/sec execute 466 sec latency 152.705 ms 2024-04-25T00:41:57.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 361056 32.86 MB/sec execute 467 sec latency 98.716 ms 2024-04-25T00:41:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:57 smithi106 ceph-mon[32510]: pgmap v1086: 97 pgs: 97 active+clean; 677 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 819 op/s 2024-04-25T00:41:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:57 smithi042 ceph-mon[28130]: pgmap v1086: 97 pgs: 97 active+clean; 677 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 819 op/s 2024-04-25T00:41:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:57 smithi136 ceph-mon[32923]: pgmap v1086: 97 pgs: 97 active+clean; 677 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 13 MiB/s wr, 819 op/s 2024-04-25T00:41:58.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 361690 32.87 MB/sec execute 468 sec latency 70.456 ms 2024-04-25T00:41:59.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 362413 32.88 MB/sec execute 469 sec latency 66.489 ms 2024-04-25T00:42:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:41:59 smithi106 ceph-mon[32510]: pgmap v1087: 97 pgs: 97 active+clean; 659 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.8 MiB/s wr, 607 op/s 2024-04-25T00:42:00.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:41:59 smithi042 ceph-mon[28130]: pgmap v1087: 97 pgs: 97 active+clean; 659 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.8 MiB/s wr, 607 op/s 2024-04-25T00:42:00.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:41:59 smithi136 ceph-mon[32923]: pgmap v1087: 97 pgs: 97 active+clean; 659 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 8.8 MiB/s wr, 607 op/s 2024-04-25T00:42:00.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 363171 32.89 MB/sec execute 470 sec latency 52.651 ms 2024-04-25T00:42:01.808 INFO:tasks.workunit.client.0.smithi042.stdout: 10 363915 32.92 MB/sec execute 471 sec latency 55.338 ms 2024-04-25T00:42:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:01 smithi106 ceph-mon[32510]: pgmap v1088: 97 pgs: 97 active+clean; 647 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 7.8 MiB/s wr, 538 op/s 2024-04-25T00:42:02.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:01 smithi042 ceph-mon[28130]: pgmap v1088: 97 pgs: 97 active+clean; 647 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 7.8 MiB/s wr, 538 op/s 2024-04-25T00:42:02.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:01 smithi136 ceph-mon[32923]: pgmap v1088: 97 pgs: 97 active+clean; 647 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 7.8 MiB/s wr, 538 op/s 2024-04-25T00:42:02.809 INFO:tasks.workunit.client.0.smithi042.stdout: 10 364501 32.91 MB/sec execute 472 sec latency 76.554 ms 2024-04-25T00:42:03.809 INFO:tasks.workunit.client.0.smithi042.stdout: 10 365103 32.90 MB/sec execute 473 sec latency 68.687 ms 2024-04-25T00:42:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:03 smithi106 ceph-mon[32510]: pgmap v1089: 97 pgs: 97 active+clean; 657 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 674 op/s 2024-04-25T00:42:04.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:03 smithi042 ceph-mon[28130]: pgmap v1089: 97 pgs: 97 active+clean; 657 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 674 op/s 2024-04-25T00:42:04.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:03 smithi136 ceph-mon[32923]: pgmap v1089: 97 pgs: 97 active+clean; 657 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 8.6 MiB/s wr, 674 op/s 2024-04-25T00:42:04.809 INFO:tasks.workunit.client.0.smithi042.stdout: 10 365947 32.94 MB/sec execute 474 sec latency 55.109 ms 2024-04-25T00:42:05.809 INFO:tasks.workunit.client.0.smithi042.stdout: 10 366666 32.94 MB/sec execute 475 sec latency 51.297 ms 2024-04-25T00:42:06.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:05 smithi106 ceph-mon[32510]: pgmap v1090: 97 pgs: 97 active+clean; 659 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.9 MiB/s wr, 591 op/s 2024-04-25T00:42:06.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:05 smithi042 ceph-mon[28130]: pgmap v1090: 97 pgs: 97 active+clean; 659 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.9 MiB/s wr, 591 op/s 2024-04-25T00:42:06.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:05 smithi136 ceph-mon[32923]: pgmap v1090: 97 pgs: 97 active+clean; 659 MiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 5.9 MiB/s wr, 591 op/s 2024-04-25T00:42:06.809 INFO:tasks.workunit.client.0.smithi042.stdout: 10 367372 32.97 MB/sec execute 476 sec latency 58.693 ms 2024-04-25T00:42:07.809 INFO:tasks.workunit.client.0.smithi042.stdout: 10 367946 32.95 MB/sec execute 477 sec latency 66.424 ms 2024-04-25T00:42:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:07 smithi106 ceph-mon[32510]: pgmap v1091: 97 pgs: 97 active+clean; 668 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-04-25T00:42:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:07 smithi042 ceph-mon[28130]: pgmap v1091: 97 pgs: 97 active+clean; 668 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-04-25T00:42:08.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:07 smithi136 ceph-mon[32923]: pgmap v1091: 97 pgs: 97 active+clean; 668 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.7 MiB/s wr, 598 op/s 2024-04-25T00:42:08.809 INFO:tasks.workunit.client.0.smithi042.stdout: 10 368492 32.95 MB/sec execute 478 sec latency 86.263 ms 2024-04-25T00:42:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.620+0000 7f1709c00640 1 -- 172.21.15.42:0/1052771945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17041039c0 msgr2=0x7f1704103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.620+0000 7f1709c00640 1 --2- 172.21.15.42:0/1052771945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17041039c0 0x7f1704103de0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009ca0 tx=0x7f16fc02a8b0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.621+0000 7f1709c00640 1 -- 172.21.15.42:0/1052771945 shutdown_connections 2024-04-25T00:42:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.621+0000 7f1709c00640 1 --2- 172.21.15.42:0/1052771945 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1704105570 0x7f170410be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.621+0000 7f1709c00640 1 --2- 172.21.15.42:0/1052771945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1704104bb0 0x7f1704105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.621+0000 7f1709c00640 1 --2- 172.21.15.42:0/1052771945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17041039c0 0x7f1704103de0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.621+0000 7f1709c00640 1 -- 172.21.15.42:0/1052771945 >> 172.21.15.42:0/1052771945 conn(0x7f17040fef30 msgr2=0x7f1704101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.621+0000 7f1709c00640 1 -- 172.21.15.42:0/1052771945 shutdown_connections 2024-04-25T00:42:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.621+0000 7f1709c00640 1 -- 172.21.15.42:0/1052771945 wait complete. 2024-04-25T00:42:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.623+0000 7f1709c00640 1 Processor -- start 2024-04-25T00:42:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.623+0000 7f1709c00640 1 -- start start 2024-04-25T00:42:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.623+0000 7f1709c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17041039c0 0x7f1704199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.623+0000 7f1709c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1704104bb0 0x7f170419a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1709c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1704105570 0x7f17041a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1709c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f170410f660 con 0x7f1704104bb0 2024-04-25T00:42:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1709c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f170410f4e0 con 0x7f1704105570 2024-04-25T00:42:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1709c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f170410f7e0 con 0x7f17041039c0 2024-04-25T00:42:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1702a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17041039c0 0x7f1704199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1702a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17041039c0 0x7f1704199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59666/0 (socket says 172.21.15.42:59666) 2024-04-25T00:42:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1702a00640 1 -- 172.21.15.42:0/2980617660 learned_addr learned my addr 172.21.15.42:0/2980617660 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1702000640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1704104bb0 0x7f170419a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1702a00640 1 -- 172.21.15.42:0/2980617660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1704105570 msgr2=0x7f17041a11d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:42:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1702a00640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1704105570 0x7f17041a11d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1702a00640 1 -- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1704104bb0 msgr2=0x7f170419a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.624+0000 7f1702a00640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1704104bb0 0x7f170419a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.625+0000 7f1702a00640 1 -- 172.21.15.42:0/2980617660 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17041a1870 con 0x7f17041039c0 2024-04-25T00:42:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.625+0000 7f1702000640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1704104bb0 0x7f170419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.625+0000 7f1702a00640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17041039c0 0x7f1704199c60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f16fc02aea0 tx=0x7f16fc003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.625+0000 7f16fb400640 1 -- 172.21.15.42:0/2980617660 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc004240 con 0x7f17041039c0 2024-04-25T00:42:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.626+0000 7f16fb400640 1 -- 172.21.15.42:0/2980617660 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16fc040070 con 0x7f17041039c0 2024-04-25T00:42:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.626+0000 7f1709c00640 1 -- 172.21.15.42:0/2980617660 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17041a1b00 con 0x7f17041039c0 2024-04-25T00:42:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.626+0000 7f16fb400640 1 -- 172.21.15.42:0/2980617660 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc045e30 con 0x7f17041039c0 2024-04-25T00:42:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.626+0000 7f1709c00640 1 -- 172.21.15.42:0/2980617660 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17041a2050 con 0x7f17041039c0 2024-04-25T00:42:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.631+0000 7f16fb400640 1 -- 172.21.15.42:0/2980617660 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f16fc0043e0 con 0x7f17041039c0 2024-04-25T00:42:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.631+0000 7f170ca13740 1 Processor -- start 2024-04-25T00:42:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.631+0000 7f170ca13740 1 -- start start 2024-04-25T00:42:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.632+0000 7f170ca13740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 0x556a417808d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.632+0000 7f170ca13740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41780ea0 0x556a417859a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.632+0000 7f170ca13740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41786010 0x556a417884a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.632+0000 7f170ca13740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556a416daab0 con 0x556a41780ea0 2024-04-25T00:42:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.632+0000 7f170ca13740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556a41683640 con 0x556a41786010 2024-04-25T00:42:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.632+0000 7f170ca13740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556a416d6a70 con 0x556a417804b0 2024-04-25T00:42:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.633+0000 7f1702000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41786010 0x556a417884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.633+0000 7f1702000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41786010 0x556a417884a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42182/0 (socket says 172.21.15.42:42182) 2024-04-25T00:42:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.633+0000 7f1702000640 1 -- 172.21.15.42:0/1039127400 learned_addr learned my addr 172.21.15.42:0/1039127400 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.633+0000 7f1703400640 1 --2- 172.21.15.42:0/1039127400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 0x556a417808d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.633+0000 7f1702a00640 1 --2- 172.21.15.42:0/1039127400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41780ea0 0x556a417859a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.634+0000 7f1702000640 1 -- 172.21.15.42:0/1039127400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 msgr2=0x556a417808d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.634+0000 7f1702000640 1 --2- 172.21.15.42:0/1039127400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 0x556a417808d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.634+0000 7f1702000640 1 -- 172.21.15.42:0/1039127400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41780ea0 msgr2=0x556a417859a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.634+0000 7f1702000640 1 --2- 172.21.15.42:0/1039127400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41780ea0 0x556a417859a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.634+0000 7f1702000640 1 -- 172.21.15.42:0/1039127400 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a413ef210 con 0x556a41786010 2024-04-25T00:42:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.634+0000 7f1702a00640 1 --2- 172.21.15.42:0/1039127400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41780ea0 0x556a417859a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.634+0000 7f16fb400640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f16e80a2df0 0x7f16e80a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.634+0000 7f16fb400640 1 -- 172.21.15.42:0/2980617660 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f16fc0f0390 con 0x7f17041039c0 2024-04-25T00:42:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.635+0000 7f1702000640 1 --2- 172.21.15.42:0/1039127400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41786010 0x556a417884a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f16f4005fd0 tx=0x7f16f400e6c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55181834ed5b7c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.635+0000 7f1703400640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f16e80a2df0 0x7f16e80a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.635+0000 7f16f3e00640 1 -- 172.21.15.42:0/1039127400 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f4016070 con 0x556a41786010 2024-04-25T00:42:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.635+0000 7f16f3e00640 1 -- 172.21.15.42:0/1039127400 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f4005500 con 0x556a41786010 2024-04-25T00:42:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.635+0000 7f16f3e00640 1 -- 172.21.15.42:0/1039127400 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16f40116d0 con 0x556a41786010 2024-04-25T00:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.636+0000 7f170ca13740 1 -- 172.21.15.42:0/1039127400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41786010 msgr2=0x556a417884a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.636+0000 7f170ca13740 1 --2- 172.21.15.42:0/1039127400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41786010 0x556a417884a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f16f4005fd0 tx=0x7f16f400e6c0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.636+0000 7f1703400640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f16e80a2df0 0x7f16e80a52b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x556a41781700 tx=0x7f16ec005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.636+0000 7f170ca13740 1 -- 172.21.15.42:0/1039127400 shutdown_connections 2024-04-25T00:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.636+0000 7f170ca13740 1 --2- 172.21.15.42:0/1039127400 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41786010 0x556a417884a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.636+0000 7f170ca13740 1 --2- 172.21.15.42:0/1039127400 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41780ea0 0x556a417859a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.636+0000 7f170ca13740 1 --2- 172.21.15.42:0/1039127400 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 0x556a417808d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.636+0000 7f170ca13740 1 -- 172.21.15.42:0/1039127400 >> 172.21.15.42:0/1039127400 conn(0x556a4177bb20 msgr2=0x556a4177bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.637+0000 7f170ca13740 1 -- 172.21.15.42:0/1039127400 shutdown_connections 2024-04-25T00:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.637+0000 7f170ca13740 1 -- 172.21.15.42:0/1039127400 wait complete. 2024-04-25T00:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.637+0000 7f170ca13740 1 -- start start 2024-04-25T00:42:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.638+0000 7f170ca13740 1 Processor -- start 2024-04-25T00:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f170ca13740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 0x556a4178a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f170ca13740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a4178ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f170ca13740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 0x556a41791c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f170ca13740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x556a41788fc0 con 0x556a417804b0 2024-04-25T00:42:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f170ca13740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x556a41696990 con 0x556a41780ea0 2024-04-25T00:42:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f170ca13740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556a41789140 con 0x556a41786010 2024-04-25T00:42:09.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f170ca13740 10 client.0 _ll_register_callbacks cb 0x556a41698950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:42:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f1703400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 0x556a41791c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f1702000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a4178ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f1703400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 0x556a41791c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59684/0 (socket says 172.21.15.42:59684) 2024-04-25T00:42:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f1703400640 1 -- 172.21.15.42:0/2936633483 learned_addr learned my addr 172.21.15.42:0/2936633483 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.639+0000 7f1702a00640 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 0x556a4178a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.640+0000 7f1703400640 1 -- 172.21.15.42:0/2936633483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 msgr2=0x556a4178ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.640+0000 7f1703400640 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a4178ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.640+0000 7f1703400640 1 -- 172.21.15.42:0/2936633483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 msgr2=0x556a4178a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.640+0000 7f1703400640 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 0x556a4178a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.640+0000 7f1703400640 1 -- 172.21.15.42:0/2936633483 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a414bb620 con 0x556a41786010 2024-04-25T00:42:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.640+0000 7f1703400640 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 0x556a41791c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00bbe0 tx=0x7f16ec004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81dc444c40a0f2b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.640+0000 7f16f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:42:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.641+0000 7f16f0c00640 1 -- 172.21.15.42:0/2936633483 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16ec01e420 con 0x556a41786010 2024-04-25T00:42:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.641+0000 7f16f0c00640 1 -- 172.21.15.42:0/2936633483 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f16ec01e5c0 con 0x556a41786010 2024-04-25T00:42:09.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.641+0000 7f170ca13740 1 -- 172.21.15.42:0/2936633483 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16f4005500 con 0x556a41786010 2024-04-25T00:42:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.641+0000 7f16f0c00640 1 -- 172.21.15.42:0/2936633483 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f16ec02d070 con 0x556a41786010 2024-04-25T00:42:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.641+0000 7f170ca13740 10 client.26929 fetch_fsmap learned FSMap version 45 2024-04-25T00:42:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.641+0000 7f170ca13740 1 -- 172.21.15.42:0/2936633483 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556a41794c80 con 0x556a41786010 2024-04-25T00:42:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.641+0000 7f16f0c00640 1 -- 172.21.15.42:0/2936633483 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16f4005500 con 0x556a41786010 2024-04-25T00:42:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.642+0000 7f16f0c00640 1 -- 172.21.15.42:0/2936633483 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f16ec026a40 con 0x556a41786010 2024-04-25T00:42:09.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.642+0000 7f170ca13740 10 client.26929 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:42:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.642+0000 7f170ca13740 10 client.26929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:42:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.642+0000 7f170ca13740 20 client.26929 populate_metadata read hostname 'smithi042' 2024-04-25T00:42:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.642+0000 7f170ca13740 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556a417963c0 0x556a417987f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.642+0000 7f170ca13740 4 client.26929 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:42:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.642+0000 7f170ca13740 1 -- 172.21.15.42:0/2936633483 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a416e7ff0 con 0x556a417963c0 2024-04-25T00:42:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.643+0000 7f1702a00640 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556a417963c0 0x556a417987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.644+0000 7f16f0c00640 10 client.26929 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:09.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.644+0000 7f1702a00640 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556a417963c0 0x556a417987f0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d869e2f0bdd7f2d server_cookie=4a2253affec09384 in_seq=0 out_seq=0 2024-04-25T00:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.647+0000 7f16f0c00640 1 -- 172.21.15.42:0/2936633483 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a416e7ff0 con 0x556a417963c0 2024-04-25T00:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.647+0000 7f16f0c00640 10 client.26929 handle_command_reply: tid=0 2024-04-25T00:42:09.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.647+0000 7f170ca13740 1 client.26929 shutdown 2024-04-25T00:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.648+0000 7f170ca13740 1 -- 172.21.15.42:0/2936633483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 msgr2=0x556a41791c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.648+0000 7f170ca13740 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 0x556a41791c10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f16ec00bbe0 tx=0x7f16ec004730 comp rx=0 tx=0).stop 2024-04-25T00:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.648+0000 7f170ca13740 1 -- 172.21.15.42:0/2936633483 shutdown_connections 2024-04-25T00:42:09.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.648+0000 7f170ca13740 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556a417963c0 0x556a417987f0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.648+0000 7f170ca13740 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 0x556a41791c10 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.648+0000 7f170ca13740 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a4178ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.648+0000 7f170ca13740 1 --2- 172.21.15.42:0/2936633483 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 0x556a4178a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.648+0000 7f170ca13740 1 -- 172.21.15.42:0/2936633483 >> 172.21.15.42:0/2936633483 conn(0x556a4177bb20 msgr2=0x556a41772480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.648+0000 7f16f0c00640 0 client.26929 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:09.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.649+0000 7f170ca13740 1 -- 172.21.15.42:0/2936633483 shutdown_connections 2024-04-25T00:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.649+0000 7f170ca13740 1 -- 172.21.15.42:0/2936633483 wait complete. 2024-04-25T00:42:09.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.651+0000 7f170ca13740 20 client.26929 trim_cache size 0 max 16384 2024-04-25T00:42:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.685+0000 7f170ca13740 1 Processor -- start 2024-04-25T00:42:09.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.685+0000 7f170ca13740 1 -- start start 2024-04-25T00:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.685+0000 7f170ca13740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 0x556a4176e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.685+0000 7f170ca13740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a41791bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f170ca13740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 0x556a4177ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f170ca13740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17041a9110 con 0x556a417804b0 2024-04-25T00:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f170ca13740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17041a69e0 con 0x556a41780ea0 2024-04-25T00:42:09.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f170ca13740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f17041a4850 con 0x556a41786010 2024-04-25T00:42:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f1702a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 0x556a4176e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f1702000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a41791bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f1702000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a41791bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42212/0 (socket says 172.21.15.42:42212) 2024-04-25T00:42:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f1702000640 1 -- 172.21.15.42:0/3344288523 learned_addr learned my addr 172.21.15.42:0/3344288523 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:09.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f1703400640 1 --2- 172.21.15.42:0/3344288523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 0x556a4177ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f1702a00640 1 -- 172.21.15.42:0/3344288523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 msgr2=0x556a4177ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f1702a00640 1 --2- 172.21.15.42:0/3344288523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 0x556a4177ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.686+0000 7f1702a00640 1 -- 172.21.15.42:0/3344288523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 msgr2=0x556a41791bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.687+0000 7f1702a00640 1 --2- 172.21.15.42:0/3344288523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a41791bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.687+0000 7f1702a00640 1 -- 172.21.15.42:0/3344288523 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a41794aa0 con 0x556a417804b0 2024-04-25T00:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.687+0000 7f1702000640 1 --2- 172.21.15.42:0/3344288523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a41791bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.687+0000 7f1702a00640 1 --2- 172.21.15.42:0/3344288523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 0x556a4176e110 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0edf10 tx=0x7f16fc0edd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c2a04618488a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.687+0000 7f16f3400640 1 -- 172.21.15.42:0/3344288523 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0bd320 con 0x556a417804b0 2024-04-25T00:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.687+0000 7f16f3400640 1 -- 172.21.15.42:0/3344288523 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16fc036040 con 0x556a417804b0 2024-04-25T00:42:09.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.687+0000 7f16f3400640 1 -- 172.21.15.42:0/3344288523 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0046d0 con 0x556a417804b0 2024-04-25T00:42:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.688+0000 7f170ca13740 1 -- 172.21.15.42:0/3344288523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 msgr2=0x556a4176e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.688+0000 7f170ca13740 1 --2- 172.21.15.42:0/3344288523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 0x556a4176e110 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0edf10 tx=0x7f16fc0edd90 comp rx=0 tx=0).stop 2024-04-25T00:42:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.689+0000 7f170ca13740 1 -- 172.21.15.42:0/3344288523 shutdown_connections 2024-04-25T00:42:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.689+0000 7f170ca13740 1 --2- 172.21.15.42:0/3344288523 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a41786010 0x556a4177ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.689+0000 7f170ca13740 1 --2- 172.21.15.42:0/3344288523 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a41791bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.689+0000 7f170ca13740 1 --2- 172.21.15.42:0/3344288523 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a417804b0 0x556a4176e110 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.689+0000 7f170ca13740 1 -- 172.21.15.42:0/3344288523 >> 172.21.15.42:0/3344288523 conn(0x556a4177bb20 msgr2=0x556a417820c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:09.690 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.689+0000 7f170ca13740 1 -- 172.21.15.42:0/3344288523 shutdown_connections 2024-04-25T00:42:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.689+0000 7f170ca13740 1 -- 172.21.15.42:0/3344288523 wait complete. 2024-04-25T00:42:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.689+0000 7f170ca13740 1 -- start start 2024-04-25T00:42:09.691 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.690+0000 7f170ca13740 1 Processor -- start 2024-04-25T00:42:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.690+0000 7f170ca13740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 0x556a4178f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.690+0000 7f170ca13740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a4178ff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.690+0000 7f170ca13740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41786010 0x556a41790f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.690+0000 7f170ca13740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f17041a4ea0 con 0x556a41786010 2024-04-25T00:42:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.690+0000 7f170ca13740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f17041a6310 con 0x556a41780ea0 2024-04-25T00:42:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f170ca13740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f17041a5570 con 0x556a417804b0 2024-04-25T00:42:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f170ca13740 10 client.0 _ll_register_callbacks cb 0x7f16ec0042a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:42:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1702000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a4178ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.692 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1702000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a4178ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42214/0 (socket says 172.21.15.42:42214) 2024-04-25T00:42:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1702000640 1 -- 172.21.15.42:0/1433983973 learned_addr learned my addr 172.21.15.42:0/1433983973 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1702a00640 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 0x556a4178f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1703400640 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41786010 0x556a41790f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1702a00640 1 -- 172.21.15.42:0/1433983973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 msgr2=0x556a4178ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:09.693 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1702a00640 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a4178ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1702a00640 1 -- 172.21.15.42:0/1433983973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41786010 msgr2=0x556a41790f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1702a00640 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41786010 0x556a41790f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1702a00640 1 -- 172.21.15.42:0/1433983973 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a41791520 con 0x556a417804b0 2024-04-25T00:42:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.691+0000 7f1703400640 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41786010 0x556a41790f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.692+0000 7f1702a00640 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 0x556a4178f9e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0c5a50 tx=0x7f16fc045470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0d75ad8b1c0c4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.692+0000 7f16f0c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:42:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.692+0000 7f16f0c00640 1 -- 172.21.15.42:0/1433983973 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16fc0d9030 con 0x556a417804b0 2024-04-25T00:42:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.692+0000 7f16f0c00640 1 -- 172.21.15.42:0/1433983973 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f16fc039e00 con 0x556a417804b0 2024-04-25T00:42:09.694 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.692+0000 7f170ca13740 1 -- 172.21.15.42:0/1433983973 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16fc036040 con 0x556a417804b0 2024-04-25T00:42:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f16f0c00640 1 -- 172.21.15.42:0/1433983973 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f16fc0bcc20 con 0x556a417804b0 2024-04-25T00:42:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f170ca13740 10 client.26935 fetch_fsmap learned FSMap version 45 2024-04-25T00:42:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f170ca13740 1 -- 172.21.15.42:0/1433983973 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x556a41773410 con 0x556a417804b0 2024-04-25T00:42:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f16f0c00640 1 -- 172.21.15.42:0/1433983973 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16fc036040 con 0x556a417804b0 2024-04-25T00:42:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f16f0c00640 1 -- 172.21.15.42:0/1433983973 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f16fc04d030 con 0x556a417804b0 2024-04-25T00:42:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f170ca13740 10 client.26935 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:42:09.695 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f170ca13740 10 client.26935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:42:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f170ca13740 20 client.26935 populate_metadata read hostname 'smithi042' 2024-04-25T00:42:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f170ca13740 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556a41773d30 0x556a41776150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f170ca13740 4 client.26935 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:42:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.693+0000 7f170ca13740 1 -- 172.21.15.42:0/1433983973 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a416e2340 con 0x556a41773d30 2024-04-25T00:42:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.694+0000 7f1702000640 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556a41773d30 0x556a41776150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.695+0000 7f1702000640 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556a41773d30 0x556a41776150 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b83f22f3914172c server_cookie=f396a9d00276521f in_seq=0 out_seq=0 2024-04-25T00:42:09.696 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:09.695+0000 7f16f0c00640 10 client.26935 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:09.809 INFO:tasks.workunit.client.0.smithi042.stdout: 10 369175 32.95 MB/sec execute 479 sec latency 108.848 ms 2024-04-25T00:42:10.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:09 smithi106 ceph-mon[32510]: pgmap v1092: 97 pgs: 97 active+clean; 675 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.7 MiB/s wr, 488 op/s 2024-04-25T00:42:10.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:09 smithi042 ceph-mon[28130]: pgmap v1092: 97 pgs: 97 active+clean; 675 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.7 MiB/s wr, 488 op/s 2024-04-25T00:42:10.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:09 smithi136 ceph-mon[32923]: pgmap v1092: 97 pgs: 97 active+clean; 675 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.7 MiB/s wr, 488 op/s 2024-04-25T00:42:10.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 369191 32.88 MB/sec execute 480 sec latency 902.322 ms 2024-04-25T00:42:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.874+0000 7f16f0c00640 1 -- 172.21.15.42:0/1433983973 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a416e2340 con 0x556a41773d30 2024-04-25T00:42:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.874+0000 7f16f0c00640 10 client.26935 handle_command_reply: tid=0 2024-04-25T00:42:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.874+0000 7f170ca13740 1 client.26935 shutdown 2024-04-25T00:42:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.874+0000 7f170ca13740 1 -- 172.21.15.42:0/1433983973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 msgr2=0x556a4178f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.874+0000 7f170ca13740 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 0x556a4178f9e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0c5a50 tx=0x7f16fc045470 comp rx=0 tx=0).stop 2024-04-25T00:42:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.875+0000 7f170ca13740 1 -- 172.21.15.42:0/1433983973 shutdown_connections 2024-04-25T00:42:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.875+0000 7f170ca13740 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x556a41773d30 0x556a41776150 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:10.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.875+0000 7f170ca13740 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x556a41786010 0x556a41790f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.875+0000 7f170ca13740 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x556a41780ea0 0x556a4178ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.875+0000 7f170ca13740 1 --2- 172.21.15.42:0/1433983973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a417804b0 0x556a4178f9e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.875+0000 7f170ca13740 1 -- 172.21.15.42:0/1433983973 >> 172.21.15.42:0/1433983973 conn(0x556a4177bb20 msgr2=0x556a4178a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.875+0000 7f16f0c00640 0 client.26935 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.875+0000 7f170ca13740 1 -- 172.21.15.42:0/1433983973 shutdown_connections 2024-04-25T00:42:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.876+0000 7f170ca13740 1 -- 172.21.15.42:0/1433983973 wait complete. 2024-04-25T00:42:10.877 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.876+0000 7f170ca13740 20 client.26935 trim_cache size 0 max 16384 2024-04-25T00:42:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:42:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:42:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:42:10.878 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:42:10.882 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.881+0000 7f1709c00640 1 -- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f16e80a2df0 msgr2=0x7f16e80a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.882+0000 7f1709c00640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f16e80a2df0 0x7f16e80a52b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x556a41781700 tx=0x7f16ec005f50 comp rx=0 tx=0).stop 2024-04-25T00:42:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.882+0000 7f1709c00640 1 -- 172.21.15.42:0/2980617660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17041039c0 msgr2=0x7f1704199c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.882+0000 7f1709c00640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17041039c0 0x7f1704199c60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f16fc02aea0 tx=0x7f16fc003ba0 comp rx=0 tx=0).stop 2024-04-25T00:42:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.883+0000 7f1709c00640 1 -- 172.21.15.42:0/2980617660 shutdown_connections 2024-04-25T00:42:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.883+0000 7f1709c00640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f16e80a2df0 0x7f16e80a52b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.883+0000 7f1709c00640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1704105570 0x7f17041a11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.883+0000 7f1709c00640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1704104bb0 0x7f170419a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.883+0000 7f1709c00640 1 --2- 172.21.15.42:0/2980617660 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f17041039c0 0x7f1704199c60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:10.884 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.883+0000 7f1709c00640 1 -- 172.21.15.42:0/2980617660 >> 172.21.15.42:0/2980617660 conn(0x7f17040fef30 msgr2=0x7f1704107e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.883+0000 7f1709c00640 1 -- 172.21.15.42:0/2980617660 shutdown_connections 2024-04-25T00:42:10.885 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:10.883+0000 7f1709c00640 1 -- 172.21.15.42:0/2980617660 wait complete. 2024-04-25T00:42:11.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 369397 32.83 MB/sec execute 481 sec latency 597.769 ms 2024-04-25T00:42:12.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:11 smithi106 ceph-mon[32510]: pgmap v1093: 97 pgs: 97 active+clean; 681 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.0 MiB/s wr, 462 op/s 2024-04-25T00:42:12.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:11 smithi042 ceph-mon[28130]: pgmap v1093: 97 pgs: 97 active+clean; 681 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.0 MiB/s wr, 462 op/s 2024-04-25T00:42:12.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:11 smithi136 ceph-mon[32923]: pgmap v1093: 97 pgs: 97 active+clean; 681 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.0 MiB/s wr, 462 op/s 2024-04-25T00:42:12.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 369851 32.81 MB/sec execute 482 sec latency 176.345 ms 2024-04-25T00:42:13.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 370559 32.84 MB/sec execute 483 sec latency 55.733 ms 2024-04-25T00:42:14.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:13 smithi106 ceph-mon[32510]: pgmap v1094: 97 pgs: 97 active+clean; 748 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 745 op/s 2024-04-25T00:42:14.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:13 smithi042 ceph-mon[28130]: pgmap v1094: 97 pgs: 97 active+clean; 748 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 745 op/s 2024-04-25T00:42:14.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:13 smithi136 ceph-mon[32923]: pgmap v1094: 97 pgs: 97 active+clean; 748 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.6 MiB/s rd, 12 MiB/s wr, 745 op/s 2024-04-25T00:42:14.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 371140 32.86 MB/sec execute 484 sec latency 48.306 ms 2024-04-25T00:42:15.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 371650 32.82 MB/sec execute 485 sec latency 56.189 ms 2024-04-25T00:42:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:15 smithi106 ceph-mon[32510]: pgmap v1095: 97 pgs: 97 active+clean; 748 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 594 op/s 2024-04-25T00:42:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:15 smithi042 ceph-mon[28130]: pgmap v1095: 97 pgs: 97 active+clean; 748 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 594 op/s 2024-04-25T00:42:16.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:15 smithi136 ceph-mon[32923]: pgmap v1095: 97 pgs: 97 active+clean; 748 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 11 MiB/s wr, 594 op/s 2024-04-25T00:42:16.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 372163 32.81 MB/sec execute 486 sec latency 101.012 ms 2024-04-25T00:42:17.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 372975 32.83 MB/sec execute 487 sec latency 65.197 ms 2024-04-25T00:42:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:17 smithi106 ceph-mon[32510]: pgmap v1096: 97 pgs: 97 active+clean; 669 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 764 op/s 2024-04-25T00:42:18.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:17 smithi042 ceph-mon[28130]: pgmap v1096: 97 pgs: 97 active+clean; 669 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 764 op/s 2024-04-25T00:42:18.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:17 smithi136 ceph-mon[32923]: pgmap v1096: 97 pgs: 97 active+clean; 669 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 12 MiB/s wr, 764 op/s 2024-04-25T00:42:18.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 373533 32.81 MB/sec execute 488 sec latency 385.613 ms 2024-04-25T00:42:19.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 374205 32.84 MB/sec execute 489 sec latency 51.311 ms 2024-04-25T00:42:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:19 smithi106 ceph-mon[32510]: pgmap v1097: 97 pgs: 97 active+clean; 648 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 720 op/s 2024-04-25T00:42:20.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:19 smithi042 ceph-mon[28130]: pgmap v1097: 97 pgs: 97 active+clean; 648 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 720 op/s 2024-04-25T00:42:20.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:19 smithi136 ceph-mon[32923]: pgmap v1097: 97 pgs: 97 active+clean; 648 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 720 op/s 2024-04-25T00:42:20.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 374804 32.86 MB/sec execute 490 sec latency 44.106 ms 2024-04-25T00:42:21.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 375290 32.82 MB/sec execute 491 sec latency 53.871 ms 2024-04-25T00:42:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:21 smithi136 ceph-mon[32923]: pgmap v1098: 97 pgs: 97 active+clean; 648 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 651 op/s 2024-04-25T00:42:22.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:21 smithi106 ceph-mon[32510]: pgmap v1098: 97 pgs: 97 active+clean; 648 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 651 op/s 2024-04-25T00:42:22.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:21 smithi042 ceph-mon[28130]: pgmap v1098: 97 pgs: 97 active+clean; 648 MiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 11 MiB/s wr, 651 op/s 2024-04-25T00:42:22.812 INFO:tasks.workunit.client.0.smithi042.stdout: 10 375911 32.82 MB/sec execute 492 sec latency 105.828 ms 2024-04-25T00:42:23.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 376729 32.84 MB/sec execute 493 sec latency 49.996 ms 2024-04-25T00:42:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.060+0000 7f39fd600640 1 -- 172.21.15.42:0/328584313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39f8104bb0 msgr2=0x7f39f8105030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.060+0000 7f39fd600640 1 --2- 172.21.15.42:0/328584313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39f8104bb0 0x7f39f8105030 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009900 tx=0x7f39e402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.061+0000 7f39fd600640 1 -- 172.21.15.42:0/328584313 shutdown_connections 2024-04-25T00:42:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.061+0000 7f39fd600640 1 --2- 172.21.15.42:0/328584313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f8105570 0x7f39f810be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.061+0000 7f39fd600640 1 --2- 172.21.15.42:0/328584313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39f8104bb0 0x7f39f8105030 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.061+0000 7f39fd600640 1 --2- 172.21.15.42:0/328584313 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39f81039c0 0x7f39f8103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.061+0000 7f39fd600640 1 -- 172.21.15.42:0/328584313 >> 172.21.15.42:0/328584313 conn(0x7f39f80fef30 msgr2=0x7f39f8101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.061+0000 7f39fd600640 1 -- 172.21.15.42:0/328584313 shutdown_connections 2024-04-25T00:42:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.061+0000 7f39fd600640 1 -- 172.21.15.42:0/328584313 wait complete. 2024-04-25T00:42:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.062+0000 7f39fd600640 1 Processor -- start 2024-04-25T00:42:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.062+0000 7f39fd600640 1 -- start start 2024-04-25T00:42:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.062+0000 7f39fd600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f81039c0 0x7f39f8199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.062+0000 7f39fd600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39f8104bb0 0x7f39f819a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.062+0000 7f39fd600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39f8105570 0x7f39f81a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39fd600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f39f810f690 con 0x7f39f8105570 2024-04-25T00:42:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39fd600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f39f810f510 con 0x7f39f81039c0 2024-04-25T00:42:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39fd600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f39f810f810 con 0x7f39f8104bb0 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39f6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39f8104bb0 0x7f39f819a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39f6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f81039c0 0x7f39f8199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39f6000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39f8104bb0 0x7f39f819a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:34350/0 (socket says 172.21.15.42:34350) 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39f6000640 1 -- 172.21.15.42:0/2440712713 learned_addr learned my addr 172.21.15.42:0/2440712713 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39f6a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f81039c0 0x7f39f8199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44388/0 (socket says 172.21.15.42:44388) 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39f7400640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39f8105570 0x7f39f81a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39f6a00640 1 -- 172.21.15.42:0/2440712713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39f8104bb0 msgr2=0x7f39f819a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39f6a00640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39f8104bb0 0x7f39f819a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.063+0000 7f39f6a00640 1 -- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39f8105570 msgr2=0x7f39f81a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.064+0000 7f39f6a00640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39f8105570 0x7f39f81a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.064+0000 7f39f6a00640 1 -- 172.21.15.42:0/2440712713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f81a1850 con 0x7f39f81039c0 2024-04-25T00:42:24.065 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.064+0000 7f39f7400640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39f8105570 0x7f39f81a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.064+0000 7f39f6a00640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f81039c0 0x7f39f8199c40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f39e800e9c0 tx=0x7f39e800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.064+0000 7f39ef400640 1 -- 172.21.15.42:0/2440712713 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e800ccc0 con 0x7f39f81039c0 2024-04-25T00:42:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.064+0000 7f39fd600640 1 -- 172.21.15.42:0/2440712713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f81a1b40 con 0x7f39f81039c0 2024-04-25T00:42:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.064+0000 7f39ef400640 1 -- 172.21.15.42:0/2440712713 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e80035f0 con 0x7f39f81039c0 2024-04-25T00:42:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.064+0000 7f39ef400640 1 -- 172.21.15.42:0/2440712713 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e800ce60 con 0x7f39f81039c0 2024-04-25T00:42:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.064+0000 7f39fd600640 1 -- 172.21.15.42:0/2440712713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f81a2090 con 0x7f39f81039c0 2024-04-25T00:42:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.069+0000 7f3a005a9740 1 Processor -- start 2024-04-25T00:42:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.069+0000 7f39ef400640 1 -- 172.21.15.42:0/2440712713 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f39e80037a0 con 0x7f39f81039c0 2024-04-25T00:42:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.069+0000 7f3a005a9740 1 -- start start 2024-04-25T00:42:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.069+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 0x55b5d58a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.069+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a2ea0 0x55b5d58a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.069+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 0x55b5d58aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.069+0000 7f3a005a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b5d57fcab0 con 0x55b5d58a24b0 2024-04-25T00:42:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.069+0000 7f3a005a9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b5d57a5640 con 0x55b5d58a8010 2024-04-25T00:42:24.071 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.069+0000 7f3a005a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b5d57f8a70 con 0x55b5d58a2ea0 2024-04-25T00:42:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.070+0000 7f39f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a2ea0 0x55b5d58a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.070+0000 7f39f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a2ea0 0x55b5d58a79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:34354/0 (socket says 172.21.15.42:34354) 2024-04-25T00:42:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.070+0000 7f39f7400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 0x55b5d58a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.070+0000 7f39f6a00640 1 -- 172.21.15.42:0/1248581579 learned_addr learned my addr 172.21.15.42:0/1248581579 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.070+0000 7f39f6000640 1 --2- 172.21.15.42:0/1248581579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 0x55b5d58aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.070+0000 7f39f6a00640 1 -- 172.21.15.42:0/1248581579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 msgr2=0x55b5d58aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.070+0000 7f39f6a00640 1 --2- 172.21.15.42:0/1248581579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 0x55b5d58aa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.071+0000 7f39f6a00640 1 -- 172.21.15.42:0/1248581579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 msgr2=0x55b5d58a28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.071+0000 7f39f6a00640 1 --2- 172.21.15.42:0/1248581579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 0x55b5d58a28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.071+0000 7f39f6a00640 1 -- 172.21.15.42:0/1248581579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d5511210 con 0x55b5d58a2ea0 2024-04-25T00:42:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.071+0000 7f39f6000640 1 --2- 172.21.15.42:0/1248581579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 0x55b5d58aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.071+0000 7f39f7400640 1 --2- 172.21.15.42:0/1248581579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 0x55b5d58a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.071+0000 7f39f6a00640 1 --2- 172.21.15.42:0/1248581579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a2ea0 0x55b5d58a79a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f39e80d73f0 tx=0x7f39e80d74e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=146a927d6fc4e75f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.071+0000 7f39ef400640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f39dc0a2c30 0x7f39dc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.071+0000 7f39e3400640 1 -- 172.21.15.42:0/1248581579 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e80e3070 con 0x55b5d58a2ea0 2024-04-25T00:42:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.072+0000 7f39e3400640 1 -- 172.21.15.42:0/1248581579 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e8087e70 con 0x55b5d58a2ea0 2024-04-25T00:42:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.072+0000 7f39e3400640 1 -- 172.21.15.42:0/1248581579 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e809b7d0 con 0x55b5d58a2ea0 2024-04-25T00:42:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.072+0000 7f39ef400640 1 -- 172.21.15.42:0/2440712713 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f39e80cf400 con 0x7f39f81039c0 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.072+0000 7f39f7400640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f39dc0a2c30 0x7f39dc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.072+0000 7f3a005a9740 1 -- 172.21.15.42:0/1248581579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a2ea0 msgr2=0x55b5d58a79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.072+0000 7f3a005a9740 1 --2- 172.21.15.42:0/1248581579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a2ea0 0x55b5d58a79a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f39e80d73f0 tx=0x7f39e80d74e0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.073+0000 7f39f7400640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f39dc0a2c30 0x7f39dc0a50f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55b5d58a3700 tx=0x7f39f0009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.073+0000 7f3a005a9740 1 -- 172.21.15.42:0/1248581579 shutdown_connections 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.073+0000 7f3a005a9740 1 --2- 172.21.15.42:0/1248581579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 0x55b5d58aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.073+0000 7f3a005a9740 1 --2- 172.21.15.42:0/1248581579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a2ea0 0x55b5d58a79a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.073+0000 7f3a005a9740 1 --2- 172.21.15.42:0/1248581579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 0x55b5d58a28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.073+0000 7f3a005a9740 1 -- 172.21.15.42:0/1248581579 >> 172.21.15.42:0/1248581579 conn(0x55b5d589db20 msgr2=0x55b5d589df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.073+0000 7f3a005a9740 1 -- 172.21.15.42:0/1248581579 shutdown_connections 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.073+0000 7f3a005a9740 1 -- 172.21.15.42:0/1248581579 wait complete. 2024-04-25T00:42:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.073+0000 7f3a005a9740 1 -- start start 2024-04-25T00:42:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.073+0000 7f3a005a9740 1 Processor -- start 2024-04-25T00:42:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 0x55b5d58ac730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 0x55b5d58acc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a8010 0x55b5d58b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f3a005a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55b5d58aafc0 con 0x55b5d58a24b0 2024-04-25T00:42:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f3a005a9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55b5d57b8990 con 0x55b5d58a2ea0 2024-04-25T00:42:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f3a005a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b5d58ab140 con 0x55b5d58a8010 2024-04-25T00:42:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f39f7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a8010 0x55b5d58b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f39f7400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a8010 0x55b5d58b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:34368/0 (socket says 172.21.15.42:34368) 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f3a005a9740 10 client.0 _ll_register_callbacks cb 0x55b5d57ba950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f39f7400640 1 -- 172.21.15.42:0/1666163175 learned_addr learned my addr 172.21.15.42:0/1666163175 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f39f6a00640 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 0x55b5d58ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.074+0000 7f39f6000640 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 0x55b5d58acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39f7400640 1 -- 172.21.15.42:0/1666163175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 msgr2=0x55b5d58acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39f7400640 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 0x55b5d58acc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39f7400640 1 -- 172.21.15.42:0/1666163175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 msgr2=0x55b5d58ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39f7400640 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 0x55b5d58ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39f7400640 1 -- 172.21.15.42:0/1666163175 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d55dd620 con 0x55b5d58a8010 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39f6000640 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 0x55b5d58acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39f7400640 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a8010 0x55b5d58b3cb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f39f00116f0 tx=0x7f39f0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73bdbe348d2f97a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39d7e00640 1 -- 172.21.15.42:0/1666163175 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f002c060 con 0x55b5d58a8010 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f3a005a9740 1 -- 172.21.15.42:0/1666163175 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e8087e70 con 0x55b5d58a8010 2024-04-25T00:42:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39d7e00640 1 -- 172.21.15.42:0/1666163175 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f39f0017030 con 0x55b5d58a8010 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.075+0000 7f39d7e00640 1 -- 172.21.15.42:0/1666163175 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f39f001f8d0 con 0x55b5d58a8010 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.076+0000 7f3a005a9740 10 client.26947 fetch_fsmap learned FSMap version 45 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.076+0000 7f3a005a9740 1 -- 172.21.15.42:0/1666163175 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b5d58b6db0 con 0x55b5d58a8010 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.076+0000 7f39d7e00640 1 -- 172.21.15.42:0/1666163175 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e8087e70 con 0x55b5d58a8010 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.076+0000 7f39d7e00640 1 -- 172.21.15.42:0/1666163175 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f39f0010ca0 con 0x55b5d58a8010 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.076+0000 7f3a005a9740 10 client.26947 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.076+0000 7f3a005a9740 10 client.26947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.076+0000 7f3a005a9740 20 client.26947 populate_metadata read hostname 'smithi042' 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.076+0000 7f3a005a9740 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b5d58b84f0 0x55b5d58ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.076+0000 7f3a005a9740 4 client.26947 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.076+0000 7f3a005a9740 1 -- 172.21.15.42:0/1666163175 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5d5809ff0 con 0x55b5d58b84f0 2024-04-25T00:42:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.077+0000 7f39f6a00640 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b5d58b84f0 0x55b5d58ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.077+0000 7f39f6a00640 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b5d58b84f0 0x55b5d58ba8e0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed75e9ab7aac13c9 server_cookie=a47e71ac40ba95a3 in_seq=0 out_seq=0 2024-04-25T00:42:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.077+0000 7f39d7e00640 10 client.26947 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.078+0000 7f39d7e00640 1 -- 172.21.15.42:0/1666163175 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b5d5809ff0 con 0x55b5d58b84f0 2024-04-25T00:42:24.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.078+0000 7f39d7e00640 10 client.26947 handle_command_reply: tid=0 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.078+0000 7f3a005a9740 1 client.26947 shutdown 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 1 -- 172.21.15.42:0/1666163175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a8010 msgr2=0x55b5d58b3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a8010 0x55b5d58b3cb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f39f00116f0 tx=0x7f39f0004ab0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 1 -- 172.21.15.42:0/1666163175 shutdown_connections 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b5d58b84f0 0x55b5d58ba8e0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a8010 0x55b5d58b3cb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 0x55b5d58acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f39d7e00640 0 client.26947 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 1 --2- 172.21.15.42:0/1666163175 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a24b0 0x55b5d58ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 1 -- 172.21.15.42:0/1666163175 >> 172.21.15.42:0/1666163175 conn(0x55b5d589db20 msgr2=0x55b5d5894480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 1 -- 172.21.15.42:0/1666163175 shutdown_connections 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 1 -- 172.21.15.42:0/1666163175 wait complete. 2024-04-25T00:42:24.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.079+0000 7f3a005a9740 20 client.26947 trim_cache size 0 max 16384 2024-04-25T00:42:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.104+0000 7f3a005a9740 1 Processor -- start 2024-04-25T00:42:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.104+0000 7f3a005a9740 1 -- start start 2024-04-25T00:42:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.104+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d5894300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.104+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a2ea0 0x55b5d58b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.104+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 0x55b5d58bf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.104+0000 7f3a005a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81a9220 con 0x55b5d58a2ea0 2024-04-25T00:42:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.104+0000 7f3a005a9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81a6af0 con 0x55b5d58a8010 2024-04-25T00:42:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.104+0000 7f3a005a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81a4940 con 0x55b5d58a24b0 2024-04-25T00:42:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 0x55b5d58bf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d5894300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f7400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 0x55b5d58bf1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:44424/0 (socket says 172.21.15.42:44424) 2024-04-25T00:42:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f7400640 1 -- 172.21.15.42:0/2500022915 learned_addr learned my addr 172.21.15.42:0/2500022915 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f6000640 1 --2- 172.21.15.42:0/2500022915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a2ea0 0x55b5d58b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f6a00640 1 -- 172.21.15.42:0/2500022915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 msgr2=0x55b5d58bf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f6a00640 1 --2- 172.21.15.42:0/2500022915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 0x55b5d58bf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f6a00640 1 -- 172.21.15.42:0/2500022915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a2ea0 msgr2=0x55b5d58b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f6a00640 1 --2- 172.21.15.42:0/2500022915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a2ea0 0x55b5d58b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f6a00640 1 -- 172.21.15.42:0/2500022915 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5d5893890 con 0x55b5d58a24b0 2024-04-25T00:42:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.105+0000 7f39f6000640 1 --2- 172.21.15.42:0/2500022915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a2ea0 0x55b5d58b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.106+0000 7f39f6a00640 1 --2- 172.21.15.42:0/2500022915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d5894300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f39e80d73f0 tx=0x7f39e8003e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b743cec82c6d28a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.106+0000 7f39e2a00640 1 -- 172.21.15.42:0/2500022915 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8048620 con 0x55b5d58a24b0 2024-04-25T00:42:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.106+0000 7f39e2a00640 1 -- 172.21.15.42:0/2500022915 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e80d2ca0 con 0x55b5d58a24b0 2024-04-25T00:42:24.108 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.106+0000 7f39e2a00640 1 -- 172.21.15.42:0/2500022915 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e8047610 con 0x55b5d58a24b0 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.107+0000 7f3a005a9740 1 -- 172.21.15.42:0/2500022915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 msgr2=0x55b5d5894300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.107+0000 7f3a005a9740 1 --2- 172.21.15.42:0/2500022915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d5894300 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f39e80d73f0 tx=0x7f39e8003e10 comp rx=0 tx=0).stop 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.107+0000 7f3a005a9740 1 -- 172.21.15.42:0/2500022915 shutdown_connections 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.107+0000 7f3a005a9740 1 --2- 172.21.15.42:0/2500022915 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a8010 0x55b5d58bf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.107+0000 7f3a005a9740 1 --2- 172.21.15.42:0/2500022915 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a2ea0 0x55b5d58b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.107+0000 7f3a005a9740 1 --2- 172.21.15.42:0/2500022915 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d5894300 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.107+0000 7f3a005a9740 1 -- 172.21.15.42:0/2500022915 >> 172.21.15.42:0/2500022915 conn(0x55b5d589db20 msgr2=0x55b5d5893bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.107+0000 7f3a005a9740 1 -- 172.21.15.42:0/2500022915 shutdown_connections 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.107+0000 7f3a005a9740 1 -- 172.21.15.42:0/2500022915 wait complete. 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.107+0000 7f3a005a9740 1 -- start start 2024-04-25T00:42:24.109 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.108+0000 7f3a005a9740 1 Processor -- start 2024-04-25T00:42:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.108+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d58b1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.108+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 0x55b5d58b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.108+0000 7f3a005a9740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a8010 0x55b5d58b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.108+0000 7f3a005a9740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81a4fb0 con 0x55b5d58a8010 2024-04-25T00:42:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.108+0000 7f3a005a9740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81a6420 con 0x55b5d58a2ea0 2024-04-25T00:42:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.108+0000 7f3a005a9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f39f81a5680 con 0x55b5d58a24b0 2024-04-25T00:42:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.108+0000 7f3a005a9740 10 client.0 _ll_register_callbacks cb 0x7f39f0003c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:42:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.108+0000 7f39f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d58b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.108+0000 7f39f6a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d58b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:34386/0 (socket says 172.21.15.42:34386) 2024-04-25T00:42:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.109+0000 7f39f6a00640 1 -- 172.21.15.42:0/3543153103 learned_addr learned my addr 172.21.15.42:0/3543153103 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.109+0000 7f39f6000640 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 0x55b5d58b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.109+0000 7f39f7400640 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a8010 0x55b5d58b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.110+0000 7f39f6a00640 1 -- 172.21.15.42:0/3543153103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 msgr2=0x55b5d58b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.110+0000 7f39f6a00640 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 0x55b5d58b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.110+0000 7f39f6a00640 1 -- 172.21.15.42:0/3543153103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a8010 msgr2=0x55b5d58b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.110+0000 7f39f6a00640 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a8010 0x55b5d58b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.110+0000 7f39f6a00640 1 -- 172.21.15.42:0/3543153103 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5d58b2ce0 con 0x55b5d58a24b0 2024-04-25T00:42:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.110+0000 7f39f6000640 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 0x55b5d58b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.110+0000 7f39f7400640 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a8010 0x55b5d58b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.110+0000 7f39f6a00640 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d58b1280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f39e809bbd0 tx=0x7f39e80d74e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ff5a6a0f15b324 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.110+0000 7f39d7e00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:42:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.111+0000 7f39d7e00640 1 -- 172.21.15.42:0/3543153103 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e80477e0 con 0x55b5d58a24b0 2024-04-25T00:42:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.111+0000 7f39d7e00640 1 -- 172.21.15.42:0/3543153103 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f39e80d29e0 con 0x55b5d58a24b0 2024-04-25T00:42:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.111+0000 7f3a005a9740 1 -- 172.21.15.42:0/3543153103 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e80d2ca0 con 0x55b5d58a24b0 2024-04-25T00:42:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f39d7e00640 1 -- 172.21.15.42:0/3543153103 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f39e80477e0 con 0x55b5d58a24b0 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f3a005a9740 10 client.26953 fetch_fsmap learned FSMap version 45 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f3a005a9740 1 -- 172.21.15.42:0/3543153103 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55b5d591ea60 con 0x55b5d58a24b0 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f39d7e00640 1 -- 172.21.15.42:0/3543153103 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39e80d2ca0 con 0x55b5d58a24b0 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f39d7e00640 1 -- 172.21.15.42:0/3543153103 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f39e8098270 con 0x55b5d58a24b0 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f3a005a9740 10 client.26953 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f3a005a9740 10 client.26953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f3a005a9740 20 client.26953 populate_metadata read hostname 'smithi042' 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f3a005a9740 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b5d591ed60 0x55b5d5921150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f3a005a9740 4 client.26953 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.112+0000 7f3a005a9740 1 -- 172.21.15.42:0/3543153103 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5d5804340 con 0x55b5d591ed60 2024-04-25T00:42:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.113+0000 7f39f6000640 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b5d591ed60 0x55b5d5921150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.114+0000 7f39f6000640 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b5d591ed60 0x55b5d5921150 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a41e6ed18383f298 server_cookie=a4ab7e575b3bc03f in_seq=0 out_seq=0 2024-04-25T00:42:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.114+0000 7f39d7e00640 10 client.26953 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:23 smithi136 ceph-mon[32923]: pgmap v1099: 97 pgs: 97 active+clean; 660 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 784 op/s 2024-04-25T00:42:24.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:23 smithi106 ceph-mon[32510]: pgmap v1099: 97 pgs: 97 active+clean; 660 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 784 op/s 2024-04-25T00:42:24.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:23 smithi042 ceph-mon[28130]: pgmap v1099: 97 pgs: 97 active+clean; 660 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 11 MiB/s wr, 784 op/s 2024-04-25T00:42:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.462+0000 7f39d7e00640 1 -- 172.21.15.42:0/3543153103 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b5d5804340 con 0x55b5d591ed60 2024-04-25T00:42:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.462+0000 7f39d7e00640 10 client.26953 handle_command_reply: tid=0 2024-04-25T00:42:24.464 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 client.26953 shutdown 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 -- 172.21.15.42:0/3543153103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 msgr2=0x55b5d58b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d58b1280 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f39e809bbd0 tx=0x7f39e80d74e0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 -- 172.21.15.42:0/3543153103 shutdown_connections 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55b5d591ed60 0x55b5d5921150 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55b5d58a8010 0x55b5d58b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55b5d58a2ea0 0x55b5d58b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 --2- 172.21.15.42:0/3543153103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b5d58a24b0 0x55b5d58b1280 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 -- 172.21.15.42:0/3543153103 >> 172.21.15.42:0/3543153103 conn(0x55b5d589db20 msgr2=0x55b5d58a12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f39d7e00640 0 client.26953 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 -- 172.21.15.42:0/3543153103 shutdown_connections 2024-04-25T00:42:24.465 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 1 -- 172.21.15.42:0/3543153103 wait complete. 2024-04-25T00:42:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.463+0000 7f3a005a9740 20 client.26953 trim_cache size 0 max 16384 2024-04-25T00:42:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:42:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:42:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:42:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:42:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.465+0000 7f39fd600640 1 -- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f39dc0a2c30 msgr2=0x7f39dc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:24.466 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.465+0000 7f39fd600640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f39dc0a2c30 0x7f39dc0a50f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55b5d58a3700 tx=0x7f39f0009500 comp rx=0 tx=0).stop 2024-04-25T00:42:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.465+0000 7f39fd600640 1 -- 172.21.15.42:0/2440712713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f81039c0 msgr2=0x7f39f8199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.465+0000 7f39fd600640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f81039c0 0x7f39f8199c40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f39e800e9c0 tx=0x7f39e800ee80 comp rx=0 tx=0).stop 2024-04-25T00:42:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.466+0000 7f39fd600640 1 -- 172.21.15.42:0/2440712713 shutdown_connections 2024-04-25T00:42:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.466+0000 7f39fd600640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f39dc0a2c30 0x7f39dc0a50f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.466+0000 7f39fd600640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f39f8105570 0x7f39f81a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.466+0000 7f39fd600640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f39f8104bb0 0x7f39f819a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.466+0000 7f39fd600640 1 --2- 172.21.15.42:0/2440712713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f39f81039c0 0x7f39f8199c40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:24.467 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.466+0000 7f39fd600640 1 -- 172.21.15.42:0/2440712713 >> 172.21.15.42:0/2440712713 conn(0x7f39f80fef30 msgr2=0x7f39f8107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.466+0000 7f39fd600640 1 -- 172.21.15.42:0/2440712713 shutdown_connections 2024-04-25T00:42:24.468 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:24.466+0000 7f39fd600640 1 -- 172.21.15.42:0/2440712713 wait complete. 2024-04-25T00:42:24.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 377333 32.83 MB/sec execute 494 sec latency 240.632 ms 2024-04-25T00:42:25.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 377908 32.85 MB/sec execute 495 sec latency 163.762 ms 2024-04-25T00:42:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:25 smithi136 ceph-mon[32923]: pgmap v1100: 97 pgs: 97 active+clean; 660 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.4 MiB/s wr, 463 op/s 2024-04-25T00:42:26.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:25 smithi106 ceph-mon[32510]: pgmap v1100: 97 pgs: 97 active+clean; 660 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.4 MiB/s wr, 463 op/s 2024-04-25T00:42:26.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:25 smithi042 ceph-mon[28130]: pgmap v1100: 97 pgs: 97 active+clean; 660 MiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 3.4 MiB/s wr, 463 op/s 2024-04-25T00:42:26.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 378417 32.85 MB/sec execute 496 sec latency 121.809 ms 2024-04-25T00:42:27.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 378895 32.81 MB/sec execute 497 sec latency 67.730 ms 2024-04-25T00:42:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:27 smithi136 ceph-mon[32923]: pgmap v1101: 97 pgs: 97 active+clean; 680 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.9 MiB/s wr, 675 op/s 2024-04-25T00:42:28.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:27 smithi106 ceph-mon[32510]: pgmap v1101: 97 pgs: 97 active+clean; 680 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.9 MiB/s wr, 675 op/s 2024-04-25T00:42:28.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:27 smithi042 ceph-mon[28130]: pgmap v1101: 97 pgs: 97 active+clean; 680 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 5.9 MiB/s wr, 675 op/s 2024-04-25T00:42:28.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 379477 32.80 MB/sec execute 498 sec latency 83.978 ms 2024-04-25T00:42:29.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 380201 32.82 MB/sec execute 499 sec latency 55.241 ms 2024-04-25T00:42:30.178 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:29 smithi042 ceph-mon[28130]: pgmap v1102: 97 pgs: 97 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.2 MiB/s wr, 526 op/s 2024-04-25T00:42:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:29 smithi136 ceph-mon[32923]: pgmap v1102: 97 pgs: 97 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.2 MiB/s wr, 526 op/s 2024-04-25T00:42:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:29 smithi106 ceph-mon[32510]: pgmap v1102: 97 pgs: 97 active+clean; 676 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 MiB/s rd, 5.2 MiB/s wr, 526 op/s 2024-04-25T00:42:30.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 380836 32.82 MB/sec execute 500 sec latency 66.782 ms 2024-04-25T00:42:31.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 381118 32.79 MB/sec execute 501 sec latency 524.508 ms 2024-04-25T00:42:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:31 smithi136 ceph-mon[32923]: pgmap v1103: 97 pgs: 97 active+clean; 677 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.9 MiB/s wr, 463 op/s 2024-04-25T00:42:32.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:31 smithi106 ceph-mon[32510]: pgmap v1103: 97 pgs: 97 active+clean; 677 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.9 MiB/s wr, 463 op/s 2024-04-25T00:42:32.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:31 smithi042 ceph-mon[28130]: pgmap v1103: 97 pgs: 97 active+clean; 677 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 4.9 MiB/s wr, 463 op/s 2024-04-25T00:42:32.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 381422 32.77 MB/sec execute 502 sec latency 760.502 ms 2024-04-25T00:42:33.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 381810 32.74 MB/sec execute 503 sec latency 127.527 ms 2024-04-25T00:42:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:33 smithi136 ceph-mon[32923]: pgmap v1104: 97 pgs: 97 active+clean; 691 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.2 MiB/s wr, 648 op/s 2024-04-25T00:42:34.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:33 smithi106 ceph-mon[32510]: pgmap v1104: 97 pgs: 97 active+clean; 691 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.2 MiB/s wr, 648 op/s 2024-04-25T00:42:34.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:33 smithi042 ceph-mon[28130]: pgmap v1104: 97 pgs: 97 active+clean; 691 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 7.2 MiB/s wr, 648 op/s 2024-04-25T00:42:34.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 382458 32.73 MB/sec execute 504 sec latency 56.397 ms 2024-04-25T00:42:35.813 INFO:tasks.workunit.client.0.smithi042.stdout: 10 382964 32.72 MB/sec execute 505 sec latency 113.161 ms 2024-04-25T00:42:36.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:35 smithi136 ceph-mon[32923]: pgmap v1105: 97 pgs: 97 active+clean; 687 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.4 MiB/s wr, 508 op/s 2024-04-25T00:42:36.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:35 smithi106 ceph-mon[32510]: pgmap v1105: 97 pgs: 97 active+clean; 687 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.4 MiB/s wr, 508 op/s 2024-04-25T00:42:36.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:35 smithi042 ceph-mon[28130]: pgmap v1105: 97 pgs: 97 active+clean; 687 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 6.4 MiB/s wr, 508 op/s 2024-04-25T00:42:36.814 INFO:tasks.workunit.client.0.smithi042.stdout: 10 383614 32.75 MB/sec execute 506 sec latency 59.007 ms 2024-04-25T00:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.631+0000 7f3802200640 1 -- 172.21.15.42:0/1727780249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37fc1031b0 msgr2=0x7f37fc109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.631+0000 7f3802200640 1 --2- 172.21.15.42:0/1727780249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37fc1031b0 0x7f37fc109a40 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009ca0 tx=0x7f37f402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.632+0000 7f3802200640 1 -- 172.21.15.42:0/1727780249 shutdown_connections 2024-04-25T00:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.632+0000 7f3802200640 1 --2- 172.21.15.42:0/1727780249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37fc1031b0 0x7f37fc109a40 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.632+0000 7f3802200640 1 --2- 172.21.15.42:0/1727780249 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37fc1027f0 0x7f37fc102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.632+0000 7f3802200640 1 --2- 172.21.15.42:0/1727780249 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37fc101600 0x7f37fc101a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.632+0000 7f3802200640 1 -- 172.21.15.42:0/1727780249 >> 172.21.15.42:0/1727780249 conn(0x7f37fc0fcb70 msgr2=0x7f37fc0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.632+0000 7f3802200640 1 -- 172.21.15.42:0/1727780249 shutdown_connections 2024-04-25T00:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.632+0000 7f3802200640 1 -- 172.21.15.42:0/1727780249 wait complete. 2024-04-25T00:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.633+0000 7f3802200640 1 Processor -- start 2024-04-25T00:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.633+0000 7f3802200640 1 -- start start 2024-04-25T00:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.633+0000 7f3802200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37fc101600 0x7f37fc199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f3802200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37fc1027f0 0x7f37fc199fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f3802200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37fc1031b0 0x7f37fc1a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f3802200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc10d2b0 con 0x7f37fc101600 2024-04-25T00:42:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f3802200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc10d130 con 0x7f37fc1031b0 2024-04-25T00:42:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f3802200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc10d430 con 0x7f37fc1027f0 2024-04-25T00:42:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f37fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37fc1027f0 0x7f37fc199fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f3800e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37fc101600 0x7f37fc199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f37fbe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37fc1027f0 0x7f37fc199fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:42760/0 (socket says 172.21.15.42:42760) 2024-04-25T00:42:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f37fbe00640 1 -- 172.21.15.42:0/2612734038 learned_addr learned my addr 172.21.15.42:0/2612734038 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:37.637 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f3801800640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37fc1031b0 0x7f37fc1a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.638 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f37fbe00640 1 -- 172.21.15.42:0/2612734038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37fc1031b0 msgr2=0x7f37fc1a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f37fbe00640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37fc1031b0 0x7f37fc1a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f37fbe00640 1 -- 172.21.15.42:0/2612734038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37fc101600 msgr2=0x7f37fc199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f37fbe00640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37fc101600 0x7f37fc199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.634+0000 7f37fbe00640 1 -- 172.21.15.42:0/2612734038 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37fc1a16a0 con 0x7f37fc1027f0 2024-04-25T00:42:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.635+0000 7f37fbe00640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37fc1027f0 0x7f37fc199fc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f37e800e9f0 tx=0x7f37e800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.635+0000 7f37f9600640 1 -- 172.21.15.42:0/2612734038 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e800cd80 con 0x7f37fc1027f0 2024-04-25T00:42:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.635+0000 7f37f9600640 1 -- 172.21.15.42:0/2612734038 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37e8010910 con 0x7f37fc1027f0 2024-04-25T00:42:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.635+0000 7f3802200640 1 -- 172.21.15.42:0/2612734038 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37fc1a1990 con 0x7f37fc1027f0 2024-04-25T00:42:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.635+0000 7f37f9600640 1 -- 172.21.15.42:0/2612734038 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80188f0 con 0x7f37fc1027f0 2024-04-25T00:42:37.639 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.635+0000 7f3802200640 1 -- 172.21.15.42:0/2612734038 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37fc1a1ee0 con 0x7f37fc1027f0 2024-04-25T00:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.640+0000 7f3806583740 1 Processor -- start 2024-04-25T00:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.640+0000 7f37f9600640 1 -- 172.21.15.42:0/2612734038 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37e8010ac0 con 0x7f37fc1027f0 2024-04-25T00:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.640+0000 7f3806583740 1 -- start start 2024-04-25T00:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.640+0000 7f3806583740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58da270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.640+0000 7f3806583740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58da840 0x560ad58df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.640+0000 7f3806583740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58df9b0 0x560ad58e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.640+0000 7f3806583740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560ad583cab0 con 0x560ad58df9b0 2024-04-25T00:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3806583740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560ad57e5640 con 0x560ad58d9e50 2024-04-25T00:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3806583740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560ad5838a70 con 0x560ad58da840 2024-04-25T00:42:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3801800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58da270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3800e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58da840 0x560ad58df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3801800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58da270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39592/0 (socket says 172.21.15.42:39592) 2024-04-25T00:42:37.643 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3801800640 1 -- 172.21.15.42:0/257530816 learned_addr learned my addr 172.21.15.42:0/257530816 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f37fbe00640 1 --2- 172.21.15.42:0/257530816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58df9b0 0x560ad58e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3801800640 1 -- 172.21.15.42:0/257530816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58da840 msgr2=0x560ad58df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3801800640 1 --2- 172.21.15.42:0/257530816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58da840 0x560ad58df340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3801800640 1 -- 172.21.15.42:0/257530816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58df9b0 msgr2=0x560ad58e1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3801800640 1 --2- 172.21.15.42:0/257530816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58df9b0 0x560ad58e1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f3801800640 1 -- 172.21.15.42:0/257530816 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad5551210 con 0x560ad58d9e50 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.641+0000 7f37fbe00640 1 --2- 172.21.15.42:0/257530816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58df9b0 0x560ad58e1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.642+0000 7f3801800640 1 --2- 172.21.15.42:0/257530816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58da270 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009ca0 tx=0x7f37f40093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12e67523e48f9d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.642+0000 7f37ee000640 1 -- 172.21.15.42:0/257530816 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4031650 con 0x560ad58d9e50 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.642+0000 7f37ee000640 1 -- 172.21.15.42:0/257530816 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f4036030 con 0x560ad58d9e50 2024-04-25T00:42:37.644 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.642+0000 7f37ee000640 1 -- 172.21.15.42:0/257530816 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f4004740 con 0x560ad58d9e50 2024-04-25T00:42:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.643+0000 7f37f9600640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37e40a2b80 0x7f37e40a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.643+0000 7f3806583740 1 -- 172.21.15.42:0/257530816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 msgr2=0x560ad58da270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.643+0000 7f3806583740 1 --2- 172.21.15.42:0/257530816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58da270 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f37f4009ca0 tx=0x7f37f40093c0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.643+0000 7f37f9600640 1 -- 172.21.15.42:0/2612734038 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f37e80cf400 con 0x7f37fc1027f0 2024-04-25T00:42:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.643+0000 7f3800e00640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37e40a2b80 0x7f37e40a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.643+0000 7f3806583740 1 -- 172.21.15.42:0/257530816 shutdown_connections 2024-04-25T00:42:37.645 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.643+0000 7f3806583740 1 --2- 172.21.15.42:0/257530816 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58df9b0 0x560ad58e1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.644+0000 7f3806583740 1 --2- 172.21.15.42:0/257530816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58da840 0x560ad58df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.644+0000 7f3806583740 1 --2- 172.21.15.42:0/257530816 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58da270 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.644+0000 7f3806583740 1 -- 172.21.15.42:0/257530816 >> 172.21.15.42:0/257530816 conn(0x560ad58d54c0 msgr2=0x560ad58d58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.644+0000 7f3806583740 1 -- 172.21.15.42:0/257530816 shutdown_connections 2024-04-25T00:42:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.644+0000 7f3806583740 1 -- 172.21.15.42:0/257530816 wait complete. 2024-04-25T00:42:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.645+0000 7f3806583740 1 -- start start 2024-04-25T00:42:37.646 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.645+0000 7f3800e00640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37e40a2b80 0x7f37e40a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x560ad58d6390 tx=0x7f37f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.645+0000 7f3806583740 1 Processor -- start 2024-04-25T00:42:37.647 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.646+0000 7f3806583740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58d9e50 0x560ad58e3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.646+0000 7f3806583740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 0x560ad58e4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.646+0000 7f3806583740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58eb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.646+0000 7f3806583740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x560ad58e2780 con 0x560ad58da840 2024-04-25T00:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.646+0000 7f3806583740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x560ad57f8990 con 0x560ad58df9b0 2024-04-25T00:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.646+0000 7f3806583740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560ad58e2900 con 0x560ad58d9e50 2024-04-25T00:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.646+0000 7f3806583740 10 client.0 _ll_register_callbacks cb 0x560ad57fa950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.646+0000 7f3800e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58d9e50 0x560ad58e3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.646+0000 7f3801800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58eb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.647+0000 7f3801800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58eb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39594/0 (socket says 172.21.15.42:39594) 2024-04-25T00:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.647+0000 7f3801800640 1 -- 172.21.15.42:0/807369305 learned_addr learned my addr 172.21.15.42:0/807369305 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.647+0000 7f37fbe00640 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 0x560ad58e4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.647+0000 7f3801800640 1 -- 172.21.15.42:0/807369305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58d9e50 msgr2=0x560ad58e3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.647+0000 7f3801800640 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58d9e50 0x560ad58e3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.647+0000 7f3801800640 1 -- 172.21.15.42:0/807369305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 msgr2=0x560ad58e4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.647+0000 7f3801800640 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 0x560ad58e4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.647+0000 7f3801800640 1 -- 172.21.15.42:0/807369305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad561d620 con 0x560ad58df9b0 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.647+0000 7f37fbe00640 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 0x560ad58e4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.648+0000 7f3801800640 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58eb470 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f37f402ade0 tx=0x7f37f4031680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f441ab2b359689c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.648+0000 7f37e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.648+0000 7f37e2a00640 1 -- 172.21.15.42:0/807369305 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f404a040 con 0x560ad58df9b0 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.648+0000 7f37e2a00640 1 -- 172.21.15.42:0/807369305 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f37f4003970 con 0x560ad58df9b0 2024-04-25T00:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.648+0000 7f3806583740 1 -- 172.21.15.42:0/807369305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f4036030 con 0x560ad58df9b0 2024-04-25T00:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f37e2a00640 1 -- 172.21.15.42:0/807369305 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f37f4003b70 con 0x560ad58df9b0 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f3806583740 10 client.27011 fetch_fsmap learned FSMap version 45 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f3806583740 1 -- 172.21.15.42:0/807369305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560ad58e2780 con 0x560ad58df9b0 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f37e2a00640 1 -- 172.21.15.42:0/807369305 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f4036030 con 0x560ad58df9b0 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f37e2a00640 1 -- 172.21.15.42:0/807369305 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f37f404d030 con 0x560ad58df9b0 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f3806583740 10 client.27011 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f3806583740 10 client.27011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f3806583740 20 client.27011 populate_metadata read hostname 'smithi042' 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f3806583740 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560ad58ef7c0 0x560ad58f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f3806583740 4 client.27011 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.649+0000 7f3806583740 1 -- 172.21.15.42:0/807369305 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ad5849ff0 con 0x560ad58ef7c0 2024-04-25T00:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.650+0000 7f3800e00640 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560ad58ef7c0 0x560ad58f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.651+0000 7f3800e00640 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560ad58ef7c0 0x560ad58f1bb0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a58d931b4d4d223 server_cookie=3dbd2972a4e496df in_seq=0 out_seq=0 2024-04-25T00:42:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.651+0000 7f37e2a00640 10 client.27011 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f37e2a00640 1 -- 172.21.15.42:0/807369305 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ad5849ff0 con 0x560ad58ef7c0 2024-04-25T00:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f37e2a00640 10 client.27011 handle_command_reply: tid=0 2024-04-25T00:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f3806583740 1 client.27011 shutdown 2024-04-25T00:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f3806583740 1 -- 172.21.15.42:0/807369305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 msgr2=0x560ad58eb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f3806583740 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58eb470 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f37f402ade0 tx=0x7f37f4031680 comp rx=0 tx=0).stop 2024-04-25T00:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f3806583740 1 -- 172.21.15.42:0/807369305 shutdown_connections 2024-04-25T00:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f3806583740 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560ad58ef7c0 0x560ad58f1bb0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f3806583740 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58eb470 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f3806583740 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 0x560ad58e4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f3806583740 1 --2- 172.21.15.42:0/807369305 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58d9e50 0x560ad58e3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.653+0000 7f37e2a00640 0 client.27011 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.654+0000 7f3806583740 1 -- 172.21.15.42:0/807369305 >> 172.21.15.42:0/807369305 conn(0x560ad58d54c0 msgr2=0x560ad58d4620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.654+0000 7f3806583740 1 -- 172.21.15.42:0/807369305 shutdown_connections 2024-04-25T00:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.654+0000 7f3806583740 1 -- 172.21.15.42:0/807369305 wait complete. 2024-04-25T00:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.654+0000 7f3806583740 20 client.27011 trim_cache size 0 max 16384 2024-04-25T00:42:37.678 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.677+0000 7f3806583740 1 Processor -- start 2024-04-25T00:42:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.677+0000 7f3806583740 1 -- start start 2024-04-25T00:42:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f3806583740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58d9e50 0x560ad58d8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f3806583740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58da840 0x560ad58eb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f3806583740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58f6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.679 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f3806583740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1ad450 con 0x560ad58d9e50 2024-04-25T00:42:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f3806583740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1aad20 con 0x560ad58df9b0 2024-04-25T00:42:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f3806583740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1a8b70 con 0x560ad58da840 2024-04-25T00:42:37.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f3801800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f3800e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58d9e50 0x560ad58d8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f3801800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39604/0 (socket says 172.21.15.42:39604) 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f3801800640 1 -- 172.21.15.42:0/2317778533 learned_addr learned my addr 172.21.15.42:0/2317778533 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.678+0000 7f37fbe00640 1 --2- 172.21.15.42:0/2317778533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58da840 0x560ad58eb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.679+0000 7f3801800640 1 -- 172.21.15.42:0/2317778533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58da840 msgr2=0x560ad58eb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.679+0000 7f3801800640 1 --2- 172.21.15.42:0/2317778533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58da840 0x560ad58eb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.679+0000 7f3801800640 1 -- 172.21.15.42:0/2317778533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58d9e50 msgr2=0x560ad58d8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.679+0000 7f3801800640 1 --2- 172.21.15.42:0/2317778533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58d9e50 0x560ad58d8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.679+0000 7f3801800640 1 -- 172.21.15.42:0/2317778533 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ad58d3a10 con 0x560ad58df9b0 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.679+0000 7f3801800640 1 --2- 172.21.15.42:0/2317778533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58f6420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f37f4031ae0 tx=0x7f37f4005d80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13b6ea4bccc4f3d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.679+0000 7f37ed600640 1 -- 172.21.15.42:0/2317778533 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f40316f0 con 0x560ad58df9b0 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.679+0000 7f37ed600640 1 -- 172.21.15.42:0/2317778533 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f4036070 con 0x560ad58df9b0 2024-04-25T00:42:37.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.679+0000 7f37ed600640 1 -- 172.21.15.42:0/2317778533 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f40305c0 con 0x560ad58df9b0 2024-04-25T00:42:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.680+0000 7f3806583740 1 -- 172.21.15.42:0/2317778533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 msgr2=0x560ad58f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.680+0000 7f3806583740 1 --2- 172.21.15.42:0/2317778533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58f6420 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f37f4031ae0 tx=0x7f37f4005d80 comp rx=0 tx=0).stop 2024-04-25T00:42:37.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.680+0000 7f3806583740 1 -- 172.21.15.42:0/2317778533 shutdown_connections 2024-04-25T00:42:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.680+0000 7f3806583740 1 --2- 172.21.15.42:0/2317778533 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58df9b0 0x560ad58f6420 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.680+0000 7f3806583740 1 --2- 172.21.15.42:0/2317778533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58da840 0x560ad58eb390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.680+0000 7f3806583740 1 --2- 172.21.15.42:0/2317778533 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58d9e50 0x560ad58d8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.680+0000 7f3806583740 1 -- 172.21.15.42:0/2317778533 >> 172.21.15.42:0/2317778533 conn(0x560ad58d54c0 msgr2=0x560ad58d83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.680+0000 7f3806583740 1 -- 172.21.15.42:0/2317778533 shutdown_connections 2024-04-25T00:42:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.680+0000 7f3806583740 1 -- 172.21.15.42:0/2317778533 wait complete. 2024-04-25T00:42:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.681+0000 7f3806583740 1 -- start start 2024-04-25T00:42:37.683 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.681+0000 7f3806583740 1 Processor -- start 2024-04-25T00:42:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3806583740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58e8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3806583740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 0x560ad58e8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3806583740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58df9b0 0x560ad58e9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3806583740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1a91e0 con 0x560ad58da840 2024-04-25T00:42:37.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3806583740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1aa650 con 0x560ad58d9e50 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3806583740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f37fc1a98b0 con 0x560ad58df9b0 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3806583740 10 client.0 _ll_register_callbacks cb 0x7f37f00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3800e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58e8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f37fbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 0x560ad58e8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3800e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58e8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:39618/0 (socket says 172.21.15.42:39618) 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3800e00640 1 -- 172.21.15.42:0/1868561219 learned_addr learned my addr 172.21.15.42:0/1868561219 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3801800640 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58df9b0 0x560ad58e9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f37fbe00640 1 -- 172.21.15.42:0/1868561219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58df9b0 msgr2=0x560ad58e9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f37fbe00640 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58df9b0 0x560ad58e9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f37fbe00640 1 -- 172.21.15.42:0/1868561219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 msgr2=0x560ad58e8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f37fbe00640 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58e8940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f37fbe00640 1 -- 172.21.15.42:0/1868561219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ad58ea480 con 0x560ad58da840 2024-04-25T00:42:37.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.682+0000 7f3800e00640 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.683+0000 7f37fbe00640 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 0x560ad58e8e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c6a0 tx=0x7f37e809ffe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ca6f91db9d611c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.683+0000 7f37e2a00640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.683+0000 7f37e2a00640 1 -- 172.21.15.42:0/1868561219 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e80cd0a0 con 0x560ad58da840 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.683+0000 7f37e2a00640 1 -- 172.21.15.42:0/1868561219 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f37e8010430 con 0x560ad58da840 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.683+0000 7f3806583740 1 -- 172.21.15.42:0/1868561219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37f4036070 con 0x560ad58da840 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.683+0000 7f37e2a00640 1 -- 172.21.15.42:0/1868561219 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f37e8010630 con 0x560ad58da840 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.683+0000 7f3806583740 10 client.16710 fetch_fsmap learned FSMap version 45 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.683+0000 7f3806583740 1 -- 172.21.15.42:0/1868561219 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x560ad5955ca0 con 0x560ad58da840 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.684+0000 7f37e2a00640 1 -- 172.21.15.42:0/1868561219 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f37f4036070 con 0x560ad58da840 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.684+0000 7f37e2a00640 1 -- 172.21.15.42:0/1868561219 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f37e8018a90 con 0x560ad58da840 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.684+0000 7f3806583740 10 client.16710 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.684+0000 7f3806583740 10 client.16710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.684+0000 7f3806583740 20 client.16710 populate_metadata read hostname 'smithi042' 2024-04-25T00:42:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.684+0000 7f3806583740 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560ad5956440 0x560ad5958860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.684+0000 7f3806583740 4 client.16710 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.684+0000 7f3806583740 1 -- 172.21.15.42:0/1868561219 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560ad5844340 con 0x560ad5956440 2024-04-25T00:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.684+0000 7f3800e00640 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560ad5956440 0x560ad5958860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.685+0000 7f3800e00640 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560ad5956440 0x560ad5958860 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1bb227562bd7e16 server_cookie=fdc48a33ed8c6578 in_seq=0 out_seq=0 2024-04-25T00:42:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:37.685+0000 7f37e2a00640 10 client.16710 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:37.814 INFO:tasks.workunit.client.0.smithi042.stdout: 10 384223 32.73 MB/sec execute 507 sec latency 85.697 ms 2024-04-25T00:42:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.078+0000 7f37e2a00640 1 -- 172.21.15.42:0/1868561219 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560ad5844340 con 0x560ad5956440 2024-04-25T00:42:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.078+0000 7f37e2a00640 10 client.16710 handle_command_reply: tid=0 2024-04-25T00:42:38.080 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.078+0000 7f3806583740 1 client.16710 shutdown 2024-04-25T00:42:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.079+0000 7f3806583740 1 -- 172.21.15.42:0/1868561219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 msgr2=0x560ad58e8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.079+0000 7f3806583740 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 0x560ad58e8e80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f37e800c6a0 tx=0x7f37e809ffe0 comp rx=0 tx=0).stop 2024-04-25T00:42:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.079+0000 7f3806583740 1 -- 172.21.15.42:0/1868561219 shutdown_connections 2024-04-25T00:42:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.079+0000 7f3806583740 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x560ad5956440 0x560ad5958860 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.079+0000 7f3806583740 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ad58df9b0 0x560ad58e9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.079+0000 7f3806583740 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x560ad58da840 0x560ad58e8e80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:38.081 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.079+0000 7f3806583740 1 --2- 172.21.15.42:0/1868561219 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x560ad58d9e50 0x560ad58e8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.079+0000 7f3806583740 1 -- 172.21.15.42:0/1868561219 >> 172.21.15.42:0/1868561219 conn(0x560ad58d54c0 msgr2=0x560ad58d83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.079+0000 7f37e2a00640 0 client.16710 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.079+0000 7f3806583740 1 -- 172.21.15.42:0/1868561219 shutdown_connections 2024-04-25T00:42:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.080+0000 7f3806583740 1 -- 172.21.15.42:0/1868561219 wait complete. 2024-04-25T00:42:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.080+0000 7f3806583740 20 client.16710 trim_cache size 0 max 16384 2024-04-25T00:42:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:42:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:42:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:42:38.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:42:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 -- 172.21.15.42:0/2612734038 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37e40a2b80 msgr2=0x7f37e40a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37e40a2b80 0x7f37e40a5040 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x560ad58d6390 tx=0x7f37f0005f90 comp rx=0 tx=0).stop 2024-04-25T00:42:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 -- 172.21.15.42:0/2612734038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37fc1027f0 msgr2=0x7f37fc199fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37fc1027f0 0x7f37fc199fc0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f37e800e9f0 tx=0x7f37e800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:42:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 -- 172.21.15.42:0/2612734038 shutdown_connections 2024-04-25T00:42:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f37e40a2b80 0x7f37e40a5040 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37fc1031b0 0x7f37fc1a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:38.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37fc1027f0 0x7f37fc199fc0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 --2- 172.21.15.42:0/2612734038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f37fc101600 0x7f37fc199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 -- 172.21.15.42:0/2612734038 >> 172.21.15.42:0/2612734038 conn(0x7f37fc0fcb70 msgr2=0x7f37fc073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.081+0000 7f3802200640 1 -- 172.21.15.42:0/2612734038 shutdown_connections 2024-04-25T00:42:38.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:38.082+0000 7f3802200640 1 -- 172.21.15.42:0/2612734038 wait complete. 2024-04-25T00:42:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:37 smithi136 ceph-mon[32923]: pgmap v1106: 97 pgs: 97 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 736 op/s 2024-04-25T00:42:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:37 smithi106 ceph-mon[32510]: pgmap v1106: 97 pgs: 97 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 736 op/s 2024-04-25T00:42:38.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:37 smithi042 ceph-mon[28130]: pgmap v1106: 97 pgs: 97 active+clean; 666 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 11 MiB/s wr, 736 op/s 2024-04-25T00:42:38.814 INFO:tasks.workunit.client.0.smithi042.stdout: 10 384703 32.72 MB/sec execute 508 sec latency 248.852 ms 2024-04-25T00:42:39.814 INFO:tasks.workunit.client.0.smithi042.stdout: 10 385360 32.74 MB/sec execute 509 sec latency 66.177 ms 2024-04-25T00:42:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:39 smithi136 ceph-mon[32923]: pgmap v1107: 97 pgs: 97 active+clean; 652 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.1 MiB/s wr, 536 op/s 2024-04-25T00:42:40.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:39 smithi106 ceph-mon[32510]: pgmap v1107: 97 pgs: 97 active+clean; 652 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.1 MiB/s wr, 536 op/s 2024-04-25T00:42:40.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:39 smithi042 ceph-mon[28130]: pgmap v1107: 97 pgs: 97 active+clean; 652 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.1 MiB/s wr, 536 op/s 2024-04-25T00:42:40.814 INFO:tasks.workunit.client.0.smithi042.stdout: 10 385986 32.72 MB/sec execute 510 sec latency 71.316 ms 2024-04-25T00:42:41.814 INFO:tasks.workunit.client.0.smithi042.stdout: 10 386638 32.73 MB/sec execute 511 sec latency 76.324 ms 2024-04-25T00:42:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:41 smithi136 ceph-mon[32923]: pgmap v1108: 97 pgs: 97 active+clean; 653 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.0 MiB/s wr, 528 op/s 2024-04-25T00:42:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:41 smithi106 ceph-mon[32510]: pgmap v1108: 97 pgs: 97 active+clean; 653 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.0 MiB/s wr, 528 op/s 2024-04-25T00:42:42.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:41 smithi042 ceph-mon[28130]: pgmap v1108: 97 pgs: 97 active+clean; 653 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 8.0 MiB/s wr, 528 op/s 2024-04-25T00:42:42.814 INFO:tasks.workunit.client.0.smithi042.stdout: 10 387308 32.74 MB/sec execute 512 sec latency 95.095 ms 2024-04-25T00:42:43.814 INFO:tasks.workunit.client.0.smithi042.stdout: 10 387961 32.74 MB/sec execute 513 sec latency 47.591 ms 2024-04-25T00:42:44.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:43 smithi136 ceph-mon[32923]: pgmap v1109: 97 pgs: 97 active+clean; 657 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.1 MiB/s wr, 712 op/s 2024-04-25T00:42:44.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:43 smithi106 ceph-mon[32510]: pgmap v1109: 97 pgs: 97 active+clean; 657 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.1 MiB/s wr, 712 op/s 2024-04-25T00:42:44.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:43 smithi042 ceph-mon[28130]: pgmap v1109: 97 pgs: 97 active+clean; 657 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 9.1 MiB/s wr, 712 op/s 2024-04-25T00:42:44.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 388514 32.74 MB/sec execute 514 sec latency 72.052 ms 2024-04-25T00:42:45.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 389212 32.74 MB/sec execute 515 sec latency 155.423 ms 2024-04-25T00:42:46.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:45 smithi136 ceph-mon[32923]: pgmap v1110: 97 pgs: 97 active+clean; 660 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 568 op/s 2024-04-25T00:42:46.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:45 smithi106 ceph-mon[32510]: pgmap v1110: 97 pgs: 97 active+clean; 660 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 568 op/s 2024-04-25T00:42:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:45 smithi042 ceph-mon[28130]: pgmap v1110: 97 pgs: 97 active+clean; 660 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.0 MiB/s wr, 568 op/s 2024-04-25T00:42:46.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 389782 32.74 MB/sec execute 516 sec latency 76.806 ms 2024-04-25T00:42:47.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 390254 32.73 MB/sec execute 517 sec latency 244.117 ms 2024-04-25T00:42:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:47 smithi136 ceph-mon[32923]: pgmap v1111: 97 pgs: 97 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.5 MiB/s wr, 650 op/s 2024-04-25T00:42:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:47 smithi106 ceph-mon[32510]: pgmap v1111: 97 pgs: 97 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.5 MiB/s wr, 650 op/s 2024-04-25T00:42:48.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:47 smithi042 ceph-mon[28130]: pgmap v1111: 97 pgs: 97 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 7.5 MiB/s wr, 650 op/s 2024-04-25T00:42:48.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 390573 32.70 MB/sec execute 518 sec latency 404.699 ms 2024-04-25T00:42:49.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 391250 32.71 MB/sec execute 519 sec latency 53.551 ms 2024-04-25T00:42:50.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:49 smithi136 ceph-mon[32923]: pgmap v1112: 97 pgs: 97 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.4 MiB/s wr, 455 op/s 2024-04-25T00:42:50.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:49 smithi106 ceph-mon[32510]: pgmap v1112: 97 pgs: 97 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.4 MiB/s wr, 455 op/s 2024-04-25T00:42:50.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:49 smithi042 ceph-mon[28130]: pgmap v1112: 97 pgs: 97 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.4 MiB/s wr, 455 op/s 2024-04-25T00:42:50.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 391840 32.71 MB/sec execute 520 sec latency 54.080 ms 2024-04-25T00:42:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.259+0000 7fe52fe00640 1 -- 172.21.15.42:0/1665491964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5301049f0 msgr2=0x7fe530106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.259+0000 7fe52fe00640 1 --2- 172.21.15.42:0/1665491964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5301049f0 0x7fe530106de0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009900 tx=0x7fe51c02a400 comp rx=0 tx=0).stop 2024-04-25T00:42:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.259+0000 7fe52fe00640 1 -- 172.21.15.42:0/1665491964 shutdown_connections 2024-04-25T00:42:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.259+0000 7fe52fe00640 1 --2- 172.21.15.42:0/1665491964 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe530107320 0x7fe530109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.259+0000 7fe52fe00640 1 --2- 172.21.15.42:0/1665491964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5301049f0 0x7fe530106de0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.259+0000 7fe52fe00640 1 --2- 172.21.15.42:0/1665491964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5301020c0 0x7fe5301044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.259+0000 7fe52fe00640 1 -- 172.21.15.42:0/1665491964 >> 172.21.15.42:0/1665491964 conn(0x7fe5300fba30 msgr2=0x7fe5300fde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.259+0000 7fe52fe00640 1 -- 172.21.15.42:0/1665491964 shutdown_connections 2024-04-25T00:42:51.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.260+0000 7fe52fe00640 1 -- 172.21.15.42:0/1665491964 wait complete. 2024-04-25T00:42:51.262 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.261+0000 7fe52fe00640 1 Processor -- start 2024-04-25T00:42:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.261+0000 7fe52fe00640 1 -- start start 2024-04-25T00:42:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52fe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5301020c0 0x7fe5301956d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52fe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5301049f0 0x7fe530195c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52fe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe530107320 0x7fe5301a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52fe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe53010d270 con 0x7fe5301049f0 2024-04-25T00:42:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52fe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe53010d0f0 con 0x7fe530107320 2024-04-25T00:42:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52fe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe53010d3f0 con 0x7fe5301020c0 2024-04-25T00:42:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe530107320 0x7fe5301a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5301049f0 0x7fe530195c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe530107320 0x7fe5301a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60822/0 (socket says 172.21.15.42:60822) 2024-04-25T00:42:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52f400640 1 -- 172.21.15.42:0/2729474877 learned_addr learned my addr 172.21.15.42:0/2729474877 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.262+0000 7fe52ea00640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5301020c0 0x7fe5301956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.263+0000 7fe52f400640 1 -- 172.21.15.42:0/2729474877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5301020c0 msgr2=0x7fe5301956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.263+0000 7fe52f400640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5301020c0 0x7fe5301956d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.263+0000 7fe52f400640 1 -- 172.21.15.42:0/2729474877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5301049f0 msgr2=0x7fe530195c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.263+0000 7fe52f400640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5301049f0 0x7fe530195c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.263+0000 7fe52f400640 1 -- 172.21.15.42:0/2729474877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5301a5b10 con 0x7fe530107320 2024-04-25T00:42:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.263+0000 7fe52f400640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe530107320 0x7fe5301a55a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe52800eb10 tx=0x7fe52800c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.263+0000 7fe527400640 1 -- 172.21.15.42:0/2729474877 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528004b20 con 0x7fe530107320 2024-04-25T00:42:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.264+0000 7fe527400640 1 -- 172.21.15.42:0/2729474877 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe528004cc0 con 0x7fe530107320 2024-04-25T00:42:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.264+0000 7fe527400640 1 -- 172.21.15.42:0/2729474877 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5280107b0 con 0x7fe530107320 2024-04-25T00:42:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.264+0000 7fe52fe00640 1 -- 172.21.15.42:0/2729474877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5301a5e00 con 0x7fe530107320 2024-04-25T00:42:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.264+0000 7fe52fe00640 1 -- 172.21.15.42:0/2729474877 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5301a6270 con 0x7fe530107320 2024-04-25T00:42:51.270 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.268+0000 7fe527400640 1 -- 172.21.15.42:0/2729474877 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe528005000 con 0x7fe530107320 2024-04-25T00:42:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.271+0000 7fe538b77740 1 Processor -- start 2024-04-25T00:42:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.271+0000 7fe538b77740 1 -- start start 2024-04-25T00:42:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.271+0000 7fe538b77740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 0x564a2bded270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.271+0000 7fe538b77740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 0x564a2bdf2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.271+0000 7fe538b77740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bdf29b0 0x564a2bdf4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.271+0000 7fe538b77740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564a2bd4fab0 con 0x564a2bdece50 2024-04-25T00:42:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.271+0000 7fe538b77740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564a2bcf8640 con 0x564a2bded840 2024-04-25T00:42:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.271+0000 7fe538b77740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564a2bd4ba70 con 0x564a2bdf29b0 2024-04-25T00:42:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.272+0000 7fe52e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 0x564a2bded270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.272+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 0x564a2bdf2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.272+0000 7fe52e000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 0x564a2bded270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:58322/0 (socket says 172.21.15.42:58322) 2024-04-25T00:42:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.272+0000 7fe52e000640 1 -- 172.21.15.42:0/1981567146 learned_addr learned my addr 172.21.15.42:0/1981567146 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.272+0000 7fe52e000640 1 --2- 172.21.15.42:0/1981567146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bdf29b0 0x564a2bdf4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.272+0000 7fe527400640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5140a2c40 0x7fe5140a5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.274 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.273+0000 7fe52e000640 1 -- 172.21.15.42:0/1981567146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bdf29b0 msgr2=0x564a2bdf4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.273+0000 7fe52e000640 1 --2- 172.21.15.42:0/1981567146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bdf29b0 0x564a2bdf4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.273+0000 7fe52e000640 1 -- 172.21.15.42:0/1981567146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 msgr2=0x564a2bdf2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.273+0000 7fe52e000640 1 --2- 172.21.15.42:0/1981567146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 0x564a2bdf2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.273+0000 7fe52e000640 1 -- 172.21.15.42:0/1981567146 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2ba64210 con 0x564a2bdece50 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.273+0000 7fe527400640 1 -- 172.21.15.42:0/2729474877 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fe5280cf610 con 0x7fe530107320 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.273+0000 7fe52e000640 1 --2- 172.21.15.42:0/1981567146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 0x564a2bded270 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0057c0 tx=0x7fe51c02a400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0f8b5fa6d388e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.273+0000 7fe52f400640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5140a2c40 0x7fe5140a5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.273+0000 7fe51be00640 1 -- 172.21.15.42:0/1981567146 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c035070 con 0x564a2bdece50 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.274+0000 7fe51be00640 1 -- 172.21.15.42:0/1981567146 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c0327c0 con 0x564a2bdece50 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.274+0000 7fe51be00640 1 -- 172.21.15.42:0/1981567146 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c004a50 con 0x564a2bdece50 2024-04-25T00:42:51.275 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.274+0000 7fe538b77740 1 -- 172.21.15.42:0/1981567146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 msgr2=0x564a2bded270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.274+0000 7fe52f400640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5140a2c40 0x7fe5140a5100 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe5280a45e0 tx=0x7fe528009200 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.274+0000 7fe538b77740 1 --2- 172.21.15.42:0/1981567146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 0x564a2bded270 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe51c0057c0 tx=0x7fe51c02a400 comp rx=0 tx=0).stop 2024-04-25T00:42:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.275+0000 7fe538b77740 1 -- 172.21.15.42:0/1981567146 shutdown_connections 2024-04-25T00:42:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.275+0000 7fe538b77740 1 --2- 172.21.15.42:0/1981567146 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bdf29b0 0x564a2bdf4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.275+0000 7fe538b77740 1 --2- 172.21.15.42:0/1981567146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 0x564a2bdf2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.276 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.275+0000 7fe538b77740 1 --2- 172.21.15.42:0/1981567146 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 0x564a2bded270 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.275+0000 7fe538b77740 1 -- 172.21.15.42:0/1981567146 >> 172.21.15.42:0/1981567146 conn(0x564a2bde84c0 msgr2=0x564a2bde88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.275+0000 7fe538b77740 1 -- 172.21.15.42:0/1981567146 shutdown_connections 2024-04-25T00:42:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.275+0000 7fe538b77740 1 -- 172.21.15.42:0/1981567146 wait complete. 2024-04-25T00:42:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.276+0000 7fe538b77740 1 -- start start 2024-04-25T00:42:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.277+0000 7fe538b77740 1 Processor -- start 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe538b77740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bdece50 0x564a2bdf70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe538b77740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdf7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe538b77740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 0x564a2bdfe640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe538b77740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x564a2bdf5900 con 0x564a2bdf29b0 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe538b77740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x564a2bd0b990 con 0x564a2bdece50 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe538b77740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564a2bdf5a80 con 0x564a2bded840 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdf7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdf7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:36420/0 (socket says 172.21.15.42:36420) 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe52ea00640 1 -- 172.21.15.42:0/485219509 learned_addr learned my addr 172.21.15.42:0/485219509 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe538b77740 10 client.0 _ll_register_callbacks cb 0x564a2bd0d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe52ea00640 1 -- 172.21.15.42:0/485219509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bdece50 msgr2=0x564a2bdf70c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:42:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe52ea00640 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bdece50 0x564a2bdf70c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.278+0000 7fe52ea00640 1 -- 172.21.15.42:0/485219509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 msgr2=0x564a2bdfe640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.279+0000 7fe52e000640 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 0x564a2bdfe640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.279+0000 7fe52ea00640 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 0x564a2bdfe640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.279+0000 7fe52ea00640 1 -- 172.21.15.42:0/485219509 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2bb30620 con 0x564a2bded840 2024-04-25T00:42:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.279+0000 7fe52e000640 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 0x564a2bdfe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:42:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.279+0000 7fe52ea00640 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdf7600 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe520013710 tx=0x7fe520013bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5a66bf0ab5bd82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.279+0000 7fe518c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:42:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.279+0000 7fe518c00640 1 -- 172.21.15.42:0/485219509 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520011e30 con 0x564a2bded840 2024-04-25T00:42:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.279+0000 7fe538b77740 1 -- 172.21.15.42:0/485219509 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51c0327c0 con 0x564a2bded840 2024-04-25T00:42:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.279+0000 7fe518c00640 1 -- 172.21.15.42:0/485219509 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fe520004020 con 0x564a2bded840 2024-04-25T00:42:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.280+0000 7fe518c00640 1 -- 172.21.15.42:0/485219509 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe520004220 con 0x564a2bded840 2024-04-25T00:42:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.280+0000 7fe538b77740 10 client.26980 fetch_fsmap learned FSMap version 45 2024-04-25T00:42:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.280+0000 7fe538b77740 1 -- 172.21.15.42:0/485219509 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564a2be01720 con 0x564a2bded840 2024-04-25T00:42:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.280+0000 7fe518c00640 1 -- 172.21.15.42:0/485219509 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c0327c0 con 0x564a2bded840 2024-04-25T00:42:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.281+0000 7fe518c00640 1 -- 172.21.15.42:0/485219509 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fe52001a070 con 0x564a2bded840 2024-04-25T00:42:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.281+0000 7fe538b77740 10 client.26980 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:42:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.281+0000 7fe538b77740 10 client.26980 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:42:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.281+0000 7fe538b77740 20 client.26980 populate_metadata read hostname 'smithi042' 2024-04-25T00:42:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.281+0000 7fe538b77740 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564a2be02e10 0x564a2be052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.281+0000 7fe538b77740 4 client.26980 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:42:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.281+0000 7fe538b77740 1 -- 172.21.15.42:0/485219509 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a2bd5cff0 con 0x564a2be02e10 2024-04-25T00:42:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.282+0000 7fe52e000640 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564a2be02e10 0x564a2be052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.283 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.282+0000 7fe52e000640 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564a2be02e10 0x564a2be052d0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=401b3929761fc934 server_cookie=c751da0f804253ec in_seq=0 out_seq=0 2024-04-25T00:42:51.284 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.282+0000 7fe518c00640 10 client.26980 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:51.285 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe518c00640 1 -- 172.21.15.42:0/485219509 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a2bd5cff0 con 0x564a2be02e10 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe518c00640 10 client.26980 handle_command_reply: tid=0 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe538b77740 1 client.26980 shutdown 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe538b77740 1 -- 172.21.15.42:0/485219509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 msgr2=0x564a2bdf7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe538b77740 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdf7600 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe520013710 tx=0x7fe520013bd0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe538b77740 1 -- 172.21.15.42:0/485219509 shutdown_connections 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe538b77740 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564a2be02e10 0x564a2be052d0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe538b77740 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 0x564a2bdfe640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe538b77740 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdf7600 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe538b77740 1 --2- 172.21.15.42:0/485219509 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bdece50 0x564a2bdf70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe538b77740 1 -- 172.21.15.42:0/485219509 >> 172.21.15.42:0/485219509 conn(0x564a2bde84c0 msgr2=0x564a2bde7620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe518c00640 0 client.26980 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.284+0000 7fe538b77740 1 -- 172.21.15.42:0/485219509 shutdown_connections 2024-04-25T00:42:51.286 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.285+0000 7fe538b77740 1 -- 172.21.15.42:0/485219509 wait complete. 2024-04-25T00:42:51.287 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.285+0000 7fe538b77740 20 client.26980 trim_cache size 0 max 16384 2024-04-25T00:42:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.311+0000 7fe538b77740 1 Processor -- start 2024-04-25T00:42:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.311+0000 7fe538b77740 1 -- start start 2024-04-25T00:42:51.312 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.311+0000 7fe538b77740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 0x564a2bdeb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.311+0000 7fe538b77740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 0x564a2bdfe600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.311+0000 7fe538b77740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bdf29b0 0x564a2bdeb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.311+0000 7fe538b77740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301b1850 con 0x564a2bdece50 2024-04-25T00:42:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.311+0000 7fe538b77740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301af120 con 0x564a2bded840 2024-04-25T00:42:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.311+0000 7fe538b77740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301acf70 con 0x564a2bdf29b0 2024-04-25T00:42:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.312+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 0x564a2bdfe600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.313 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.312+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 0x564a2bdfe600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:60848/0 (socket says 172.21.15.42:60848) 2024-04-25T00:42:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.312+0000 7fe52ea00640 1 -- 172.21.15.42:0/1883788996 learned_addr learned my addr 172.21.15.42:0/1883788996 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.312+0000 7fe52e000640 1 --2- 172.21.15.42:0/1883788996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 0x564a2bdeb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.312+0000 7fe52e000640 1 --2- 172.21.15.42:0/1883788996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bdf29b0 0x564a2bdeb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.312+0000 7fe52ea00640 1 -- 172.21.15.42:0/1883788996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bdf29b0 msgr2=0x564a2bdeb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.312+0000 7fe52ea00640 1 --2- 172.21.15.42:0/1883788996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bdf29b0 0x564a2bdeb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.312+0000 7fe52ea00640 1 -- 172.21.15.42:0/1883788996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 msgr2=0x564a2bdeb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.312+0000 7fe52ea00640 1 --2- 172.21.15.42:0/1883788996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 0x564a2bdeb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.314 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.312+0000 7fe52ea00640 1 -- 172.21.15.42:0/1883788996 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a2bde82f0 con 0x564a2bded840 2024-04-25T00:42:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.313+0000 7fe52ea00640 1 --2- 172.21.15.42:0/1883788996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 0x564a2bdfe600 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe5200097b0 tx=0x7fe5200092c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f718bab314b36056 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.313+0000 7fe51b400640 1 -- 172.21.15.42:0/1883788996 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5200117b0 con 0x564a2bded840 2024-04-25T00:42:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.313+0000 7fe51b400640 1 -- 172.21.15.42:0/1883788996 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520003e60 con 0x564a2bded840 2024-04-25T00:42:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.313+0000 7fe51b400640 1 -- 172.21.15.42:0/1883788996 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520026840 con 0x564a2bded840 2024-04-25T00:42:51.315 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.314+0000 7fe538b77740 1 -- 172.21.15.42:0/1883788996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 msgr2=0x564a2bdfe600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.314+0000 7fe538b77740 1 --2- 172.21.15.42:0/1883788996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 0x564a2bdfe600 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe5200097b0 tx=0x7fe5200092c0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.314+0000 7fe538b77740 1 -- 172.21.15.42:0/1883788996 shutdown_connections 2024-04-25T00:42:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.314+0000 7fe538b77740 1 --2- 172.21.15.42:0/1883788996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bdf29b0 0x564a2bdeb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.314+0000 7fe538b77740 1 --2- 172.21.15.42:0/1883788996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bded840 0x564a2bdfe600 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.314+0000 7fe538b77740 1 --2- 172.21.15.42:0/1883788996 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdece50 0x564a2bdeb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.314+0000 7fe538b77740 1 -- 172.21.15.42:0/1883788996 >> 172.21.15.42:0/1883788996 conn(0x564a2bde84c0 msgr2=0x564a2bde3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.314+0000 7fe538b77740 1 -- 172.21.15.42:0/1883788996 shutdown_connections 2024-04-25T00:42:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.314+0000 7fe538b77740 1 -- 172.21.15.42:0/1883788996 wait complete. 2024-04-25T00:42:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.315+0000 7fe538b77740 1 -- start start 2024-04-25T00:42:51.316 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.315+0000 7fe538b77740 1 Processor -- start 2024-04-25T00:42:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.315+0000 7fe538b77740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bdece50 0x564a2bdfb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe538b77740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdfbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe538b77740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 0x564a2bdfce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe538b77740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301ad5e0 con 0x564a2bdf29b0 2024-04-25T00:42:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe538b77740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301aea50 con 0x564a2bdece50 2024-04-25T00:42:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe538b77740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5301adcb0 con 0x564a2bded840 2024-04-25T00:42:51.317 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe538b77740 10 client.0 _ll_register_callbacks cb 0x7fe5200096e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:42:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe52e000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bdece50 0x564a2bdfb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdfbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe52ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdfbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:36436/0 (socket says 172.21.15.42:36436) 2024-04-25T00:42:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe52ea00640 1 -- 172.21.15.42:0/3632605275 learned_addr learned my addr 172.21.15.42:0/3632605275 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:42:51.318 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.316+0000 7fe52e000640 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 0x564a2bdfce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.317+0000 7fe52ea00640 1 -- 172.21.15.42:0/3632605275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bdece50 msgr2=0x564a2bdfb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.317+0000 7fe52ea00640 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bdece50 0x564a2bdfb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.317+0000 7fe52ea00640 1 -- 172.21.15.42:0/3632605275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 msgr2=0x564a2bdfce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:42:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.317+0000 7fe52ea00640 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 0x564a2bdfce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.317+0000 7fe52ea00640 1 -- 172.21.15.42:0/3632605275 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a2bde9190 con 0x564a2bded840 2024-04-25T00:42:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.317+0000 7fe52ea00640 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdfbe90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe520000c00 tx=0x7fe520003a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e02a910c94a5b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:42:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.317+0000 7fe518c00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:42:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.318+0000 7fe518c00640 1 -- 172.21.15.42:0/3632605275 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520011c30 con 0x564a2bded840 2024-04-25T00:42:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.318+0000 7fe518c00640 1 -- 172.21.15.42:0/3632605275 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fe5200267e0 con 0x564a2bded840 2024-04-25T00:42:51.319 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.318+0000 7fe538b77740 1 -- 172.21.15.42:0/3632605275 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe520003e60 con 0x564a2bded840 2024-04-25T00:42:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.318+0000 7fe518c00640 1 -- 172.21.15.42:0/3632605275 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fe5200269e0 con 0x564a2bded840 2024-04-25T00:42:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.318+0000 7fe538b77740 10 client.26986 fetch_fsmap learned FSMap version 45 2024-04-25T00:42:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.318+0000 7fe538b77740 1 -- 172.21.15.42:0/3632605275 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x564a2bdfe560 con 0x564a2bded840 2024-04-25T00:42:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.318+0000 7fe518c00640 1 -- 172.21.15.42:0/3632605275 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe520003e60 con 0x564a2bded840 2024-04-25T00:42:51.320 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.319+0000 7fe518c00640 1 -- 172.21.15.42:0/3632605275 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fe52001a070 con 0x564a2bded840 2024-04-25T00:42:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.319+0000 7fe538b77740 10 client.26986 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:42:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.319+0000 7fe538b77740 10 client.26986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:42:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.319+0000 7fe538b77740 20 client.26986 populate_metadata read hostname 'smithi042' 2024-04-25T00:42:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.319+0000 7fe538b77740 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564a2be696f0 0x564a2be6bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:42:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.319+0000 7fe538b77740 4 client.26986 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:42:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.319+0000 7fe538b77740 1 -- 172.21.15.42:0/3632605275 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a2bd57340 con 0x564a2be696f0 2024-04-25T00:42:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.319+0000 7fe52e000640 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564a2be696f0 0x564a2be6bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:42:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.320+0000 7fe52e000640 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564a2be696f0 0x564a2be6bb10 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cfeaaf1b3179f59 server_cookie=9e2a8919fc765d4c in_seq=0 out_seq=0 2024-04-25T00:42:51.321 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.320+0000 7fe518c00640 10 client.26986 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.679+0000 7fe518c00640 1 -- 172.21.15.42:0/3632605275 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a2bd57340 con 0x564a2be696f0 2024-04-25T00:42:51.680 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.679+0000 7fe518c00640 10 client.26986 handle_command_reply: tid=0 2024-04-25T00:42:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.679+0000 7fe538b77740 1 client.26986 shutdown 2024-04-25T00:42:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.679+0000 7fe538b77740 1 -- 172.21.15.42:0/3632605275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 msgr2=0x564a2bdfbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.679+0000 7fe538b77740 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdfbe90 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe520000c00 tx=0x7fe520003a40 comp rx=0 tx=0).stop 2024-04-25T00:42:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.679+0000 7fe538b77740 1 -- 172.21.15.42:0/3632605275 shutdown_connections 2024-04-25T00:42:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.680+0000 7fe538b77740 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x564a2be696f0 0x564a2be6bb10 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.680+0000 7fe518c00640 0 client.26986 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:42:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.680+0000 7fe538b77740 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x564a2bdf29b0 0x564a2bdfce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.680+0000 7fe538b77740 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564a2bded840 0x564a2bdfbe90 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.681 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.680+0000 7fe538b77740 1 --2- 172.21.15.42:0/3632605275 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564a2bdece50 0x564a2bdfb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.680+0000 7fe538b77740 1 -- 172.21.15.42:0/3632605275 >> 172.21.15.42:0/3632605275 conn(0x564a2bde84c0 msgr2=0x564a2bdeb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.681+0000 7fe538b77740 1 -- 172.21.15.42:0/3632605275 shutdown_connections 2024-04-25T00:42:51.682 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.681+0000 7fe538b77740 1 -- 172.21.15.42:0/3632605275 wait complete. 2024-04-25T00:42:51.684 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.683+0000 7fe538b77740 20 client.26986 trim_cache size 0 max 16384 2024-04-25T00:42:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:42:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:42:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:42:51.685 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:42:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.685+0000 7fe52fe00640 1 -- 172.21.15.42:0/2729474877 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5140a2c40 msgr2=0x7fe5140a5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.685+0000 7fe52fe00640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5140a2c40 0x7fe5140a5100 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fe5280a45e0 tx=0x7fe528009200 comp rx=0 tx=0).stop 2024-04-25T00:42:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.685+0000 7fe52fe00640 1 -- 172.21.15.42:0/2729474877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe530107320 msgr2=0x7fe5301a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:42:51.686 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.685+0000 7fe52fe00640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe530107320 0x7fe5301a55a0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe52800eb10 tx=0x7fe52800c2d0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.686+0000 7fe52fe00640 1 -- 172.21.15.42:0/2729474877 shutdown_connections 2024-04-25T00:42:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.686+0000 7fe52fe00640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe5140a2c40 0x7fe5140a5100 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.686+0000 7fe52fe00640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe530107320 0x7fe5301a55a0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.686+0000 7fe52fe00640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe5301049f0 0x7fe530195c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.686+0000 7fe52fe00640 1 --2- 172.21.15.42:0/2729474877 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5301020c0 0x7fe5301956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:42:51.688 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.686+0000 7fe52fe00640 1 -- 172.21.15.42:0/2729474877 >> 172.21.15.42:0/2729474877 conn(0x7fe5300fba30 msgr2=0x7fe530101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:42:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.687+0000 7fe52fe00640 1 -- 172.21.15.42:0/2729474877 shutdown_connections 2024-04-25T00:42:51.689 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:42:51.687+0000 7fe52fe00640 1 -- 172.21.15.42:0/2729474877 wait complete. 2024-04-25T00:42:51.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 392296 32.70 MB/sec execute 521 sec latency 247.062 ms 2024-04-25T00:42:52.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:51 smithi136 ceph-mon[32923]: pgmap v1113: 97 pgs: 97 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.7 MiB/s wr, 439 op/s 2024-04-25T00:42:52.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:51 smithi106 ceph-mon[32510]: pgmap v1113: 97 pgs: 97 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.7 MiB/s wr, 439 op/s 2024-04-25T00:42:52.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:51 smithi042 ceph-mon[28130]: pgmap v1113: 97 pgs: 97 active+clean; 672 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.7 MiB/s wr, 439 op/s 2024-04-25T00:42:52.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 392618 32.66 MB/sec execute 522 sec latency 466.533 ms 2024-04-25T00:42:53.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:52 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:42:53.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:52 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:42:53.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:52 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:42:53.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 392808 32.62 MB/sec execute 523 sec latency 1061.481 ms 2024-04-25T00:42:54.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:53 smithi136 ceph-mon[32923]: pgmap v1114: 97 pgs: 97 active+clean; 678 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.3 MiB/s wr, 623 op/s 2024-04-25T00:42:54.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:53 smithi106 ceph-mon[32510]: pgmap v1114: 97 pgs: 97 active+clean; 678 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.3 MiB/s wr, 623 op/s 2024-04-25T00:42:54.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:53 smithi042 ceph-mon[28130]: pgmap v1114: 97 pgs: 97 active+clean; 678 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 6.3 MiB/s wr, 623 op/s 2024-04-25T00:42:54.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 393040 32.57 MB/sec execute 524 sec latency 245.330 ms 2024-04-25T00:42:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:42:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:42:55.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:54 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:42:55.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:42:55.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:42:55.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:54 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:42:55.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:42:55.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:42:55.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:54 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:42:55.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 393426 32.55 MB/sec execute 525 sec latency 187.251 ms 2024-04-25T00:42:56.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:55 smithi136 ceph-mon[32923]: pgmap v1115: 97 pgs: 97 active+clean; 681 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.5 MiB/s wr, 455 op/s 2024-04-25T00:42:56.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:55 smithi106 ceph-mon[32510]: pgmap v1115: 97 pgs: 97 active+clean; 681 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.5 MiB/s wr, 455 op/s 2024-04-25T00:42:56.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:55 smithi042 ceph-mon[28130]: pgmap v1115: 97 pgs: 97 active+clean; 681 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 5.5 MiB/s wr, 455 op/s 2024-04-25T00:42:56.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 393959 32.55 MB/sec execute 526 sec latency 72.599 ms 2024-04-25T00:42:57.815 INFO:tasks.workunit.client.0.smithi042.stdout: 10 394530 32.55 MB/sec execute 527 sec latency 114.344 ms 2024-04-25T00:42:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:57 smithi136 ceph-mon[32923]: pgmap v1116: 97 pgs: 97 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 10 MiB/s wr, 665 op/s 2024-04-25T00:42:58.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:57 smithi106 ceph-mon[32510]: pgmap v1116: 97 pgs: 97 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 10 MiB/s wr, 665 op/s 2024-04-25T00:42:58.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:57 smithi042 ceph-mon[28130]: pgmap v1116: 97 pgs: 97 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 10 MiB/s wr, 665 op/s 2024-04-25T00:42:58.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 395254 32.55 MB/sec execute 528 sec latency 60.395 ms 2024-04-25T00:42:59.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 395999 32.57 MB/sec execute 529 sec latency 45.018 ms 2024-04-25T00:43:00.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:42:59 smithi136 ceph-mon[32923]: pgmap v1117: 97 pgs: 97 active+clean; 671 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 678 op/s 2024-04-25T00:43:00.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:42:59 smithi106 ceph-mon[32510]: pgmap v1117: 97 pgs: 97 active+clean; 671 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 678 op/s 2024-04-25T00:43:00.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:42:59 smithi042 ceph-mon[28130]: pgmap v1117: 97 pgs: 97 active+clean; 671 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 11 MiB/s wr, 678 op/s 2024-04-25T00:43:00.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 396644 32.59 MB/sec execute 530 sec latency 51.535 ms 2024-04-25T00:43:01.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 397136 32.56 MB/sec execute 531 sec latency 49.936 ms 2024-04-25T00:43:02.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:02 smithi136 ceph-mon[32923]: pgmap v1118: 97 pgs: 97 active+clean; 673 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 691 op/s 2024-04-25T00:43:02.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:02 smithi106 ceph-mon[32510]: pgmap v1118: 97 pgs: 97 active+clean; 673 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 691 op/s 2024-04-25T00:43:02.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:02 smithi042 ceph-mon[28130]: pgmap v1118: 97 pgs: 97 active+clean; 673 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 11 MiB/s wr, 691 op/s 2024-04-25T00:43:02.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 397680 32.56 MB/sec execute 532 sec latency 88.076 ms 2024-04-25T00:43:03.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 398405 32.57 MB/sec execute 533 sec latency 90.563 ms 2024-04-25T00:43:04.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:04 smithi136 ceph-mon[32923]: pgmap v1119: 97 pgs: 97 active+clean; 660 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 801 op/s 2024-04-25T00:43:04.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:04 smithi106 ceph-mon[32510]: pgmap v1119: 97 pgs: 97 active+clean; 660 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 801 op/s 2024-04-25T00:43:04.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:04 smithi042 ceph-mon[28130]: pgmap v1119: 97 pgs: 97 active+clean; 660 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 12 MiB/s wr, 801 op/s 2024-04-25T00:43:04.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 399104 32.57 MB/sec execute 534 sec latency 71.181 ms 2024-04-25T00:43:04.887 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.886+0000 7f073f000640 1 -- 172.21.15.42:0/2138178052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f073810d390 msgr2=0x7f073810d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.888 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.886+0000 7f073f000640 1 --2- 172.21.15.42:0/2138178052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f073810d390 0x7f073810d810 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f0724009900 tx=0x7f072402a410 comp rx=0 tx=0).stop 2024-04-25T00:43:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.887+0000 7f073f000640 1 -- 172.21.15.42:0/2138178052 shutdown_connections 2024-04-25T00:43:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.887+0000 7f073f000640 1 --2- 172.21.15.42:0/2138178052 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073810dd50 0x7f07381145e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.887+0000 7f073f000640 1 --2- 172.21.15.42:0/2138178052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f073810d390 0x7f073810d810 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.887+0000 7f073f000640 1 --2- 172.21.15.42:0/2138178052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f073810c1a0 0x7f073810c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.887+0000 7f073f000640 1 -- 172.21.15.42:0/2138178052 >> 172.21.15.42:0/2138178052 conn(0x7f073806c340 msgr2=0x7f073806c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.887+0000 7f073f000640 1 -- 172.21.15.42:0/2138178052 shutdown_connections 2024-04-25T00:43:04.889 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.887+0000 7f073f000640 1 -- 172.21.15.42:0/2138178052 wait complete. 2024-04-25T00:43:04.890 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.889+0000 7f073f000640 1 Processor -- start 2024-04-25T00:43:04.891 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.890+0000 7f073f000640 1 -- start start 2024-04-25T00:43:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.891+0000 7f073f000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f073810c1a0 0x7f07381a23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.891+0000 7f073f000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f073810d390 0x7f07381a2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.891+0000 7f073f000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073810dd50 0x7f07381a9970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.892 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.891+0000 7f073f000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0738117e80 con 0x7f073810dd50 2024-04-25T00:43:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.891+0000 7f073f000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f0738117d00 con 0x7f073810c1a0 2024-04-25T00:43:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.891+0000 7f073f000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0738118000 con 0x7f073810d390 2024-04-25T00:43:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.892+0000 7f0737e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f073810c1a0 0x7f07381a23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.892+0000 7f0737400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f073810d390 0x7f07381a2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.892+0000 7f0737400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f073810d390 0x7f07381a2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:50494/0 (socket says 172.21.15.42:50494) 2024-04-25T00:43:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.892+0000 7f0737400640 1 -- 172.21.15.42:0/3345196867 learned_addr learned my addr 172.21.15.42:0/3345196867 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:04.893 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.892+0000 7f073d000640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073810dd50 0x7f07381a9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.892+0000 7f0737400640 1 -- 172.21.15.42:0/3345196867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f073810c1a0 msgr2=0x7f07381a23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.892+0000 7f0737400640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f073810c1a0 0x7f07381a23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.893+0000 7f0737400640 1 -- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073810dd50 msgr2=0x7f07381a9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.894 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.893+0000 7f0737400640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073810dd50 0x7f07381a9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.893+0000 7f0737400640 1 -- 172.21.15.42:0/3345196867 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07381a9fe0 con 0x7f073810d390 2024-04-25T00:43:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.893+0000 7f073d000640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073810dd50 0x7f07381a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:43:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.893+0000 7f0737400640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f073810d390 0x7f07381a2900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0724031610 tx=0x7f0724032c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.893+0000 7f0737e00640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f073810c1a0 0x7f07381a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:43:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.894+0000 7f0734c00640 1 -- 172.21.15.42:0/3345196867 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072403f030 con 0x7f073810d390 2024-04-25T00:43:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.894+0000 7f0734c00640 1 -- 172.21.15.42:0/3345196867 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0724003ad0 con 0x7f073810d390 2024-04-25T00:43:04.895 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.894+0000 7f073f000640 1 -- 172.21.15.42:0/3345196867 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07381aa270 con 0x7f073810d390 2024-04-25T00:43:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.894+0000 7f0734c00640 1 -- 172.21.15.42:0/3345196867 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724035070 con 0x7f073810d390 2024-04-25T00:43:04.896 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.894+0000 7f073f000640 1 -- 172.21.15.42:0/3345196867 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07381aa760 con 0x7f073810d390 2024-04-25T00:43:04.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.901+0000 7f0741d6e740 1 Processor -- start 2024-04-25T00:43:04.903 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.902+0000 7f0734c00640 1 -- 172.21.15.42:0/3345196867 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0724042030 con 0x7f073810d390 2024-04-25T00:43:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.904+0000 7f0741d6e740 1 -- start start 2024-04-25T00:43:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.904+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf41c4b0 0x55a0cf41c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.904+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf4219a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.904+0000 7f0734c00640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f071c0a2df0 0x7f071c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.904+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf422010 0x55a0cf4244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.904+0000 7f0741d6e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a0cf376ab0 con 0x55a0cf41cea0 2024-04-25T00:43:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.904+0000 7f0741d6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a0cf31f640 con 0x55a0cf422010 2024-04-25T00:43:04.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.904+0000 7f0741d6e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0cf372a70 con 0x55a0cf41c4b0 2024-04-25T00:43:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.905+0000 7f073d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf422010 0x55a0cf4244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.905+0000 7f0737e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf4219a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.905+0000 7f073d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf422010 0x55a0cf4244a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35250/0 (socket says 172.21.15.42:35250) 2024-04-25T00:43:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.905+0000 7f073d000640 1 -- 172.21.15.42:0/4083461003 learned_addr learned my addr 172.21.15.42:0/4083461003 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.905+0000 7f0734c00640 1 -- 172.21.15.42:0/3345196867 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f07240ef620 con 0x7f073810d390 2024-04-25T00:43:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.906+0000 7f0737e00640 1 -- 172.21.15.42:0/4083461003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf41c4b0 msgr2=0x55a0cf41c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.906+0000 7f0737e00640 1 --2- 172.21.15.42:0/4083461003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf41c4b0 0x55a0cf41c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.906+0000 7f0737e00640 1 -- 172.21.15.42:0/4083461003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf422010 msgr2=0x55a0cf4244a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.906+0000 7f0737400640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f071c0a2df0 0x7f071c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.907+0000 7f0737e00640 1 --2- 172.21.15.42:0/4083461003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf422010 0x55a0cf4244a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.907+0000 7f0737e00640 1 -- 172.21.15.42:0/4083461003 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0cf08b210 con 0x55a0cf41cea0 2024-04-25T00:43:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.907+0000 7f073d000640 1 --2- 172.21.15.42:0/4083461003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf422010 0x55a0cf4244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:43:04.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.907+0000 7f0737400640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f071c0a2df0 0x7f071c0a52b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f07240b8280 tx=0x7f0724003f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.908+0000 7f0737e00640 1 --2- 172.21.15.42:0/4083461003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf4219a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f073810d1f0 tx=0x7f072800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b83ba085c503c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.910+0000 7f072d600640 1 -- 172.21.15.42:0/4083461003 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728015070 con 0x55a0cf41cea0 2024-04-25T00:43:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.910+0000 7f072d600640 1 -- 172.21.15.42:0/4083461003 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f072801a940 con 0x55a0cf41cea0 2024-04-25T00:43:04.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.910+0000 7f072d600640 1 -- 172.21.15.42:0/4083461003 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728011980 con 0x55a0cf41cea0 2024-04-25T00:43:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.911+0000 7f0741d6e740 1 -- 172.21.15.42:0/4083461003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 msgr2=0x55a0cf4219a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.911+0000 7f0741d6e740 1 --2- 172.21.15.42:0/4083461003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf4219a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f073810d1f0 tx=0x7f072800f730 comp rx=0 tx=0).stop 2024-04-25T00:43:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.912+0000 7f0741d6e740 1 -- 172.21.15.42:0/4083461003 shutdown_connections 2024-04-25T00:43:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.912+0000 7f0741d6e740 1 --2- 172.21.15.42:0/4083461003 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf422010 0x55a0cf4244a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.912+0000 7f0741d6e740 1 --2- 172.21.15.42:0/4083461003 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf4219a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.912+0000 7f0741d6e740 1 --2- 172.21.15.42:0/4083461003 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf41c4b0 0x55a0cf41c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.912+0000 7f0741d6e740 1 -- 172.21.15.42:0/4083461003 >> 172.21.15.42:0/4083461003 conn(0x55a0cf40f310 msgr2=0x55a0cf40f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.913+0000 7f0741d6e740 1 -- 172.21.15.42:0/4083461003 shutdown_connections 2024-04-25T00:43:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.913+0000 7f0741d6e740 1 -- 172.21.15.42:0/4083461003 wait complete. 2024-04-25T00:43:04.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.913+0000 7f0741d6e740 1 -- start start 2024-04-25T00:43:04.916 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.915+0000 7f0741d6e740 1 Processor -- start 2024-04-25T00:43:04.917 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.916+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.918 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.917+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf415b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.917+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf41abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.917+0000 7f0741d6e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55a0cf413e00 con 0x55a0cf41cea0 2024-04-25T00:43:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.917+0000 7f0741d6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55a0cf332990 con 0x55a0cf41c4b0 2024-04-25T00:43:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.917+0000 7f0741d6e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0cf413f80 con 0x55a0cf422010 2024-04-25T00:43:04.919 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.917+0000 7f0741d6e740 10 client.0 _ll_register_callbacks cb 0x55a0cf334950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:43:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.918+0000 7f073d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.918+0000 7f073d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35254/0 (socket says 172.21.15.42:35254) 2024-04-25T00:43:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.918+0000 7f073d000640 1 -- 172.21.15.42:0/1761467577 learned_addr learned my addr 172.21.15.42:0/1761467577 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.919+0000 7f073d000640 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf41abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.919+0000 7f073d000640 1 -- 172.21.15.42:0/1761467577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 msgr2=0x55a0cf41abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.919+0000 7f073d000640 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf41abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.919+0000 7f073d000640 1 -- 172.21.15.42:0/1761467577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 msgr2=0x55a0cf415b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.920 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.919+0000 7f073d000640 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf415b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.919+0000 7f073d000640 1 -- 172.21.15.42:0/1761467577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0cf157620 con 0x55a0cf41c4b0 2024-04-25T00:43:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.919+0000 7f073d000640 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415640 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0730011f10 tx=0x7f0730009d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8e88f44f4a7f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.921 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.919+0000 7f0722000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:43:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.920+0000 7f0722000640 1 -- 172.21.15.42:0/1761467577 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730004b20 con 0x55a0cf41c4b0 2024-04-25T00:43:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.920+0000 7f0722000640 1 -- 172.21.15.42:0/1761467577 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f0730004cc0 con 0x55a0cf41c4b0 2024-04-25T00:43:04.922 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.920+0000 7f0741d6e740 1 -- 172.21.15.42:0/1761467577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f072801a940 con 0x55a0cf41c4b0 2024-04-25T00:43:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.924+0000 7f0722000640 1 -- 172.21.15.42:0/1761467577 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0730025050 con 0x55a0cf41c4b0 2024-04-25T00:43:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.924+0000 7f0741d6e740 10 client.27044 fetch_fsmap learned FSMap version 45 2024-04-25T00:43:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.924+0000 7f0741d6e740 1 -- 172.21.15.42:0/1761467577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a0cf439570 con 0x55a0cf41c4b0 2024-04-25T00:43:04.925 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.924+0000 7f0722000640 1 -- 172.21.15.42:0/1761467577 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f072801a940 con 0x55a0cf41c4b0 2024-04-25T00:43:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.925+0000 7f0722000640 1 -- 172.21.15.42:0/1761467577 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f073001e070 con 0x55a0cf41c4b0 2024-04-25T00:43:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.925+0000 7f0741d6e740 10 client.27044 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:43:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.925+0000 7f0741d6e740 10 client.27044 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:43:04.926 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.925+0000 7f0741d6e740 20 client.27044 populate_metadata read hostname 'smithi042' 2024-04-25T00:43:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.925+0000 7f0741d6e740 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0cf43ac90 0x55a0cf43d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.925+0000 7f0741d6e740 4 client.27044 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:43:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.925+0000 7f0741d6e740 1 -- 172.21.15.42:0/1761467577 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0cf383ff0 con 0x55a0cf43ac90 2024-04-25T00:43:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.926+0000 7f0737e00640 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0cf43ac90 0x55a0cf43d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.927 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.926+0000 7f0737e00640 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0cf43ac90 0x55a0cf43d0c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7281d85d8b79855 server_cookie=71c7693a6e7f175d in_seq=0 out_seq=0 2024-04-25T00:43:04.928 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.926+0000 7f0722000640 10 client.27044 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.931+0000 7f0722000640 1 -- 172.21.15.42:0/1761467577 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0cf383ff0 con 0x55a0cf43ac90 2024-04-25T00:43:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.931+0000 7f0722000640 10 client.27044 handle_command_reply: tid=0 2024-04-25T00:43:04.932 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.931+0000 7f0741d6e740 1 client.27044 shutdown 2024-04-25T00:43:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.931+0000 7f0741d6e740 1 -- 172.21.15.42:0/1761467577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 msgr2=0x55a0cf415640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.931+0000 7f0741d6e740 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415640 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f0730011f10 tx=0x7f0730009d10 comp rx=0 tx=0).stop 2024-04-25T00:43:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.932+0000 7f0741d6e740 1 -- 172.21.15.42:0/1761467577 shutdown_connections 2024-04-25T00:43:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.932+0000 7f0741d6e740 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0cf43ac90 0x55a0cf43d0c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.932+0000 7f0741d6e740 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf41abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.932+0000 7f0741d6e740 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf415b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.932+0000 7f0741d6e740 1 --2- 172.21.15.42:0/1761467577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415640 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.932+0000 7f0722000640 0 client.27044 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:04.933 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.932+0000 7f0741d6e740 1 -- 172.21.15.42:0/1761467577 >> 172.21.15.42:0/1761467577 conn(0x55a0cf40f310 msgr2=0x55a0cf40e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.932+0000 7f0741d6e740 1 -- 172.21.15.42:0/1761467577 shutdown_connections 2024-04-25T00:43:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.933+0000 7f0741d6e740 1 -- 172.21.15.42:0/1761467577 wait complete. 2024-04-25T00:43:04.934 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.933+0000 7f0741d6e740 20 client.27044 trim_cache size 0 max 16384 2024-04-25T00:43:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.955+0000 7f0741d6e740 1 Processor -- start 2024-04-25T00:43:04.956 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.955+0000 7f0741d6e740 1 -- start start 2024-04-25T00:43:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.956+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.956+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf412220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.956+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf412760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.956+0000 7f0741d6e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f07381b1970 con 0x55a0cf41cea0 2024-04-25T00:43:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.956+0000 7f0741d6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f07381af240 con 0x55a0cf41c4b0 2024-04-25T00:43:04.957 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.956+0000 7f0741d6e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07381ad0b0 con 0x55a0cf422010 2024-04-25T00:43:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.956+0000 7f073d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.956+0000 7f073d000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35268/0 (socket says 172.21.15.42:35268) 2024-04-25T00:43:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.956+0000 7f073d000640 1 -- 172.21.15.42:0/1994921814 learned_addr learned my addr 172.21.15.42:0/1994921814 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.957+0000 7f073d000640 1 --2- 172.21.15.42:0/1994921814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf412760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.958 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.957+0000 7f0737e00640 1 --2- 172.21.15.42:0/1994921814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf412220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.957+0000 7f073d000640 1 -- 172.21.15.42:0/1994921814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 msgr2=0x55a0cf412760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.957+0000 7f073d000640 1 --2- 172.21.15.42:0/1994921814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf412760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.957+0000 7f073d000640 1 -- 172.21.15.42:0/1994921814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 msgr2=0x55a0cf412220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.957+0000 7f073d000640 1 --2- 172.21.15.42:0/1994921814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf412220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.959 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.957+0000 7f073d000640 1 -- 172.21.15.42:0/1994921814 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0cf41dcb0 con 0x55a0cf41c4b0 2024-04-25T00:43:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.958+0000 7f073d000640 1 --2- 172.21.15.42:0/1994921814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415960 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0730020e00 tx=0x7f0730021f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e08e4d55bab8d3b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.958+0000 7f072cc00640 1 -- 172.21.15.42:0/1994921814 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730020e60 con 0x55a0cf41c4b0 2024-04-25T00:43:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.958+0000 7f072cc00640 1 -- 172.21.15.42:0/1994921814 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f073001b040 con 0x55a0cf41c4b0 2024-04-25T00:43:04.960 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.958+0000 7f072cc00640 1 -- 172.21.15.42:0/1994921814 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730005460 con 0x55a0cf41c4b0 2024-04-25T00:43:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.959+0000 7f0741d6e740 1 -- 172.21.15.42:0/1994921814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 msgr2=0x55a0cf415960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.959+0000 7f0741d6e740 1 --2- 172.21.15.42:0/1994921814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415960 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0730020e00 tx=0x7f0730021f90 comp rx=0 tx=0).stop 2024-04-25T00:43:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.959+0000 7f0741d6e740 1 -- 172.21.15.42:0/1994921814 shutdown_connections 2024-04-25T00:43:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.959+0000 7f0741d6e740 1 --2- 172.21.15.42:0/1994921814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf412760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.959+0000 7f0741d6e740 1 --2- 172.21.15.42:0/1994921814 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41cea0 0x55a0cf412220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.959+0000 7f0741d6e740 1 --2- 172.21.15.42:0/1994921814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41c4b0 0x55a0cf415960 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.959+0000 7f0741d6e740 1 -- 172.21.15.42:0/1994921814 >> 172.21.15.42:0/1994921814 conn(0x55a0cf40f310 msgr2=0x55a0cf415250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.959+0000 7f0741d6e740 1 -- 172.21.15.42:0/1994921814 shutdown_connections 2024-04-25T00:43:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.959+0000 7f0741d6e740 1 -- 172.21.15.42:0/1994921814 wait complete. 2024-04-25T00:43:04.961 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.960+0000 7f0741d6e740 1 -- start start 2024-04-25T00:43:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.960+0000 7f0741d6e740 1 Processor -- start 2024-04-25T00:43:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41c4b0 0x55a0cf44e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.962 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41cea0 0x55a0cf44c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f0741d6e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf44cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f0741d6e740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f07381ad700 con 0x55a0cf41c4b0 2024-04-25T00:43:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f0741d6e740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f07381aeb70 con 0x55a0cf41cea0 2024-04-25T00:43:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f0741d6e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f07381addd0 con 0x55a0cf422010 2024-04-25T00:43:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f0741d6e740 10 client.0 _ll_register_callbacks cb 0x7f0730009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:43:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f0737e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41cea0 0x55a0cf44c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f0737e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41cea0 0x55a0cf44c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:35272/0 (socket says 172.21.15.42:35272) 2024-04-25T00:43:04.963 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f0737e00640 1 -- 172.21.15.42:0/999260501 learned_addr learned my addr 172.21.15.42:0/999260501 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.961+0000 7f073d000640 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41c4b0 0x55a0cf44e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f0737e00640 1 -- 172.21.15.42:0/999260501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 msgr2=0x55a0cf44cd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:43:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f0737e00640 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf44cd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f0737e00640 1 -- 172.21.15.42:0/999260501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41c4b0 msgr2=0x55a0cf44e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:04.964 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f0737e00640 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41c4b0 0x55a0cf44e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f0737e00640 1 -- 172.21.15.42:0/999260501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0cf44d2a0 con 0x55a0cf41cea0 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f0737e00640 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41cea0 0x55a0cf44c7f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f07280058c0 tx=0x7f072800b7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1ac68d715ba536 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f073d000640 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41c4b0 0x55a0cf44e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f0722000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f0722000640 1 -- 172.21.15.42:0/999260501 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728015040 con 0x55a0cf41cea0 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f0722000640 1 -- 172.21.15.42:0/999260501 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f0728005450 con 0x55a0cf41cea0 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.962+0000 7f0741d6e740 1 -- 172.21.15.42:0/999260501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f073001b040 con 0x55a0cf41cea0 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.963+0000 7f0722000640 1 -- 172.21.15.42:0/999260501 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0728004680 con 0x55a0cf41cea0 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.963+0000 7f0741d6e740 10 client.27050 fetch_fsmap learned FSMap version 45 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.963+0000 7f0741d6e740 1 -- 172.21.15.42:0/999260501 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55a0cf4a1b70 con 0x55a0cf41cea0 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.963+0000 7f0722000640 1 -- 172.21.15.42:0/999260501 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f073001b040 con 0x55a0cf41cea0 2024-04-25T00:43:04.965 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.963+0000 7f0722000640 1 -- 172.21.15.42:0/999260501 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f0728026490 con 0x55a0cf41cea0 2024-04-25T00:43:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.964+0000 7f0741d6e740 10 client.27050 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:43:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.964+0000 7f0741d6e740 10 client.27050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:43:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.964+0000 7f0741d6e740 20 client.27050 populate_metadata read hostname 'smithi042' 2024-04-25T00:43:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.964+0000 7f0741d6e740 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0cf4a2240 0x55a0cf4a4650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.964+0000 7f0741d6e740 4 client.27050 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:43:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.964+0000 7f0741d6e740 1 -- 172.21.15.42:0/999260501 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0cf37e340 con 0x55a0cf4a2240 2024-04-25T00:43:04.966 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.964+0000 7f073d000640 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0cf4a2240 0x55a0cf4a4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.965+0000 7f073d000640 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0cf4a2240 0x55a0cf4a4650 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a52aad2c95171c6 server_cookie=9d51045beea8c381 in_seq=0 out_seq=0 2024-04-25T00:43:04.967 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:04.965+0000 7f0722000640 10 client.27050 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.338+0000 7f0722000640 1 -- 172.21.15.42:0/999260501 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0cf37e340 con 0x55a0cf4a2240 2024-04-25T00:43:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.338+0000 7f0722000640 10 client.27050 handle_command_reply: tid=0 2024-04-25T00:43:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.338+0000 7f0741d6e740 1 client.27050 shutdown 2024-04-25T00:43:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.339+0000 7f0741d6e740 1 -- 172.21.15.42:0/999260501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41cea0 msgr2=0x55a0cf44c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.340 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.339+0000 7f0741d6e740 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41cea0 0x55a0cf44c7f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f07280058c0 tx=0x7f072800b7c0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.339+0000 7f0741d6e740 1 -- 172.21.15.42:0/999260501 shutdown_connections 2024-04-25T00:43:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.339+0000 7f0741d6e740 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55a0cf4a2240 0x55a0cf4a4650 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.339+0000 7f0741d6e740 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0cf422010 0x55a0cf44cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.339+0000 7f0741d6e740 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55a0cf41cea0 0x55a0cf44c7f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.339+0000 7f0741d6e740 1 --2- 172.21.15.42:0/999260501 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55a0cf41c4b0 0x55a0cf44e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.339+0000 7f0741d6e740 1 -- 172.21.15.42:0/999260501 >> 172.21.15.42:0/999260501 conn(0x55a0cf40f310 msgr2=0x55a0cf412220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:05.341 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.340+0000 7f0722000640 0 client.27050 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.340+0000 7f0741d6e740 1 -- 172.21.15.42:0/999260501 shutdown_connections 2024-04-25T00:43:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.340+0000 7f0741d6e740 1 -- 172.21.15.42:0/999260501 wait complete. 2024-04-25T00:43:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.340+0000 7f0741d6e740 20 client.27050 trim_cache size 0 max 16384 2024-04-25T00:43:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:43:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:43:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:43:05.342 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:43:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.341+0000 7f073f000640 1 -- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f071c0a2df0 msgr2=0x7f071c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.341+0000 7f073f000640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f071c0a2df0 0x7f071c0a52b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f07240b8280 tx=0x7f0724003f60 comp rx=0 tx=0).stop 2024-04-25T00:43:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.341+0000 7f073f000640 1 -- 172.21.15.42:0/3345196867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f073810d390 msgr2=0x7f07381a2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.341+0000 7f073f000640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f073810d390 0x7f07381a2900 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0724031610 tx=0x7f0724032c80 comp rx=0 tx=0).stop 2024-04-25T00:43:05.343 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.341+0000 7f073f000640 1 -- 172.21.15.42:0/3345196867 shutdown_connections 2024-04-25T00:43:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.341+0000 7f073f000640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f071c0a2df0 0x7f071c0a52b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.341+0000 7f073f000640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f073810dd50 0x7f07381a9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.341+0000 7f073f000640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f073810d390 0x7f07381a2900 secure :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f0724031610 tx=0x7f0724032c80 comp rx=0 tx=0).stop 2024-04-25T00:43:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.341+0000 7f073f000640 1 --2- 172.21.15.42:0/3345196867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f073810c1a0 0x7f07381a23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.342+0000 7f073f000640 1 -- 172.21.15.42:0/3345196867 >> 172.21.15.42:0/3345196867 conn(0x7f073806c340 msgr2=0x7f0738110630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.342+0000 7f073f000640 1 -- 172.21.15.42:0/3345196867 shutdown_connections 2024-04-25T00:43:05.344 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:05.342+0000 7f073f000640 1 -- 172.21.15.42:0/3345196867 wait complete. 2024-04-25T00:43:05.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 399609 32.57 MB/sec execute 535 sec latency 227.027 ms 2024-04-25T00:43:06.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:06 smithi136 ceph-mon[32923]: pgmap v1120: 97 pgs: 97 active+clean; 660 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.2 MiB/s wr, 591 op/s 2024-04-25T00:43:06.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:06 smithi106 ceph-mon[32510]: pgmap v1120: 97 pgs: 97 active+clean; 660 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.2 MiB/s wr, 591 op/s 2024-04-25T00:43:06.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:06 smithi042 ceph-mon[28130]: pgmap v1120: 97 pgs: 97 active+clean; 660 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 9.2 MiB/s wr, 591 op/s 2024-04-25T00:43:06.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 400255 32.59 MB/sec execute 536 sec latency 45.221 ms 2024-04-25T00:43:07.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 400802 32.56 MB/sec execute 537 sec latency 49.430 ms 2024-04-25T00:43:08.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:08 smithi136 ceph-mon[32923]: pgmap v1121: 97 pgs: 97 active+clean; 665 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.8 MiB/s wr, 746 op/s 2024-04-25T00:43:08.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:08 smithi106 ceph-mon[32510]: pgmap v1121: 97 pgs: 97 active+clean; 665 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.8 MiB/s wr, 746 op/s 2024-04-25T00:43:08.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:08 smithi042 ceph-mon[28130]: pgmap v1121: 97 pgs: 97 active+clean; 665 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 9.8 MiB/s wr, 746 op/s 2024-04-25T00:43:08.816 INFO:tasks.workunit.client.0.smithi042.stdout: 10 401435 32.57 MB/sec execute 538 sec latency 72.288 ms 2024-04-25T00:43:09.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 402261 32.58 MB/sec execute 539 sec latency 43.494 ms 2024-04-25T00:43:10.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:10 smithi106 ceph-mon[32510]: pgmap v1122: 97 pgs: 97 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.1 MiB/s wr, 542 op/s 2024-04-25T00:43:10.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:10 smithi042 ceph-mon[28130]: pgmap v1122: 97 pgs: 97 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.1 MiB/s wr, 542 op/s 2024-04-25T00:43:10.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:10 smithi136 ceph-mon[32923]: pgmap v1122: 97 pgs: 97 active+clean; 669 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 5.1 MiB/s wr, 542 op/s 2024-04-25T00:43:10.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 402829 32.60 MB/sec execute 540 sec latency 182.848 ms 2024-04-25T00:43:11.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:11 smithi106 ceph-mon[32510]: pgmap v1123: 97 pgs: 97 active+clean; 670 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 494 op/s 2024-04-25T00:43:11.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:11 smithi042 ceph-mon[28130]: pgmap v1123: 97 pgs: 97 active+clean; 670 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 494 op/s 2024-04-25T00:43:11.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:11 smithi136 ceph-mon[32923]: pgmap v1123: 97 pgs: 97 active+clean; 670 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.4 MiB/s wr, 494 op/s 2024-04-25T00:43:11.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 403515 32.61 MB/sec execute 541 sec latency 57.648 ms 2024-04-25T00:43:12.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 403970 32.60 MB/sec execute 542 sec latency 72.232 ms 2024-04-25T00:43:13.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 404406 32.59 MB/sec execute 543 sec latency 220.124 ms 2024-04-25T00:43:14.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:13 smithi136 ceph-mon[32923]: pgmap v1124: 97 pgs: 97 active+clean; 680 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.1 MiB/s wr, 565 op/s 2024-04-25T00:43:14.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:13 smithi106 ceph-mon[32510]: pgmap v1124: 97 pgs: 97 active+clean; 680 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.1 MiB/s wr, 565 op/s 2024-04-25T00:43:14.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:13 smithi042 ceph-mon[28130]: pgmap v1124: 97 pgs: 97 active+clean; 680 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 4.1 MiB/s wr, 565 op/s 2024-04-25T00:43:14.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 404873 32.59 MB/sec execute 544 sec latency 125.193 ms 2024-04-25T00:43:15.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 405597 32.58 MB/sec execute 545 sec latency 113.895 ms 2024-04-25T00:43:16.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:15 smithi136 ceph-mon[32923]: pgmap v1125: 97 pgs: 97 active+clean; 675 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 438 op/s 2024-04-25T00:43:16.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:15 smithi106 ceph-mon[32510]: pgmap v1125: 97 pgs: 97 active+clean; 675 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 438 op/s 2024-04-25T00:43:16.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:15 smithi042 ceph-mon[28130]: pgmap v1125: 97 pgs: 97 active+clean; 675 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 3.2 MiB/s wr, 438 op/s 2024-04-25T00:43:16.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 406273 32.59 MB/sec execute 546 sec latency 67.955 ms 2024-04-25T00:43:17.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 407051 32.64 MB/sec execute 547 sec latency 46.982 ms 2024-04-25T00:43:18.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:17 smithi136 ceph-mon[32923]: pgmap v1126: 97 pgs: 97 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.6 MiB/s wr, 610 op/s 2024-04-25T00:43:18.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:17 smithi106 ceph-mon[32510]: pgmap v1126: 97 pgs: 97 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.6 MiB/s wr, 610 op/s 2024-04-25T00:43:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:17 smithi042 ceph-mon[28130]: pgmap v1126: 97 pgs: 97 active+clean; 668 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 5.6 MiB/s wr, 610 op/s 2024-04-25T00:43:18.817 INFO:tasks.workunit.client.0.smithi042.stdout: 10 407232 32.59 MB/sec execute 548 sec latency 412.239 ms 2024-04-25T00:43:19.818 INFO:tasks.workunit.client.0.smithi042.stdout: 10 407243 32.54 MB/sec execute 549 sec latency 1188.047 ms 2024-04-25T00:43:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.823+0000 7f5d43400640 1 -- 172.21.15.42:0/954696642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c108fc0 msgr2=0x7f5d3c109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.823+0000 7f5d43400640 1 --2- 172.21.15.42:0/954696642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c108fc0 0x7f5d3c109440 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5d28009860 tx=0x7f5d2802a340 comp rx=0 tx=0).stop 2024-04-25T00:43:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.824+0000 7f5d43400640 1 -- 172.21.15.42:0/954696642 shutdown_connections 2024-04-25T00:43:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.824+0000 7f5d43400640 1 --2- 172.21.15.42:0/954696642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d3c109980 0x7f5d3c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.824+0000 7f5d43400640 1 --2- 172.21.15.42:0/954696642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c108fc0 0x7f5d3c109440 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.824+0000 7f5d43400640 1 --2- 172.21.15.42:0/954696642 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d3c107dd0 0x7f5d3c1081f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.824+0000 7f5d43400640 1 -- 172.21.15.42:0/954696642 >> 172.21.15.42:0/954696642 conn(0x7f5d3c076d90 msgr2=0x7f5d3c0791b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.824+0000 7f5d43400640 1 -- 172.21.15.42:0/954696642 shutdown_connections 2024-04-25T00:43:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.825+0000 7f5d43400640 1 -- 172.21.15.42:0/954696642 wait complete. 2024-04-25T00:43:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.826+0000 7f5d43400640 1 Processor -- start 2024-04-25T00:43:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.827+0000 7f5d43400640 1 -- start start 2024-04-25T00:43:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.828+0000 7f5d43400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d3c107dd0 0x7f5d3c07eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.828+0000 7f5d43400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d3c108fc0 0x7f5d3c07f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.828+0000 7f5d43400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c109980 0x7f5d3c1ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.828+0000 7f5d43400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c113a30 con 0x7f5d3c108fc0 2024-04-25T00:43:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.828+0000 7f5d43400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1138b0 con 0x7f5d3c109980 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.828+0000 7f5d43400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c113bb0 con 0x7f5d3c107dd0 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.828+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d3c107dd0 0x7f5d3c07eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.829+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d3c107dd0 0x7f5d3c07eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44078/0 (socket says 172.21.15.42:44078) 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.829+0000 7f5d40a00640 1 -- 172.21.15.42:0/2621804954 learned_addr learned my addr 172.21.15.42:0/2621804954 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.829+0000 7f5d41400640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c109980 0x7f5d3c1ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.829+0000 7f5d3be00640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d3c108fc0 0x7f5d3c07f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.829+0000 7f5d40a00640 1 -- 172.21.15.42:0/2621804954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c109980 msgr2=0x7f5d3c1ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.829+0000 7f5d40a00640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c109980 0x7f5d3c1ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.829+0000 7f5d40a00640 1 -- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d3c108fc0 msgr2=0x7f5d3c07f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.829+0000 7f5d40a00640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d3c108fc0 0x7f5d3c07f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.829+0000 7f5d40a00640 1 -- 172.21.15.42:0/2621804954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c07fa50 con 0x7f5d3c107dd0 2024-04-25T00:43:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.829+0000 7f5d3be00640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d3c108fc0 0x7f5d3c07f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.830+0000 7f5d40a00640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d3c107dd0 0x7f5d3c07eea0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000d770 tx=0x7f5d3000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.830+0000 7f5d39600640 1 -- 172.21.15.42:0/2621804954 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30017070 con 0x7f5d3c107dd0 2024-04-25T00:43:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.830+0000 7f5d39600640 1 -- 172.21.15.42:0/2621804954 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d30009c30 con 0x7f5d3c107dd0 2024-04-25T00:43:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.830+0000 7f5d39600640 1 -- 172.21.15.42:0/2621804954 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30003dd0 con 0x7f5d3c107dd0 2024-04-25T00:43:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.831+0000 7f5d43400640 1 -- 172.21.15.42:0/2621804954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1ae740 con 0x7f5d3c107dd0 2024-04-25T00:43:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.831+0000 7f5d43400640 1 -- 172.21.15.42:0/2621804954 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c1aec40 con 0x7f5d3c107dd0 2024-04-25T00:43:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.837+0000 7f5d461a8740 1 Processor -- start 2024-04-25T00:43:19.837 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.837+0000 7f5d461a8740 1 -- start start 2024-04-25T00:43:19.838 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.838+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.838+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc4690 0x55f2cebc9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.838+0000 7f5d41400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.838+0000 7f5d41400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34688/0 (socket says 172.21.15.42:34688) 2024-04-25T00:43:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.838+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc9800 0x55f2cebcbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.839+0000 7f5d41400640 1 -- 172.21.15.42:0/1549676926 learned_addr learned my addr 172.21.15.42:0/1549676926 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:19.839 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.839+0000 7f5d461a8740 1 -- 172.21.15.42:0/1549676926 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f2ceb26ab0 con 0x55f2cebc9800 2024-04-25T00:43:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.839+0000 7f5d461a8740 1 -- 172.21.15.42:0/1549676926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f2ceacf640 con 0x55f2cebc3ca0 2024-04-25T00:43:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.839+0000 7f5d461a8740 1 -- 172.21.15.42:0/1549676926 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f2ceb22a70 con 0x55f2cebc4690 2024-04-25T00:43:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.839+0000 7f5d3be00640 1 --2- 172.21.15.42:0/1549676926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc9800 0x55f2cebcbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.840+0000 7f5d41400640 1 -- 172.21.15.42:0/1549676926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc4690 msgr2=0x55f2cebc9190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T00:43:19.840 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.840+0000 7f5d41400640 1 --2- 172.21.15.42:0/1549676926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc4690 0x55f2cebc9190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.840+0000 7f5d41400640 1 -- 172.21.15.42:0/1549676926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc9800 msgr2=0x55f2cebcbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.840+0000 7f5d41400640 1 --2- 172.21.15.42:0/1549676926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc9800 0x55f2cebcbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.840+0000 7f5d41400640 1 -- 172.21.15.42:0/1549676926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2ce83b210 con 0x55f2cebc3ca0 2024-04-25T00:43:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.840+0000 7f5d41400640 1 --2- 172.21.15.42:0/1549676926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc40c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400e450 tx=0x7f5d340095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8515158ac2206d96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.840+0000 7f5d39600640 1 -- 172.21.15.42:0/2621804954 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5d30003f70 con 0x7f5d3c107dd0 2024-04-25T00:43:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.841+0000 7f5d2dc00640 1 -- 172.21.15.42:0/1549676926 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d3400d910 con 0x55f2cebc3ca0 2024-04-25T00:43:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.841+0000 7f5d2dc00640 1 -- 172.21.15.42:0/1549676926 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3401c070 con 0x55f2cebc3ca0 2024-04-25T00:43:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.841+0000 7f5d2dc00640 1 -- 172.21.15.42:0/1549676926 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34016380 con 0x55f2cebc3ca0 2024-04-25T00:43:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.842+0000 7f5d461a8740 1 -- 172.21.15.42:0/1549676926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 msgr2=0x55f2cebc40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.842+0000 7f5d461a8740 1 --2- 172.21.15.42:0/1549676926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc40c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5d3400e450 tx=0x7f5d340095c0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.842+0000 7f5d461a8740 1 -- 172.21.15.42:0/1549676926 shutdown_connections 2024-04-25T00:43:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.842+0000 7f5d461a8740 1 --2- 172.21.15.42:0/1549676926 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc9800 0x55f2cebcbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.842+0000 7f5d461a8740 1 --2- 172.21.15.42:0/1549676926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc4690 0x55f2cebc9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.842+0000 7f5d461a8740 1 --2- 172.21.15.42:0/1549676926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc40c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.842+0000 7f5d461a8740 1 -- 172.21.15.42:0/1549676926 >> 172.21.15.42:0/1549676926 conn(0x55f2cebbf310 msgr2=0x55f2cebbf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.844+0000 7f5d39600640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5d240a2e20 0x7f5d240a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.844+0000 7f5d39600640 1 -- 172.21.15.42:0/2621804954 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f5d300ce8c0 con 0x7f5d3c107dd0 2024-04-25T00:43:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.844+0000 7f5d3be00640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5d240a2e20 0x7f5d240a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.845+0000 7f5d461a8740 1 -- 172.21.15.42:0/1549676926 shutdown_connections 2024-04-25T00:43:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.845+0000 7f5d461a8740 1 -- 172.21.15.42:0/1549676926 wait complete. 2024-04-25T00:43:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.846+0000 7f5d3be00640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5d240a2e20 0x7f5d240a52e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5d28000c00 tx=0x7f5d28030380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.846+0000 7f5d461a8740 1 -- start start 2024-04-25T00:43:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.846+0000 7f5d461a8740 1 Processor -- start 2024-04-25T00:43:19.847 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.847+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc3ca0 0x55f2cebcdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.847+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 0x55f2cebce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.847+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc9800 0x55f2cebd5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.847+0000 7f5d461a8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55f2cebcc800 con 0x55f2cebc4690 2024-04-25T00:43:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.847+0000 7f5d461a8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55f2ceae2990 con 0x55f2cebc9800 2024-04-25T00:43:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.847+0000 7f5d461a8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f2cebcc980 con 0x55f2cebc3ca0 2024-04-25T00:43:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.847+0000 7f5d461a8740 10 client.0 _ll_register_callbacks cb 0x55f2ceae4950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:43:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.847+0000 7f5d41400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc9800 0x55f2cebd5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.847+0000 7f5d41400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc9800 0x55f2cebd5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34692/0 (socket says 172.21.15.42:34692) 2024-04-25T00:43:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.847+0000 7f5d41400640 1 -- 172.21.15.42:0/3129373349 learned_addr learned my addr 172.21.15.42:0/3129373349 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.848+0000 7f5d3be00640 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 0x55f2cebce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.848+0000 7f5d40a00640 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc3ca0 0x55f2cebcdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.848+0000 7f5d41400640 1 -- 172.21.15.42:0/3129373349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc3ca0 msgr2=0x55f2cebcdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.848+0000 7f5d41400640 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc3ca0 0x55f2cebcdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.848+0000 7f5d41400640 1 -- 172.21.15.42:0/3129373349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 msgr2=0x55f2cebce4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.848+0000 7f5d41400640 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 0x55f2cebce4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.848+0000 7f5d41400640 1 -- 172.21.15.42:0/3129373349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2ce907620 con 0x55f2cebc9800 2024-04-25T00:43:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.848+0000 7f5d3be00640 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 0x55f2cebce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:43:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.850+0000 7f5d41400640 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc9800 0x55f2cebd5480 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5d34017650 tx=0x7f5d34017680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=134b801b36616fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.850+0000 7f5d22000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:43:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.853+0000 7f5d22000640 1 -- 172.21.15.42:0/3129373349 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d34004e50 con 0x55f2cebc9800 2024-04-25T00:43:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.854+0000 7f5d22000640 1 -- 172.21.15.42:0/3129373349 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f5d34003970 con 0x55f2cebc9800 2024-04-25T00:43:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.854+0000 7f5d461a8740 1 -- 172.21.15.42:0/3129373349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d3401c070 con 0x55f2cebc9800 2024-04-25T00:43:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.854+0000 7f5d22000640 1 -- 172.21.15.42:0/3129373349 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5d34003b70 con 0x55f2cebc9800 2024-04-25T00:43:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.855+0000 7f5d461a8740 10 client.27062 fetch_fsmap learned FSMap version 45 2024-04-25T00:43:19.855 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.855+0000 7f5d461a8740 1 -- 172.21.15.42:0/3129373349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f2cebd8460 con 0x55f2cebc9800 2024-04-25T00:43:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.855+0000 7f5d22000640 1 -- 172.21.15.42:0/3129373349 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3401c070 con 0x55f2cebc9800 2024-04-25T00:43:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.856+0000 7f5d22000640 1 -- 172.21.15.42:0/3129373349 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f5d3400b040 con 0x55f2cebc9800 2024-04-25T00:43:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.856+0000 7f5d461a8740 10 client.27062 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:43:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.856+0000 7f5d461a8740 10 client.27062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:43:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.856+0000 7f5d461a8740 20 client.27062 populate_metadata read hostname 'smithi042' 2024-04-25T00:43:19.856 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.856+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f2cebd9870 0x55f2cebdbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.856+0000 7f5d461a8740 4 client.27062 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:43:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.856+0000 7f5d461a8740 1 -- 172.21.15.42:0/3129373349 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2ceb33ff0 con 0x55f2cebd9870 2024-04-25T00:43:19.857 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.857+0000 7f5d40a00640 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f2cebd9870 0x55f2cebdbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.858 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.858+0000 7f5d40a00640 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f2cebd9870 0x55f2cebdbc60 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3018fa3e2f46c48e server_cookie=efbfe0787d585464 in_seq=0 out_seq=0 2024-04-25T00:43:19.861 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.861+0000 7f5d22000640 10 client.27062 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.861+0000 7f5d22000640 1 -- 172.21.15.42:0/3129373349 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2ceb33ff0 con 0x55f2cebd9870 2024-04-25T00:43:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.861+0000 7f5d22000640 10 client.27062 handle_command_reply: tid=0 2024-04-25T00:43:19.862 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.862+0000 7f5d461a8740 1 client.27062 shutdown 2024-04-25T00:43:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.863+0000 7f5d461a8740 1 -- 172.21.15.42:0/3129373349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc9800 msgr2=0x55f2cebd5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.863+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc9800 0x55f2cebd5480 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5d34017650 tx=0x7f5d34017680 comp rx=0 tx=0).stop 2024-04-25T00:43:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.863+0000 7f5d461a8740 1 -- 172.21.15.42:0/3129373349 shutdown_connections 2024-04-25T00:43:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.863+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f2cebd9870 0x55f2cebdbc60 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.863 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.863+0000 7f5d22000640 0 client.27062 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.863+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc9800 0x55f2cebd5480 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.863+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 0x55f2cebce4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.863+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3129373349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc3ca0 0x55f2cebcdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.863+0000 7f5d461a8740 1 -- 172.21.15.42:0/3129373349 >> 172.21.15.42:0/3129373349 conn(0x55f2cebbf310 msgr2=0x55f2cebbe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.864+0000 7f5d461a8740 1 -- 172.21.15.42:0/3129373349 shutdown_connections 2024-04-25T00:43:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.864+0000 7f5d461a8740 1 -- 172.21.15.42:0/3129373349 wait complete. 2024-04-25T00:43:19.864 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.864+0000 7f5d461a8740 20 client.27062 trim_cache size 0 max 16384 2024-04-25T00:43:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.898+0000 7f5d461a8740 1 Processor -- start 2024-04-25T00:43:19.899 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.899+0000 7f5d461a8740 1 -- start start 2024-04-25T00:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.899+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.899+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 0x55f2cebd5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.899+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc9800 0x55f2cebe0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.900+0000 7f5d461a8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1ba360 con 0x55f2cebc4690 2024-04-25T00:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.900+0000 7f5d461a8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1b7c30 con 0x55f2cebc3ca0 2024-04-25T00:43:19.900 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.900+0000 7f5d461a8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1b5a80 con 0x55f2cebc9800 2024-04-25T00:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.900+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.900+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc2700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34704/0 (socket says 172.21.15.42:34704) 2024-04-25T00:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.900+0000 7f5d40a00640 1 -- 172.21.15.42:0/752602628 learned_addr learned my addr 172.21.15.42:0/752602628 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.901+0000 7f5d41400640 1 --2- 172.21.15.42:0/752602628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc9800 0x55f2cebe0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.901 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.901+0000 7f5d3be00640 1 --2- 172.21.15.42:0/752602628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 0x55f2cebd5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.901+0000 7f5d40a00640 1 -- 172.21.15.42:0/752602628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc9800 msgr2=0x55f2cebe0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.901+0000 7f5d40a00640 1 --2- 172.21.15.42:0/752602628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc9800 0x55f2cebe0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.901+0000 7f5d40a00640 1 -- 172.21.15.42:0/752602628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 msgr2=0x55f2cebd5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.901+0000 7f5d40a00640 1 --2- 172.21.15.42:0/752602628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 0x55f2cebd5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.901+0000 7f5d40a00640 1 -- 172.21.15.42:0/752602628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2cebbedb0 con 0x55f2cebc3ca0 2024-04-25T00:43:19.902 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.902+0000 7f5d40a00640 1 --2- 172.21.15.42:0/752602628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc2700 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000d740 tx=0x7f5d3001d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=249f31560c43edb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.903+0000 7f5d2d200640 1 -- 172.21.15.42:0/752602628 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d300b3030 con 0x55f2cebc3ca0 2024-04-25T00:43:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.903+0000 7f5d2d200640 1 -- 172.21.15.42:0/752602628 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3009f520 con 0x55f2cebc3ca0 2024-04-25T00:43:19.904 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.903+0000 7f5d2d200640 1 -- 172.21.15.42:0/752602628 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d300977e0 con 0x55f2cebc3ca0 2024-04-25T00:43:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.905+0000 7f5d461a8740 1 -- 172.21.15.42:0/752602628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 msgr2=0x55f2cebc2700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:19.905 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.905+0000 7f5d461a8740 1 --2- 172.21.15.42:0/752602628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc2700 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000d740 tx=0x7f5d3001d040 comp rx=0 tx=0).stop 2024-04-25T00:43:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.905+0000 7f5d461a8740 1 -- 172.21.15.42:0/752602628 shutdown_connections 2024-04-25T00:43:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.905+0000 7f5d461a8740 1 --2- 172.21.15.42:0/752602628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc9800 0x55f2cebe0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.905+0000 7f5d461a8740 1 --2- 172.21.15.42:0/752602628 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc4690 0x55f2cebd5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.905+0000 7f5d461a8740 1 --2- 172.21.15.42:0/752602628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebc2700 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.905+0000 7f5d461a8740 1 -- 172.21.15.42:0/752602628 >> 172.21.15.42:0/752602628 conn(0x55f2cebbf310 msgr2=0x55f2cebc1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:19.906 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.906+0000 7f5d461a8740 1 -- 172.21.15.42:0/752602628 shutdown_connections 2024-04-25T00:43:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.906+0000 7f5d461a8740 1 -- 172.21.15.42:0/752602628 wait complete. 2024-04-25T00:43:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.906+0000 7f5d461a8740 1 -- start start 2024-04-25T00:43:19.907 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.907+0000 7f5d461a8740 1 Processor -- start 2024-04-25T00:43:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.908+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebd2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.908+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc4690 0x55f2cebd32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.908+0000 7f5d461a8740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc9800 0x55f2cebd42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.908+0000 7f5d461a8740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1b60f0 con 0x55f2cebc9800 2024-04-25T00:43:19.908 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.908+0000 7f5d461a8740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1b7560 con 0x55f2cebc3ca0 2024-04-25T00:43:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.908+0000 7f5d461a8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c1b67c0 con 0x55f2cebc4690 2024-04-25T00:43:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.908+0000 7f5d461a8740 10 client.0 _ll_register_callbacks cb 0x7f5d28035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:43:19.909 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.908+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebd2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.909+0000 7f5d40a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebd2db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:34708/0 (socket says 172.21.15.42:34708) 2024-04-25T00:43:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.909+0000 7f5d40a00640 1 -- 172.21.15.42:0/3610213799 learned_addr learned my addr 172.21.15.42:0/3610213799 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.909+0000 7f5d3be00640 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc4690 0x55f2cebd32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.909+0000 7f5d41400640 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc9800 0x55f2cebd42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.909+0000 7f5d40a00640 1 -- 172.21.15.42:0/3610213799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc4690 msgr2=0x55f2cebd32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.909+0000 7f5d40a00640 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc4690 0x55f2cebd32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.909+0000 7f5d40a00640 1 -- 172.21.15.42:0/3610213799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc9800 msgr2=0x55f2cebd42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.909+0000 7f5d40a00640 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc9800 0x55f2cebd42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:19.910 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.909+0000 7f5d40a00640 1 -- 172.21.15.42:0/3610213799 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2cebd48a0 con 0x55f2cebc3ca0 2024-04-25T00:43:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.910+0000 7f5d40a00640 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebd2db0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5d3009b100 tx=0x7f5d30003870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=31b71a9211871a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.910+0000 7f5d22000640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:43:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.910+0000 7f5d22000640 1 -- 172.21.15.42:0/3610213799 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d300b3070 con 0x55f2cebc3ca0 2024-04-25T00:43:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.911+0000 7f5d22000640 1 -- 172.21.15.42:0/3610213799 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f5d3009a5c0 con 0x55f2cebc3ca0 2024-04-25T00:43:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.911+0000 7f5d461a8740 1 -- 172.21.15.42:0/3610213799 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d3009f520 con 0x55f2cebc3ca0 2024-04-25T00:43:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.911+0000 7f5d22000640 1 -- 172.21.15.42:0/3610213799 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f5d30099210 con 0x55f2cebc3ca0 2024-04-25T00:43:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.911+0000 7f5d461a8740 10 client.27068 fetch_fsmap learned FSMap version 45 2024-04-25T00:43:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.912+0000 7f5d461a8740 1 -- 172.21.15.42:0/3610213799 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55f2cec3ff50 con 0x55f2cebc3ca0 2024-04-25T00:43:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.912+0000 7f5d22000640 1 -- 172.21.15.42:0/3610213799 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5d3009f520 con 0x55f2cebc3ca0 2024-04-25T00:43:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.912+0000 7f5d22000640 1 -- 172.21.15.42:0/3610213799 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f5d30095ce0 con 0x55f2cebc3ca0 2024-04-25T00:43:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.913+0000 7f5d461a8740 10 client.27068 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:43:19.913 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.913+0000 7f5d461a8740 10 client.27068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:43:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.913+0000 7f5d461a8740 20 client.27068 populate_metadata read hostname 'smithi042' 2024-04-25T00:43:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.913+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f2cebd9870 0x55f2cec428b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.914+0000 7f5d461a8740 4 client.27068 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:43:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.914+0000 7f5d461a8740 1 -- 172.21.15.42:0/3610213799 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2ceb2e340 con 0x55f2cebd9870 2024-04-25T00:43:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.914+0000 7f5d3be00640 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f2cebd9870 0x55f2cec428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.915+0000 7f5d3be00640 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f2cebd9870 0x55f2cec428b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d2ba6ed2511468 server_cookie=53b3e33e3f4a0806 in_seq=0 out_seq=0 2024-04-25T00:43:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:19.915+0000 7f5d22000640 10 client.27068 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:20.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:19 smithi136 ceph-mon[32923]: pgmap v1127: 97 pgs: 97 active+clean; 655 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.1 MiB/s wr, 463 op/s 2024-04-25T00:43:20.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:19 smithi106 ceph-mon[32510]: pgmap v1127: 97 pgs: 97 active+clean; 655 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.1 MiB/s wr, 463 op/s 2024-04-25T00:43:20.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:19 smithi042 ceph-mon[28130]: pgmap v1127: 97 pgs: 97 active+clean; 655 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 5.1 MiB/s wr, 463 op/s 2024-04-25T00:43:20.818 INFO:tasks.workunit.client.0.smithi042.stdout: 10 407244 32.48 MB/sec execute 550 sec latency 2188.125 ms 2024-04-25T00:43:21.818 INFO:tasks.workunit.client.0.smithi042.stdout: 10 407244 32.42 MB/sec execute 551 sec latency 3188.262 ms 2024-04-25T00:43:22.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:21 smithi136 ceph-mon[32923]: pgmap v1128: 97 pgs: 97 active+clean; 687 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.6 MiB/s wr, 476 op/s 2024-04-25T00:43:22.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:21 smithi106 ceph-mon[32510]: pgmap v1128: 97 pgs: 97 active+clean; 687 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.6 MiB/s wr, 476 op/s 2024-04-25T00:43:22.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:21 smithi042 ceph-mon[28130]: pgmap v1128: 97 pgs: 97 active+clean; 687 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 7.6 MiB/s wr, 476 op/s 2024-04-25T00:43:22.818 INFO:tasks.workunit.client.0.smithi042.stdout: 10 407285 32.36 MB/sec execute 552 sec latency 3736.307 ms 2024-04-25T00:43:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.259+0000 7f5d22000640 1 -- 172.21.15.42:0/3610213799 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2ceb2e340 con 0x55f2cebd9870 2024-04-25T00:43:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.259+0000 7f5d22000640 10 client.27068 handle_command_reply: tid=0 2024-04-25T00:43:23.259 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.259+0000 7f5d461a8740 1 client.27068 shutdown 2024-04-25T00:43:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.259+0000 7f5d461a8740 1 -- 172.21.15.42:0/3610213799 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 msgr2=0x55f2cebd2db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.259+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebd2db0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f5d3009b100 tx=0x7f5d30003870 comp rx=0 tx=0).stop 2024-04-25T00:43:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.259+0000 7f5d461a8740 1 -- 172.21.15.42:0/3610213799 shutdown_connections 2024-04-25T00:43:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.259+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55f2cebd9870 0x55f2cec428b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.260+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55f2cebc9800 0x55f2cebd42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.260+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f2cebc4690 0x55f2cebd32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.260+0000 7f5d461a8740 1 --2- 172.21.15.42:0/3610213799 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55f2cebc3ca0 0x55f2cebd2db0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.260+0000 7f5d461a8740 1 -- 172.21.15.42:0/3610213799 >> 172.21.15.42:0/3610213799 conn(0x55f2cebbf310 msgr2=0x55f2cebc2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.260+0000 7f5d22000640 0 client.27068 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.260+0000 7f5d461a8740 1 -- 172.21.15.42:0/3610213799 shutdown_connections 2024-04-25T00:43:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.260+0000 7f5d461a8740 1 -- 172.21.15.42:0/3610213799 wait complete. 2024-04-25T00:43:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.261+0000 7f5d461a8740 20 client.27068 trim_cache size 0 max 16384 2024-04-25T00:43:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:43:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:43:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:43:23.263 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:43:23.264 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.264+0000 7f5d43400640 1 -- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5d240a2e20 msgr2=0x7f5d240a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.264+0000 7f5d43400640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5d240a2e20 0x7f5d240a52e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5d28000c00 tx=0x7f5d28030380 comp rx=0 tx=0).stop 2024-04-25T00:43:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.264+0000 7f5d43400640 1 -- 172.21.15.42:0/2621804954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d3c107dd0 msgr2=0x7f5d3c07eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.264+0000 7f5d43400640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d3c107dd0 0x7f5d3c07eea0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000d770 tx=0x7f5d3000dc30 comp rx=0 tx=0).stop 2024-04-25T00:43:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.264+0000 7f5d43400640 1 -- 172.21.15.42:0/2621804954 shutdown_connections 2024-04-25T00:43:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.264+0000 7f5d43400640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f5d240a2e20 0x7f5d240a52e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.264+0000 7f5d43400640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d3c109980 0x7f5d3c1ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.264+0000 7f5d43400640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5d3c108fc0 0x7f5d3c07f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.264+0000 7f5d43400640 1 --2- 172.21.15.42:0/2621804954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5d3c107dd0 0x7f5d3c07eea0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.264+0000 7f5d43400640 1 -- 172.21.15.42:0/2621804954 >> 172.21.15.42:0/2621804954 conn(0x7f5d3c076d90 msgr2=0x7f5d3c10c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:23.265 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.265+0000 7f5d43400640 1 -- 172.21.15.42:0/2621804954 shutdown_connections 2024-04-25T00:43:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:23.265+0000 7f5d43400640 1 -- 172.21.15.42:0/2621804954 wait complete. 2024-04-25T00:43:23.818 INFO:tasks.workunit.client.0.smithi042.stdout: 10 407500 32.31 MB/sec execute 553 sec latency 356.194 ms 2024-04-25T00:43:24.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:23 smithi136 ceph-mon[32923]: pgmap v1129: 97 pgs: 97 active+clean; 780 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 16 MiB/s wr, 578 op/s 2024-04-25T00:43:24.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:23 smithi106 ceph-mon[32510]: pgmap v1129: 97 pgs: 97 active+clean; 780 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 16 MiB/s wr, 578 op/s 2024-04-25T00:43:24.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:23 smithi042 ceph-mon[28130]: pgmap v1129: 97 pgs: 97 active+clean; 780 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 16 MiB/s wr, 578 op/s 2024-04-25T00:43:24.818 INFO:tasks.workunit.client.0.smithi042.stdout: 10 407739 32.27 MB/sec execute 554 sec latency 117.618 ms 2024-04-25T00:43:25.819 INFO:tasks.workunit.client.0.smithi042.stdout: 10 408280 32.27 MB/sec execute 555 sec latency 81.790 ms 2024-04-25T00:43:26.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:25 smithi136 ceph-mon[32923]: pgmap v1130: 97 pgs: 97 active+clean; 777 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 15 MiB/s wr, 483 op/s 2024-04-25T00:43:26.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:25 smithi106 ceph-mon[32510]: pgmap v1130: 97 pgs: 97 active+clean; 777 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 15 MiB/s wr, 483 op/s 2024-04-25T00:43:26.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:25 smithi042 ceph-mon[28130]: pgmap v1130: 97 pgs: 97 active+clean; 777 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 15 MiB/s wr, 483 op/s 2024-04-25T00:43:26.819 INFO:tasks.workunit.client.0.smithi042.stdout: 10 408975 32.27 MB/sec execute 556 sec latency 69.473 ms 2024-04-25T00:43:27.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 409651 32.28 MB/sec execute 557 sec latency 53.223 ms 2024-04-25T00:43:28.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:27 smithi136 ceph-mon[32923]: pgmap v1131: 97 pgs: 97 active+clean; 688 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 746 op/s 2024-04-25T00:43:28.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:27 smithi106 ceph-mon[32510]: pgmap v1131: 97 pgs: 97 active+clean; 688 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 746 op/s 2024-04-25T00:43:28.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:27 smithi042 ceph-mon[28130]: pgmap v1131: 97 pgs: 97 active+clean; 688 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 16 MiB/s wr, 746 op/s 2024-04-25T00:43:28.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 410263 32.29 MB/sec execute 558 sec latency 79.755 ms 2024-04-25T00:43:29.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 410761 32.29 MB/sec execute 559 sec latency 63.715 ms 2024-04-25T00:43:30.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:29 smithi136 ceph-mon[32923]: pgmap v1132: 97 pgs: 97 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 15 MiB/s wr, 665 op/s 2024-04-25T00:43:30.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:29 smithi106 ceph-mon[32510]: pgmap v1132: 97 pgs: 97 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 15 MiB/s wr, 665 op/s 2024-04-25T00:43:30.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:29 smithi042 ceph-mon[28130]: pgmap v1132: 97 pgs: 97 active+clean; 659 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 15 MiB/s wr, 665 op/s 2024-04-25T00:43:30.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 411249 32.26 MB/sec execute 560 sec latency 72.322 ms 2024-04-25T00:43:31.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 411716 32.25 MB/sec execute 561 sec latency 120.624 ms 2024-04-25T00:43:32.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:31 smithi136 ceph-mon[32923]: pgmap v1133: 97 pgs: 97 active+clean; 657 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 14 MiB/s wr, 643 op/s 2024-04-25T00:43:32.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:31 smithi106 ceph-mon[32510]: pgmap v1133: 97 pgs: 97 active+clean; 657 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 14 MiB/s wr, 643 op/s 2024-04-25T00:43:32.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:31 smithi042 ceph-mon[28130]: pgmap v1133: 97 pgs: 97 active+clean; 657 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 14 MiB/s wr, 643 op/s 2024-04-25T00:43:32.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 412365 32.25 MB/sec execute 562 sec latency 101.951 ms 2024-04-25T00:43:33.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 412928 32.26 MB/sec execute 563 sec latency 87.872 ms 2024-04-25T00:43:34.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:33 smithi136 ceph-mon[32923]: pgmap v1134: 97 pgs: 97 active+clean; 666 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 720 op/s 2024-04-25T00:43:34.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:33 smithi106 ceph-mon[32510]: pgmap v1134: 97 pgs: 97 active+clean; 666 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 720 op/s 2024-04-25T00:43:34.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:33 smithi042 ceph-mon[28130]: pgmap v1134: 97 pgs: 97 active+clean; 666 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 13 MiB/s wr, 720 op/s 2024-04-25T00:43:34.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 413562 32.26 MB/sec execute 564 sec latency 58.450 ms 2024-04-25T00:43:35.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 414083 32.26 MB/sec execute 565 sec latency 74.052 ms 2024-04-25T00:43:36.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:35 smithi136 ceph-mon[32923]: pgmap v1135: 97 pgs: 97 active+clean; 666 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 559 op/s 2024-04-25T00:43:36.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:35 smithi106 ceph-mon[32510]: pgmap v1135: 97 pgs: 97 active+clean; 666 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 559 op/s 2024-04-25T00:43:36.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:35 smithi042 ceph-mon[28130]: pgmap v1135: 97 pgs: 97 active+clean; 666 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 4.3 MiB/s wr, 559 op/s 2024-04-25T00:43:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.474+0000 7fcf22c00640 1 -- 172.21.15.42:0/2353845103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf1c104bc0 msgr2=0x7fcf1c105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.474+0000 7fcf22c00640 1 --2- 172.21.15.42:0/2353845103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf1c104bc0 0x7fcf1c105040 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fcf08009900 tx=0x7fcf0802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.475 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.475+0000 7fcf22c00640 1 -- 172.21.15.42:0/2353845103 shutdown_connections 2024-04-25T00:43:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.475+0000 7fcf22c00640 1 --2- 172.21.15.42:0/2353845103 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf1c105580 0x7fcf1c10be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.475+0000 7fcf22c00640 1 --2- 172.21.15.42:0/2353845103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf1c104bc0 0x7fcf1c105040 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.475+0000 7fcf22c00640 1 --2- 172.21.15.42:0/2353845103 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf1c1039d0 0x7fcf1c103df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.475+0000 7fcf22c00640 1 -- 172.21.15.42:0/2353845103 >> 172.21.15.42:0/2353845103 conn(0x7fcf1c0fef60 msgr2=0x7fcf1c101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.476+0000 7fcf22c00640 1 -- 172.21.15.42:0/2353845103 shutdown_connections 2024-04-25T00:43:36.476 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.476+0000 7fcf22c00640 1 -- 172.21.15.42:0/2353845103 wait complete. 2024-04-25T00:43:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.478+0000 7fcf22c00640 1 Processor -- start 2024-04-25T00:43:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.478+0000 7fcf22c00640 1 -- start start 2024-04-25T00:43:36.478 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.478+0000 7fcf22c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf1c1039d0 0x7fcf1c19e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.478+0000 7fcf22c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf1c104bc0 0x7fcf1c19e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.478+0000 7fcf22c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf1c105580 0x7fcf1c1a5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.478+0000 7fcf22c00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c10f720 con 0x7fcf1c105580 2024-04-25T00:43:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.478+0000 7fcf22c00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c10f5a0 con 0x7fcf1c104bc0 2024-04-25T00:43:36.479 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.478+0000 7fcf22c00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c10f8a0 con 0x7fcf1c1039d0 2024-04-25T00:43:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.479+0000 7fcf1be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf1c1039d0 0x7fcf1c19e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.479+0000 7fcf1be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf1c1039d0 0x7fcf1c19e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59022/0 (socket says 172.21.15.42:59022) 2024-04-25T00:43:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.479+0000 7fcf1be00640 1 -- 172.21.15.42:0/2539829367 learned_addr learned my addr 172.21.15.42:0/2539829367 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.479+0000 7fcf1be00640 1 -- 172.21.15.42:0/2539829367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf1c104bc0 msgr2=0x7fcf1c19e5f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T00:43:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.479+0000 7fcf1be00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf1c104bc0 0x7fcf1c19e5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.480 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.479+0000 7fcf1be00640 1 -- 172.21.15.42:0/2539829367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf1c105580 msgr2=0x7fcf1c1a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.479+0000 7fcf1be00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf1c105580 0x7fcf1c1a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.479+0000 7fcf1be00640 1 -- 172.21.15.42:0/2539829367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf1c1a5cd0 con 0x7fcf1c1039d0 2024-04-25T00:43:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.480+0000 7fcf1be00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf1c1039d0 0x7fcf1c19e0b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00e9c0 tx=0x7fcf0c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.480+0000 7fcf18c00640 1 -- 172.21.15.42:0/2539829367 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00cd80 con 0x7fcf1c1039d0 2024-04-25T00:43:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.480+0000 7fcf18c00640 1 -- 172.21.15.42:0/2539829367 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf0c0035f0 con 0x7fcf1c1039d0 2024-04-25T00:43:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.480+0000 7fcf18c00640 1 -- 172.21.15.42:0/2539829367 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c010690 con 0x7fcf1c1039d0 2024-04-25T00:43:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.480+0000 7fcf22c00640 1 -- 172.21.15.42:0/2539829367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf1c1a5fc0 con 0x7fcf1c1039d0 2024-04-25T00:43:36.481 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.481+0000 7fcf22c00640 1 -- 172.21.15.42:0/2539829367 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf1c1a6480 con 0x7fcf1c1039d0 2024-04-25T00:43:36.486 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.486+0000 7fcf18c00640 1 -- 172.21.15.42:0/2539829367 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf0c0037a0 con 0x7fcf1c1039d0 2024-04-25T00:43:36.488 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.488+0000 7fcf25a33740 1 Processor -- start 2024-04-25T00:43:36.489 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.489+0000 7fcf18c00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcf000a2cb0 0x7fcf000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.490+0000 7fcf18c00640 1 -- 172.21.15.42:0/2539829367 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fcf0c0cf440 con 0x7fcf1c1039d0 2024-04-25T00:43:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.490+0000 7fcf25a33740 1 -- start start 2024-04-25T00:43:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.491+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 0x55c335ff20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.491+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff2690 0x55c335ff7190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.491+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff7800 0x55c335ff9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.491+0000 7fcf25a33740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c335f54ab0 con 0x55c335ff2690 2024-04-25T00:43:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.491+0000 7fcf25a33740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c335efd640 con 0x55c335ff1ca0 2024-04-25T00:43:36.491 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.491+0000 7fcf25a33740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c335f50a70 con 0x55c335ff7800 2024-04-25T00:43:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.491+0000 7fcf20c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff7800 0x55c335ff9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.491+0000 7fcf20c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff7800 0x55c335ff9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59026/0 (socket says 172.21.15.42:59026) 2024-04-25T00:43:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.491+0000 7fcf20c00640 1 -- 172.21.15.42:0/166431793 learned_addr learned my addr 172.21.15.42:0/166431793 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:36.492 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.492+0000 7fcf20c00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcf000a2cb0 0x7fcf000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.493 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.492+0000 7fcf20c00640 1 -- 172.21.15.42:0/166431793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 msgr2=0x55c335ff20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.492+0000 7fcf20c00640 1 --2- 172.21.15.42:0/166431793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 0x55c335ff20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.492+0000 7fcf20c00640 1 -- 172.21.15.42:0/166431793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff2690 msgr2=0x55c335ff7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.492+0000 7fcf20c00640 1 --2- 172.21.15.42:0/166431793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff2690 0x55c335ff7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.492+0000 7fcf20c00640 1 -- 172.21.15.42:0/166431793 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c335c69210 con 0x55c335ff7800 2024-04-25T00:43:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.492+0000 7fcf20c00640 1 --2- 172.21.15.42:0/166431793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff7800 0x55c335ff9c90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf140122e0 tx=0x7fcf140127a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd76726500ca2370 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.493+0000 7fcf11200640 1 -- 172.21.15.42:0/166431793 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14018480 con 0x55c335ff7800 2024-04-25T00:43:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.493+0000 7fcf11200640 1 -- 172.21.15.42:0/166431793 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf1401d070 con 0x55c335ff7800 2024-04-25T00:43:36.494 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.493+0000 7fcf11200640 1 -- 172.21.15.42:0/166431793 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf14017800 con 0x55c335ff7800 2024-04-25T00:43:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.494+0000 7fcf20c00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcf000a2cb0 0x7fcf000a5170 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf1401c000 tx=0x7fcf14018000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.494+0000 7fcf25a33740 1 -- 172.21.15.42:0/166431793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff7800 msgr2=0x55c335ff9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.494+0000 7fcf25a33740 1 --2- 172.21.15.42:0/166431793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff7800 0x55c335ff9c90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fcf140122e0 tx=0x7fcf140127a0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.494+0000 7fcf25a33740 1 -- 172.21.15.42:0/166431793 shutdown_connections 2024-04-25T00:43:36.495 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.494+0000 7fcf25a33740 1 --2- 172.21.15.42:0/166431793 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff7800 0x55c335ff9c90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.494+0000 7fcf25a33740 1 --2- 172.21.15.42:0/166431793 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff2690 0x55c335ff7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.494+0000 7fcf25a33740 1 --2- 172.21.15.42:0/166431793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 0x55c335ff20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.494+0000 7fcf25a33740 1 -- 172.21.15.42:0/166431793 >> 172.21.15.42:0/166431793 conn(0x55c335fed310 msgr2=0x55c335fed710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.495+0000 7fcf25a33740 1 -- 172.21.15.42:0/166431793 shutdown_connections 2024-04-25T00:43:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.495+0000 7fcf25a33740 1 -- 172.21.15.42:0/166431793 wait complete. 2024-04-25T00:43:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.495+0000 7fcf25a33740 1 -- start start 2024-04-25T00:43:36.496 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.495+0000 7fcf25a33740 1 Processor -- start 2024-04-25T00:43:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 0x55c335ffbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff2690 0x55c335ffc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff7800 0x55c336003510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf25a33740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55c335ffa7b0 con 0x55c335ff1ca0 2024-04-25T00:43:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf25a33740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55c335f10990 con 0x55c335ff7800 2024-04-25T00:43:36.497 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf25a33740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c335ffa930 con 0x55c335ff2690 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf25a33740 10 client.0 _ll_register_callbacks cb 0x55c335f12950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff2690 0x55c335ffc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf1be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 0x55c335ffbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf20c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff7800 0x55c336003510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf20c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff7800 0x55c336003510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:32898/0 (socket says 172.21.15.42:32898) 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.496+0000 7fcf20c00640 1 -- 172.21.15.42:0/2232588745 learned_addr learned my addr 172.21.15.42:0/2232588745 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.497+0000 7fcf20c00640 1 -- 172.21.15.42:0/2232588745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff2690 msgr2=0x55c335ffc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.497+0000 7fcf20c00640 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff2690 0x55c335ffc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.497+0000 7fcf20c00640 1 -- 172.21.15.42:0/2232588745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 msgr2=0x55c335ffbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.497+0000 7fcf20c00640 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 0x55c335ffbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.497+0000 7fcf20c00640 1 -- 172.21.15.42:0/2232588745 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c335d35620 con 0x55c335ff7800 2024-04-25T00:43:36.498 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.497+0000 7fcf1be00640 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 0x55c335ffbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:43:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.497+0000 7fcf20c00640 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff7800 0x55c336003510 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcf14006010 tx=0x7fcf14017c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45d879fc7700c6bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.497+0000 7fcf05600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:43:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.498+0000 7fcf05600640 1 -- 172.21.15.42:0/2232588745 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf140175a0 con 0x55c335ff7800 2024-04-25T00:43:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.498+0000 7fcf05600640 1 -- 172.21.15.42:0/2232588745 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fcf14017740 con 0x55c335ff7800 2024-04-25T00:43:36.499 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.498+0000 7fcf25a33740 1 -- 172.21.15.42:0/2232588745 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf1401d070 con 0x55c335ff7800 2024-04-25T00:43:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.498+0000 7fcf05600640 1 -- 172.21.15.42:0/2232588745 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcf14018690 con 0x55c335ff7800 2024-04-25T00:43:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.498+0000 7fcf25a33740 10 client.27074 fetch_fsmap learned FSMap version 45 2024-04-25T00:43:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.498+0000 7fcf25a33740 1 -- 172.21.15.42:0/2232588745 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c336006560 con 0x55c335ff7800 2024-04-25T00:43:36.500 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.499+0000 7fcf05600640 1 -- 172.21.15.42:0/2232588745 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf1401d070 con 0x55c335ff7800 2024-04-25T00:43:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.500+0000 7fcf05600640 1 -- 172.21.15.42:0/2232588745 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fcf14023760 con 0x55c335ff7800 2024-04-25T00:43:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.500+0000 7fcf25a33740 10 client.27074 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:43:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.500+0000 7fcf25a33740 10 client.27074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:43:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.500+0000 7fcf25a33740 20 client.27074 populate_metadata read hostname 'smithi042' 2024-04-25T00:43:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.500+0000 7fcf25a33740 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c336007af0 0x55c336009ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.500+0000 7fcf25a33740 4 client.27074 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:43:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.500+0000 7fcf25a33740 1 -- 172.21.15.42:0/2232588745 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c335f61ff0 con 0x55c336007af0 2024-04-25T00:43:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.500+0000 7fcf1be00640 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c336007af0 0x55c336009ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.501 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.501+0000 7fcf1be00640 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c336007af0 0x55c336009ee0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b010fd4997fe38 server_cookie=72857e506a3b727c in_seq=0 out_seq=0 2024-04-25T00:43:36.502 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.501+0000 7fcf05600640 10 client.27074 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf05600640 1 -- 172.21.15.42:0/2232588745 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c335f61ff0 con 0x55c336007af0 2024-04-25T00:43:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf05600640 10 client.27074 handle_command_reply: tid=0 2024-04-25T00:43:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf25a33740 1 client.27074 shutdown 2024-04-25T00:43:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf25a33740 1 -- 172.21.15.42:0/2232588745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff7800 msgr2=0x55c336003510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:36.503 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf25a33740 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff7800 0x55c336003510 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcf14006010 tx=0x7fcf14017c90 comp rx=0 tx=0).stop 2024-04-25T00:43:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf25a33740 1 -- 172.21.15.42:0/2232588745 shutdown_connections 2024-04-25T00:43:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf25a33740 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c336007af0 0x55c336009ee0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf25a33740 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff7800 0x55c336003510 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf05600640 0 client.27074 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf25a33740 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335ff2690 0x55c335ffc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf25a33740 1 --2- 172.21.15.42:0/2232588745 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 0x55c335ffbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf25a33740 1 -- 172.21.15.42:0/2232588745 >> 172.21.15.42:0/2232588745 conn(0x55c335fed310 msgr2=0x55c335fec480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.502+0000 7fcf25a33740 1 -- 172.21.15.42:0/2232588745 shutdown_connections 2024-04-25T00:43:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.503+0000 7fcf25a33740 1 -- 172.21.15.42:0/2232588745 wait complete. 2024-04-25T00:43:36.504 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.503+0000 7fcf25a33740 20 client.27074 trim_cache size 0 max 16384 2024-04-25T00:43:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.527+0000 7fcf25a33740 1 Processor -- start 2024-04-25T00:43:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.527+0000 7fcf25a33740 1 -- start start 2024-04-25T00:43:36.527 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.527+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 0x55c335ff3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.527+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 0x55c3360034d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.527+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff2690 0x55c335ff3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.527+0000 7fcf25a33740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c1b1b90 con 0x55c335ff2690 2024-04-25T00:43:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.527+0000 7fcf25a33740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c1af460 con 0x55c335ff1ca0 2024-04-25T00:43:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.527+0000 7fcf25a33740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c1ad2b0 con 0x55c335fed310 2024-04-25T00:43:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.527+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 0x55c3360034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.528+0000 7fcf1b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 0x55c3360034d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:32912/0 (socket says 172.21.15.42:32912) 2024-04-25T00:43:36.528 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.528+0000 7fcf1b400640 1 -- 172.21.15.42:0/4225468266 learned_addr learned my addr 172.21.15.42:0/4225468266 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.528+0000 7fcf1be00640 1 --2- 172.21.15.42:0/4225468266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 0x55c335ff3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.528+0000 7fcf20c00640 1 --2- 172.21.15.42:0/4225468266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff2690 0x55c335ff3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.528+0000 7fcf1b400640 1 -- 172.21.15.42:0/4225468266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 msgr2=0x55c335ff3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.528+0000 7fcf1b400640 1 --2- 172.21.15.42:0/4225468266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 0x55c335ff3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.528+0000 7fcf1b400640 1 -- 172.21.15.42:0/4225468266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff2690 msgr2=0x55c335ff3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.528+0000 7fcf1b400640 1 --2- 172.21.15.42:0/4225468266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff2690 0x55c335ff3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.528+0000 7fcf1b400640 1 -- 172.21.15.42:0/4225468266 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c33600a250 con 0x55c335ff1ca0 2024-04-25T00:43:36.529 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.529+0000 7fcf1b400640 1 --2- 172.21.15.42:0/4225468266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 0x55c3360034d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcf0802a9e0 tx=0x7fcf08000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f152b2da13710ab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.530+0000 7fcf07e00640 1 -- 172.21.15.42:0/4225468266 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08040070 con 0x55c335ff1ca0 2024-04-25T00:43:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.530+0000 7fcf07e00640 1 -- 172.21.15.42:0/4225468266 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf08005500 con 0x55c335ff1ca0 2024-04-25T00:43:36.530 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.530+0000 7fcf07e00640 1 -- 172.21.15.42:0/4225468266 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf08040070 con 0x55c335ff1ca0 2024-04-25T00:43:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.531+0000 7fcf25a33740 1 -- 172.21.15.42:0/4225468266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 msgr2=0x55c3360034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:36.531 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.531+0000 7fcf25a33740 1 --2- 172.21.15.42:0/4225468266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 0x55c3360034d0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcf0802a9e0 tx=0x7fcf08000bc0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.531+0000 7fcf25a33740 1 -- 172.21.15.42:0/4225468266 shutdown_connections 2024-04-25T00:43:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.531+0000 7fcf25a33740 1 --2- 172.21.15.42:0/4225468266 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff2690 0x55c335ff3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.531+0000 7fcf25a33740 1 --2- 172.21.15.42:0/4225468266 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff1ca0 0x55c3360034d0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.531+0000 7fcf25a33740 1 --2- 172.21.15.42:0/4225468266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 0x55c335ff3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.531+0000 7fcf25a33740 1 -- 172.21.15.42:0/4225468266 >> 172.21.15.42:0/4225468266 conn(0x55c335ff72d0 msgr2=0x55c335fe7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:36.532 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.532+0000 7fcf25a33740 1 -- 172.21.15.42:0/4225468266 shutdown_connections 2024-04-25T00:43:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.532+0000 7fcf25a33740 1 -- 172.21.15.42:0/4225468266 wait complete. 2024-04-25T00:43:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.532+0000 7fcf25a33740 1 -- start start 2024-04-25T00:43:36.533 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.533+0000 7fcf25a33740 1 Processor -- start 2024-04-25T00:43:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.533+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 0x55c336000170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.533+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 0x55c3360006b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf25a33740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff2690 0x55c33606bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf25a33740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c1ad920 con 0x55c335ff1ca0 2024-04-25T00:43:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf25a33740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c1aed90 con 0x55c335ff2690 2024-04-25T00:43:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf25a33740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf1c1adff0 con 0x55c335fed310 2024-04-25T00:43:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf25a33740 10 client.0 _ll_register_callbacks cb 0x7fcf1400bf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:43:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf1be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 0x55c336000170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf1be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 0x55c336000170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:59052/0 (socket says 172.21.15.42:59052) 2024-04-25T00:43:36.534 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf1be00640 1 -- 172.21.15.42:0/3196646454 learned_addr learned my addr 172.21.15.42:0/3196646454 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf20c00640 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff2690 0x55c33606bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf1b400640 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 0x55c3360006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf1be00640 1 -- 172.21.15.42:0/3196646454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff2690 msgr2=0x55c33606bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf1be00640 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff2690 0x55c33606bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf1be00640 1 -- 172.21.15.42:0/3196646454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 msgr2=0x55c3360006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf1be00640 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 0x55c3360006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.534+0000 7fcf1be00640 1 -- 172.21.15.42:0/3196646454 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c336000ca0 con 0x55c335fed310 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.535+0000 7fcf1be00640 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 0x55c336000170 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0986a0 tx=0x7fcf0c09bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b82ef837c69b324e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.535+0000 7fcf05600640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.535+0000 7fcf05600640 1 -- 172.21.15.42:0/3196646454 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c0b8070 con 0x55c335fed310 2024-04-25T00:43:36.536 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.535+0000 7fcf05600640 1 -- 172.21.15.42:0/3196646454 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fcf0c099fd0 con 0x55c335fed310 2024-04-25T00:43:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.535+0000 7fcf25a33740 1 -- 172.21.15.42:0/3196646454 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf08005500 con 0x55c335fed310 2024-04-25T00:43:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.535+0000 7fcf05600640 1 -- 172.21.15.42:0/3196646454 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fcf0c081070 con 0x55c335fed310 2024-04-25T00:43:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.535+0000 7fcf25a33740 10 client.27031 fetch_fsmap learned FSMap version 45 2024-04-25T00:43:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.535+0000 7fcf25a33740 1 -- 172.21.15.42:0/3196646454 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55c33606d670 con 0x55c335fed310 2024-04-25T00:43:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.536+0000 7fcf05600640 1 -- 172.21.15.42:0/3196646454 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf08005500 con 0x55c335fed310 2024-04-25T00:43:36.537 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.536+0000 7fcf05600640 1 -- 172.21.15.42:0/3196646454 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fcf0c0a4b90 con 0x55c335fed310 2024-04-25T00:43:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.536+0000 7fcf25a33740 10 client.27031 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:43:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.536+0000 7fcf25a33740 10 client.27031 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:43:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.536+0000 7fcf25a33740 20 client.27031 populate_metadata read hostname 'smithi042' 2024-04-25T00:43:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.536+0000 7fcf25a33740 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c335ff7800 0x55c336070030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.536+0000 7fcf25a33740 4 client.27031 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:43:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.536+0000 7fcf25a33740 1 -- 172.21.15.42:0/3196646454 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c335f5c340 con 0x55c335ff7800 2024-04-25T00:43:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.537+0000 7fcf1b400640 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c335ff7800 0x55c336070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.537+0000 7fcf1b400640 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c335ff7800 0x55c336070030 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d021516d3652ca5 server_cookie=c7e54c939e7afaab in_seq=0 out_seq=0 2024-04-25T00:43:36.538 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.537+0000 7fcf05600640 10 client.27031 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:36.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 414543 32.24 MB/sec execute 566 sec latency 201.427 ms 2024-04-25T00:43:36.871 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.871+0000 7fcf05600640 1 -- 172.21.15.42:0/3196646454 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c335f5c340 con 0x55c335ff7800 2024-04-25T00:43:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.871+0000 7fcf05600640 10 client.27031 handle_command_reply: tid=0 2024-04-25T00:43:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.871+0000 7fcf25a33740 1 client.27031 shutdown 2024-04-25T00:43:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.872+0000 7fcf25a33740 1 -- 172.21.15.42:0/3196646454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 msgr2=0x55c336000170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.872+0000 7fcf25a33740 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 0x55c336000170 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c0986a0 tx=0x7fcf0c09bd90 comp rx=0 tx=0).stop 2024-04-25T00:43:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.872+0000 7fcf25a33740 1 -- 172.21.15.42:0/3196646454 shutdown_connections 2024-04-25T00:43:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.872+0000 7fcf25a33740 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55c335ff7800 0x55c336070030 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.872+0000 7fcf25a33740 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c335ff2690 0x55c33606bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.872+0000 7fcf25a33740 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55c335ff1ca0 0x55c3360006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.872 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.872+0000 7fcf25a33740 1 --2- 172.21.15.42:0/3196646454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c335fed310 0x55c336000170 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.872+0000 7fcf25a33740 1 -- 172.21.15.42:0/3196646454 >> 172.21.15.42:0/3196646454 conn(0x55c335ff72d0 msgr2=0x55c335ff6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.872+0000 7fcf05600640 0 client.27031 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.872+0000 7fcf25a33740 1 -- 172.21.15.42:0/3196646454 shutdown_connections 2024-04-25T00:43:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.873+0000 7fcf25a33740 1 -- 172.21.15.42:0/3196646454 wait complete. 2024-04-25T00:43:36.873 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.873+0000 7fcf25a33740 20 client.27031 trim_cache size 0 max 16384 2024-04-25T00:43:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:43:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:43:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:43:36.874 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:43:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.874+0000 7fcf22c00640 1 -- 172.21.15.42:0/2539829367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcf000a2cb0 msgr2=0x7fcf000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.874+0000 7fcf22c00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcf000a2cb0 0x7fcf000a5170 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcf1401c000 tx=0x7fcf14018000 comp rx=0 tx=0).stop 2024-04-25T00:43:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.874+0000 7fcf22c00640 1 -- 172.21.15.42:0/2539829367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf1c1039d0 msgr2=0x7fcf1c19e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.874+0000 7fcf22c00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf1c1039d0 0x7fcf1c19e0b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00e9c0 tx=0x7fcf0c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:43:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.874+0000 7fcf22c00640 1 -- 172.21.15.42:0/2539829367 shutdown_connections 2024-04-25T00:43:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.874+0000 7fcf22c00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fcf000a2cb0 0x7fcf000a5170 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.874+0000 7fcf22c00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fcf1c105580 0x7fcf1c1a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.874+0000 7fcf22c00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcf1c104bc0 0x7fcf1c19e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.875 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.874+0000 7fcf22c00640 1 --2- 172.21.15.42:0/2539829367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf1c1039d0 0x7fcf1c19e0b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.874+0000 7fcf22c00640 1 -- 172.21.15.42:0/2539829367 >> 172.21.15.42:0/2539829367 conn(0x7fcf1c0fef60 msgr2=0x7fcf1c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.875+0000 7fcf22c00640 1 -- 172.21.15.42:0/2539829367 shutdown_connections 2024-04-25T00:43:36.876 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:36.875+0000 7fcf22c00640 1 -- 172.21.15.42:0/2539829367 wait complete. 2024-04-25T00:43:37.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 415084 32.24 MB/sec execute 567 sec latency 215.843 ms 2024-04-25T00:43:38.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:37 smithi136 ceph-mon[32923]: pgmap v1136: 97 pgs: 97 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.9 MiB/s wr, 642 op/s 2024-04-25T00:43:38.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:37 smithi106 ceph-mon[32510]: pgmap v1136: 97 pgs: 97 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.9 MiB/s wr, 642 op/s 2024-04-25T00:43:38.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:37 smithi042 ceph-mon[28130]: pgmap v1136: 97 pgs: 97 active+clean; 675 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 4.9 MiB/s wr, 642 op/s 2024-04-25T00:43:38.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 415870 32.25 MB/sec execute 568 sec latency 73.790 ms 2024-04-25T00:43:39.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 416611 32.27 MB/sec execute 569 sec latency 47.439 ms 2024-04-25T00:43:40.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:39 smithi136 ceph-mon[32923]: pgmap v1137: 97 pgs: 97 active+clean; 679 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 438 op/s 2024-04-25T00:43:40.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:39 smithi106 ceph-mon[32510]: pgmap v1137: 97 pgs: 97 active+clean; 679 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 438 op/s 2024-04-25T00:43:40.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:39 smithi042 ceph-mon[28130]: pgmap v1137: 97 pgs: 97 active+clean; 679 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 3.7 MiB/s wr, 438 op/s 2024-04-25T00:43:40.820 INFO:tasks.workunit.client.0.smithi042.stdout: 10 417210 32.27 MB/sec execute 570 sec latency 47.397 ms 2024-04-25T00:43:41.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 417885 32.28 MB/sec execute 571 sec latency 58.051 ms 2024-04-25T00:43:42.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:41 smithi136 ceph-mon[32923]: pgmap v1138: 97 pgs: 97 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 404 op/s 2024-04-25T00:43:42.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:41 smithi106 ceph-mon[32510]: pgmap v1138: 97 pgs: 97 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 404 op/s 2024-04-25T00:43:42.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:41 smithi042 ceph-mon[28130]: pgmap v1138: 97 pgs: 97 active+clean; 682 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.1 MiB/s wr, 404 op/s 2024-04-25T00:43:42.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 418568 32.27 MB/sec execute 572 sec latency 45.461 ms 2024-04-25T00:43:43.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 418881 32.25 MB/sec execute 573 sec latency 364.249 ms 2024-04-25T00:43:44.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:43 smithi136 ceph-mon[32923]: pgmap v1139: 97 pgs: 97 active+clean; 687 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 531 op/s 2024-04-25T00:43:44.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:43 smithi106 ceph-mon[32510]: pgmap v1139: 97 pgs: 97 active+clean; 687 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 531 op/s 2024-04-25T00:43:44.378 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:43 smithi042 ceph-mon[28130]: pgmap v1139: 97 pgs: 97 active+clean; 687 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.9 MiB/s wr, 531 op/s 2024-04-25T00:43:44.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 419599 32.27 MB/sec execute 574 sec latency 75.595 ms 2024-04-25T00:43:45.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 420143 32.25 MB/sec execute 575 sec latency 93.278 ms 2024-04-25T00:43:46.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:45 smithi136 ceph-mon[32923]: pgmap v1140: 97 pgs: 97 active+clean; 692 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.2 MiB/s wr, 447 op/s 2024-04-25T00:43:46.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:45 smithi106 ceph-mon[32510]: pgmap v1140: 97 pgs: 97 active+clean; 692 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.2 MiB/s wr, 447 op/s 2024-04-25T00:43:46.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:45 smithi042 ceph-mon[28130]: pgmap v1140: 97 pgs: 97 active+clean; 692 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 3.2 MiB/s wr, 447 op/s 2024-04-25T00:43:46.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 420790 32.27 MB/sec execute 576 sec latency 64.336 ms 2024-04-25T00:43:47.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 421590 32.32 MB/sec execute 577 sec latency 51.165 ms 2024-04-25T00:43:48.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:47 smithi136 ceph-mon[32923]: pgmap v1141: 97 pgs: 97 active+clean; 701 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.1 MiB/s wr, 544 op/s 2024-04-25T00:43:48.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:47 smithi106 ceph-mon[32510]: pgmap v1141: 97 pgs: 97 active+clean; 701 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.1 MiB/s wr, 544 op/s 2024-04-25T00:43:48.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:47 smithi042 ceph-mon[28130]: pgmap v1141: 97 pgs: 97 active+clean; 701 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.1 MiB/s wr, 544 op/s 2024-04-25T00:43:48.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 422206 32.32 MB/sec execute 578 sec latency 70.241 ms 2024-04-25T00:43:49.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 422781 32.32 MB/sec execute 579 sec latency 65.515 ms 2024-04-25T00:43:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.072+0000 7fab5c000640 1 -- 172.21.15.42:0/3132789065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5410de50 msgr2=0x7fab54110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.072+0000 7fab5c000640 1 --2- 172.21.15.42:0/3132789065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5410de50 0x7fab54110310 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fab4c009be0 tx=0x7fab4c02a6d0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.073+0000 7fab5c000640 1 -- 172.21.15.42:0/3132789065 shutdown_connections 2024-04-25T00:43:50.073 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.073+0000 7fab5c000640 1 --2- 172.21.15.42:0/3132789065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5410de50 0x7fab54110310 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.073+0000 7fab5c000640 1 --2- 172.21.15.42:0/3132789065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab5410b370 0x7fab5410d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.073+0000 7fab5c000640 1 --2- 172.21.15.42:0/3132789065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab541089b0 0x7fab5410ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.073+0000 7fab5c000640 1 -- 172.21.15.42:0/3132789065 >> 172.21.15.42:0/3132789065 conn(0x7fab5406e970 msgr2=0x7fab54070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.073+0000 7fab5c000640 1 -- 172.21.15.42:0/3132789065 shutdown_connections 2024-04-25T00:43:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.073+0000 7fab5c000640 1 -- 172.21.15.42:0/3132789065 wait complete. 2024-04-25T00:43:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.074+0000 7fab5c000640 1 Processor -- start 2024-04-25T00:43:50.074 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.074+0000 7fab5c000640 1 -- start start 2024-04-25T00:43:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.075+0000 7fab5c000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab541089b0 0x7fab541a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.075+0000 7fab5c000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5410b370 0x7fab541a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.075 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.075+0000 7fab5c000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5410de50 0x7fab541a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.075+0000 7fab5c000640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fab54113a30 con 0x7fab541089b0 2024-04-25T00:43:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.075+0000 7fab5c000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab541138b0 con 0x7fab5410b370 2024-04-25T00:43:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.075+0000 7fab5c000640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab54113bb0 con 0x7fab5410de50 2024-04-25T00:43:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.075+0000 7fab58c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5410b370 0x7fab541a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.076 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.075+0000 7fab59600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab541089b0 0x7fab541a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.075+0000 7fab58c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5410b370 0x7fab541a28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42490/0 (socket says 172.21.15.42:42490) 2024-04-25T00:43:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.076+0000 7fab58c00640 1 -- 172.21.15.42:0/3830164475 learned_addr learned my addr 172.21.15.42:0/3830164475 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.075+0000 7fab59600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab541089b0 0x7fab541a2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:33570/0 (socket says 172.21.15.42:33570) 2024-04-25T00:43:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.076+0000 7fab5a000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5410de50 0x7fab541a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.076+0000 7fab58c00640 1 -- 172.21.15.42:0/3830164475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5410de50 msgr2=0x7fab541a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.076+0000 7fab58c00640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5410de50 0x7fab541a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.076+0000 7fab58c00640 1 -- 172.21.15.42:0/3830164475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab541089b0 msgr2=0x7fab541a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.076+0000 7fab58c00640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab541089b0 0x7fab541a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.076+0000 7fab58c00640 1 -- 172.21.15.42:0/3830164475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab541a9fb0 con 0x7fab5410b370 2024-04-25T00:43:50.077 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.077+0000 7fab58c00640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5410b370 0x7fab541a28d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fab4800c970 tx=0x7fab4800cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.077+0000 7fab52000640 1 -- 172.21.15.42:0/3830164475 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48014070 con 0x7fab5410b370 2024-04-25T00:43:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.077+0000 7fab52000640 1 -- 172.21.15.42:0/3830164475 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab48004440 con 0x7fab5410b370 2024-04-25T00:43:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.077+0000 7fab52000640 1 -- 172.21.15.42:0/3830164475 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab48003e10 con 0x7fab5410b370 2024-04-25T00:43:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.077+0000 7fab5c000640 1 -- 172.21.15.42:0/3830164475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab541aa2a0 con 0x7fab5410b370 2024-04-25T00:43:50.078 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.077+0000 7fab5c000640 1 -- 172.21.15.42:0/3830164475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab541aa7a0 con 0x7fab5410b370 2024-04-25T00:43:50.082 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.082+0000 7fab5ef8b740 1 Processor -- start 2024-04-25T00:43:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.083+0000 7fab52000640 1 -- 172.21.15.42:0/3830164475 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fab48017070 con 0x7fab5410b370 2024-04-25T00:43:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.083+0000 7fab5ef8b740 1 -- start start 2024-04-25T00:43:50.083 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.083+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f754b0 0x55d699f758d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.083+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.083+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f7b010 0x55d699f7d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.083+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d699ecfab0 con 0x55d699f754b0 2024-04-25T00:43:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.083+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d699e78640 con 0x55d699f75ea0 2024-04-25T00:43:50.084 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.083+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d699ecba70 con 0x55d699f7b010 2024-04-25T00:43:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.084+0000 7fab59600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.084+0000 7fab59600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42502/0 (socket says 172.21.15.42:42502) 2024-04-25T00:43:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.084+0000 7fab59600640 1 -- 172.21.15.42:0/325889698 learned_addr learned my addr 172.21.15.42:0/325889698 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.084+0000 7fab58c00640 1 --2- 172.21.15.42:0/325889698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f7b010 0x55d699f7d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.084+0000 7fab59600640 1 -- 172.21.15.42:0/325889698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f7b010 msgr2=0x55d699f7d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.084+0000 7fab59600640 1 --2- 172.21.15.42:0/325889698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f7b010 0x55d699f7d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.084+0000 7fab59600640 1 -- 172.21.15.42:0/325889698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f754b0 msgr2=0x55d699f758d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.084+0000 7fab59600640 1 --2- 172.21.15.42:0/325889698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f754b0 0x55d699f758d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.085 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.084+0000 7fab59600640 1 -- 172.21.15.42:0/325889698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d699be4210 con 0x55d699f75ea0 2024-04-25T00:43:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.085+0000 7fab59600640 1 --2- 172.21.15.42:0/325889698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7a9a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fab4000a810 tx=0x7fab4000ac00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7845ffda962a8846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.085+0000 7fab52000640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab3c0a2d00 0x7fab3c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.086 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.085+0000 7fab46600640 1 -- 172.21.15.42:0/325889698 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40009e50 con 0x55d699f75ea0 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.085+0000 7fab52000640 1 -- 172.21.15.42:0/3830164475 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fab480cee70 con 0x7fab5410b370 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.085+0000 7fab46600640 1 -- 172.21.15.42:0/325889698 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4001b070 con 0x55d699f75ea0 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.085+0000 7fab46600640 1 -- 172.21.15.42:0/325889698 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40016480 con 0x55d699f75ea0 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.086+0000 7fab5ef8b740 1 -- 172.21.15.42:0/325889698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 msgr2=0x55d699f7a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.086+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/325889698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7a9a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fab4000a810 tx=0x7fab4000ac00 comp rx=0 tx=0).stop 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.086+0000 7fab5a000640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab3c0a2d00 0x7fab3c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.086+0000 7fab5ef8b740 1 -- 172.21.15.42:0/325889698 shutdown_connections 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.086+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/325889698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f7b010 0x55d699f7d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.086+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/325889698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7a9a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.086+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/325889698 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f754b0 0x55d699f758d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.087 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.086+0000 7fab5ef8b740 1 -- 172.21.15.42:0/325889698 >> 172.21.15.42:0/325889698 conn(0x55d699f70b20 msgr2=0x55d699f70f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.087+0000 7fab5ef8b740 1 -- 172.21.15.42:0/325889698 shutdown_connections 2024-04-25T00:43:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.088+0000 7fab5ef8b740 1 -- 172.21.15.42:0/325889698 wait complete. 2024-04-25T00:43:50.088 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.088+0000 7fab5ef8b740 1 -- start start 2024-04-25T00:43:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.088+0000 7fab5ef8b740 1 Processor -- start 2024-04-25T00:43:50.089 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.088+0000 7fab5a000640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab3c0a2d00 0x7fab3c0a51c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fab4c006010 tx=0x7fab4c005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f754b0 0x55d699f7f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f7b010 0x55d699f86cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.090 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x55d699f7dfc0 con 0x55d699f754b0 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x55d699e8b990 con 0x55d699f75ea0 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d699f7e140 con 0x55d699f7b010 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab5ef8b740 10 client.0 _ll_register_callbacks cb 0x55d699e8d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab58c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab58c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42512/0 (socket says 172.21.15.42:42512) 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab58c00640 1 -- 172.21.15.42:0/981151302 learned_addr learned my addr 172.21.15.42:0/981151302 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.089+0000 7fab5a000640 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f7b010 0x55d699f86cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.090+0000 7fab58c00640 1 -- 172.21.15.42:0/981151302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f7b010 msgr2=0x55d699f86cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.090+0000 7fab58c00640 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f7b010 0x55d699f86cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.090+0000 7fab58c00640 1 -- 172.21.15.42:0/981151302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f754b0 msgr2=0x55d699f7f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.090+0000 7fab58c00640 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f754b0 0x55d699f7f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.090+0000 7fab58c00640 1 -- 172.21.15.42:0/981151302 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d699cb0620 con 0x55d699f75ea0 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.090+0000 7fab58c00640 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7fc70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fab48004630 tx=0x7fab4800a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edb4039953e45cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:50.091 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.090+0000 7fab3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.091+0000 7fab3aa00640 1 -- 172.21.15.42:0/981151302 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab480cce60 con 0x55d699f75ea0 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.091+0000 7fab3aa00640 1 -- 172.21.15.42:0/981151302 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fab480a3ee0 con 0x55d699f75ea0 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.091+0000 7fab5ef8b740 1 -- 172.21.15.42:0/981151302 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab4001b070 con 0x55d699f75ea0 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.091+0000 7fab3aa00640 1 -- 172.21.15.42:0/981151302 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fab4809b240 con 0x55d699f75ea0 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.091+0000 7fab5ef8b740 10 client.27092 fetch_fsmap learned FSMap version 45 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.091+0000 7fab5ef8b740 1 -- 172.21.15.42:0/981151302 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d699f6a7c0 con 0x55d699f75ea0 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.091+0000 7fab3aa00640 1 -- 172.21.15.42:0/981151302 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4001b070 con 0x55d699f75ea0 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.092+0000 7fab3aa00640 1 -- 172.21.15.42:0/981151302 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fab4809ab50 con 0x55d699f75ea0 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.092+0000 7fab5ef8b740 10 client.27092 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.092+0000 7fab5ef8b740 10 client.27092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.092+0000 7fab5ef8b740 20 client.27092 populate_metadata read hostname 'smithi042' 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.092+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d699f6be40 0x55d699f6e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.093 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.092+0000 7fab5ef8b740 4 client.27092 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:43:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.092+0000 7fab5ef8b740 1 -- 172.21.15.42:0/981151302 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d699edcff0 con 0x55d699f6be40 2024-04-25T00:43:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.093+0000 7fab59600640 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d699f6be40 0x55d699f6e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.093+0000 7fab59600640 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d699f6be40 0x55d699f6e2b0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d1a74b67360865a server_cookie=8e2b47a13e2e408 in_seq=0 out_seq=0 2024-04-25T00:43:50.094 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.093+0000 7fab3aa00640 10 client.27092 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.094+0000 7fab3aa00640 1 -- 172.21.15.42:0/981151302 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d699edcff0 con 0x55d699f6be40 2024-04-25T00:43:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.094+0000 7fab3aa00640 10 client.27092 handle_command_reply: tid=0 2024-04-25T00:43:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab5ef8b740 1 client.27092 shutdown 2024-04-25T00:43:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab5ef8b740 1 -- 172.21.15.42:0/981151302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 msgr2=0x55d699f7fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:50.095 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7fc70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fab48004630 tx=0x7fab4800a630 comp rx=0 tx=0).stop 2024-04-25T00:43:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab5ef8b740 1 -- 172.21.15.42:0/981151302 shutdown_connections 2024-04-25T00:43:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d699f6be40 0x55d699f6e2b0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f7b010 0x55d699f86cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f7fc70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab3aa00640 0 client.27092 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/981151302 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f754b0 0x55d699f7f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab5ef8b740 1 -- 172.21.15.42:0/981151302 >> 172.21.15.42:0/981151302 conn(0x55d699f70b20 msgr2=0x55d699f67480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:50.096 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.095+0000 7fab5ef8b740 1 -- 172.21.15.42:0/981151302 shutdown_connections 2024-04-25T00:43:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.096+0000 7fab5ef8b740 1 -- 172.21.15.42:0/981151302 wait complete. 2024-04-25T00:43:50.097 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.096+0000 7fab5ef8b740 20 client.27092 trim_cache size 0 max 16384 2024-04-25T00:43:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.125+0000 7fab5ef8b740 1 Processor -- start 2024-04-25T00:43:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.125+0000 7fab5ef8b740 1 -- start start 2024-04-25T00:43:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.125+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f754b0 0x55d699f74220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.125+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f75ea0 0x55d699f86c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.125+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f7b010 0x55d699f7e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.125+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fab541b1a70 con 0x55d699f7b010 2024-04-25T00:43:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.125+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab541af340 con 0x55d699f754b0 2024-04-25T00:43:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.126+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab541ad190 con 0x55d699f75ea0 2024-04-25T00:43:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.126+0000 7fab59600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f754b0 0x55d699f74220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.126+0000 7fab59600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f754b0 0x55d699f74220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:42518/0 (socket says 172.21.15.42:42518) 2024-04-25T00:43:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.126+0000 7fab59600640 1 -- 172.21.15.42:0/2439873038 learned_addr learned my addr 172.21.15.42:0/2439873038 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.126+0000 7fab58c00640 1 --2- 172.21.15.42:0/2439873038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f75ea0 0x55d699f86c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.126+0000 7fab5a000640 1 --2- 172.21.15.42:0/2439873038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f7b010 0x55d699f7e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.126+0000 7fab58c00640 1 -- 172.21.15.42:0/2439873038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f754b0 msgr2=0x55d699f74220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.126+0000 7fab58c00640 1 --2- 172.21.15.42:0/2439873038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f754b0 0x55d699f74220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.126+0000 7fab58c00640 1 -- 172.21.15.42:0/2439873038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f7b010 msgr2=0x55d699f7e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.126+0000 7fab58c00640 1 --2- 172.21.15.42:0/2439873038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f7b010 0x55d699f7e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.127+0000 7fab58c00640 1 -- 172.21.15.42:0/2439873038 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d699f7ffe0 con 0x55d699f75ea0 2024-04-25T00:43:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.127+0000 7fab58c00640 1 --2- 172.21.15.42:0/2439873038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f75ea0 0x55d699f86c70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fab4800c940 tx=0x7fab48000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d7e55b9cc64ae53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.128+0000 7fab45c00640 1 -- 172.21.15.42:0/2439873038 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab480cce60 con 0x55d699f75ea0 2024-04-25T00:43:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.128+0000 7fab45c00640 1 -- 172.21.15.42:0/2439873038 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4809fab0 con 0x55d699f75ea0 2024-04-25T00:43:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.128+0000 7fab45c00640 1 -- 172.21.15.42:0/2439873038 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4809b060 con 0x55d699f75ea0 2024-04-25T00:43:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.129+0000 7fab5ef8b740 1 -- 172.21.15.42:0/2439873038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f75ea0 msgr2=0x55d699f86c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.129+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/2439873038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f75ea0 0x55d699f86c70 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fab4800c940 tx=0x7fab48000bc0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.129 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.129+0000 7fab5ef8b740 1 -- 172.21.15.42:0/2439873038 shutdown_connections 2024-04-25T00:43:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.129+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/2439873038 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f7b010 0x55d699f7e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.129+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/2439873038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f75ea0 0x55d699f86c70 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.129+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/2439873038 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f754b0 0x55d699f74220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.129+0000 7fab5ef8b740 1 -- 172.21.15.42:0/2439873038 >> 172.21.15.42:0/2439873038 conn(0x55d699f70b20 msgr2=0x55d699f73b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.129+0000 7fab5ef8b740 1 -- 172.21.15.42:0/2439873038 shutdown_connections 2024-04-25T00:43:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.129+0000 7fab5ef8b740 1 -- 172.21.15.42:0/2439873038 wait complete. 2024-04-25T00:43:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.130+0000 7fab5ef8b740 1 -- start start 2024-04-25T00:43:50.130 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.130+0000 7fab5ef8b740 1 Processor -- start 2024-04-25T00:43:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f754b0 0x55d699f82b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f830a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab5ef8b740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f7b010 0x55d699f860c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fab541ad800 con 0x55d699f7b010 2024-04-25T00:43:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab541aec70 con 0x55d699f75ea0 2024-04-25T00:43:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab5ef8b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab541aded0 con 0x55d699f754b0 2024-04-25T00:43:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab5ef8b740 10 client.0 _ll_register_callbacks cb 0x7fab480d2640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:43:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab59600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f754b0 0x55d699f82b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab59600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f754b0 0x55d699f82b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:44702/0 (socket says 172.21.15.42:44702) 2024-04-25T00:43:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab59600640 1 -- 172.21.15.42:0/3085743579 learned_addr learned my addr 172.21.15.42:0/3085743579 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:43:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab59600640 1 -- 172.21.15.42:0/3085743579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 msgr2=0x55d699f830a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.131+0000 7fab58c00640 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f830a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.132+0000 7fab59600640 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f830a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.132+0000 7fab59600640 1 -- 172.21.15.42:0/3085743579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f7b010 msgr2=0x55d699f860c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:43:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.132+0000 7fab59600640 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f7b010 0x55d699f860c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.132+0000 7fab59600640 1 -- 172.21.15.42:0/3085743579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d699f63620 con 0x55d699f754b0 2024-04-25T00:43:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.132+0000 7fab58c00640 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:43:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.132+0000 7fab59600640 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f754b0 0x55d699f82b60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55d699f76910 tx=0x7fab4000a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1992a8fb7107f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:43:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.132+0000 7fab3aa00640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-04-25T00:43:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.133+0000 7fab3aa00640 1 -- 172.21.15.42:0/3085743579 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40009c50 con 0x55d699f754b0 2024-04-25T00:43:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.133+0000 7fab3aa00640 1 -- 172.21.15.42:0/3085743579 <== mon.1 v2:172.21.15.136:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fab40009df0 con 0x55d699f754b0 2024-04-25T00:43:50.134 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.133+0000 7fab5ef8b740 1 -- 172.21.15.42:0/3085743579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab4809fab0 con 0x55d699f754b0 2024-04-25T00:43:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.133+0000 7fab3aa00640 1 -- 172.21.15.42:0/3085743579 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fab40027b50 con 0x55d699f754b0 2024-04-25T00:43:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.133+0000 7fab5ef8b740 10 client.27049 fetch_fsmap learned FSMap version 45 2024-04-25T00:43:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.133+0000 7fab5ef8b740 1 -- 172.21.15.42:0/3085743579 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55d699ffa100 con 0x55d699f754b0 2024-04-25T00:43:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.134+0000 7fab3aa00640 1 -- 172.21.15.42:0/3085743579 <== mon.1 v2:172.21.15.136:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab4809fab0 con 0x55d699f754b0 2024-04-25T00:43:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.134+0000 7fab3aa00640 1 -- 172.21.15.42:0/3085743579 <== mon.1 v2:172.21.15.136:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7fab40027310 con 0x55d699f754b0 2024-04-25T00:43:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.135+0000 7fab5ef8b740 10 client.27049 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:43:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.135+0000 7fab5ef8b740 10 client.27049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:43:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.135+0000 7fab5ef8b740 20 client.27049 populate_metadata read hostname 'smithi042' 2024-04-25T00:43:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.135+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d699ffa850 0x55d699ffcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:43:50.135 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.135+0000 7fab5ef8b740 4 client.27049 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:43:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.135+0000 7fab5ef8b740 1 -- 172.21.15.42:0/3085743579 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d699ed7340 con 0x55d699ffa850 2024-04-25T00:43:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.135+0000 7fab58c00640 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d699ffa850 0x55d699ffcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:43:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.136+0000 7fab58c00640 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d699ffa850 0x55d699ffcc70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dedb6e1534c647cd server_cookie=13b5914f9bd80559 in_seq=0 out_seq=0 2024-04-25T00:43:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:50.136+0000 7fab3aa00640 10 client.27049 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:50.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:49 smithi136 ceph-mon[32923]: pgmap v1142: 97 pgs: 97 active+clean; 702 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.3 MiB/s wr, 452 op/s 2024-04-25T00:43:50.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:49 smithi106 ceph-mon[32510]: pgmap v1142: 97 pgs: 97 active+clean; 702 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.3 MiB/s wr, 452 op/s 2024-04-25T00:43:50.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:49 smithi042 ceph-mon[28130]: pgmap v1142: 97 pgs: 97 active+clean; 702 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 3.3 MiB/s wr, 452 op/s 2024-04-25T00:43:50.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 423002 32.28 MB/sec execute 580 sec latency 616.716 ms 2024-04-25T00:43:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.185+0000 7fab3aa00640 1 -- 172.21.15.42:0/3085743579 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d699ed7340 con 0x55d699ffa850 2024-04-25T00:43:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.185+0000 7fab3aa00640 10 client.27049 handle_command_reply: tid=0 2024-04-25T00:43:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.186+0000 7fab5ef8b740 1 client.27049 shutdown 2024-04-25T00:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.186+0000 7fab5ef8b740 1 -- 172.21.15.42:0/3085743579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f754b0 msgr2=0x55d699f82b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.186+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f754b0 0x55d699f82b60 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x55d699f76910 tx=0x7fab4000a430 comp rx=0 tx=0).stop 2024-04-25T00:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.186+0000 7fab5ef8b740 1 -- 172.21.15.42:0/3085743579 shutdown_connections 2024-04-25T00:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.186+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x55d699ffa850 0x55d699ffcc70 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.187+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x55d699f7b010 0x55d699f860c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.187+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55d699f75ea0 0x55d699f830a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.187+0000 7fab5ef8b740 1 --2- 172.21.15.42:0/3085743579 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d699f754b0 0x55d699f82b60 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.187+0000 7fab3aa00640 0 client.27049 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:43:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.187+0000 7fab5ef8b740 1 -- 172.21.15.42:0/3085743579 >> 172.21.15.42:0/3085743579 conn(0x55d699f70b20 msgr2=0x55d699f743a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.187+0000 7fab5ef8b740 1 -- 172.21.15.42:0/3085743579 shutdown_connections 2024-04-25T00:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.187+0000 7fab5ef8b740 1 -- 172.21.15.42:0/3085743579 wait complete. 2024-04-25T00:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.187+0000 7fab5ef8b740 20 client.27049 trim_cache size 0 max 16384 2024-04-25T00:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:43:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.189+0000 7fab5c000640 1 -- 172.21.15.42:0/3830164475 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab3c0a2d00 msgr2=0x7fab3c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.189+0000 7fab5c000640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab3c0a2d00 0x7fab3c0a51c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fab4c006010 tx=0x7fab4c005ec0 comp rx=0 tx=0).stop 2024-04-25T00:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.189+0000 7fab5c000640 1 -- 172.21.15.42:0/3830164475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5410b370 msgr2=0x7fab541a28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:43:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.189+0000 7fab5c000640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5410b370 0x7fab541a28d0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fab4800c970 tx=0x7fab4800cd70 comp rx=0 tx=0).stop 2024-04-25T00:43:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.191+0000 7fab5c000640 1 -- 172.21.15.42:0/3830164475 shutdown_connections 2024-04-25T00:43:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.191+0000 7fab5c000640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab3c0a2d00 0x7fab3c0a51c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.191+0000 7fab5c000640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab5410de50 0x7fab541a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.191+0000 7fab5c000640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab5410b370 0x7fab541a28d0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.191+0000 7fab5c000640 1 --2- 172.21.15.42:0/3830164475 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab541089b0 0x7fab541a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:43:51.191 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.191+0000 7fab5c000640 1 -- 172.21.15.42:0/3830164475 >> 172.21.15.42:0/3830164475 conn(0x7fab5406e970 msgr2=0x7fab54109270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:43:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.191+0000 7fab5c000640 1 -- 172.21.15.42:0/3830164475 shutdown_connections 2024-04-25T00:43:51.192 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:43:51.191+0000 7fab5c000640 1 -- 172.21.15.42:0/3830164475 wait complete. 2024-04-25T00:43:51.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 423458 32.26 MB/sec execute 581 sec latency 730.869 ms 2024-04-25T00:43:52.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:51 smithi136 ceph-mon[32923]: pgmap v1143: 97 pgs: 97 active+clean; 704 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 433 op/s 2024-04-25T00:43:52.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:51 smithi106 ceph-mon[32510]: pgmap v1143: 97 pgs: 97 active+clean; 704 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 433 op/s 2024-04-25T00:43:52.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:51 smithi042 ceph-mon[28130]: pgmap v1143: 97 pgs: 97 active+clean; 704 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 3.4 MiB/s wr, 433 op/s 2024-04-25T00:43:52.821 INFO:tasks.workunit.client.0.smithi042.stdout: 10 423985 32.25 MB/sec execute 582 sec latency 85.050 ms 2024-04-25T00:43:53.822 INFO:tasks.workunit.client.0.smithi042.stdout: 10 424591 32.25 MB/sec execute 583 sec latency 191.335 ms 2024-04-25T00:43:54.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:53 smithi136 ceph-mon[32923]: pgmap v1144: 97 pgs: 97 active+clean; 712 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 659 op/s 2024-04-25T00:43:54.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:53 smithi106 ceph-mon[32510]: pgmap v1144: 97 pgs: 97 active+clean; 712 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 659 op/s 2024-04-25T00:43:54.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:53 smithi042 ceph-mon[28130]: pgmap v1144: 97 pgs: 97 active+clean; 712 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 7.3 MiB/s wr, 659 op/s 2024-04-25T00:43:54.822 INFO:tasks.workunit.client.0.smithi042.stdout: 10 425154 32.26 MB/sec execute 584 sec latency 302.221 ms 2024-04-25T00:43:55.825 INFO:tasks.workunit.client.0.smithi042.stdout: 10 425611 32.24 MB/sec execute 585 sec latency 280.711 ms 2024-04-25T00:43:56.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:56 smithi136 ceph-mon[32923]: pgmap v1145: 97 pgs: 97 active+clean; 703 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 540 op/s 2024-04-25T00:43:56.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:56 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:43:56.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:56 smithi106 ceph-mon[32510]: pgmap v1145: 97 pgs: 97 active+clean; 703 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 540 op/s 2024-04-25T00:43:56.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:56 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:43:56.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:56 smithi042 ceph-mon[28130]: pgmap v1145: 97 pgs: 97 active+clean; 703 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 6.6 MiB/s wr, 540 op/s 2024-04-25T00:43:56.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:56 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:43:56.823 INFO:tasks.workunit.client.0.smithi042.stdout: 10 425635 32.19 MB/sec execute 586 sec latency 1039.486 ms 2024-04-25T00:43:57.823 INFO:tasks.workunit.client.0.smithi042.stdout: 10 425788 32.14 MB/sec execute 587 sec latency 1309.895 ms 2024-04-25T00:43:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:58 smithi136 ceph-mon[32923]: pgmap v1146: 97 pgs: 97 active+clean; 672 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.7 MiB/s wr, 654 op/s 2024-04-25T00:43:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:43:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:43:58.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:43:58 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:43:58.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:58 smithi106 ceph-mon[32510]: pgmap v1146: 97 pgs: 97 active+clean; 672 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.7 MiB/s wr, 654 op/s 2024-04-25T00:43:58.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:43:58.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:43:58.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:43:58 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:43:58.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:58 smithi042 ceph-mon[28130]: pgmap v1146: 97 pgs: 97 active+clean; 672 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 8.7 MiB/s wr, 654 op/s 2024-04-25T00:43:58.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:43:58.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:43:58.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:43:58 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:43:58.823 INFO:tasks.workunit.client.0.smithi042.stdout: 10 426082 32.12 MB/sec execute 588 sec latency 185.592 ms 2024-04-25T00:43:59.824 INFO:tasks.workunit.client.0.smithi042.stdout: 10 426513 32.11 MB/sec execute 589 sec latency 150.699 ms 2024-04-25T00:44:00.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:00 smithi136 ceph-mon[32923]: pgmap v1147: 97 pgs: 97 active+clean; 689 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 613 op/s 2024-04-25T00:44:00.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:00 smithi106 ceph-mon[32510]: pgmap v1147: 97 pgs: 97 active+clean; 689 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 613 op/s 2024-04-25T00:44:00.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:00 smithi042 ceph-mon[28130]: pgmap v1147: 97 pgs: 97 active+clean; 689 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 10 MiB/s wr, 613 op/s 2024-04-25T00:44:00.824 INFO:tasks.workunit.client.0.smithi042.stdout: 10 427119 32.11 MB/sec execute 590 sec latency 72.671 ms 2024-04-25T00:44:01.824 INFO:tasks.workunit.client.0.smithi042.stdout: 10 427702 32.10 MB/sec execute 591 sec latency 127.056 ms 2024-04-25T00:44:02.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:02 smithi106 ceph-mon[32510]: pgmap v1148: 97 pgs: 97 active+clean; 691 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 665 op/s 2024-04-25T00:44:02.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:02 smithi042 ceph-mon[28130]: pgmap v1148: 97 pgs: 97 active+clean; 691 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 665 op/s 2024-04-25T00:44:02.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:02 smithi136 ceph-mon[32923]: pgmap v1148: 97 pgs: 97 active+clean; 691 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 12 MiB/s wr, 665 op/s 2024-04-25T00:44:02.824 INFO:tasks.workunit.client.0.smithi042.stdout: 10 428234 32.09 MB/sec execute 592 sec latency 138.945 ms 2024-04-25T00:44:03.824 INFO:tasks.workunit.client.0.smithi042.stdout: 10 428815 32.10 MB/sec execute 593 sec latency 132.475 ms 2024-04-25T00:44:04.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:04 smithi106 ceph-mon[32510]: pgmap v1149: 97 pgs: 97 active+clean; 655 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 848 op/s 2024-04-25T00:44:04.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:04 smithi042 ceph-mon[28130]: pgmap v1149: 97 pgs: 97 active+clean; 655 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 848 op/s 2024-04-25T00:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.380+0000 7f8a66400640 1 -- 172.21.15.42:0/460432599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6010b0b0 msgr2=0x7f8a6010d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.380+0000 7f8a66400640 1 --2- 172.21.15.42:0/460432599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6010b0b0 0x7f8a6010d520 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c009860 tx=0x7f8a4c02a340 comp rx=0 tx=0).stop 2024-04-25T00:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.380+0000 7f8a66400640 1 -- 172.21.15.42:0/460432599 shutdown_connections 2024-04-25T00:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.380+0000 7f8a66400640 1 --2- 172.21.15.42:0/460432599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a6010da60 0x7f8a6010dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.380+0000 7f8a66400640 1 --2- 172.21.15.42:0/460432599 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6010b0b0 0x7f8a6010d520 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.380+0000 7f8a66400640 1 --2- 172.21.15.42:0/460432599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a60108740 0x7f8a6010ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.380+0000 7f8a66400640 1 -- 172.21.15.42:0/460432599 >> 172.21.15.42:0/460432599 conn(0x7f8a6006e7b0 msgr2=0x7f8a60070bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.380+0000 7f8a66400640 1 -- 172.21.15.42:0/460432599 shutdown_connections 2024-04-25T00:44:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.380+0000 7f8a66400640 1 -- 172.21.15.42:0/460432599 wait complete. 2024-04-25T00:44:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.381+0000 7f8a66400640 1 Processor -- start 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.382+0000 7f8a66400640 1 -- start start 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.382+0000 7f8a66400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a60108740 0x7f8a60114d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.382+0000 7f8a66400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a6010b0b0 0x7f8a601152d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.382+0000 7f8a66400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6010da60 0x7f8a601b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.382+0000 7f8a66400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6011c230 con 0x7f8a6010da60 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.382+0000 7f8a66400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6011c0b0 con 0x7f8a60108740 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.382+0000 7f8a66400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a6011c3b0 con 0x7f8a6010b0b0 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.382+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a60108740 0x7f8a60114d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.382+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a60108740 0x7f8a60114d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58832/0 (socket says 172.21.15.42:58832) 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.382+0000 7f8a5f400640 1 -- 172.21.15.42:0/1204493973 learned_addr learned my addr 172.21.15.42:0/1204493973 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:44:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.383+0000 7f8a5fe00640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6010da60 0x7f8a601b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.383+0000 7f8a5f400640 1 -- 172.21.15.42:0/1204493973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a6010b0b0 msgr2=0x7f8a601152d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.383+0000 7f8a5f400640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a6010b0b0 0x7f8a601152d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.383+0000 7f8a5f400640 1 -- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6010da60 msgr2=0x7f8a601b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.383+0000 7f8a5f400640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6010da60 0x7f8a601b6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.383+0000 7f8a5f400640 1 -- 172.21.15.42:0/1204493973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a60118400 con 0x7f8a60108740 2024-04-25T00:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.383+0000 7f8a5fe00640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6010da60 0x7f8a601b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T00:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.383+0000 7f8a5f400640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a60108740 0x7f8a60114d90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000e9f0 tx=0x7f8a5000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:04.384 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.384+0000 7f8a57e00640 1 -- 172.21.15.42:0/1204493973 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5000ccc0 con 0x7f8a60108740 2024-04-25T00:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.384+0000 7f8a57e00640 1 -- 172.21.15.42:0/1204493973 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a50005500 con 0x7f8a60108740 2024-04-25T00:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.384+0000 7f8a66400640 1 -- 172.21.15.42:0/1204493973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a601b6f00 con 0x7f8a60108740 2024-04-25T00:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.384+0000 7f8a57e00640 1 -- 172.21.15.42:0/1204493973 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50010690 con 0x7f8a60108740 2024-04-25T00:44:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.384+0000 7f8a66400640 1 -- 172.21.15.42:0/1204493973 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a601b7310 con 0x7f8a60108740 2024-04-25T00:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.390+0000 7f8a693be740 1 Processor -- start 2024-04-25T00:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.390+0000 7f8a693be740 1 -- start start 2024-04-25T00:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.390+0000 7f8a693be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.390+0000 7f8a693be740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 0x557ea69bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.390+0000 7f8a693be740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69bc800 0x557ea69bec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.390+0000 7f8a693be740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557ea6919ab0 con 0x557ea69bc800 2024-04-25T00:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.390+0000 7f8a693be740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ea68c2640 con 0x557ea69b7690 2024-04-25T00:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.390+0000 7f8a693be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557ea6915a70 con 0x557ea69b6ca0 2024-04-25T00:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.391+0000 7f8a57e00640 1 -- 172.21.15.42:0/1204493973 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8a500046c0 con 0x7f8a60108740 2024-04-25T00:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.391+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 0x557ea69bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.391+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 0x557ea69bc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58838/0 (socket says 172.21.15.42:58838) 2024-04-25T00:44:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.391+0000 7f8a5f400640 1 -- 172.21.15.42:0/2759906284 learned_addr learned my addr 172.21.15.42:0/2759906284 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:44:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.391+0000 7f8a5fe00640 1 --2- 172.21.15.42:0/2759906284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.391+0000 7f8a5ea00640 1 --2- 172.21.15.42:0/2759906284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69bc800 0x557ea69bec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.391+0000 7f8a5f400640 1 -- 172.21.15.42:0/2759906284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 msgr2=0x557ea69b70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.391+0000 7f8a5f400640 1 --2- 172.21.15.42:0/2759906284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69b70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.391+0000 7f8a5f400640 1 -- 172.21.15.42:0/2759906284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69bc800 msgr2=0x557ea69bec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.391+0000 7f8a5f400640 1 --2- 172.21.15.42:0/2759906284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69bc800 0x557ea69bec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.392+0000 7f8a5f400640 1 -- 172.21.15.42:0/2759906284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea662e210 con 0x557ea69b7690 2024-04-25T00:44:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.392+0000 7f8a5ea00640 1 --2- 172.21.15.42:0/2759906284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69bc800 0x557ea69bec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:44:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.392+0000 7f8a5f400640 1 --2- 172.21.15.42:0/2759906284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 0x557ea69bc190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8a500d73c0 tx=0x7f8a500d74b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3b12c017abe3208 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.392+0000 7f8a54c00640 1 -- 172.21.15.42:0/2759906284 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a500e2070 con 0x557ea69b7690 2024-04-25T00:44:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.392+0000 7f8a54c00640 1 -- 172.21.15.42:0/2759906284 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a500d1b70 con 0x557ea69b7690 2024-04-25T00:44:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.392+0000 7f8a54c00640 1 -- 172.21.15.42:0/2759906284 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a500dd8a0 con 0x557ea69b7690 2024-04-25T00:44:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.394+0000 7f8a57e00640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a440a2d30 0x7f8a440a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.394+0000 7f8a57e00640 1 -- 172.21.15.42:0/1204493973 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f8a500cf450 con 0x7f8a60108740 2024-04-25T00:44:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.394+0000 7f8a693be740 1 -- 172.21.15.42:0/2759906284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 msgr2=0x557ea69bc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.394+0000 7f8a693be740 1 --2- 172.21.15.42:0/2759906284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 0x557ea69bc190 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8a500d73c0 tx=0x7f8a500d74b0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.395+0000 7f8a5fe00640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a440a2d30 0x7f8a440a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.395+0000 7f8a693be740 1 -- 172.21.15.42:0/2759906284 shutdown_connections 2024-04-25T00:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.395+0000 7f8a693be740 1 --2- 172.21.15.42:0/2759906284 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69bc800 0x557ea69bec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.395+0000 7f8a693be740 1 --2- 172.21.15.42:0/2759906284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 0x557ea69bc190 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.395 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.395+0000 7f8a693be740 1 --2- 172.21.15.42:0/2759906284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69b70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.395+0000 7f8a693be740 1 -- 172.21.15.42:0/2759906284 >> 172.21.15.42:0/2759906284 conn(0x557ea69b2310 msgr2=0x557ea69b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:04.396 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.396+0000 7f8a693be740 1 -- 172.21.15.42:0/2759906284 shutdown_connections 2024-04-25T00:44:04.397 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.397+0000 7f8a693be740 1 -- 172.21.15.42:0/2759906284 wait complete. 2024-04-25T00:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.397+0000 7f8a5fe00640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a440a2d30 0x7f8a440a51f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x557ea69b7ef0 tx=0x7f8a58009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.397+0000 7f8a693be740 1 -- start start 2024-04-25T00:44:04.398 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.398+0000 7f8a693be740 1 Processor -- start 2024-04-25T00:44:04.399 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.398+0000 7f8a693be740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b6ca0 0x557ea69c1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.398+0000 7f8a693be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b7690 0x557ea69c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.398+0000 7f8a693be740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69bc800 0x557ea69c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.398+0000 7f8a693be740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x557ea69bf7b0 con 0x557ea69bc800 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.398+0000 7f8a693be740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x557ea68d5990 con 0x557ea69b6ca0 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.398+0000 7f8a693be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557ea69bf930 con 0x557ea69b7690 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.398+0000 7f8a693be740 10 client.0 _ll_register_callbacks cb 0x557ea68d7950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.399+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b6ca0 0x557ea69c1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.399+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b6ca0 0x557ea69c1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58848/0 (socket says 172.21.15.42:58848) 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.399+0000 7f8a5f400640 1 -- 172.21.15.42:0/1943632265 learned_addr learned my addr 172.21.15.42:0/1943632265 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.399+0000 7f8a5f400640 1 -- 172.21.15.42:0/1943632265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b7690 msgr2=0x557ea69c1540 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.399+0000 7f8a5f400640 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b7690 0x557ea69c1540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.399+0000 7f8a5f400640 1 -- 172.21.15.42:0/1943632265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69bc800 msgr2=0x557ea69c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:04.400 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.399+0000 7f8a5f400640 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69bc800 0x557ea69c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.399+0000 7f8a5f400640 1 -- 172.21.15.42:0/1943632265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea66fa620 con 0x557ea69b6ca0 2024-04-25T00:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.399+0000 7f8a5f400640 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b6ca0 0x557ea69c1000 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8a500980e0 tx=0x7f8a50009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd6150432a36df86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.399+0000 7f8a49600640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2024-04-25T00:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.400+0000 7f8a49600640 1 -- 172.21.15.42:0/1943632265 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a50097f20 con 0x557ea69b6ca0 2024-04-25T00:44:04.401 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.400+0000 7f8a49600640 1 -- 172.21.15.42:0/1943632265 <== mon.2 v2:172.21.15.106:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f8a50016dd0 con 0x557ea69b6ca0 2024-04-25T00:44:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.400+0000 7f8a693be740 1 -- 172.21.15.42:0/1943632265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a500d1b70 con 0x557ea69b6ca0 2024-04-25T00:44:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.400+0000 7f8a49600640 1 -- 172.21.15.42:0/1943632265 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8a500056b0 con 0x557ea69b6ca0 2024-04-25T00:44:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.400+0000 7f8a693be740 10 client.27107 fetch_fsmap learned FSMap version 45 2024-04-25T00:44:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.400+0000 7f8a693be740 1 -- 172.21.15.42:0/1943632265 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557ea69bf7b0 con 0x557ea69b6ca0 2024-04-25T00:44:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.401+0000 7f8a49600640 1 -- 172.21.15.42:0/1943632265 <== mon.2 v2:172.21.15.106:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a500d1b70 con 0x557ea69b6ca0 2024-04-25T00:44:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.401+0000 7f8a49600640 1 -- 172.21.15.42:0/1943632265 <== mon.2 v2:172.21.15.106:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f8a5009bee0 con 0x557ea69b6ca0 2024-04-25T00:44:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.401+0000 7f8a693be740 10 client.27107 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:44:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.402+0000 7f8a693be740 10 client.27107 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:44:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.402+0000 7f8a693be740 20 client.27107 populate_metadata read hostname 'smithi042' 2024-04-25T00:44:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.402+0000 7f8a693be740 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ea69cc650 0x557ea69cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.402+0000 7f8a693be740 4 client.27107 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T00:44:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.402+0000 7f8a693be740 1 -- 172.21.15.42:0/1943632265 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ea6926ff0 con 0x557ea69cc650 2024-04-25T00:44:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.402+0000 7f8a5ea00640 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ea69cc650 0x557ea69cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.403+0000 7f8a49600640 10 client.27107 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:44:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.405+0000 7f8a5ea00640 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ea69cc650 0x557ea69cea40 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482247e8e4b654f1 server_cookie=e19e61f843edf340 in_seq=0 out_seq=0 2024-04-25T00:44:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.405+0000 7f8a49600640 1 -- 172.21.15.42:0/1943632265 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ea6926ff0 con 0x557ea69cc650 2024-04-25T00:44:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.405+0000 7f8a49600640 10 client.27107 handle_command_reply: tid=0 2024-04-25T00:44:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.406+0000 7f8a693be740 1 client.27107 shutdown 2024-04-25T00:44:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.406+0000 7f8a693be740 1 -- 172.21.15.42:0/1943632265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b6ca0 msgr2=0x557ea69c1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.406+0000 7f8a693be740 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b6ca0 0x557ea69c1000 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8a500980e0 tx=0x7f8a50009f50 comp rx=0 tx=0).stop 2024-04-25T00:44:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.406+0000 7f8a693be740 1 -- 172.21.15.42:0/1943632265 shutdown_connections 2024-04-25T00:44:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.406+0000 7f8a693be740 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ea69cc650 0x557ea69cea40 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.406+0000 7f8a49600640 0 client.27107 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:44:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.406+0000 7f8a693be740 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69bc800 0x557ea69c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.406+0000 7f8a693be740 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b7690 0x557ea69c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.407+0000 7f8a693be740 1 --2- 172.21.15.42:0/1943632265 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b6ca0 0x557ea69c1000 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.407 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.407+0000 7f8a693be740 1 -- 172.21.15.42:0/1943632265 >> 172.21.15.42:0/1943632265 conn(0x557ea69b2310 msgr2=0x557ea69b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.407+0000 7f8a693be740 1 -- 172.21.15.42:0/1943632265 shutdown_connections 2024-04-25T00:44:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.407+0000 7f8a693be740 1 -- 172.21.15.42:0/1943632265 wait complete. 2024-04-25T00:44:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.408+0000 7f8a693be740 20 client.27107 trim_cache size 0 max 16384 2024-04-25T00:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.432+0000 7f8a693be740 1 Processor -- start 2024-04-25T00:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.433+0000 7f8a693be740 1 -- start start 2024-04-25T00:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.433+0000 7f8a693be740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b2310 0x557ea69b5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.433+0000 7f8a693be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69c8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.433+0000 7f8a693be740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b7690 0x557ea69d32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.433+0000 7f8a693be740 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601c29f0 con 0x557ea69b7690 2024-04-25T00:44:04.433 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.433+0000 7f8a693be740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601c02c0 con 0x557ea69b2310 2024-04-25T00:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.433+0000 7f8a693be740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601be110 con 0x557ea69b6ca0 2024-04-25T00:44:04.434 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.434+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.434+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b2310 0x557ea69b5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.435 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.434+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b2310 0x557ea69b5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:58852/0 (socket says 172.21.15.42:58852) 2024-04-25T00:44:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.434+0000 7f8a5f400640 1 -- 172.21.15.42:0/1148366770 learned_addr learned my addr 172.21.15.42:0/1148366770 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:44:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.434+0000 7f8a5fe00640 1 --2- 172.21.15.42:0/1148366770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b7690 0x557ea69d32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.435+0000 7f8a5f400640 1 -- 172.21.15.42:0/1148366770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 msgr2=0x557ea69c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.435+0000 7f8a5f400640 1 --2- 172.21.15.42:0/1148366770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.435+0000 7f8a5f400640 1 -- 172.21.15.42:0/1148366770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b7690 msgr2=0x557ea69d32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.435+0000 7f8a5f400640 1 --2- 172.21.15.42:0/1148366770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b7690 0x557ea69d32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.435+0000 7f8a5f400640 1 -- 172.21.15.42:0/1148366770 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea69b0890 con 0x557ea69b2310 2024-04-25T00:44:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.435+0000 7f8a5fe00640 1 --2- 172.21.15.42:0/1148366770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b7690 0x557ea69d32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:44:04.436 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.435+0000 7f8a5f400640 1 --2- 172.21.15.42:0/1148366770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b2310 0x557ea69b5880 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8a50098650 tx=0x7f8a500d74b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10c516a1b39095e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.436+0000 7f8a4be00640 1 -- 172.21.15.42:0/1148366770 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a500b9080 con 0x557ea69b2310 2024-04-25T00:44:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.436+0000 7f8a4be00640 1 -- 172.21.15.42:0/1148366770 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5009c380 con 0x557ea69b2310 2024-04-25T00:44:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.436+0000 7f8a4be00640 1 -- 172.21.15.42:0/1148366770 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5009b5e0 con 0x557ea69b2310 2024-04-25T00:44:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.437+0000 7f8a693be740 1 -- 172.21.15.42:0/1148366770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b2310 msgr2=0x557ea69b5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:04.437 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.437+0000 7f8a693be740 1 --2- 172.21.15.42:0/1148366770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b2310 0x557ea69b5880 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8a50098650 tx=0x7f8a500d74b0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.437+0000 7f8a693be740 1 -- 172.21.15.42:0/1148366770 shutdown_connections 2024-04-25T00:44:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.437+0000 7f8a693be740 1 --2- 172.21.15.42:0/1148366770 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b7690 0x557ea69d32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.437+0000 7f8a693be740 1 --2- 172.21.15.42:0/1148366770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.437+0000 7f8a693be740 1 --2- 172.21.15.42:0/1148366770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b2310 0x557ea69b5880 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.437+0000 7f8a693be740 1 -- 172.21.15.42:0/1148366770 >> 172.21.15.42:0/1148366770 conn(0x557ea69b8290 msgr2=0x557ea69c1280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.438+0000 7f8a693be740 1 -- 172.21.15.42:0/1148366770 shutdown_connections 2024-04-25T00:44:04.438 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.438+0000 7f8a693be740 1 -- 172.21.15.42:0/1148366770 wait complete. 2024-04-25T00:44:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.438+0000 7f8a693be740 1 -- start start 2024-04-25T00:44:04.439 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.439+0000 7f8a693be740 1 Processor -- start 2024-04-25T00:44:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.440+0000 7f8a693be740 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b2310 0x557ea69c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.440+0000 7f8a693be740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69c8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.440+0000 7f8a693be740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 0x557ea69c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.440 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.440+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.440+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b2310 0x557ea69c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.440+0000 7f8a5ea00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69c8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:58808/0 (socket says 172.21.15.42:58808) 2024-04-25T00:44:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.441+0000 7f8a5f400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b2310 0x557ea69c7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:50360/0 (socket says 172.21.15.42:50360) 2024-04-25T00:44:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.441+0000 7f8a5f400640 1 -- 172.21.15.42:0/3720964198 learned_addr learned my addr 172.21.15.42:0/3720964198 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:44:04.441 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.441+0000 7f8a5fe00640 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 0x557ea69c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.441+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601be780 con 0x557ea69b2310 2024-04-25T00:44:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.441+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601bfbf0 con 0x557ea69b7690 2024-04-25T00:44:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.441+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a601bee50 con 0x557ea69b6ca0 2024-04-25T00:44:04.442 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.442+0000 7f8a693be740 10 client.0 _ll_register_callbacks cb 0x7f8a500e0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T00:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.442+0000 7f8a5f400640 1 -- 172.21.15.42:0/3720964198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 msgr2=0x557ea69c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.442+0000 7f8a5f400640 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.443+0000 7f8a5f400640 1 -- 172.21.15.42:0/3720964198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 msgr2=0x557ea69c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T00:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.443+0000 7f8a5f400640 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 0x557ea69c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.443+0000 7f8a5f400640 1 -- 172.21.15.42:0/3720964198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea69c6ca0 con 0x557ea69b2310 2024-04-25T00:44:04.443 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.443+0000 7f8a5f400640 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b2310 0x557ea69c7f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a5009be40 tx=0x7f8a50004ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55102cfa5db28dc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.443+0000 7f8a49600640 10 client.0 ms_handle_connect on v2:172.21.15.42:3300/0 2024-04-25T00:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.444+0000 7f8a49600640 1 -- 172.21.15.42:0/3720964198 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a5009b7c0 con 0x557ea69b2310 2024-04-25T00:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.444+0000 7f8a49600640 1 -- 172.21.15.42:0/3720964198 <== mon.0 v2:172.21.15.42:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f8a500a4660 con 0x557ea69b2310 2024-04-25T00:44:04.444 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.444+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a5009c380 con 0x557ea69b2310 2024-04-25T00:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.445+0000 7f8a49600640 1 -- 172.21.15.42:0/3720964198 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f8a5009ba50 con 0x557ea69b2310 2024-04-25T00:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.445+0000 7f8a693be740 10 client.16776 fetch_fsmap learned FSMap version 45 2024-04-25T00:44:04.445 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.445+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x557ea69c6ca0 con 0x557ea69b2310 2024-04-25T00:44:04.446 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.446+0000 7f8a49600640 1 -- 172.21.15.42:0/3720964198 <== mon.0 v2:172.21.15.42:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a5009c380 con 0x557ea69b2310 2024-04-25T00:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.447+0000 7f8a49600640 1 -- 172.21.15.42:0/3720964198 <== mon.0 v2:172.21.15.42:3300/0 5 ==== fsmap(e 45) ==== 8338+0+0 (secure 0 0 0) 0x7f8a500a4b40 con 0x557ea69b2310 2024-04-25T00:44:04.447 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.447+0000 7f8a693be740 10 client.16776 fetch_fsmap finished waiting for FSMap version 45 2024-04-25T00:44:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.447+0000 7f8a693be740 10 client.16776 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T00:44:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.447+0000 7f8a693be740 20 client.16776 populate_metadata read hostname 'smithi042' 2024-04-25T00:44:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.447+0000 7f8a693be740 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ea69bc800 0x557ea6a35aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.447+0000 7f8a693be740 4 client.16776 mds_command: new command op to 24569 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T00:44:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.447+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 --> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ea69b2160 con 0x557ea69bc800 2024-04-25T00:44:04.448 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.448+0000 7f8a5ea00640 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ea69bc800 0x557ea6a35aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.449+0000 7f8a5ea00640 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ea69bc800 0x557ea6a35aa0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73ea16ef4b68eee0 server_cookie=efc86c74df7300ea in_seq=0 out_seq=0 2024-04-25T00:44:04.449 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.449+0000 7f8a49600640 10 client.16776 ms_handle_connect on v2:172.21.15.106:6838/3349564106 2024-04-25T00:44:04.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:04 smithi136 ceph-mon[32923]: pgmap v1149: 97 pgs: 97 active+clean; 655 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 16 MiB/s wr, 848 op/s 2024-04-25T00:44:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.794+0000 7f8a49600640 1 -- 172.21.15.42:0/3720964198 <== mds.0 v2:172.21.15.106:6838/3349564106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ea69b2160 con 0x557ea69bc800 2024-04-25T00:44:04.794 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.794+0000 7f8a49600640 10 client.16776 handle_command_reply: tid=0 2024-04-25T00:44:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.795+0000 7f8a693be740 1 client.16776 shutdown 2024-04-25T00:44:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.795+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b2310 msgr2=0x557ea69c7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:04.795 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.795+0000 7f8a693be740 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b2310 0x557ea69c7f00 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8a5009be40 tx=0x7f8a50004ba0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.795+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 shutdown_connections 2024-04-25T00:44:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.795+0000 7f8a693be740 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.106:6838/3349564106,v1:172.21.15.106:6839/3349564106] conn(0x557ea69bc800 0x557ea6a35aa0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.796 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.795+0000 7f8a693be740 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x557ea69b7690 0x557ea69c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.795+0000 7f8a693be740 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557ea69b6ca0 0x557ea69c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.795+0000 7f8a693be740 1 --2- 172.21.15.42:0/3720964198 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x557ea69b2310 0x557ea69c7f00 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.795+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 >> 172.21.15.42:0/3720964198 conn(0x557ea69b8290 msgr2=0x557ea69b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.796+0000 7f8a49600640 0 client.16776 ms_handle_reset on v2:172.21.15.106:6838/3349564106 2024-04-25T00:44:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.796+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 shutdown_connections 2024-04-25T00:44:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.796+0000 7f8a693be740 1 -- 172.21.15.42:0/3720964198 wait complete. 2024-04-25T00:44:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.796+0000 7f8a693be740 20 client.16776 trim_cache size 0 max 16384 2024-04-25T00:44:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:{ 2024-04-25T00:44:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "message": "", 2024-04-25T00:44:04.797 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout: "return_code": 0 2024-04-25T00:44:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stdout:} 2024-04-25T00:44:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.797+0000 7f8a66400640 1 -- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a440a2d30 msgr2=0x7f8a440a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.797+0000 7f8a66400640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a440a2d30 0x7f8a440a51f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x557ea69b7ef0 tx=0x7f8a58009500 comp rx=0 tx=0).stop 2024-04-25T00:44:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.797+0000 7f8a66400640 1 -- 172.21.15.42:0/1204493973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a60108740 msgr2=0x7f8a60114d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.797+0000 7f8a66400640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a60108740 0x7f8a60114d90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000e9f0 tx=0x7f8a5000eeb0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.798 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.797+0000 7f8a66400640 1 -- 172.21.15.42:0/1204493973 shutdown_connections 2024-04-25T00:44:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.797+0000 7f8a66400640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f8a440a2d30 0x7f8a440a51f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.797+0000 7f8a66400640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f8a6010da60 0x7f8a601b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.797+0000 7f8a66400640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a6010b0b0 0x7f8a601152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.797+0000 7f8a66400640 1 --2- 172.21.15.42:0/1204493973 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8a60108740 0x7f8a60114d90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.797+0000 7f8a66400640 1 -- 172.21.15.42:0/1204493973 >> 172.21.15.42:0/1204493973 conn(0x7f8a6006e7b0 msgr2=0x7f8a6010a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.798+0000 7f8a66400640 1 -- 172.21.15.42:0/1204493973 shutdown_connections 2024-04-25T00:44:04.799 INFO:teuthology.task.background_exec.ubuntu@smithi042.front.sepia.ceph.com.smithi042.stderr:2024-04-25T00:44:04.798+0000 7f8a66400640 1 -- 172.21.15.42:0/1204493973 wait complete. 2024-04-25T00:44:04.824 INFO:tasks.workunit.client.0.smithi042.stdout: 10 429181 32.08 MB/sec execute 594 sec latency 245.192 ms 2024-04-25T00:44:05.824 INFO:tasks.workunit.client.0.smithi042.stdout: 10 429657 32.05 MB/sec execute 595 sec latency 75.118 ms 2024-04-25T00:44:06.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:06 smithi106 ceph-mon[32510]: pgmap v1150: 97 pgs: 97 active+clean; 656 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 598 op/s 2024-04-25T00:44:06.379 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:06 smithi042 ceph-mon[28130]: pgmap v1150: 97 pgs: 97 active+clean; 656 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 598 op/s 2024-04-25T00:44:06.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:06 smithi136 ceph-mon[32923]: pgmap v1150: 97 pgs: 97 active+clean; 656 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 12 MiB/s wr, 598 op/s 2024-04-25T00:44:06.824 INFO:tasks.workunit.client.0.smithi042.stdout: 10 430204 32.06 MB/sec execute 596 sec latency 99.447 ms 2024-04-25T00:44:07.824 INFO:tasks.workunit.client.0.smithi042.stdout: 10 430788 32.07 MB/sec execute 597 sec latency 72.599 ms 2024-04-25T00:44:08.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:08 smithi106 ceph-mon[32510]: pgmap v1151: 97 pgs: 97 active+clean; 662 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 747 op/s 2024-04-25T00:44:08.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:08 smithi042 ceph-mon[28130]: pgmap v1151: 97 pgs: 97 active+clean; 662 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 747 op/s 2024-04-25T00:44:08.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:08 smithi136 ceph-mon[32923]: pgmap v1151: 97 pgs: 97 active+clean; 662 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 13 MiB/s wr, 747 op/s 2024-04-25T00:44:08.826 INFO:tasks.workunit.client.0.smithi042.stdout: 10 431381 32.06 MB/sec execute 598 sec latency 91.521 ms 2024-04-25T00:44:09.826 INFO:tasks.workunit.client.0.smithi042.stdout: 10 431955 32.06 MB/sec execute 599 sec latency 69.942 ms 2024-04-25T00:44:10.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:10 smithi106 ceph-mon[32510]: pgmap v1152: 97 pgs: 97 active+clean; 660 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 608 op/s 2024-04-25T00:44:10.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:10 smithi042 ceph-mon[28130]: pgmap v1152: 97 pgs: 97 active+clean; 660 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 608 op/s 2024-04-25T00:44:10.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:10 smithi136 ceph-mon[32923]: pgmap v1152: 97 pgs: 97 active+clean; 660 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 10 MiB/s wr, 608 op/s 2024-04-25T00:44:10.826 INFO:tasks.workunit.client.0.smithi042.stdout: 10 cleanup 600 sec 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: 0 cleanup 601 sec 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: Operation Count AvgLat MaxLat 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: ---------------------------------------- 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: NTCreateX 615797 4.333 3632.921 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: Close 452215 0.008 3.703 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: Rename 26096 18.754 1309.880 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: Unlink 124460 4.970 539.773 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: Deltree 4 123.056 216.639 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: Mkdir 2 0.022 0.023 2024-04-25T00:44:11.737 INFO:tasks.workunit.client.0.smithi042.stdout: Qpathinfo 558426 2.715 1537.405 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout: Qfileinfo 97486 0.003 1.818 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout: Qfsinfo 102428 0.464 126.670 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout: Sfileinfo 50178 0.181 281.394 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout: Find 215831 0.356 993.065 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout: WriteX 304976 0.106 3736.295 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout: ReadX 966179 0.045 796.896 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout: LockX 2008 2.743 51.615 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout: UnlockX 2008 2.403 33.392 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout: Flush 43175 10.842 1301.304 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout: 2024-04-25T00:44:11.738 INFO:tasks.workunit.client.0.smithi042.stdout:Throughput 32.0633 MB/sec 10 clients 10 procs max_latency=3736.307 ms 2024-04-25T00:44:11.833 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T00:44:11.833 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T00:44:12.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:12 smithi106 ceph-mon[32510]: pgmap v1153: 97 pgs: 97 active+clean; 660 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.0 MiB/s wr, 564 op/s 2024-04-25T00:44:12.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:12 smithi042 ceph-mon[28130]: pgmap v1153: 97 pgs: 97 active+clean; 660 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.0 MiB/s wr, 564 op/s 2024-04-25T00:44:12.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:12 smithi136 ceph-mon[32923]: pgmap v1153: 97 pgs: 97 active+clean; 660 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 8.0 MiB/s wr, 564 op/s 2024-04-25T00:44:13.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:13 smithi042 ceph-mon[28130]: pgmap v1154: 97 pgs: 97 active+clean; 668 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-04-25T00:44:13.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:13 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:13 smithi136 ceph-mon[32923]: pgmap v1154: 97 pgs: 97 active+clean; 668 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-04-25T00:44:13.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:13 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:13 smithi106 ceph-mon[32510]: pgmap v1154: 97 pgs: 97 active+clean; 668 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 7.0 MiB/s wr, 621 op/s 2024-04-25T00:44:13.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:13 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:13.826 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-04-25T00:44:13.827 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T00:44:15.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:15 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:15.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:15 smithi042 ceph-mon[28130]: pgmap v1155: 97 pgs: 97 active+clean; 668 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.7 MiB/s wr, 391 op/s 2024-04-25T00:44:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:15 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:15.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:15 smithi136 ceph-mon[32923]: pgmap v1155: 97 pgs: 97 active+clean; 668 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.7 MiB/s wr, 391 op/s 2024-04-25T00:44:15.549 DEBUG:teuthology.parallel:result is None 2024-04-25T00:44:15.549 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T00:44:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:15 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:15.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:15 smithi106 ceph-mon[32510]: pgmap v1155: 97 pgs: 97 active+clean; 668 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.7 MiB/s wr, 391 op/s 2024-04-25T00:44:15.641 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T00:44:15.641 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T00:44:15.653 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi042.front.sepia.ceph.com 2024-04-25T00:44:15.679 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T00:44:15.689 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:44:15.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.945+0000 7f951cc00640 1 -- 172.21.15.42:0/142276132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9518109620 msgr2=0x7f951810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:15.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.945+0000 7f951cc00640 1 --2- 172.21.15.42:0/142276132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9518109620 0x7f951810bac0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9510009860 tx=0x7f951002a3d0 comp rx=0 tx=0).stop 2024-04-25T00:44:15.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.946+0000 7f951cc00640 1 -- 172.21.15.42:0/142276132 shutdown_connections 2024-04-25T00:44:15.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.946+0000 7f951cc00640 1 --2- 172.21.15.42:0/142276132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9518109620 0x7f951810bac0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:15.947 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.946+0000 7f951cc00640 1 --2- 172.21.15.42:0/142276132 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9518074140 0x7f95180745a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:15.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.946+0000 7f951cc00640 1 --2- 172.21.15.42:0/142276132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9518075720 0x7f9518073b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:15.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.946+0000 7f951cc00640 1 -- 172.21.15.42:0/142276132 >> 172.21.15.42:0/142276132 conn(0x7f95180fcd20 msgr2=0x7f95180ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:15.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.946+0000 7f951cc00640 1 -- 172.21.15.42:0/142276132 shutdown_connections 2024-04-25T00:44:15.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.946+0000 7f951cc00640 1 -- 172.21.15.42:0/142276132 wait complete. 2024-04-25T00:44:15.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.947+0000 7f951cc00640 1 Processor -- start 2024-04-25T00:44:15.948 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.947+0000 7f951cc00640 1 -- start start 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.947+0000 7f951cc00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9518074140 0x7f9518199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.947+0000 7f951cc00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9518075720 0x7f951819a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.947+0000 7f951cc00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9518109620 0x7f95181a11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.947+0000 7f951cc00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f951810f660 con 0x7f9518075720 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.947+0000 7f951cc00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f951810f4e0 con 0x7f9518074140 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.947+0000 7f951cc00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f951810f7e0 con 0x7f9518109620 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9515400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9518075720 0x7f951819a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9515e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9518074140 0x7f9518199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9515400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9518075720 0x7f951819a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37888/0 (socket says 172.21.15.42:37888) 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9515400640 1 -- 172.21.15.42:0/3876031367 learned_addr learned my addr 172.21.15.42:0/3876031367 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:44:15.950 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9516800640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9518109620 0x7f95181a11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9515e00640 1 -- 172.21.15.42:0/3876031367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9518109620 msgr2=0x7f95181a11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9515e00640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9518109620 0x7f95181a11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9515e00640 1 -- 172.21.15.42:0/3876031367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9518075720 msgr2=0x7f951819a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9515e00640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9518075720 0x7f951819a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9515e00640 1 -- 172.21.15.42:0/3876031367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95181a1880 con 0x7f9518074140 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.948+0000 7f9515400640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9518075720 0x7f951819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.949+0000 7f9515e00640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9518074140 0x7f9518199c70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9508009fd0 tx=0x7f950800ec20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.949+0000 7f950ea00640 1 -- 172.21.15.42:0/3876031367 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f950800ccc0 con 0x7f9518074140 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.949+0000 7f950ea00640 1 -- 172.21.15.42:0/3876031367 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f950800ee70 con 0x7f9518074140 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.949+0000 7f951cc00640 1 -- 172.21.15.42:0/3876031367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95181a1b70 con 0x7f9518074140 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.949+0000 7f950ea00640 1 -- 172.21.15.42:0/3876031367 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9508004a00 con 0x7f9518074140 2024-04-25T00:44:15.951 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.949+0000 7f951cc00640 1 -- 172.21.15.42:0/3876031367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95181a2070 con 0x7f9518074140 2024-04-25T00:44:15.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.950+0000 7f951cc00640 1 -- 172.21.15.42:0/3876031367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95181083e0 con 0x7f9518074140 2024-04-25T00:44:15.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.952+0000 7f950ea00640 1 -- 172.21.15.42:0/3876031367 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9508004ba0 con 0x7f9518074140 2024-04-25T00:44:15.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.953+0000 7f950ea00640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94f80a2b90 0x7f94f80a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:15.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.954+0000 7f950ea00640 1 -- 172.21.15.42:0/3876031367 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f95080cefe0 con 0x7f9518074140 2024-04-25T00:44:15.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.954+0000 7f9515400640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94f80a2b90 0x7f94f80a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:15.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.955+0000 7f9515400640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94f80a2b90 0x7f94f80a5050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f950000f7d0 tx=0x7f9500009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:15.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:15.958+0000 7f950ea00640 1 -- 172.21.15.42:0/3876031367 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f950809bf50 con 0x7f9518074140 2024-04-25T00:44:16.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.174+0000 7f951cc00640 1 -- 172.21.15.42:0/3876031367 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f951802d170 con 0x7f9518074140 2024-04-25T00:44:16.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.176+0000 7f950ea00640 1 -- 172.21.15.42:0/3876031367 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22547 (secure 0 0 0) 0x7f950809b850 con 0x7f9518074140 2024-04-25T00:44:16.177 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:44:16.177 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":91,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:34:20.134521+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:44:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.179+0000 7f951cc00640 1 -- 172.21.15.42:0/3876031367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94f80a2b90 msgr2=0x7f94f80a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.179+0000 7f951cc00640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94f80a2b90 0x7f94f80a5050 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f950000f7d0 tx=0x7f9500009500 comp rx=0 tx=0).stop 2024-04-25T00:44:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.179+0000 7f951cc00640 1 -- 172.21.15.42:0/3876031367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9518074140 msgr2=0x7f9518199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.179+0000 7f951cc00640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9518074140 0x7f9518199c70 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f9508009fd0 tx=0x7f950800ec20 comp rx=0 tx=0).stop 2024-04-25T00:44:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.180+0000 7f951cc00640 1 -- 172.21.15.42:0/3876031367 shutdown_connections 2024-04-25T00:44:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.180+0000 7f951cc00640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f94f80a2b90 0x7f94f80a5050 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.180+0000 7f951cc00640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9518109620 0x7f95181a11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.180+0000 7f951cc00640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f9518075720 0x7f951819a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.180+0000 7f951cc00640 1 --2- 172.21.15.42:0/3876031367 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9518074140 0x7f9518199c70 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.180 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.180+0000 7f951cc00640 1 -- 172.21.15.42:0/3876031367 >> 172.21.15.42:0/3876031367 conn(0x7f95180fcd20 msgr2=0x7f95181055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:16.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.180+0000 7f951cc00640 1 -- 172.21.15.42:0/3876031367 shutdown_connections 2024-04-25T00:44:16.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.180+0000 7f951cc00640 1 -- 172.21.15.42:0/3876031367 wait complete. 2024-04-25T00:44:16.201 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:11:14.472869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:16:18.168243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:16:19.289585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:44:16.202 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:44:16.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.410+0000 7ff877200640 1 -- 172.21.15.42:0/1321005740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8781016d0 msgr2=0x7ff878101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:16.411 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.410+0000 7ff877200640 1 --2- 172.21.15.42:0/1321005740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8781016d0 0x7ff878101af0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff868009900 tx=0x7ff86802a410 comp rx=0 tx=0).stop 2024-04-25T00:44:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.411+0000 7ff877200640 1 -- 172.21.15.42:0/1321005740 shutdown_connections 2024-04-25T00:44:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.411+0000 7ff877200640 1 --2- 172.21.15.42:0/1321005740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff878103280 0x7ff878109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.411+0000 7ff877200640 1 --2- 172.21.15.42:0/1321005740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8781028c0 0x7ff878102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.411+0000 7ff877200640 1 --2- 172.21.15.42:0/1321005740 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8781016d0 0x7ff878101af0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.411+0000 7ff877200640 1 -- 172.21.15.42:0/1321005740 >> 172.21.15.42:0/1321005740 conn(0x7ff8780fcc40 msgr2=0x7ff8780ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.411+0000 7ff877200640 1 -- 172.21.15.42:0/1321005740 shutdown_connections 2024-04-25T00:44:16.412 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.411+0000 7ff877200640 1 -- 172.21.15.42:0/1321005740 wait complete. 2024-04-25T00:44:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.412+0000 7ff877200640 1 Processor -- start 2024-04-25T00:44:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.412+0000 7ff877200640 1 -- start start 2024-04-25T00:44:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.412+0000 7ff877200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8781016d0 0x7ff8780754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:16.413 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.412+0000 7ff877200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8781028c0 0x7ff878073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.412+0000 7ff877200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff878103280 0x7ff878074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.412+0000 7ff877200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff87810d2e0 con 0x7ff8781016d0 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.412+0000 7ff877200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff87810d160 con 0x7ff878103280 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.412+0000 7ff877200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff87810d460 con 0x7ff8781028c0 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.413+0000 7ff875e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8781016d0 0x7ff8780754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.413+0000 7ff875e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8781016d0 0x7ff8780754f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:37914/0 (socket says 172.21.15.42:37914) 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.413+0000 7ff875e00640 1 -- 172.21.15.42:0/2677218405 learned_addr learned my addr 172.21.15.42:0/2677218405 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.413+0000 7ff876800640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff878103280 0x7ff878074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.413+0000 7ff875400640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8781028c0 0x7ff878073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.413+0000 7ff876800640 1 -- 172.21.15.42:0/2677218405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8781028c0 msgr2=0x7ff878073b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.413+0000 7ff876800640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8781028c0 0x7ff878073b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.413+0000 7ff876800640 1 -- 172.21.15.42:0/2677218405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8781016d0 msgr2=0x7ff8780754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.413+0000 7ff876800640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8781016d0 0x7ff8780754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.414 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.413+0000 7ff876800640 1 -- 172.21.15.42:0/2677218405 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff878074900 con 0x7ff878103280 2024-04-25T00:44:16.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.414+0000 7ff876800640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff878103280 0x7ff878074070 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff870009fd0 tx=0x7ff87000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:16.415 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.414+0000 7ff86ea00640 1 -- 172.21.15.42:0/2677218405 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87000ce20 con 0x7ff878103280 2024-04-25T00:44:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.414+0000 7ff877200640 1 -- 172.21.15.42:0/2677218405 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8781a1ab0 con 0x7ff878103280 2024-04-25T00:44:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.414+0000 7ff86ea00640 1 -- 172.21.15.42:0/2677218405 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff870005020 con 0x7ff878103280 2024-04-25T00:44:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.414+0000 7ff86ea00640 1 -- 172.21.15.42:0/2677218405 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff870010720 con 0x7ff878103280 2024-04-25T00:44:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.414+0000 7ff877200640 1 -- 172.21.15.42:0/2677218405 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8781a2050 con 0x7ff878103280 2024-04-25T00:44:16.416 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.415+0000 7ff877200640 1 -- 172.21.15.42:0/2677218405 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8781089f0 con 0x7ff878103280 2024-04-25T00:44:16.417 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.417+0000 7ff86ea00640 1 -- 172.21.15.42:0/2677218405 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff870005540 con 0x7ff878103280 2024-04-25T00:44:16.418 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.419+0000 7ff86ea00640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff85c0a2b90 0x7ff85c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:16.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.419+0000 7ff86ea00640 1 -- 172.21.15.42:0/2677218405 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ff8700cf730 con 0x7ff878103280 2024-04-25T00:44:16.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.419+0000 7ff875e00640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff85c0a2b90 0x7ff85c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:16.419 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.420+0000 7ff875e00640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff85c0a2b90 0x7ff85c0a5050 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff868000c00 tx=0x7ff8680046c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:16.423 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.423+0000 7ff86ea00640 1 -- 172.21.15.42:0/2677218405 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff87009c660 con 0x7ff878103280 2024-04-25T00:44:16.629 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.629+0000 7ff877200640 1 -- 172.21.15.42:0/2677218405 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff878062340 con 0x7ff878103280 2024-04-25T00:44:16.630 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.630+0000 7ff86ea00640 1 -- 172.21.15.42:0/2677218405 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7ff87009bf60 con 0x7ff878103280 2024-04-25T00:44:16.630 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 -- 172.21.15.42:0/2677218405 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff85c0a2b90 msgr2=0x7ff85c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff85c0a2b90 0x7ff85c0a5050 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff868000c00 tx=0x7ff8680046c0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 -- 172.21.15.42:0/2677218405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff878103280 msgr2=0x7ff878074070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff878103280 0x7ff878074070 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff870009fd0 tx=0x7ff87000ef10 comp rx=0 tx=0).stop 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 -- 172.21.15.42:0/2677218405 shutdown_connections 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff85c0a2b90 0x7ff85c0a5050 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff878103280 0x7ff878074070 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff8781028c0 0x7ff878073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 --2- 172.21.15.42:0/2677218405 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff8781016d0 0x7ff8780754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 -- 172.21.15.42:0/2677218405 >> 172.21.15.42:0/2677218405 conn(0x7ff8780fcc40 msgr2=0x7ff878105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.633+0000 7ff877200640 1 -- 172.21.15.42:0/2677218405 shutdown_connections 2024-04-25T00:44:16.634 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.634+0000 7ff877200640 1 -- 172.21.15.42:0/2677218405 wait complete. 2024-04-25T00:44:16.654 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:44:16.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.951+0000 7ff614c00640 1 -- 172.21.15.42:0/2952438122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6080a4c20 msgr2=0x7ff6080a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:16.952 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.951+0000 7ff614c00640 1 --2- 172.21.15.42:0/2952438122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6080a4c20 0x7ff6080a5040 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff610067440 tx=0x7ff61008dcc0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.953 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.952+0000 7ff614c00640 1 -- 172.21.15.42:0/2952438122 shutdown_connections 2024-04-25T00:44:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.952+0000 7ff614c00640 1 --2- 172.21.15.42:0/2952438122 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6080a66f0 0x7ff6080aafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.952+0000 7ff614c00640 1 --2- 172.21.15.42:0/2952438122 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6080a5d50 0x7ff6080a61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.952+0000 7ff614c00640 1 --2- 172.21.15.42:0/2952438122 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6080a4c20 0x7ff6080a5040 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.954 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.952+0000 7ff614c00640 1 -- 172.21.15.42:0/2952438122 >> 172.21.15.42:0/2952438122 conn(0x7ff6080a00b0 msgr2=0x7ff6080a2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.952+0000 7ff614c00640 1 -- 172.21.15.42:0/2952438122 shutdown_connections 2024-04-25T00:44:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.954+0000 7ff614c00640 1 -- 172.21.15.42:0/2952438122 wait complete. 2024-04-25T00:44:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.955+0000 7ff614c00640 1 Processor -- start 2024-04-25T00:44:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.956+0000 7ff614c00640 1 -- start start 2024-04-25T00:44:16.957 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.957+0000 7ff614c00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6080a4c20 0x7ff6080b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.957+0000 7ff614c00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6080a5d50 0x7ff6080b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.957+0000 7ff614c00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6080a66f0 0x7ff608153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.957+0000 7ff60f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6080a4c20 0x7ff6080b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:16.958 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.957+0000 7ff60f400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6080a4c20 0x7ff6080b1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46114/0 (socket says 172.21.15.42:46114) 2024-04-25T00:44:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.960+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff6080b8ff0 con 0x7ff6080a5d50 2024-04-25T00:44:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.960+0000 7ff60f400640 1 -- 172.21.15.42:0/3032864382 learned_addr learned my addr 172.21.15.42:0/3032864382 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:44:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.960+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff6080b8e70 con 0x7ff6080a4c20 2024-04-25T00:44:16.961 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.961+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff6080b9170 con 0x7ff6080a66f0 2024-04-25T00:44:16.962 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.961+0000 7ff616200640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6080a66f0 0x7ff608153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:16.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.962+0000 7ff60ea00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6080a5d50 0x7ff6080b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:16.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.963+0000 7ff60f400640 1 -- 172.21.15.42:0/3032864382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6080a66f0 msgr2=0x7ff608153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:16.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.963+0000 7ff60f400640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6080a66f0 0x7ff608153700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.963 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.963+0000 7ff60f400640 1 -- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6080a5d50 msgr2=0x7ff6080b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:16.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.964+0000 7ff60f400640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6080a5d50 0x7ff6080b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:16.964 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.964+0000 7ff60f400640 1 -- 172.21.15.42:0/3032864382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff608153c40 con 0x7ff6080a4c20 2024-04-25T00:44:16.966 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.965+0000 7ff60ea00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6080a5d50 0x7ff6080b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T00:44:16.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.966+0000 7ff60f400640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6080a4c20 0x7ff6080b1800 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff610050770 tx=0x7ff610096900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:16.967 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.967+0000 7ff603e00640 1 -- 172.21.15.42:0/3032864382 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff610096c80 con 0x7ff6080a4c20 2024-04-25T00:44:16.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.967+0000 7ff603e00640 1 -- 172.21.15.42:0/3032864382 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff610096e20 con 0x7ff6080a4c20 2024-04-25T00:44:16.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.967+0000 7ff603e00640 1 -- 172.21.15.42:0/3032864382 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100978e0 con 0x7ff6080a4c20 2024-04-25T00:44:16.968 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.968+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff608153e10 con 0x7ff6080a4c20 2024-04-25T00:44:16.969 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.968+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6081541c0 con 0x7ff6080a4c20 2024-04-25T00:44:16.971 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.970+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff608005250 con 0x7ff6080a4c20 2024-04-25T00:44:16.981 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.980+0000 7ff603e00640 1 -- 172.21.15.42:0/3032864382 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff61008eb30 con 0x7ff6080a4c20 2024-04-25T00:44:16.986 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.986+0000 7ff603e00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5f80a2d10 0x7ff5f80a51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:16.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.986+0000 7ff603e00640 1 -- 172.21.15.42:0/3032864382 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ff610152290 con 0x7ff6080a4c20 2024-04-25T00:44:16.987 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.987+0000 7ff60ea00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5f80a2d10 0x7ff5f80a51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:16.989 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.989+0000 7ff60ea00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5f80a2d10 0x7ff5f80a51d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc006fd0 tx=0x7ff5fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:16.991 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:16.991+0000 7ff603e00640 1 -- 172.21.15.42:0/3032864382 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff6100a5060 con 0x7ff6080a4c20 2024-04-25T00:44:17.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.261+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff608002590 con 0x7ff6080a4c20 2024-04-25T00:44:17.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.262+0000 7ff603e00640 1 -- 172.21.15.42:0/3032864382 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7ff61011f320 con 0x7ff6080a4c20 2024-04-25T00:44:17.262 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-25T00:44:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5f80a2d10 msgr2=0x7ff5f80a51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5f80a2d10 0x7ff5f80a51d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc006fd0 tx=0x7ff5fc008040 comp rx=0 tx=0).stop 2024-04-25T00:44:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6080a4c20 msgr2=0x7ff6080b1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6080a4c20 0x7ff6080b1800 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff610050770 tx=0x7ff610096900 comp rx=0 tx=0).stop 2024-04-25T00:44:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 shutdown_connections 2024-04-25T00:44:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff5f80a2d10 0x7ff5f80a51d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff6080a66f0 0x7ff608153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff6080a5d50 0x7ff6080b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 --2- 172.21.15.42:0/3032864382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6080a4c20 0x7ff6080b1800 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 >> 172.21.15.42:0/3032864382 conn(0x7ff6080a00b0 msgr2=0x7ff6080a6f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:17.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 shutdown_connections 2024-04-25T00:44:17.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.265+0000 7ff614c00640 1 -- 172.21.15.42:0/3032864382 wait complete. 2024-04-25T00:44:17.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:44:17.376 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:17 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:17.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:17 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3876031367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:44:17.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:17 smithi042 ceph-mon[28130]: pgmap v1156: 97 pgs: 97 active+clean; 573 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.8 MiB/s wr, 410 op/s 2024-04-25T00:44:17.377 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:17 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2677218405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:44:17.484 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.484+0000 7f1b8f200640 1 -- 172.21.15.42:0/915225065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b88103880 msgr2=0x7f1b88103ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.484+0000 7f1b8f200640 1 --2- 172.21.15.42:0/915225065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b88103880 0x7f1b88103ca0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c009900 tx=0x7f1b7c02a3f0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.484+0000 7f1b8f200640 1 -- 172.21.15.42:0/915225065 shutdown_connections 2024-04-25T00:44:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.484+0000 7f1b8f200640 1 --2- 172.21.15.42:0/915225065 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b88105430 0x7f1b8810bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.484+0000 7f1b8f200640 1 --2- 172.21.15.42:0/915225065 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b88104a70 0x7f1b88104ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.484+0000 7f1b8f200640 1 --2- 172.21.15.42:0/915225065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b88103880 0x7f1b88103ca0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.484+0000 7f1b8f200640 1 -- 172.21.15.42:0/915225065 >> 172.21.15.42:0/915225065 conn(0x7f1b880fedf0 msgr2=0x7f1b88101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.484+0000 7f1b8f200640 1 -- 172.21.15.42:0/915225065 shutdown_connections 2024-04-25T00:44:17.485 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.484+0000 7f1b8f200640 1 -- 172.21.15.42:0/915225065 wait complete. 2024-04-25T00:44:17.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.485+0000 7f1b8f200640 1 Processor -- start 2024-04-25T00:44:17.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.485+0000 7f1b8f200640 1 -- start start 2024-04-25T00:44:17.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.485+0000 7f1b8f200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b88103880 0x7f1b88199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:17.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.485+0000 7f1b8f200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b88104a70 0x7f1b8819a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:17.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.485+0000 7f1b8f200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b88105430 0x7f1b881a1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:17.487 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.485+0000 7f1b8f200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8810f550 con 0x7f1b88104a70 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.485+0000 7f1b8f200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8810f3d0 con 0x7f1b88103880 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.485+0000 7f1b8f200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1b8810f6d0 con 0x7f1b88105430 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b88103880 0x7f1b88199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8de00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b88103880 0x7f1b88199b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:46136/0 (socket says 172.21.15.42:46136) 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8de00640 1 -- 172.21.15.42:0/1211700486 learned_addr learned my addr 172.21.15.42:0/1211700486 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8e800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b88105430 0x7f1b881a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8d400640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b88104a70 0x7f1b8819a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8de00640 1 -- 172.21.15.42:0/1211700486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b88105430 msgr2=0x7f1b881a1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8de00640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b88105430 0x7f1b881a1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8de00640 1 -- 172.21.15.42:0/1211700486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b88104a70 msgr2=0x7f1b8819a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8de00640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b88104a70 0x7f1b8819a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8de00640 1 -- 172.21.15.42:0/1211700486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b881a1790 con 0x7f1b88103880 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.486+0000 7f1b8de00640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b88103880 0x7f1b88199b70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c031440 tx=0x7f1b7c032e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.487+0000 7f1b82a00640 1 -- 172.21.15.42:0/1211700486 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c03f030 con 0x7f1b88103880 2024-04-25T00:44:17.488 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.487+0000 7f1b82a00640 1 -- 172.21.15.42:0/1211700486 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b7c003ce0 con 0x7f1b88103880 2024-04-25T00:44:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.487+0000 7f1b82a00640 1 -- 172.21.15.42:0/1211700486 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7c035070 con 0x7f1b88103880 2024-04-25T00:44:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.487+0000 7f1b8f200640 1 -- 172.21.15.42:0/1211700486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b881a1a20 con 0x7f1b88103880 2024-04-25T00:44:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.487+0000 7f1b8f200640 1 -- 172.21.15.42:0/1211700486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b881a1ec0 con 0x7f1b88103880 2024-04-25T00:44:17.489 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.488+0000 7f1b8f200640 1 -- 172.21.15.42:0/1211700486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b8810ab40 con 0x7f1b88103880 2024-04-25T00:44:17.490 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.490+0000 7f1b82a00640 1 -- 172.21.15.42:0/1211700486 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1b7c042030 con 0x7f1b88103880 2024-04-25T00:44:17.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.491+0000 7f1b82a00640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1b700a2c10 0x7f1b700a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:17.491 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.491+0000 7f1b82a00640 1 -- 172.21.15.42:0/1211700486 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f1b7c0ef5a0 con 0x7f1b88103880 2024-04-25T00:44:17.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.492+0000 7f1b8d400640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1b700a2c10 0x7f1b700a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:17.492 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.492+0000 7f1b8d400640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1b700a2c10 0x7f1b700a50d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1b8819b080 tx=0x7f1b7800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:17.495 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.495+0000 7f1b82a00640 1 -- 172.21.15.42:0/1211700486 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f1b7c0bbe80 con 0x7f1b88103880 2024-04-25T00:44:17.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:17 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:17.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:17 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3876031367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:44:17.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:17 smithi136 ceph-mon[32923]: pgmap v1156: 97 pgs: 97 active+clean; 573 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.8 MiB/s wr, 410 op/s 2024-04-25T00:44:17.513 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:17 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2677218405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:44:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:17 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:17 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3876031367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:44:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:17 smithi106 ceph-mon[32510]: pgmap v1156: 97 pgs: 97 active+clean; 573 MiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.8 MiB/s wr, 410 op/s 2024-04-25T00:44:17.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:17 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2677218405' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:44:17.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.698+0000 7f1b8f200640 1 -- 172.21.15.42:0/1211700486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1b88062340 con 0x7f1b88103880 2024-04-25T00:44:17.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.698+0000 7f1b82a00640 1 -- 172.21.15.42:0/1211700486 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f1b7c0c0180 con 0x7f1b88103880 2024-04-25T00:44:17.699 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-25T00:44:17.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 -- 172.21.15.42:0/1211700486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1b700a2c10 msgr2=0x7f1b700a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1b700a2c10 0x7f1b700a50d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1b8819b080 tx=0x7f1b7800b410 comp rx=0 tx=0).stop 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 -- 172.21.15.42:0/1211700486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b88103880 msgr2=0x7f1b88199b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b88103880 0x7f1b88199b70 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c031440 tx=0x7f1b7c032e20 comp rx=0 tx=0).stop 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 -- 172.21.15.42:0/1211700486 shutdown_connections 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f1b700a2c10 0x7f1b700a50d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b88105430 0x7f1b881a1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1b88104a70 0x7f1b8819a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 --2- 172.21.15.42:0/1211700486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b88103880 0x7f1b88199b70 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 -- 172.21.15.42:0/1211700486 >> 172.21.15.42:0/1211700486 conn(0x7f1b880fedf0 msgr2=0x7f1b88107d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 -- 172.21.15.42:0/1211700486 shutdown_connections 2024-04-25T00:44:17.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.701+0000 7f1b8f200640 1 -- 172.21.15.42:0/1211700486 wait complete. 2024-04-25T00:44:17.720 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T00:44:17.927 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.927+0000 7f82afe00640 1 -- 172.21.15.42:0/542897414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b00741a0 msgr2=0x7f82b0074620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.927+0000 7f82afe00640 1 --2- 172.21.15.42:0/542897414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b00741a0 0x7f82b0074620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f829c009930 tx=0x7f829c02a420 comp rx=0 tx=0).stop 2024-04-25T00:44:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.927+0000 7f82afe00640 1 -- 172.21.15.42:0/542897414 shutdown_connections 2024-04-25T00:44:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.927+0000 7f82afe00640 1 --2- 172.21.15.42:0/542897414 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b010db80 0x7f82b010ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.927+0000 7f82afe00640 1 --2- 172.21.15.42:0/542897414 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b00741a0 0x7f82b0074620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.927+0000 7f82afe00640 1 --2- 172.21.15.42:0/542897414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b0075780 0x7f82b0073bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.927+0000 7f82afe00640 1 -- 172.21.15.42:0/542897414 >> 172.21.15.42:0/542897414 conn(0x7f82b00fcb20 msgr2=0x7f82b00fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.927+0000 7f82afe00640 1 -- 172.21.15.42:0/542897414 shutdown_connections 2024-04-25T00:44:17.928 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.927+0000 7f82afe00640 1 -- 172.21.15.42:0/542897414 wait complete. 2024-04-25T00:44:17.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.928+0000 7f82afe00640 1 Processor -- start 2024-04-25T00:44:17.930 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.928+0000 7f82afe00640 1 -- start start 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b00741a0 0x7f82b019def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82afe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0075780 0x7f82b019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82afe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b010db80 0x7f82b01a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82afe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82b0113850 con 0x7f82b00741a0 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82afe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01136d0 con 0x7f82b010db80 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82afe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82b01139d0 con 0x7f82b0075780 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82aea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b00741a0 0x7f82b019def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82aea00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b00741a0 0x7f82b019def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:47016/0 (socket says 172.21.15.42:47016) 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82aea00640 1 -- 172.21.15.42:0/1339755880 learned_addr learned my addr 172.21.15.42:0/1339755880 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82ae000640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0075780 0x7f82b019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82af400640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b010db80 0x7f82b01a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82aea00640 1 -- 172.21.15.42:0/1339755880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0075780 msgr2=0x7f82b019e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82aea00640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0075780 0x7f82b019e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.931 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82aea00640 1 -- 172.21.15.42:0/1339755880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b010db80 msgr2=0x7f82b01a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:17.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82aea00640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b010db80 0x7f82b01a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:17.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.929+0000 7f82aea00640 1 -- 172.21.15.42:0/1339755880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82b01a5b10 con 0x7f82b00741a0 2024-04-25T00:44:17.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.930+0000 7f82aea00640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b00741a0 0x7f82b019def0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f82a000e9f0 tx=0x7f82a000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:17.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.930+0000 7f82a7400640 1 -- 172.21.15.42:0/1339755880 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a000cd80 con 0x7f82b00741a0 2024-04-25T00:44:17.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.930+0000 7f82a7400640 1 -- 172.21.15.42:0/1339755880 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82a0010910 con 0x7f82b00741a0 2024-04-25T00:44:17.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.930+0000 7f82afe00640 1 -- 172.21.15.42:0/1339755880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82b01a5e00 con 0x7f82b00741a0 2024-04-25T00:44:17.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.930+0000 7f82a7400640 1 -- 172.21.15.42:0/1339755880 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82a0018990 con 0x7f82b00741a0 2024-04-25T00:44:17.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.930+0000 7f82afe00640 1 -- 172.21.15.42:0/1339755880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82b01a6260 con 0x7f82b00741a0 2024-04-25T00:44:17.932 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.931+0000 7f82afe00640 1 -- 172.21.15.42:0/1339755880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82b0103d50 con 0x7f82b00741a0 2024-04-25T00:44:17.933 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.933+0000 7f82a7400640 1 -- 172.21.15.42:0/1339755880 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f82a0010ac0 con 0x7f82b00741a0 2024-04-25T00:44:17.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.935+0000 7f82a7400640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82940a2b90 0x7f82940a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:44:17.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.935+0000 7f82a7400640 1 -- 172.21.15.42:0/1339755880 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f82a00cf360 con 0x7f82b00741a0 2024-04-25T00:44:17.935 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.935+0000 7f82ae000640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82940a2b90 0x7f82940a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:44:17.936 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.936+0000 7f82ae000640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82940a2b90 0x7f82940a5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f829c009900 tx=0x7f829c0315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:44:17.941 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:17.941+0000 7f82a7400640 1 -- 172.21.15.42:0/1339755880 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f82a009bc40 con 0x7f82b00741a0 2024-04-25T00:44:18.125 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:18 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3032864382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:44:18.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:18 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1211700486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:44:18.224 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.224+0000 7f82afe00640 1 -- 172.21.15.42:0/1339755880 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f82b0062410 con 0x7f82b00741a0 2024-04-25T00:44:18.249 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.247+0000 7f82a7400640 1 -- 172.21.15.42:0/1339755880 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740367 (secure 0 0 0) 0x7f82a009ff40 con 0x7f82b00741a0 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout: "epoch": 18, 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout: "active_gid": 14152, 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout: "active_name": "x", 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addrs": { 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-25T00:44:18.261 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6800", 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 384891702 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v1", 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:6801", 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 384891702 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: "active_addr": "172.21.15.42:6801/384891702", 2024-04-25T00:44:18.262 INFO:teuthology.orchestra.run.smithi042.stdout: "active_change": "2024-04-25T00:08:25.764082+0000", 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: "available": true, 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: "standbys": [ 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 24107, 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "y", 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.263 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.264 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.265 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.265 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.265 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.265 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.267 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.268 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.269 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-04-25T00:44:18.270 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-04-25T00:44:18.271 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-04-25T00:44:18.272 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-04-25T00:44:18.273 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-04-25T00:44:18.274 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.275 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:44:18.277 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.278 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.279 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-04-25T00:44:18.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-04-25T00:44:18.281 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-04-25T00:44:18.282 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.283 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-04-25T00:44:18.284 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-04-25T00:44:18.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:18.286 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.288 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.289 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.290 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.291 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-04-25T00:44:18.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-25T00:44:18.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-04-25T00:44:18.294 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.295 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.296 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.298 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.299 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.300 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.301 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.302 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.303 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.304 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-04-25T00:44:18.305 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.306 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.307 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:44:18.308 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.309 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.310 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.313 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.314 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:44:18.315 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.316 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.317 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.318 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.319 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:44:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.320 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.322 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.325 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.326 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-04-25T00:44:18.326 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-04-25T00:44:18.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.326 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.326 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.326 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.326 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.327 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.327 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:44:18.327 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.327 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.327 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.327 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-04-25T00:44:18.327 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:44:18.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.327 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:44:18.328 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-04-25T00:44:18.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-04-25T00:44:18.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.331 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.332 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.333 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.334 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.336 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.337 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.338 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.339 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.340 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.341 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.343 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.344 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.345 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.346 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.347 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:44:18.348 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.349 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.350 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.351 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.353 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-04-25T00:44:18.355 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-04-25T00:44:18.356 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-04-25T00:44:18.357 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.358 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.359 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.360 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.361 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.362 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.363 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.364 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.365 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.366 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.367 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.368 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.369 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.370 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.371 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.372 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.373 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.374 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.375 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.376 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.377 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.378 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.379 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.380 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.381 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.382 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.383 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.384 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.385 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.386 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.387 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.388 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.389 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.390 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.391 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:44:18.392 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:44:18.393 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:44:18.394 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.395 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.396 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.397 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.398 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.399 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.401 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.402 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.403 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.406 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.407 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.409 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:44:18.410 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-04-25T00:44:18.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-04-25T00:44:18.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-04-25T00:44:18.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-04-25T00:44:18.414 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-25T00:44:18.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.417 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.418 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.419 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-04-25T00:44:18.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.422 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.423 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.424 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.425 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.426 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.426 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.426 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.426 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.426 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.426 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-04-25T00:44:18.426 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-04-25T00:44:18.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-25T00:44:18.427 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.428 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.430 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.431 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.432 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.434 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.435 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.436 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-04-25T00:44:18.437 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.438 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.438 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-04-25T00:44:18.438 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-04-25T00:44:18.439 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.440 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-04-25T00:44:18.441 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.442 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.445 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.448 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.449 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.450 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.451 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.452 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.453 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.454 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-04-25T00:44:18.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-04-25T00:44:18.456 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:18.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.458 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.459 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.460 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.461 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.462 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.464 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:44:18.466 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:44:18.467 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.468 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.469 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.470 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.470 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.470 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.470 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.470 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.470 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.470 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.470 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.470 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.472 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.473 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.474 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.475 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.476 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.477 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.478 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.480 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.481 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.482 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.483 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.484 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.485 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.486 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.487 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.487 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.487 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.487 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.487 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.487 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.487 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.487 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.488 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.489 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.490 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.491 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.492 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.493 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.494 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.495 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-04-25T00:44:18.496 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.497 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.498 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.498 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-04-25T00:44:18.498 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.498 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.498 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.498 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:44:18.498 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.498 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.498 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.499 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.502 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.503 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.504 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.507 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.509 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.510 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.511 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.512 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.513 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.516 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.517 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-25T00:44:18.518 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.519 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.520 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.521 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.522 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.523 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.524 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.525 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.527 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.527 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-04-25T00:44:18.527 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.527 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.527 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.527 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.527 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.527 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.528 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.529 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.530 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.531 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.532 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.533 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.534 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.536 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.537 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.539 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.541 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:18.543 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-04-25T00:44:18.544 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.545 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.546 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.547 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.548 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.549 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.550 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.551 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-04-25T00:44:18.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:44:18.554 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.556 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.557 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.558 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.559 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.560 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.561 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.562 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.563 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.564 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-25T00:44:18.565 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-04-25T00:44:18.566 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.567 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-25T00:44:18.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-04-25T00:44:18.569 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.570 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.571 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.572 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.573 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.574 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.575 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.576 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.577 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.578 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.579 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.580 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.580 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.580 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.580 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.580 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.580 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.580 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.581 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.582 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.583 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.584 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.585 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.586 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.587 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.588 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.589 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.590 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.591 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.592 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.593 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.594 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.595 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.596 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.597 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.598 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.599 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.600 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.601 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.602 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.603 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.604 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.605 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.606 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.607 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.608 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.609 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.610 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.611 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.612 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.613 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.614 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.615 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-04-25T00:44:18.616 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.617 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.618 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.619 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.620 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.620 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.620 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.620 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.620 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.620 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.620 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.620 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.620 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.620 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.621 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.622 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.623 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.624 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.625 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.626 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.627 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.628 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.629 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.630 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.631 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.632 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.633 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.634 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.635 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.636 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.637 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.638 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.639 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.640 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.641 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.642 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.643 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.643 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.643 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-04-25T00:44:18.643 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.643 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.643 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.643 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-04-25T00:44:18.643 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-04-25T00:44:18.644 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.645 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.646 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.647 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.648 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.649 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.650 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.651 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-04-25T00:44:18.652 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-04-25T00:44:18.653 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-04-25T00:44:18.654 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.655 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.656 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.657 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.658 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.659 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-04-25T00:44:18.660 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-04-25T00:44:18.661 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.662 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.663 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.664 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.665 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.666 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.667 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.668 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.669 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.670 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.671 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-04-25T00:44:18.672 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.673 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.674 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-04-25T00:44:18.674 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-04-25T00:44:18.674 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.674 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.674 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.674 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.674 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.674 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.674 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.674 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:18.675 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.676 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.677 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.678 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.679 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.680 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.681 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.682 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.683 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.684 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.685 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.686 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-04-25T00:44:18.687 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.688 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: "gid": 24109, 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "z", 2024-04-25T00:44:18.689 INFO:teuthology.orchestra.run.smithi042.stdout: "mgr_features": 4540701547738038271, 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.690 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.691 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.692 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.692 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.692 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.692 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.692 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.692 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.692 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.692 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.692 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.692 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.693 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.693 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.693 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.693 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.693 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.693 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.693 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.693 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.693 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.693 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.694 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.695 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.696 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.697 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-04-25T00:44:18.698 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-04-25T00:44:18.699 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.700 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-04-25T00:44:18.701 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.702 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.703 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.704 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-04-25T00:44:18.705 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.706 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.707 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.708 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:44:18.709 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:44:18.709 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.709 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.709 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.709 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-04-25T00:44:18.709 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:44:18.709 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:18.709 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.709 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.709 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.710 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.711 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-04-25T00:44:18.712 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.713 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.714 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.715 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.716 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.717 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.718 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.719 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.720 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-04-25T00:44:18.721 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.722 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.723 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.724 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:18.725 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.726 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:44:18.727 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.728 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.729 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.730 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.731 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.732 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.733 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.734 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.735 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.736 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.737 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.738 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:44:18.739 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.740 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:44:18.741 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.742 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:44:18.743 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.744 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.744 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.744 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:44:18.744 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.744 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.744 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.744 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.744 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-04-25T00:44:18.744 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:44:18.744 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.745 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.745 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.745 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:44:18.745 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.745 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.745 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.745 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-04-25T00:44:18.745 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.745 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.745 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.746 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.747 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.748 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.749 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.750 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.751 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:44:18.752 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.753 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.754 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.755 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.756 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.757 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.758 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.759 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.760 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-04-25T00:44:18.761 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-04-25T00:44:18.762 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.763 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:44:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.764 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.765 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-04-25T00:44:18.766 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.767 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.768 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.769 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.770 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.771 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:44:18.772 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.773 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.774 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.775 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.776 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.777 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.778 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.779 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-04-25T00:44:18.780 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-04-25T00:44:18.781 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:44:18.782 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-04-25T00:44:18.783 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.784 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.785 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.786 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.787 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-04-25T00:44:18.788 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:44:18.789 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.790 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-04-25T00:44:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.791 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.792 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.793 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.794 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.795 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.796 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.797 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.798 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.799 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.800 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.801 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.802 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.803 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.804 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.805 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.806 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.807 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.808 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.809 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.810 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.811 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.812 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.813 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.814 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.815 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.816 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.817 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.818 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:44:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:44:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.819 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.820 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.821 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.822 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.823 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.824 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.825 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.826 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.827 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.828 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.829 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.830 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:44:18.831 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.832 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.833 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.834 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.835 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-25T00:44:18.836 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.837 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.838 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.839 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.840 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:18.841 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-04-25T00:44:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-04-25T00:44:18.842 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-04-25T00:44:18.843 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.844 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.845 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.846 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.847 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.848 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.849 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.850 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.851 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-04-25T00:44:18.852 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.853 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.854 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.855 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.856 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.857 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-04-25T00:44:18.858 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.858 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.858 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.858 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.858 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.858 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.858 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.858 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.859 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-25T00:44:18.859 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-25T00:44:18.859 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.859 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.859 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.859 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-04-25T00:44:18.859 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.859 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.859 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.860 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.860 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-04-25T00:44:18.860 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.860 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.860 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.860 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.860 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.860 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.860 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-25T00:44:18.860 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-25T00:44:18.861 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.861 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.861 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.861 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-25T00:44:18.861 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-25T00:44:18.861 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-25T00:44:18.861 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.861 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.862 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.862 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.862 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.862 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.862 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-04-25T00:44:18.862 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-04-25T00:44:18.862 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.863 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.864 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.865 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.866 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.867 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.868 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-04-25T00:44:18.869 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-04-25T00:44:18.870 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.871 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.872 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.873 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.874 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.875 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.876 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.877 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.878 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.879 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.880 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-25T00:44:18.881 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.882 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:18.883 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.884 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.886 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-04-25T00:44:18.886 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-04-25T00:44:18.886 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.886 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.886 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.886 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-04-25T00:44:18.886 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.886 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.886 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.886 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.887 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-04-25T00:44:18.888 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.889 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.890 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.891 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.892 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.893 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:44:18.894 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.895 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:44:18.896 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.897 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.898 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.899 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.900 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.901 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.902 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.903 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.904 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.905 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.906 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.907 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.908 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.909 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.910 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-04-25T00:44:18.911 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:18.912 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.913 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.914 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.915 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.916 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.917 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.918 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.919 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.920 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.921 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.922 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.923 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.924 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.925 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.925 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.925 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.925 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.925 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.925 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.925 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.925 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.925 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.925 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.926 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.927 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-04-25T00:44:18.928 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.929 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.929 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.929 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-04-25T00:44:18.929 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.929 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.929 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.929 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:44:18.930 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.930 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.930 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.930 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.930 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-04-25T00:44:18.930 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-04-25T00:44:18.930 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.930 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.930 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.930 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.931 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.932 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.933 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.934 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.935 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.936 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.936 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.936 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.936 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.936 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.936 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.936 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.936 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.936 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.936 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.937 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.938 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.939 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.940 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.941 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.942 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.943 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.944 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.945 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.946 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.947 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.948 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.949 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.950 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.951 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.952 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.953 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.954 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.955 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.956 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.956 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.956 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.956 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.956 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.956 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.956 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.956 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.956 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.957 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.958 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.959 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.960 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.961 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.961 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.961 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.961 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.961 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.961 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.961 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.961 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.961 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.961 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.962 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.963 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.964 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.965 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.966 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.967 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.968 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.969 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.969 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.969 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.969 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.969 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.969 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.970 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.971 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.972 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.973 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.974 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.975 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.976 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.977 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.978 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.979 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.980 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.981 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.982 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.982 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.982 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.982 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.982 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.982 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.982 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.982 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.982 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.983 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.984 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.985 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.986 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.987 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.988 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:18.989 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.990 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:18.991 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.992 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.993 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:18.994 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:18.995 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-04-25T00:44:18.996 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:44:18.997 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.998 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-25T00:44:18.999 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-25T00:44:19.000 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.001 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.002 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.003 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.004 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.005 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.006 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.007 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.008 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.009 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.010 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.011 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.012 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.013 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.014 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.015 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.016 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.017 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.018 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.019 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.020 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.021 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.022 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.023 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.024 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.025 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.026 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.027 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.028 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.029 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.030 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.031 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.032 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.033 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.034 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.035 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.036 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.037 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.038 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.039 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.040 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.041 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.042 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.043 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.044 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.045 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.046 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-04-25T00:44:19.047 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-04-25T00:44:19.048 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.049 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.050 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.051 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.052 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.053 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.054 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.055 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.056 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.057 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.058 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.059 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.060 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.061 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.062 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.063 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.064 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.065 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.066 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.067 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.068 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.069 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.070 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.071 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.072 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.073 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-04-25T00:44:19.074 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-04-25T00:44:19.075 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.076 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.077 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.078 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.079 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.080 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:44:19.081 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.082 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-04-25T00:44:19.083 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.084 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.085 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.086 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.087 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.088 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.089 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.090 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.091 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.092 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.093 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.094 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.095 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.096 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.097 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.098 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.099 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-04-25T00:44:19.100 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.101 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.102 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.103 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.104 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.105 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-04-25T00:44:19.106 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.107 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.108 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.109 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.110 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.111 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.112 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.113 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.114 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.115 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.116 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.117 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.118 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.119 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.120 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.121 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: "modules": [ 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard", 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: "iostat", 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: "nfs", 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: "restful" 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: "available_modules": [ 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "alerts", 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.122 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.123 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.124 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.125 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.126 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.127 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_destination": { 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_destination", 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email address to send alerts to", 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.129 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_from_name": { 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_from_name", 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "Ceph", 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Email From: name", 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.130 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_host": { 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_host", 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP server", 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.131 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_password": { 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_password", 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Password to authenticate with", 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.132 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_port": { 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_port", 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "465", 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP port", 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.133 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_sender": { 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_sender", 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SMTP envelope sender", 2024-04-25T00:44:19.134 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_ssl": { 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_ssl", 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.135 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "smtp_user": { 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "smtp_user", 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "User to authenticate as", 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.136 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "balancer", 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "active": { 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "active", 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.137 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_time": { 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_time", 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.138 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0000", 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "begin_weekday": { 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "begin_weekday", 2024-04-25T00:44:19.139 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.140 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_max_iterations": { 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_max_iterations", 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "25", 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "250", 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.141 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_metrics": { 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_metrics", 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.142 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "crush_compat_step": { 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crush_compat_step", 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.5", 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0.001", 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "0.999", 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "aggressiveness of optimization", 2024-04-25T00:44:19.143 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "end_time": { 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_time", 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2359", 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.144 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "end_weekday": { 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "end_weekday", 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "0", 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "6", 2024-04-25T00:44:19.145 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.146 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.147 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.148 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.149 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.150 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "min_score": { 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_score", 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T00:44:19.151 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "upmap", 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "crush-compat", 2024-04-25T00:44:19.152 INFO:teuthology.orchestra.run.smithi042.stdout: "none", 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "read", 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap", 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap-read" 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Balancer mode", 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_ids": { 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_ids", 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.153 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.154 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_deviation": { 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_deviation", 2024-04-25T00:44:19.155 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "upmap_max_optimizations": { 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "upmap_max_optimizations", 2024-04-25T00:44:19.156 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.157 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm", 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_down_multiplier": { 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_down_multiplier", 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.158 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_refresh_rate": { 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_refresh_rate", 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "20", 2024-04-25T00:44:19.159 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "agent_starting_port": { 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "agent_starting_port", 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4721", 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.160 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_ptrace": { 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_ptrace", 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.161 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_interval": { 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_interval", 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.162 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "autotune_memory_target_ratio": { 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0.7", 2024-04-25T00:44:19.163 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm_log_destination": { 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cephadm_log_destination", 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.164 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "file", 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "file,syslog", 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "syslog" 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "cgroups_split": { 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cgroups_split", 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.165 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "config_checks_enabled": { 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_checks_enabled", 2024-04-25T00:44:19.166 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "config_dashboard": { 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_dashboard", 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.167 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.168 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_alertmanager": { 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_alertmanager", 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.170 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_base": { 2024-04-25T00:44:19.170 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_base", 2024-04-25T00:44:19.170 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.170 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.170 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.170 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T00:44:19.170 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.170 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.170 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.170 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Container image name, without the tag", 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_elasticsearch": { 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_elasticsearch", 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.171 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "elasticsearch container image", 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_grafana": { 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_grafana", 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T00:44:19.172 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_haproxy": { 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_haproxy", 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.173 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "HAproxy container image", 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_agent": { 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_agent", 2024-04-25T00:44:19.174 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger agent container image", 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_collector": { 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_collector", 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger collector container image", 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.176 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.177 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_jaeger_query": { 2024-04-25T00:44:19.177 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_jaeger_query", 2024-04-25T00:44:19.177 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.177 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.177 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.177 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T00:44:19.177 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.177 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.177 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.177 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Jaeger query container image", 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_keepalived": { 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_keepalived", 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.178 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Keepalived container image", 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_loki": { 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_loki", 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.179 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Loki container image", 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_node_exporter": { 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_node_exporter", 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T00:44:19.180 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_nvmeof": { 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_nvmeof", 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T00:44:19.181 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Nvme-of container image", 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_prometheus": { 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_prometheus", 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.182 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Prometheus container image", 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_promtail": { 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_promtail", 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.183 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Promtail container image", 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_samba": { 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_samba", 2024-04-25T00:44:19.184 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.185 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.185 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.185 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T00:44:19.185 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.185 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.185 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.185 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Samba/SMB container image", 2024-04-25T00:44:19.185 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.185 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "container_image_snmp_gateway": { 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_image_snmp_gateway", 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.186 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "SNMP Gateway container image", 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "container_init": { 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "container_init", 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.187 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: "daemon_cache_timeout": { 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "daemon_cache_timeout", 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.188 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "default_cephadm_command_timeout": { 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.189 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "900", 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "default_registry": { 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "default_registry", 2024-04-25T00:44:19.190 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "docker.io", 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.191 INFO:teuthology.orchestra.run.smithi042.stdout: "device_cache_timeout": { 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_cache_timeout", 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1800", 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache device inventory", 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.192 INFO:teuthology.orchestra.run.smithi042.stdout: "device_enhanced_scan": { 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_enhanced_scan", 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.193 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: "facts_cache_timeout": { 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "facts_cache_timeout", 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.194 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "seconds to cache host facts data", 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: "grafana_dashboards_path": { 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "grafana_dashboards_path", 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.195 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: "host_check_interval": { 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "host_check_interval", 2024-04-25T00:44:19.196 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to perform a host check", 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.197 INFO:teuthology.orchestra.run.smithi042.stdout: "hw_monitoring": { 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hw_monitoring", 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.198 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: "inventory_list_all": { 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "inventory_list_all", 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.199 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.200 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.201 INFO:teuthology.orchestra.run.smithi042.stdout: "log_refresh_metadata": { 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_refresh_metadata", 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.202 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.203 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.204 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.205 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.206 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "*", 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.207 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "max_count_per_host": { 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_count_per_host", 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of daemons per service per host", 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.208 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "max_osd_draining_count": { 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_osd_draining_count", 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "migration_current": { 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "migration_current", 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "internal - do not modify", 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "mode": { 2024-04-25T00:44:19.210 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mode", 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "root", 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm-package", 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "root" 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.211 INFO:teuthology.orchestra.run.smithi042.stdout: "oob_default_addr": { 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "oob_default_addr", 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "169.254.1.1", 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "prometheus_alerts_path": { 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus_alerts_path", 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.212 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_insecure": { 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_insecure", 2024-04-25T00:44:19.213 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_password": { 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_password", 2024-04-25T00:44:19.214 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_url": { 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_url", 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.215 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "registry_username": { 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "registry_username", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.216 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "secure_monitoring_stack": { 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "secure_monitoring_stack", 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.217 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "service_discovery_port": { 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "service_discovery_port", 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.218 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8765", 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "cephadm service discovery port", 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "ssh_config_file": { 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssh_config_file", 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.219 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "use_agent": { 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_agent", 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.220 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "use_repo_digest": { 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "use_repo_digest", 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.221 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_failed_host_check": { 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_failed_host_check", 2024-04-25T00:44:19.222 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_daemons": { 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_daemons", 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.223 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_on_stray_hosts": { 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_on_stray_hosts", 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.224 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.225 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.226 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crash", 2024-04-25T00:44:19.226 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.226 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.226 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.226 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.226 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.226 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.226 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.226 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.226 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.227 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.228 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.229 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.230 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.231 INFO:teuthology.orchestra.run.smithi042.stdout: "retain_interval": { 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retain_interval", 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "31536000", 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.232 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.233 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.233 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.233 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_recent_interval": { 2024-04-25T00:44:19.233 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_recent_interval", 2024-04-25T00:44:19.233 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.233 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.233 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.233 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1209600", 2024-04-25T00:44:19.233 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.234 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.234 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.234 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T00:44:19.234 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.234 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.234 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.234 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.235 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.235 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.235 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.235 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dashboard", 2024-04-25T00:44:19.235 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.235 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.235 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.235 INFO:teuthology.orchestra.run.smithi042.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T00:44:19.235 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T00:44:19.236 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.236 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.236 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.236 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:19.236 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.236 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.236 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.236 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.237 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.237 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.237 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.237 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.237 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T00:44:19.237 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T00:44:19.237 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.237 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.237 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T00:44:19.238 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_ENABLED": { 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T00:44:19.239 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.240 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.241 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.242 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.243 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T00:44:19.244 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.245 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.246 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.247 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.248 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.249 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T00:44:19.250 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.251 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_URL": { 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_URL", 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.252 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-25T00:44:19.253 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.254 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.255 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.256 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.257 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.258 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.259 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.260 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.261 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.262 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.263 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.264 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.265 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.266 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T00:44:19.267 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.268 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8", 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.269 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T00:44:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T00:44:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "45", 2024-04-25T00:44:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.270 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.271 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "admin", 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.272 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.273 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.274 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.275 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T00:44:19.276 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "cross_origin_url": { 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cross_origin_url", 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.277 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "crt_file": { 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "crt_file", 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.278 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "debug": { 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "debug", 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.279 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Enable/disable debug options", 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "jwt_token_ttl": { 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "jwt_token_ttl", 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.280 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "28800", 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.281 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.282 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.283 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.284 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.285 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.286 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.287 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "motd": { 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "motd", 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "The message of the day", 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.288 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect_resolve_ip_addr": { 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.289 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.290 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8080", 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.291 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.292 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl_server_port": { 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl_server_port", 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8443", 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.293 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "redirect", 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.294 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: "redirect" 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-25T00:44:19.295 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "url_prefix": { 2024-04-25T00:44:19.296 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url_prefix", 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.297 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_monitoring": { 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_monitoring", 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.298 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "monitor device health metrics", 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.299 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.300 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.301 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.302 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.303 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "mark_out_threshold": { 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mark_out_threshold", 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "2419200", 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.304 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "pool_name": { 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pool_name", 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "device_health_metrics", 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T00:44:19.305 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "retention_period": { 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "retention_period", 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15552000", 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long to retain device health metrics", 2024-04-25T00:44:19.306 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_frequency": { 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_frequency", 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.307 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "self_heal": { 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "self_heal", 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.308 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.309 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "warn_threshold": { 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "warn_threshold", 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7257600", 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.310 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "diskprediction_local", 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.311 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.312 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.313 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.314 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.315 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "predict_interval": { 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predict_interval", 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "86400", 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.316 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "predictor_model": { 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "predictor_model", 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "prophetstor", 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.317 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "600", 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.318 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "influx", 2024-04-25T00:44:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": false, 2024-04-25T00:44:19.319 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "influxdb python module not found", 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "batch_size": { 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "batch_size", 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5000", 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.320 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "database": { 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "database", 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "ceph", 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.321 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "hostname": { 2024-04-25T00:44:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "hostname", 2024-04-25T00:44:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.322 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server hostname", 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.323 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "30", 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "5", 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.324 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.325 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.326 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.327 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.328 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.329 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "password": { 2024-04-25T00:44:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "password", 2024-04-25T00:44:19.330 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "password of InfluxDB server user", 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "port": { 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "port", 2024-04-25T00:44:19.331 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "8086", 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "InfluxDB server port", 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "ssl": { 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ssl", 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.332 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "false", 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "threads": { 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threads", 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.333 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "32", 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "username": { 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "username", 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.334 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "username of InfluxDB server user", 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "verify_ssl": { 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "verify_ssl", 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.335 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "true", 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "insights", 2024-04-25T00:44:19.336 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.337 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.338 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.339 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.340 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "iostat", 2024-04-25T00:44:19.341 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.342 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.343 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.344 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.345 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.346 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "k8sevents", 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_event_retention_days": { 2024-04-25T00:44:19.347 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "ceph_event_retention_days", 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "7", 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.348 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "config_check_secs": { 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "config_check_secs", 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "10", 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.349 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.350 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.351 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.352 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.353 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.354 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "localpool", 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.355 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "failure_domain": { 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "failure_domain", 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "host", 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "failure domain for any created local pool", 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.356 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.357 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.358 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.359 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.360 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "min_size": { 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "min_size", 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.361 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default min_size for any created local pool", 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "num_rep": { 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "num_rep", 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.362 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3", 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default replica count for any created local pool", 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_num": { 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_num", 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.363 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "128", 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "prefix": { 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prefix", 2024-04-25T00:44:19.364 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "name prefix for any created local pool", 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.365 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "subtree": { 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "subtree", 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "rack", 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.366 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mds_autoscaler", 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.367 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.368 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.369 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.370 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.371 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirroring", 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.372 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.373 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.374 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.375 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.376 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "nfs", 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.377 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.378 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.379 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.380 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.381 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.382 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "fail_fs": { 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "fail_fs", 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.383 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.384 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.385 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.386 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.387 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.388 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator": { 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "orchestrator", 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.389 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: "cephadm", 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: "rook", 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: "test_orchestrator" 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Orchestrator backend", 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.390 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_perf_query", 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.391 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.392 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.393 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.394 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.395 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.396 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "osd_support", 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.397 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.398 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.399 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.400 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.401 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "pg_autoscaler", 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.402 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.403 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.404 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.405 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.406 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.407 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.408 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "threshold": { 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "threshold", 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "3.0", 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1.0", 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "scaling threshold", 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T00:44:19.409 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "progress", 2024-04-25T00:44:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.410 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_pg_recovery_event": { 2024-04-25T00:44:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_pg_recovery_event", 2024-04-25T00:44:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T00:44:19.411 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.412 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.413 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.414 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.415 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.416 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.417 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "max_completed_events": { 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_completed_events", 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "50", 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "number of past completed events to remember", 2024-04-25T00:44:19.418 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "sleep_interval": { 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "sleep_interval", 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "5", 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "how long the module is going to sleep", 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.419 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "prometheus", 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "cache": { 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "cache", 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.420 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "exclude_perf_counters": { 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "exclude_perf_counters", 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.421 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.422 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.423 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.424 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.425 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.426 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools": { 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools", 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.427 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "300", 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.428 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "scrape_interval": { 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "scrape_interval", 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15.0", 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.429 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "::", 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.430 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "9283", 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.431 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "stale_cache_strategy": { 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stale_cache_strategy", 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "log", 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.432 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_behaviour": { 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_behaviour", 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "default", 2024-04-25T00:44:19.433 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "default", 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "error" 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "standby_error_status_code": { 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "standby_error_status_code", 2024-04-25T00:44:19.434 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "500", 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "400", 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "599", 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.435 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.436 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.437 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.438 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.439 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.440 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.441 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.442 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.443 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_snap_create": { 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_snap_create", 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.444 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10", 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "mirror_snapshot_schedule": { 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.445 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "trash_purge_schedule": { 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "trash_purge_schedule", 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.446 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "restful", 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.447 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "enable_auth": { 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enable_auth", 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.448 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "key_file": { 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "key_file", 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.449 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.450 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.451 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.452 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.453 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "server_addr": { 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_addr", 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.454 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "server_port": { 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "server_port", 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.455 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rgw", 2024-04-25T00:44:19.456 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.457 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.458 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.459 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.460 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.461 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rook", 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.462 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.463 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.464 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.465 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.466 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "storage_class": { 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "storage_class", 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.467 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "local", 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.468 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "selftest", 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.469 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.470 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.471 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.472 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.473 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.474 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "roption1": { 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption1", 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.475 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "roption2": { 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "roption2", 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.476 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "xyz", 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption1": { 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption1", 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.477 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption2": { 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption2", 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.478 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption3": { 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption3", 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "float", 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.479 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption4": { 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption4", 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.480 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption5": { 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption5", 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.481 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption6": { 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption6", 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.482 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "rwoption7": { 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rwoption7", 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "1", 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "42", 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.483 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "testkey": { 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testkey", 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.484 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "testlkey": { 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testlkey", 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.485 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "testnewline": { 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "testnewline", 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.486 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snap_schedule", 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: "allow_m_granularity": { 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "allow_m_granularity", 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.487 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "dump_on_update": { 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "dump_on_update", 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.488 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "dump database to debug log on update", 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.489 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.490 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.491 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.492 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.493 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "stats", 2024-04-25T00:44:19.494 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.495 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.496 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.497 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.498 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.499 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "status", 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.500 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.501 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.502 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.503 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.504 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.504 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.504 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.504 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.504 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.504 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.504 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.504 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.505 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telegraf", 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.506 INFO:teuthology.orchestra.run.smithi042.stdout: "address": { 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "address", 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.507 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "15", 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.508 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.509 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.510 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.511 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.512 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.513 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.513 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.513 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.513 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.513 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.513 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.513 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.513 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.513 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.513 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "telemetry", 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.514 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_basic": { 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_basic", 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.515 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_crash": { 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_crash", 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.516 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_device": { 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_device", 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.517 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_ident": { 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_ident", 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.518 INFO:teuthology.orchestra.run.smithi042.stdout: "channel_perf": { 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "channel_perf", 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "contact": { 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "contact", 2024-04-25T00:44:19.519 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "description": { 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "description", 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.520 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "device_url": { 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "device_url", 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.521 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "enabled": { 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "enabled", 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.522 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.523 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "24", 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "8", 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "last_opt_revision": { 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "last_opt_revision", 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.524 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "1", 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard": { 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard", 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.525 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "leaderboard_description": { 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "leaderboard_description", 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.526 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.527 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.528 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.529 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.530 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.531 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "organization": { 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "organization", 2024-04-25T00:44:19.532 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "proxy": { 2024-04-25T00:44:19.533 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "proxy", 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "url": { 2024-04-25T00:44:19.534 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "url", 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.535 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "test_orchestrator", 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.536 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.537 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.538 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.539 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.540 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.541 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.542 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.543 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.544 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.545 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.546 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "max_concurrent_clones": { 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "max_concurrent_clones", 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "4", 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.547 INFO:teuthology.orchestra.run.smithi042.stdout: "periodic_async_work": { 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "periodic_async_work", 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Periodically check for async work", 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_delay": { 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_delay", 2024-04-25T00:44:19.548 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "0", 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "snapshot_clone_no_wait": { 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.549 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "True", 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix", 2024-04-25T00:44:19.550 INFO:teuthology.orchestra.run.smithi042.stdout: "can_run": true, 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "error_string": "", 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "module_options": { 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "discovery_interval": { 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "discovery_interval", 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "uint", 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "100", 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.551 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "identifier": { 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "identifier", 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.552 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "interval": { 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "interval", 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "secs", 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "60", 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.553 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "log_level": { 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_level", 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.554 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster": { 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster", 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.555 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_cluster_level": { 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_cluster_level", 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.556 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "info", 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [ 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "", 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "critical", 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "debug", 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "error", 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "info", 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "warning" 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.557 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "log_to_file": { 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "log_to_file", 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "bool", 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 1, 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "False", 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.558 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_host": { 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_host", 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "", 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.559 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_port": { 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_port", 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "int", 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "10051", 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.560 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "zabbix_sender": { 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "zabbix_sender", 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "str", 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "level": "advanced", 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "flags": 0, 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "min": "", 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "max": "", 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "enum_allowed": [], 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "desc": "", 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "long_desc": "", 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "tags": [], 2024-04-25T00:44:19.561 INFO:teuthology.orchestra.run.smithi042.stdout: "see_also": [] 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: "services": { 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: "dashboard": "https://172.21.15.42:8443/" 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: "always_on_modules": { 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: "octopus": [ 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-04-25T00:44:19.562 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "pacific": [ 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-04-25T00:44:19.563 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "quincy": [ 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-04-25T00:44:19.564 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "reef": [ 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-04-25T00:44:19.565 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: ], 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "squid": [ 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "balancer", 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "crash", 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "devicehealth", 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "orchestrator", 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "pg_autoscaler", 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "progress", 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "rbd_support", 2024-04-25T00:44:19.566 INFO:teuthology.orchestra.run.smithi042.stdout: "status", 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: "telemetry", 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: "volumes" 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: "last_failure_osd_epoch": 3, 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: "active_clients": [ 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "devicehealth", 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 1157984729 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.567 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "libcephsqlite", 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 3660948450 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "rbd_support", 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-25T00:44:19.568 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 639384425 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: "name": "volumes", 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: "addrvec": [ 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: { 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: "type": "v2", 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: "addr": "172.21.15.42:0", 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: "nonce": 4114878478 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-25T00:44:19.569 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:19.570 INFO:teuthology.orchestra.run.smithi042.stdout: ] 2024-04-25T00:44:19.570 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:44:19.571 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:18 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3032864382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:44:19.571 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:18 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1211700486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:44:19.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:18 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3032864382' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:44:19.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:18 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1211700486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:44:19.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:19 smithi136 ceph-mon[32923]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:19.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:19 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1339755880' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:44:19.572 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:19 smithi136 ceph-mon[32923]: pgmap v1157: 97 pgs: 97 active+clean; 537 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-04-25T00:44:19.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.258+0000 7f82afe00640 1 -- 172.21.15.42:0/1339755880 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82940a2b90 msgr2=0x7f82940a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:19.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.258+0000 7f82afe00640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82940a2b90 0x7f82940a5050 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f829c009900 tx=0x7f829c0315b0 comp rx=0 tx=0).stop 2024-04-25T00:44:19.572 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.258+0000 7f82afe00640 1 -- 172.21.15.42:0/1339755880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b00741a0 msgr2=0x7f82b019def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:44:19.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.258+0000 7f82afe00640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b00741a0 0x7f82b019def0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f82a000e9f0 tx=0x7f82a000eeb0 comp rx=0 tx=0).stop 2024-04-25T00:44:19.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.259+0000 7f82afe00640 1 -- 172.21.15.42:0/1339755880 shutdown_connections 2024-04-25T00:44:19.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.259+0000 7f82afe00640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82940a2b90 0x7f82940a5050 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:19.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.259+0000 7f82afe00640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82b010db80 0x7f82b01a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:19.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.259+0000 7f82afe00640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82b0075780 0x7f82b019e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:19.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.259+0000 7f82afe00640 1 --2- 172.21.15.42:0/1339755880 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82b00741a0 0x7f82b019def0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:44:19.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.259+0000 7f82afe00640 1 -- 172.21.15.42:0/1339755880 >> 172.21.15.42:0/1339755880 conn(0x7f82b00fcb20 msgr2=0x7f82b0100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:44:19.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.259+0000 7f82afe00640 1 -- 172.21.15.42:0/1339755880 shutdown_connections 2024-04-25T00:44:19.573 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:44:18.259+0000 7f82afe00640 1 -- 172.21.15.42:0/1339755880 wait complete. 2024-04-25T00:44:19.579 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T00:44:19.579 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T00:44:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:19 smithi106 ceph-mon[32510]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:19 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1339755880' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:44:19.588 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:19 smithi106 ceph-mon[32510]: pgmap v1157: 97 pgs: 97 active+clean; 537 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-04-25T00:44:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:19 smithi042 ceph-mon[28130]: fsmap cephfs:3 {0=b=up:active,1=a=up:active,2=c=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T00:44:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:19 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1339755880' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T00:44:19.626 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:19 smithi042 ceph-mon[28130]: pgmap v1157: 97 pgs: 97 active+clean; 537 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 2.0 MiB/s wr, 257 op/s 2024-04-25T00:44:20.110 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:44:21.955 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:21 smithi042 ceph-mon[28130]: pgmap v1158: 97 pgs: 97 active+clean; 532 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-04-25T00:44:22.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:21 smithi136 ceph-mon[32923]: pgmap v1158: 97 pgs: 97 active+clean; 532 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-04-25T00:44:22.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:21 smithi106 ceph-mon[32510]: pgmap v1158: 97 pgs: 97 active+clean; 532 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.2 MiB/s wr, 233 op/s 2024-04-25T00:44:23.167 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:44:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 804641, 2024-04-25T00:44:23.167 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 1017938, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 230983047, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 785542064, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 13, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 5, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 139.917572023, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 100.138592659, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 52.620507562, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 2.054896349, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 1017937, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 109.903946161, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000107967 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.168 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 1159111, 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.445372516, 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000384 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 689136, 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 284592 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 43173, 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 43512, 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 10916368, 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 155606410, 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 9, 2024-04-25T00:44:23.169 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 6, 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 7.361687360, 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 2.462836377, 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 1.333607298, 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 3.007160682, 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 43511, 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 7.349937211, 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000168921 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.170 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 3184, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 690512 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 446556, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 373812, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 118172389, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 345357330, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 8, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 6, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 61.770225690, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 26.827194242, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 37.991569306, 2024-04-25T00:44:23.171 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 4.736314395, 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 373810, 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 37.094970157, 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000099234 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 712412, 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.226333716, 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000317 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:44:23.172 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 1, 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "finisher-MDSRank": { 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "queue_len": 0, 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "complete_latency": { 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 108706, 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 251.229042351, 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.002311087 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.173 INFO:teuthology.orchestra.run.smithi042.stdout: "finisher-PurgeQueue": { 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "queue_len": 0, 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "complete_latency": { 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 75165, 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 24.536256780, 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000326431 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "mds": { 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "request": 374734, 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "reply": 307615, 2024-04-25T00:44:23.174 INFO:teuthology.orchestra.run.smithi042.stdout: "reply_latency": { 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 307615, 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 1732.834098408, 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.005633126 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "slow_reply": 0, 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "forward": 67118, 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_fetch_complete": 0, 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_fetch_keys": 0, 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_commit": 151, 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_split": 14, 2024-04-25T00:44:23.175 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_merge": 19, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes": 2085, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_top": 1168, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_bottom": 516, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_pin_tail": 401, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_pinned": 416, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_expired": 363, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "inodes_with_caps": 288, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "caps": 288, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "subtrees": 4, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse": 609825, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_hit": 293855, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_forward": 67117, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_discover": 202, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:44:23.176 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_remote_ino": 0, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "traverse_lock": 5872, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "load_cent": 21521, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "q": 0, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "exported": 5, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "exported_inodes": 101, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "imported": 5, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "imported_inodes": 494, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_dir_fetch": 0, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "openino_peer_discover": 0, 2024-04-25T00:44:23.177 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rfiles": 4, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rbytes": 234, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "root_rsnaps": 0, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_set_tag": 0, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "scrub_file_inodes": 0, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_inode_file_caps": 26, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_revoke": 25322, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_grant": 183181, 2024-04-25T00:44:23.178 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_caps": 106787, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_caps_dirty": 42624, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_cap_release": 20312, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "process_request_cap_release": 221325 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_cache": { 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_update": 314, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_update_receipt": 1732, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_try_discover": 937, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_send_discover": 1694, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "dir_handle_discover": 400, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays": 391, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays_delayed": 300, 2024-04-25T00:44:23.179 INFO:teuthology.orchestra.run.smithi042.stdout: "num_strays_enqueuing": 89, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_created": 48954, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_enqueued": 48563, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_reintegrated": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "strays_migrated": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_processing": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_started": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "recovery_completed": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_exportdir": 5, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_flush": 0, 2024-04-25T00:44:23.180 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_fragmentdir": 33, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_fragstats": 0, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "ireq_inodestats": 0 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_log": { 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "evlrg": 0, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "evadd": 246872, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "evex": 140756, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "evtrm": 135191, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "ev": 111681, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "evexg": 0, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "evexd": 5565, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "segadd": 297, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "segex": 169, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "segtrm": 162, 2024-04-25T00:44:23.181 INFO:teuthology.orchestra.run.smithi042.stdout: "seg": 135, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "segmjr": 10, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "segexg": 0, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "segexd": 7, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "expos": 261431427, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "wrpos": 490898473, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "rdpos": 0, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "jlat": { 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 62549, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 592.930752780, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.009479460 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "replayed": 0 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.182 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_mem": { 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "ino": 421, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "ino+": 49152, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "ino-": 48731, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "dir": 47, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "dir+": 1114, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "dir-": 1067, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "dn": 2138, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "dn+": 96631, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "dn-": 94493, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "cap": 1, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "cap+": 55463, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "cap-": 55462, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "rss": 83576, 2024-04-25T00:44:23.183 INFO:teuthology.orchestra.run.smithi042.stdout: "heap": 223612 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_server": { 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "dispatch_client_request": 525954, 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "dispatch_server_request": 15, 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_request": 374734, 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_client_session": 6879, 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "handle_peer_request": 11654, 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "req_create_latency": { 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 48646, 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 356.635353582, 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.007331236 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getattr_latency": { 2024-04-25T00:44:23.184 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 17886, 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 260.664474838, 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.014573659 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getfilelock_latency": { 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "req_link_latency": { 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookup_latency": { 2024-04-25T00:44:23.185 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 102572, 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 284.932016994, 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.002777873 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookuphash_latency": { 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupino_latency": { 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 1, 2024-04-25T00:44:23.186 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000188044, 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000188044 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupname_latency": { 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupparent_latency": { 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.187 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lookupsnap_latency": { 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "req_lssnap_latency": { 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mkdir_latency": { 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.188 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.189 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mknod_latency": { 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "req_mksnap_latency": { 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "req_open_latency": { 2024-04-25T00:44:23.190 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "req_readdir_latency": { 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 78739, 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 509.356012538, 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.006468916 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rename_latency": { 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 9379, 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 71.272726415, 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.007599181 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "req_renamesnap_latency": { 2024-04-25T00:44:23.191 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "req_snapdiff_latency": { 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmdir_latency": { 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 30, 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.445347894, 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.014844929 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmsnap_latency": { 2024-04-25T00:44:23.192 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "req_rmxattr_latency": { 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setattr_latency": { 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 46, 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 2.084226982, 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.045309282 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.193 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setdirlayout_latency": { 2024-04-25T00:44:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.194 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setfilelock_latency": { 2024-04-25T00:44:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 1392, 2024-04-25T00:44:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 3.029006908, 2024-04-25T00:44:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.002176010 2024-04-25T00:44:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setlayout_latency": { 2024-04-25T00:44:23.195 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "req_setxattr_latency": { 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "req_symlink_latency": { 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.196 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "req_unlink_latency": { 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 48924, 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 244.414744213, 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.004995804 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "req_getvxattr_latency": { 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:23.197 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_sessions": { 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "session_count": 1, 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "session_add": 1, 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "session_remove": 0, 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "sessions_open": 1, 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "sessions_stale": 0, 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "total_load": 31331, 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "average_load": 31331, 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "avg_session_uptime": 1533, 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mdthresh_evicted": 0 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-04-25T00:44:23.198 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:23.199 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:23.200 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 951572, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 693, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 528, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 6, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:23.201 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 7432, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 156, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 2955466, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 84083, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:23.202 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "objecter": { 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "op_active": 1, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "op_laggy": 0, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send": 116667, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "op_send_bytes": 497951027, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "op_resend": 33, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "op_reply": 116666, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "op_latency": { 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 116666, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 1180.129291670, 2024-04-25T00:44:23.203 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.010115451 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "op_inflight": 0, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "oplen_avg": { 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 116667, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 118920 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "op": 116667, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "op_r": 1489, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "op_w": 115178, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "op_rmw": 0, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "op_pg": 0, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_stat": 9, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_create": 21, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_read": 1489, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_write": 64040, 2024-04-25T00:44:23.204 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writefull": 790, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_writesame": 0, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_append": 0, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_zero": 0, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_truncate": 14, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_delete": 48871, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_mapext": 0, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_sparse_read": 0, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_clonerange": 0, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_getxattr": 0, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_setxattr": 42, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:44:23.205 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_rmxattr": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_call": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_watch": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_notify": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "osdop_other": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_active": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_send": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_resend": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "linger_ping": 0, 2024-04-25T00:44:23.206 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_active": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_send": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "poolop_resend": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_active": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_send": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "poolstat_resend": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_active": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_send": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "statfs_resend": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "command_active": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "command_send": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "command_resend": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "map_epoch": 91, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "map_full": 0, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "map_inc": 5, 2024-04-25T00:44:23.207 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_sessions": 12, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_open": 12, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_session_close": 0, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_laggy": 0, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_wr": 2793, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_rd": 0, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_del": 851 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "oft": { 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_objs": 1, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_updates": 0, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "omap_total_removes": 0 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "osd-slow-ops": { 2024-04-25T00:44:23.208 INFO:teuthology.orchestra.run.smithi042.stdout: "slow_ops_count": 0 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "purge_queue": { 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_ops": 0, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_ops_high_water": 10, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing": 0, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executing_high_water": 10, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executed_ops": 48563, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_executed": 48563, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "pq_item_in_journal": 0 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-25T00:44:23.209 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 1294366, 2024-04-25T00:44:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 251955904, 2024-04-25T00:44:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 1294366, 2024-04-25T00:44:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-04-25T00:44:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-04-25T00:44:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 1294366, 2024-04-25T00:44:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 251955904, 2024-04-25T00:44:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-25T00:44:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.210 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_bytes": { 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 101, 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:23.211 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 116667, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 502855890, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 116666, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 502855789, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-objecter_ops": { 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 1, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 1024, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 0, 2024-04-25T00:44:23.212 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 0, 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 116667, 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 116667, 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 116666, 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 116666, 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:44:23.213 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 9191, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 3758096384, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 48655, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 4914155, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 48655, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 1491, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 4904964, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.214 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-write_buf_throttle-0x561160b46500": { 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 164905, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 3758096384, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 246872, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 486704169, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 246872, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 62549, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 486539264, 2024-04-25T00:44:23.215 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-25T00:44:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:23.216 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:23.216 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:44:23.878 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:23 smithi042 ceph-mon[28130]: pgmap v1159: 97 pgs: 97 active+clean; 508 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-04-25T00:44:24.014 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:23 smithi136 ceph-mon[32923]: pgmap v1159: 97 pgs: 97 active+clean; 508 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-04-25T00:44:24.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:23 smithi106 ceph-mon[32510]: pgmap v1159: 97 pgs: 97 active+clean; 508 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.8 MiB/s wr, 266 op/s 2024-04-25T00:44:24.240 INFO:tasks.check_counter:Daemon mds.a mds.exported=5 2024-04-25T00:44:24.240 INFO:tasks.check_counter:Daemon mds.a mds.imported=5 2024-04-25T00:44:24.240 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=314 2024-04-25T00:44:24.240 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=1732 2024-04-25T00:44:24.240 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T00:44:24.240 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T00:44:24.693 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:44:25.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:25 smithi042 ceph-mon[28130]: pgmap v1160: 97 pgs: 97 active+clean; 504 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.3 MiB/s wr, 153 op/s 2024-04-25T00:44:26.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:25 smithi136 ceph-mon[32923]: pgmap v1160: 97 pgs: 97 active+clean; 504 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.3 MiB/s wr, 153 op/s 2024-04-25T00:44:26.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:25 smithi106 ceph-mon[32510]: pgmap v1160: 97 pgs: 97 active+clean; 504 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.3 MiB/s wr, 153 op/s 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 0, 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 2, 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 0, 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.006005131, 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:44:27.125 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000876382, 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.126 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 6, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 350, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 3344, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 217280, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.060321147, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.049639197, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.004198913, 2024-04-25T00:44:27.127 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 349, 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.041537381, 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000119018 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 217280 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:27.128 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 458, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 455, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 546784, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 281872, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.141840787, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.064878898, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.050314050, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.021628391, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 454, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.052022915, 2024-04-25T00:44:27.129 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000114587 2024-04-25T00:44:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 546784, 2024-04-25T00:44:27.130 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 281872 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:27.131 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:27.132 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:27.133 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148704, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 29, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-04-25T00:44:27.134 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:27.135 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 460, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 482011, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 460, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 460, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 482011, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:27.136 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:27.137 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:27.137 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:44:27.877 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:27 smithi042 ceph-mon[28130]: pgmap v1161: 97 pgs: 97 active+clean; 500 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 6.6 MiB/s wr, 273 op/s 2024-04-25T00:44:27.998 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T00:44:27.998 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T00:44:27.998 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T00:44:27.998 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T00:44:27.998 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T00:44:27.999 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T00:44:28.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:27 smithi136 ceph-mon[32923]: pgmap v1161: 97 pgs: 97 active+clean; 500 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 6.6 MiB/s wr, 273 op/s 2024-04-25T00:44:28.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:27 smithi106 ceph-mon[32510]: pgmap v1161: 97 pgs: 97 active+clean; 500 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 702 KiB/s rd, 6.6 MiB/s wr, 273 op/s 2024-04-25T00:44:28.495 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:44:29.821 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:29 smithi042 ceph-mon[28130]: pgmap v1162: 97 pgs: 97 active+clean; 477 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 7.1 MiB/s wr, 258 op/s 2024-04-25T00:44:30.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:29 smithi136 ceph-mon[32923]: pgmap v1162: 97 pgs: 97 active+clean; 477 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 7.1 MiB/s wr, 258 op/s 2024-04-25T00:44:30.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:29 smithi106 ceph-mon[32510]: pgmap v1162: 97 pgs: 97 active+clean; 477 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 7.1 MiB/s wr, 258 op/s 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 0, 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 2, 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 0, 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.004436555, 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:44:30.875 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.001170667, 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:44:30.876 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 2, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 345, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 216304, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.059614350, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.048966243, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.004356452, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:30.877 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 343, 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.055032996, 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000160446 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 216304 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:30.878 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 458, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 453, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 548720, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 279760, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.137801865, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.065410951, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.046888907, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.019277096, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 453, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.044823104, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000098947 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.879 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 548720, 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 279760 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:30.880 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:30.881 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:30.882 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:30.883 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:30.883 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-04-25T00:44:30.883 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:30.883 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:30.883 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:30.883 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:30.883 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 148704, 2024-04-25T00:44:30.883 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 29, 2024-04-25T00:44:30.883 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:44:30.883 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-04-25T00:44:30.884 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 456, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 481491, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 456, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-04-25T00:44:30.885 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-04-25T00:44:30.886 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 456, 2024-04-25T00:44:30.886 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 481491, 2024-04-25T00:44:30.886 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-25T00:44:30.886 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:30.886 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:30.886 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:30.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:30.886 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:30.886 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:44:31.786 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T00:44:31.786 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T00:44:31.786 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-25T00:44:31.786 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-25T00:44:31.786 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T00:44:31.786 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T00:44:32.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:31 smithi136 ceph-mon[32923]: pgmap v1163: 97 pgs: 97 active+clean; 470 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 6.7 MiB/s wr, 249 op/s 2024-04-25T00:44:32.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:31 smithi106 ceph-mon[32510]: pgmap v1163: 97 pgs: 97 active+clean; 470 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 6.7 MiB/s wr, 249 op/s 2024-04-25T00:44:32.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:31 smithi042 ceph-mon[28130]: pgmap v1163: 97 pgs: 97 active+clean; 470 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 6.7 MiB/s wr, 249 op/s 2024-04-25T00:44:32.250 INFO:teuthology.orchestra.run.smithi042.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T00:44:33.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:33 smithi042 ceph-mon[28130]: pgmap v1164: 97 pgs: 97 active+clean; 462 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 6.4 MiB/s wr, 251 op/s 2024-04-25T00:44:34.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:33 smithi136 ceph-mon[32923]: pgmap v1164: 97 pgs: 97 active+clean; 462 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 6.4 MiB/s wr, 251 op/s 2024-04-25T00:44:34.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:33 smithi106 ceph-mon[32510]: pgmap v1164: 97 pgs: 97 active+clean; 462 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 6.4 MiB/s wr, 251 op/s 2024-04-25T00:44:34.569 INFO:teuthology.orchestra.run.smithi042.stdout:{ 2024-04-25T00:44:34.574 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 0, 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 2, 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 0, 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 0, 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.003845390, 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.000738264, 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:34.575 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:44:34.576 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 461, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 458, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 547600, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 283792, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 2, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 1, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.150537346, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.071797931, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.052944992, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.020201141, 2024-04-25T00:44:34.577 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 457, 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.061386363, 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000134324 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 547600, 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 283792 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.578 INFO:teuthology.orchestra.run.smithi042.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_messages": 6, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages": 352, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_bytes": 3344, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_bytes": 217632, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_active_connections": 2, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_total_time": 0.057284654, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_send_time": 0.046225056, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_recv_time": 0.004506729, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 351, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.045097585, 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000128483 2024-04-25T00:44:34.579 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "msgr_send_encrypted_bytes": 217632 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "cct": { 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "total_workers": 0, 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "unhealthy_workers": 0 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "mempool": { 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:34.580 INFO:teuthology.orchestra.run.smithi042.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:34.581 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:34.582 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_items": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_bytes": 149735, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_anon_items": 30, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "buffer_meta_items": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_bytes": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_items": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:34.583 INFO:teuthology.orchestra.run.smithi042.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_bytes": 0, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_items": 0, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "pgmap_items": 0, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_bytes": 928, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "mds_co_items": 2, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_1_items": 0, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "unittest_2_items": 0 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: }, 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:34.584 INFO:teuthology.orchestra.run.smithi042.stdout: "val": 0, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "max": 104857600, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "get_started": 0, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "get": 463, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "get_sum": 482401, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "get_or_fail_success": 463, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "take": 0, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "take_sum": 0, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "put": 463, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "put_sum": 482401, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "wait": { 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "avgcount": 0, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "sum": 0.000000000, 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: "avgtime": 0.000000000 2024-04-25T00:44:34.585 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:34.586 INFO:teuthology.orchestra.run.smithi042.stdout: } 2024-04-25T00:44:34.586 INFO:teuthology.orchestra.run.smithi042.stdout:} 2024-04-25T00:44:35.477 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T00:44:35.477 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T00:44:35.478 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T00:44:35.478 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T00:44:35.478 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T00:44:35.478 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T00:44:35.800 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:44:35.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:35 smithi106 ceph-mon[32510]: pgmap v1165: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-04-25T00:44:36.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:35 smithi136 ceph-mon[32923]: pgmap v1165: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-04-25T00:44:36.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:35 smithi042 ceph-mon[28130]: pgmap v1165: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 275 KiB/s rd, 4.7 MiB/s wr, 205 op/s 2024-04-25T00:44:38.005 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:37 smithi106 ceph-mon[32510]: pgmap v1166: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.1 MiB/s wr, 193 op/s 2024-04-25T00:44:38.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:37 smithi136 ceph-mon[32923]: pgmap v1166: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.1 MiB/s wr, 193 op/s 2024-04-25T00:44:38.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:37 smithi042 ceph-mon[28130]: pgmap v1166: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 4.1 MiB/s wr, 193 op/s 2024-04-25T00:44:38.300 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T00:44:38.300 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:38.300 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 410390, 2024-04-25T00:44:38.300 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 483136, 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 97020073, 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 485126035, 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 233, 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 6, 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 24.966564290, 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 10.480625196, 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 13.082212317, 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 2.983171670, 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 483138, 2024-04-25T00:44:38.301 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 41.703079773, 2024-04-25T00:44:38.302 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000086317 2024-04-25T00:44:38.302 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.302 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:38.302 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 702242, 2024-04-25T00:44:38.302 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.104547040, 2024-04-25T00:44:38.302 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000148 2024-04-25T00:44:38.302 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.302 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-25T00:44:38.302 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:44:38.302 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 509175, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 520425, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 126654362, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 380432397, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 7, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 5, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 26.436198374, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 10.273086013, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 16.261829208, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 2.656384590, 2024-04-25T00:44:38.303 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 520428, 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 61.358904420, 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000117900 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 864563, 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.116525460, 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000134 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 697168, 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 288656 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.304 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 740128, 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 1048172, 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 228933983, 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 867479777, 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 16, 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 8, 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 35.682440682, 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 17.586601906, 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 21.743401479, 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 2.062570562, 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:38.305 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1048163, 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 60.932834623, 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000058132 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 813488, 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.194339746, 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000238 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 58832, 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 810960 2024-04-25T00:44:38.306 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 197217, 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 241.151191550, 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001222770 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:38.307 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 112281, 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 15.034757650, 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000133902 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 475500, 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 406880, 2024-04-25T00:44:38.308 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 406880, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 1988.003026158, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.004885968 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 68620, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 0, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 0, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 4713, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 20, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 24, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 635, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 234, 2024-04-25T00:44:38.309 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 387, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 14, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 300, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 1659, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 2, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 2, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 4, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 748969, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 421565, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 68592, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 125, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-04-25T00:44:38.310 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 5921, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 47, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 12, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 1162, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 12, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 127, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 4, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 234, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 0, 2024-04-25T00:44:38.311 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 34, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 71452, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 243352, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 164347, 2024-04-25T00:44:38.312 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 49625, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 23096, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 267399 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 407, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 412, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 228, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 443, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 1477, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 14, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 62646, 2024-04-25T00:44:38.313 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 62632, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 12, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 44, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-04-25T00:44:38.314 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 323599, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 320614, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 320614, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 2985, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 4514, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 453, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 449, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 449, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 4, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 1, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-04-25T00:44:38.315 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 6, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 680358018, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 686078135, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 0, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 106670, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 933.110853924, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.008747640 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 0 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 45, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 63513, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 63468, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 44, 2024-04-25T00:44:38.316 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 594, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 550, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 647, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 124608, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 123961, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 2, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 69092, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 69090, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 92752, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 231804 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 666977, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 40713, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 475500, 2024-04-25T00:44:38.317 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 9138, 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 84165, 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 63129, 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 371.411672481, 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.005883376 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 45287, 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 521.122070634, 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.011507100 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.318 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 121034, 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 228.912811611, 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001891309 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-04-25T00:44:38.319 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1, 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000227752, 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000227752 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-04-25T00:44:38.320 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.321 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 119, 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.532261340, 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.004472784 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.322 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10, 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.007801663, 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000780166 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 97303, 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 538.059388285, 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.005529730 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 14712, 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 78.734165027, 2024-04-25T00:44:38.323 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.005351696 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 54, 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.708201565, 2024-04-25T00:44:38.324 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.013114843 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 37, 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.684001450, 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.018486525 2024-04-25T00:44:38.325 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2584, 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 3.426011583, 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.001325855 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.326 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 7, 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.018757342, 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.002679620 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 62603, 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 244.385655425, 2024-04-25T00:44:38.327 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.003903737 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 1, 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 2, 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 1, 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 1, 2024-04-25T00:44:38.328 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 31528, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 31528, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 1547, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 360, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 360, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:38.329 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:38.330 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 291668, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 139, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 616, 2024-04-25T00:44:38.331 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 7, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 7432, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 156, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 652131, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 9193, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-04-25T00:44:38.332 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 212273, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 732277705, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 37, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 212273, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 212273, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 5500.871799077, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.025914137 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-04-25T00:44:38.333 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 212273, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 287665 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 212273, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 1663, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 210610, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 4398, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 30931, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 1663, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 108334, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 1338, 2024-04-25T00:44:38.334 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 20, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 63633, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 0, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 61862, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-04-25T00:44:38.335 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-04-25T00:44:38.336 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 91, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 6, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 12, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 12, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 0, 2024-04-25T00:44:38.337 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 11836, 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 0, 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 3650 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 1, 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 0 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.338 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 10, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 10, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 62632, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 62632, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 1659689, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 314271826, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:38.339 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 1659689, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 1659689, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 314271826, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-25T00:44:38.340 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 212273, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 738603537, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 212273, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 738603537, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:38.341 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 212273, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 212273, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 212273, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 212273, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:44:38.342 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 62632, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 6325832, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 62632, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 1664, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 6325832, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.343 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x561a828e2500": { 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 323599, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 681883831, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 323599, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 106670, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 681883831, 2024-04-25T00:44:38.344 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:38.345 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:38.345 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:38.345 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:38.345 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:38.345 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:38.345 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T00:44:40.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:39 smithi136 ceph-mon[32923]: pgmap v1167: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 874 KiB/s wr, 73 op/s 2024-04-25T00:44:40.054 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:39 smithi106 ceph-mon[32510]: pgmap v1167: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 874 KiB/s wr, 73 op/s 2024-04-25T00:44:40.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:39 smithi042 ceph-mon[28130]: pgmap v1167: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 874 KiB/s wr, 73 op/s 2024-04-25T00:44:40.470 INFO:tasks.check_counter:Daemon mds.b mds.exported=12 2024-04-25T00:44:40.470 INFO:tasks.check_counter:Daemon mds.b mds.imported=12 2024-04-25T00:44:40.470 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=407 2024-04-25T00:44:40.470 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=412 2024-04-25T00:44:40.470 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T00:44:40.471 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T00:44:40.727 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:44:42.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:41 smithi136 ceph-mon[32923]: pgmap v1168: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 32 op/s 2024-04-25T00:44:42.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:41 smithi106 ceph-mon[32510]: pgmap v1168: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 32 op/s 2024-04-25T00:44:42.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:41 smithi042 ceph-mon[28130]: pgmap v1168: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 65 KiB/s wr, 32 op/s 2024-04-25T00:44:42.849 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T00:44:42.849 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:42.849 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 0, 2024-04-25T00:44:42.849 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 2, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 0, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 0, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 0, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.001048449, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.000219118, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:42.850 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 461, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 458, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 547600, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 283792, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 2, 2024-04-25T00:44:42.851 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 1, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.053119281, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.021375068, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.018604331, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.009614376, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 457, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.017041814, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000037290 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.852 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 547600, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 283792 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 6, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 352, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 3344, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 217632, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 2, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.015136059, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.011636041, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.001031202, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:42.853 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 351, 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.015206334, 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000043322 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 217632 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 0, 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-04-25T00:44:42.854 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:42.855 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:42.856 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 148699, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 30, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:42.857 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 928, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 2, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-04-25T00:44:42.858 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 463, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 482401, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 463, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 463, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 482401, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:42.859 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:42.860 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:42.860 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:42.860 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:42.860 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T00:44:43.557 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2024-04-25T00:44:43.557 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2024-04-25T00:44:43.557 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2024-04-25T00:44:43.557 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2024-04-25T00:44:43.557 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T00:44:43.558 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T00:44:43.798 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:44:44.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:43 smithi136 ceph-mon[32923]: pgmap v1169: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 23 op/s 2024-04-25T00:44:44.075 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:43 smithi106 ceph-mon[32510]: pgmap v1169: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 23 op/s 2024-04-25T00:44:44.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:43 smithi042 ceph-mon[28130]: pgmap v1169: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 23 op/s 2024-04-25T00:44:45.502 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T00:44:45.502 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:45.502 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 3105, 2024-04-25T00:44:45.502 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 3073, 2024-04-25T00:44:45.502 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 102147497, 2024-04-25T00:44:45.502 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 1113572, 2024-04-25T00:44:45.502 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 7, 2024-04-25T00:44:45.502 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 3, 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.298368872, 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.087272379, 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.120931807, 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.070554620, 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3072, 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.094937174, 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000030904 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.503 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 684800, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 284864 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 1235, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 1583, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 139120479, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 1255166, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 9, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 6, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.166807236, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.033968780, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.096848664, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.027429915, 2024-04-25T00:44:45.504 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 1581, 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.046818238, 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000029613 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 863328 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 2491, 2024-04-25T00:44:45.505 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 2491, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 248112091, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 791321, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 7, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 5, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.306258351, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.039052781, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.200619711, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.054343872, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 2490, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.054879938, 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000022040 2024-04-25T00:44:45.506 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 1, 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-MDSRank": { 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-25T00:44:45.507 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 10807, 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.251724477, 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000023292 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "finisher-PurgeQueue": { 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "queue_len": 0, 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "complete_latency": { 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": { 2024-04-25T00:44:45.508 INFO:teuthology.orchestra.run.smithi106.stdout: "request": 0, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "reply": 0, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "reply_latency": { 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_reply": 0, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "forward": 0, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_complete": 0, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_fetch_keys": 0, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_commit": 0, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_split": 0, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_merge": 0, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes": 49092, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_top": 17791, 2024-04-25T00:44:45.509 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_bottom": 31301, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pin_tail": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_pinned": 23676, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_expired": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "inodes_with_caps": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "subtrees": 1, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_hit": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_forward": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_discover": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_remote_ino": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "traverse_lock": 0, 2024-04-25T00:44:45.510 INFO:teuthology.orchestra.run.smithi106.stdout: "load_cent": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "q": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "exported": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "exported_inodes": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "imported": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "imported_inodes": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_dir_fetch": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "openino_peer_discover": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rfiles": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rbytes": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "root_rsnaps": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_set_tag": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:44:45.511 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "scrub_file_inodes": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_cap_release": 0, 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: "process_request_cap_release": 0 2024-04-25T00:44:45.512 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_cache": { 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_update_receipt": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_try_discover": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_send_discover": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "dir_handle_discover": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_delayed": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_created": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_enqueued": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_reintegrated": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "strays_migrated": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_processing": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:44:45.513 INFO:teuthology.orchestra.run.smithi106.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_started": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "recovery_completed": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_exportdir": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_flush": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_fragstats": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "ireq_inodestats": 0 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_log": { 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "evlrg": 0, 2024-04-25T00:44:45.514 INFO:teuthology.orchestra.run.smithi106.stdout: "evadd": 0, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "evex": 0, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "evtrm": 0, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "ev": 247253, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "evexg": 0, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "evexd": 0, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "segadd": 0, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "segex": 0, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "segtrm": 0, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "seg": 135, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "segmjr": 22, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "segexg": 0, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "segexd": 0, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "expos": 261431427, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "wrpos": 491127744, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "rdpos": 491126093, 2024-04-25T00:44:45.515 INFO:teuthology.orchestra.run.smithi106.stdout: "jlat": { 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "replayed": 247253 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_mem": { 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "ino": 24, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "ino+": 49010, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "ino-": 48986, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "dir": 22, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "dir+": 199, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "dir-": 177, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "dn": 49092, 2024-04-25T00:44:45.516 INFO:teuthology.orchestra.run.smithi106.stdout: "dn+": 49533, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "dn-": 441, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "cap": 0, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "cap+": 0, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "cap-": 0, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "rss": 100640, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "heap": 215420 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_server": { 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_client_request": 0, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "dispatch_server_request": 0, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_request": 0, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_client_session": 0, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "handle_peer_request": 0, 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "req_create_latency": { 2024-04-25T00:44:45.517 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getattr_latency": { 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getfilelock_latency": { 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "req_link_latency": { 2024-04-25T00:44:45.518 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookup_latency": { 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookuphash_latency": { 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupino_latency": { 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.519 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupname_latency": { 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupparent_latency": { 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lookupsnap_latency": { 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.520 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.521 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.521 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.521 INFO:teuthology.orchestra.run.smithi106.stdout: "req_lssnap_latency": { 2024-04-25T00:44:45.521 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.521 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.521 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.521 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mkdir_latency": { 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mknod_latency": { 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "req_mksnap_latency": { 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.522 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "req_open_latency": { 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "req_readdir_latency": { 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rename_latency": { 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.523 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "req_renamesnap_latency": { 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "req_snapdiff_latency": { 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmdir_latency": { 2024-04-25T00:44:45.524 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmsnap_latency": { 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "req_rmxattr_latency": { 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setattr_latency": { 2024-04-25T00:44:45.525 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setdirlayout_latency": { 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setfilelock_latency": { 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setlayout_latency": { 2024-04-25T00:44:45.526 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "req_setxattr_latency": { 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "req_symlink_latency": { 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.527 INFO:teuthology.orchestra.run.smithi106.stdout: "req_unlink_latency": { 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "req_getvxattr_latency": { 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_sessions": { 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "session_count": 1, 2024-04-25T00:44:45.528 INFO:teuthology.orchestra.run.smithi106.stdout: "session_add": 1, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "session_remove": 0, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_open": 1, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "sessions_stale": 0, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "total_load": 0, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "average_load": 0, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "avg_session_uptime": 3136, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "mdthresh_evicted": 0 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:45.529 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:45.530 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:45.531 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 242168, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 95, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 7432, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 156, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:45.532 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 34645730, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 197863, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "objecter": { 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "op_active": 0, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "op_laggy": 0, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send": 6325, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "op_send_bytes": 21, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "op_resend": 3, 2024-04-25T00:44:45.533 INFO:teuthology.orchestra.run.smithi106.stdout: "op_reply": 6325, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "op_latency": { 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 6325, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 16.972605120, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.002683415 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "op_inflight": 0, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "oplen_avg": { 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 6325, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 6326 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "op": 6325, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "op_r": 6325, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "op_w": 0, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "op_rmw": 0, 2024-04-25T00:44:45.534 INFO:teuthology.orchestra.run.smithi106.stdout: "op_pg": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_stat": 3345, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_create": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_read": 2978, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_write": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writefull": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_writesame": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_append": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_zero": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_truncate": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_delete": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_mapext": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_sparse_read": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_clonerange": 0, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_getxattr": 1, 2024-04-25T00:44:45.535 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_setxattr": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_rmxattr": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_call": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_watch": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_notify": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "osdop_other": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_active": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_send": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_resend": 0, 2024-04-25T00:44:45.536 INFO:teuthology.orchestra.run.smithi106.stdout: "linger_ping": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_active": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_send": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "poolop_resend": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_active": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_send": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "poolstat_resend": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_active": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_send": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "statfs_resend": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "command_active": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "command_send": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "command_resend": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "map_epoch": 91, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "map_full": 0, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "map_inc": 5, 2024-04-25T00:44:45.537 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_sessions": 11, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_open": 12, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_session_close": 1, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_laggy": 0, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_wr": 0, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_rd": 2, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_del": 0 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "oft": { 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_objs": 0, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_updates": 0, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "omap_total_removes": 0 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.538 INFO:teuthology.orchestra.run.smithi106.stdout: "osd-slow-ops": { 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "slow_ops_count": 0 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "purge_queue": { 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops": 0, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing": 0, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executing_high_water": 0, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed_ops": 0, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_executed": 0, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "pq_item_in_journal": 0 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:45.539 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 6827, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 488736960, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 6827, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 6827, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 488736960, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_bytes": { 2024-04-25T00:44:45.540 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 6325, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 486947931, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 6325, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 486947931, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.541 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-objecter_ops": { 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 1024, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 6325, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 6325, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 6325, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 6325, 2024-04-25T00:44:45.542 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-25T00:44:45.543 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 0, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 0, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-write_buf_throttle-0x55d9a2f166e0": { 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 3758096384, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 0, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 0, 2024-04-25T00:44:45.544 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 0, 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 0, 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:45.545 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T00:44:45.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:45 smithi106 ceph-mon[32510]: pgmap v1170: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-04-25T00:44:46.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:45 smithi136 ceph-mon[32923]: pgmap v1170: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-04-25T00:44:46.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:45 smithi042 ceph-mon[28130]: pgmap v1170: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-04-25T00:44:46.242 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-25T00:44:46.242 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-25T00:44:46.243 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-25T00:44:46.243 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2024-04-25T00:44:46.243 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T00:44:46.243 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T00:44:46.481 INFO:teuthology.orchestra.run.smithi106.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T00:44:47.981 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:47 smithi106 ceph-mon[32510]: pgmap v1171: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-25T00:44:48.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:47 smithi136 ceph-mon[32923]: pgmap v1171: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-25T00:44:48.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:47 smithi042 ceph-mon[28130]: pgmap v1171: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.2 KiB/s wr, 8 op/s 2024-04-25T00:44:48.250 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2024-04-25T00:44:48.250 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:48.250 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 4, 2024-04-25T00:44:48.250 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 4, 2024-04-25T00:44:48.250 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 3024, 2024-04-25T00:44:48.250 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 448, 2024-04-25T00:44:48.250 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:48.250 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 0, 2024-04-25T00:44:48.250 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.001201872, 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.000029127, 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.000414442, 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 3, 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000702892, 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000234297 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:48.251 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:48.252 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:48.252 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.252 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-25T00:44:48.252 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:44:48.252 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.252 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:48.252 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 465, 2024-04-25T00:44:48.252 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 462, 2024-04-25T00:44:48.252 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 548688, 2024-04-25T00:44:48.252 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 286352, 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 2, 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 1, 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.053931344, 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.021316768, 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.020093822, 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.009486669, 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 461, 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.016526959, 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000035850 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.253 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 548688, 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 286352 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_messages": 2, 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages": 354, 2024-04-25T00:44:48.254 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:44:48.255 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_bytes": 217888, 2024-04-25T00:44:48.255 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:48.255 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_active_connections": 2, 2024-04-25T00:44:48.255 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_total_time": 0.015167420, 2024-04-25T00:44:48.255 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_send_time": 0.011598227, 2024-04-25T00:44:48.255 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_recv_time": 0.000812357, 2024-04-25T00:44:48.255 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:48.255 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:48.255 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 352, 2024-04-25T00:44:48.255 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.015224856, 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000043252 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "msgr_send_encrypted_bytes": 217888 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "cct": { 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "total_workers": 0, 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: "unhealthy_workers": 0 2024-04-25T00:44:48.256 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "mempool": { 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:48.257 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:48.258 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_items": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_bytes": 148699, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_anon_items": 30, 2024-04-25T00:44:48.259 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:44:48.260 INFO:teuthology.orchestra.run.smithi106.stdout: "buffer_meta_items": 0, 2024-04-25T00:44:48.260 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_bytes": 0, 2024-04-25T00:44:48.260 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_items": 0, 2024-04-25T00:44:48.260 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:48.260 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:48.260 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:48.260 INFO:teuthology.orchestra.run.smithi106.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_bytes": 0, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_items": 0, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "pgmap_items": 0, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_bytes": 928, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "mds_co_items": 2, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_1_items": 0, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:48.261 INFO:teuthology.orchestra.run.smithi106.stdout: "unittest_2_items": 0 2024-04-25T00:44:48.262 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2024-04-25T00:44:48.262 INFO:teuthology.orchestra.run.smithi106.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:48.262 INFO:teuthology.orchestra.run.smithi106.stdout: "val": 0, 2024-04-25T00:44:48.262 INFO:teuthology.orchestra.run.smithi106.stdout: "max": 104857600, 2024-04-25T00:44:48.262 INFO:teuthology.orchestra.run.smithi106.stdout: "get_started": 0, 2024-04-25T00:44:48.262 INFO:teuthology.orchestra.run.smithi106.stdout: "get": 467, 2024-04-25T00:44:48.262 INFO:teuthology.orchestra.run.smithi106.stdout: "get_sum": 482921, 2024-04-25T00:44:48.262 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:48.262 INFO:teuthology.orchestra.run.smithi106.stdout: "get_or_fail_success": 467, 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout: "take": 0, 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout: "take_sum": 0, 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout: "put": 467, 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout: "put_sum": 482921, 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout: "wait": { 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout: "avgcount": 0, 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout: "sum": 0.000000000, 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout: "avgtime": 0.000000000 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout: } 2024-04-25T00:44:48.263 INFO:teuthology.orchestra.run.smithi106.stdout:} 2024-04-25T00:44:48.861 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T00:44:48.862 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T00:44:48.862 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-25T00:44:48.862 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-25T00:44:48.862 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T00:44:48.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T00:44:49.194 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:44:50.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:49 smithi136 ceph-mon[32923]: pgmap v1172: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-25T00:44:50.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:49 smithi106 ceph-mon[32510]: pgmap v1172: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-25T00:44:50.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:49 smithi042 ceph-mon[28130]: pgmap v1172: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-25T00:44:51.151 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 494232, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 482957, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 125755739, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 333794279, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 9, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 25.439252659, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 10.214700171, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 15.939577917, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 2.221494404, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 482955, 2024-04-25T00:44:51.152 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 14.131741910, 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000029260 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 863551, 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.113562303, 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000131 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.153 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 304097, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 315469, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 72185515, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 314551747, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 7, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 6, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 16.966295811, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 7.114690518, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 9.214282408, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 1.878899590, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 315471, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 11.345828208, 2024-04-25T00:44:51.154 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000035964 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 480199, 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.073036760, 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000152 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 690016, 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 292592 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 705222, 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1035135, 2024-04-25T00:44:51.155 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 222398290, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 928702460, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 15, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 8, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 34.712067918, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 16.976088839, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 21.945790374, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 2.049766637, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1035131, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 39.025265940, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000037700 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.156 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 786378, 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.202085142, 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000256 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 779488 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-04-25T00:44:51.157 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 157016, 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 234.814906700, 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001495483 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 122347, 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 18.515074697, 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000151332 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.158 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 470684, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 409721, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 409721, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2114.352031829, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.005160467 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 60963, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 229, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 22, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 25, 2024-04-25T00:44:51.159 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 2789, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 1920, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 847, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 22, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 25, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 428, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 4, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 805160, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 398312, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 60962, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 195, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-04-25T00:44:51.160 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 8652, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 9, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 118, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 9, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 760, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:44:51.161 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 34, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 37040, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 246726, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 129880, 2024-04-25T00:44:51.162 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 52265, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 20151, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 305163 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 571, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 797, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 484, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 651, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 911, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 12, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 69224, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 69212, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-04-25T00:44:51.163 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 12, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 47, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-04-25T00:44:51.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 335901, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 227548, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 221221, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 114680, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 6327, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 402, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 274, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 266, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 136, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 10, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-04-25T00:44:51.165 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 8, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 402071639, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 638133347, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 89509, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 850.695564886, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.009504022 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 38, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 69510, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 69472, 2024-04-25T00:44:51.166 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 37, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 651, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 614, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 2789, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 123021, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 120232, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 75249, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 75249, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 90072, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 223612 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 691892, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 6304, 2024-04-25T00:44:51.167 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 470684, 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 9126, 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 45264, 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 68940, 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 431.189358243, 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.006254559 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 22784, 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 362.142239903, 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.015894585 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-04-25T00:44:51.168 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 128802, 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 310.724712756, 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002412421 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.169 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.170 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-04-25T00:44:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-04-25T00:44:51.171 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.172 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-04-25T00:44:51.173 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.174 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 104400, 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 637.728001244, 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.006108505 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 13679, 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 98.882974112, 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.007228816 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.175 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 40, 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.342113690, 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.008552842 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-04-25T00:44:51.176 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 64, 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.810079768, 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.028282496 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.177 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1828, 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 4.054793201, 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002218158 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-04-25T00:44:51.178 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 69184, 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 267.477758912, 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003866179 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:44:51.179 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 1, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 0, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 36132, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 36132, 2024-04-25T00:44:51.180 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 1478, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:51.181 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:51.182 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 293773, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 141, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 616, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 7, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-04-25T00:44:51.183 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7432, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 156, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 2254921, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 32374, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-04-25T00:44:51.184 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 165276, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 649838889, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 33, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 165276, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 165276, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1646.202534533, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.009960324 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-04-25T00:44:51.185 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 165276, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 167907 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 165276, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 1769, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 163507, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 17, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 26, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 1769, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 91277, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 950, 2024-04-25T00:44:51.186 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 22, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 69650, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 52, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-04-25T00:44:51.187 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-04-25T00:44:51.188 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 91, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 5, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 12, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 3097, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 0, 2024-04-25T00:44:51.189 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 1047 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 10, 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-04-25T00:44:51.190 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 10, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 69212, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 69212, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 1503547, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 295425683, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 1503547, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T00:44:51.191 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1503547, 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 295425683, 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T00:44:51.192 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 165276, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 656829301, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 165276, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 656829301, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:51.193 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 165276, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 165276, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 165276, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 165276, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.194 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 69212, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 6990412, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 69212, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1768, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 6990412, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.195 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x559a7c0e6500": { 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 335901, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 633939043, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 335901, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 89509, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 633939043, 2024-04-25T00:44:51.196 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:51.197 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:51.197 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-25T00:44:52.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:51 smithi136 ceph-mon[32923]: pgmap v1173: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T00:44:52.060 INFO:tasks.check_counter:Daemon mds.c mds.exported=9 2024-04-25T00:44:52.060 INFO:tasks.check_counter:Daemon mds.c mds.imported=9 2024-04-25T00:44:52.060 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=571 2024-04-25T00:44:52.060 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=797 2024-04-25T00:44:52.060 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T00:44:52.061 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T00:44:52.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:51 smithi106 ceph-mon[32510]: pgmap v1173: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T00:44:52.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:51 smithi042 ceph-mon[28130]: pgmap v1173: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T00:44:52.303 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:44:53.838 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:53 smithi106 ceph-mon[32510]: pgmap v1174: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-04-25T00:44:53.933 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:53 smithi136 ceph-mon[32923]: pgmap v1174: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-04-25T00:44:54.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:53 smithi042 ceph-mon[28130]: pgmap v1174: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1485, 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1456, 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 172895379, 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 600338, 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 8, 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 3, 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.226863245, 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.041454469, 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.143695647, 2024-04-25T00:44:54.167 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.032704021, 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1455, 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.043205858, 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000029694 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 674944, 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 286784 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.168 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1974, 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 2322, 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 273134574, 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 1415045, 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 8, 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 6, 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.274781073, 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.052916402, 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.165524231, 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.043360017, 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:54.169 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2321, 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.068193225, 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000029380 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 789216 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.170 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 3520, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 3522, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 190377252, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 1119418, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.391906147, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.075821669, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.197718120, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.094710488, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3520, 2024-04-25T00:44:54.171 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.105282168, 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000029909 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-04-25T00:44:54.172 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 11000, 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.323362857, 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000029396 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-04-25T00:44:54.173 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 69466, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 31490, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 37976, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 24480, 2024-04-25T00:44:54.175 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 1, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-04-25T00:44:54.176 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:44:54.177 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-04-25T00:44:54.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:44:54.179 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-04-25T00:44:54.180 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 335901, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 136, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 29, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-04-25T00:44:54.181 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 402071639, 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 638133347, 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 638132209, 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 335901 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.182 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 25, 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 69300, 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 69275, 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 23, 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 271, 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 248, 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 69466, 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 70016, 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 550, 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-04-25T00:44:54.183 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 113536, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215420 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.184 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.185 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.186 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.187 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-04-25T00:44:54.188 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.188 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.188 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.188 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-04-25T00:44:54.188 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.188 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.188 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-04-25T00:44:54.189 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.190 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.191 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.192 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-04-25T00:44:54.194 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.195 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 1, 2024-04-25T00:44:54.196 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 0, 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 3147, 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:54.197 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:54.198 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:54.199 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 242989, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 98, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7432, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 156, 2024-04-25T00:44:54.200 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 48992324, 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 279410, 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-04-25T00:44:54.201 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-04-25T00:44:54.202 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-04-25T00:44:54.202 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 6474, 2024-04-25T00:44:54.202 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 21, 2024-04-25T00:44:54.202 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 3, 2024-04-25T00:44:54.202 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 6474, 2024-04-25T00:44:54.202 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-04-25T00:44:54.202 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6474, 2024-04-25T00:44:54.202 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 23.304483406, 2024-04-25T00:44:54.203 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003599703 2024-04-25T00:44:54.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.203 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-04-25T00:44:54.203 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-04-25T00:44:54.203 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6474, 2024-04-25T00:44:54.203 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 6475 2024-04-25T00:44:54.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.203 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 6474, 2024-04-25T00:44:54.203 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 6474, 2024-04-25T00:44:54.204 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-04-25T00:44:54.204 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-04-25T00:44:54.204 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-04-25T00:44:54.204 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 3412, 2024-04-25T00:44:54.204 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-04-25T00:44:54.204 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 3060, 2024-04-25T00:44:54.204 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-04-25T00:44:54.205 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-04-25T00:44:54.205 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-04-25T00:44:54.205 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-04-25T00:44:54.205 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-04-25T00:44:54.205 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-04-25T00:44:54.205 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-04-25T00:44:54.205 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 1, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-04-25T00:44:54.206 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-04-25T00:44:54.207 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 91, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 5, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 11, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 1, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-04-25T00:44:54.208 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:44:54.209 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 6975, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 635750804, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 6975, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T00:44:54.210 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6975, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 635750804, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T00:44:54.211 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 6474, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 633945007, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6474, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 633945007, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-04-25T00:44:54.212 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 6474, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 6474, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6474, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 6474, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:54.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:54.214 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x564a2ff8e6e0": { 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T00:44:54.215 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-04-25T00:44:54.216 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-04-25T00:44:54.216 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:54.216 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:54.216 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:54.216 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:54.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:54.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:54.216 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-25T00:44:55.102 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T00:44:55.102 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T00:44:55.102 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T00:44:55.102 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T00:44:55.102 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T00:44:55.102 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T00:44:55.340 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:44:56.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:55 smithi106 ceph-mon[32510]: pgmap v1175: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-25T00:44:56.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:55 smithi042 ceph-mon[28130]: pgmap v1175: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-25T00:44:56.164 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:55 smithi136 ceph-mon[32923]: pgmap v1175: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-25T00:44:57.006 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-25T00:44:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1249, 2024-04-25T00:44:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1251, 2024-04-25T00:44:57.006 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 134586835, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 397192, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 8, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.169371884, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.026036014, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.102045377, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.032545363, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1249, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.044372614, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000035526 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:57.007 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 822, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 952, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 149333496, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 577944, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 7, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 6, 2024-04-25T00:44:57.008 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.149910931, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.020347997, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.104006402, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.019738179, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 950, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.042435175, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000044668 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T00:44:57.009 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 316800 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 781, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 769, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 117836415, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 294233, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 5, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.127268781, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.019835142, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.085820715, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.017944862, 2024-04-25T00:44:57.010 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 769, 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.032640032, 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000042444 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 245456, 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 107248 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-04-25T00:44:57.011 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 4390, 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.422054392, 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000096139 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-04-25T00:44:57.012 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-04-25T00:44:57.013 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 40265, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 27538, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 12727, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 1114, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-04-25T00:44:57.014 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 2, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-04-25T00:44:57.015 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T00:44:57.016 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-04-25T00:44:57.017 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-04-25T00:44:57.018 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 192293, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-04-25T00:44:57.019 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 4, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 43, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 680358018, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 686078135, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 686077946, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 192293 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.020 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 31, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 39959, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 39928, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 30, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 180, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 150, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 40265, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 40517, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 252, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 89812, 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215396 2024-04-25T00:44:57.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.022 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-04-25T00:44:57.023 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.024 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-04-25T00:44:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-04-25T00:44:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.025 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.026 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.027 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-04-25T00:44:57.027 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.027 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.027 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.028 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.029 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-04-25T00:44:57.030 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-04-25T00:44:57.031 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-04-25T00:44:57.032 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-04-25T00:44:57.033 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 1, 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 0, 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 0, 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-04-25T00:44:57.034 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 3151, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:57.035 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:57.036 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 239165, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 90, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-04-25T00:44:57.037 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 7432, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 156, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 28457212, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 162814, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:57.038 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 2658, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 26, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 4, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 2658, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2658, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 15.924649115, 2024-04-25T00:44:57.039 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.005991214 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2658, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2659 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 2658, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 2658, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 1393, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 1262, 2024-04-25T00:44:57.040 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 2, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-04-25T00:44:57.041 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-04-25T00:44:57.042 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 11, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 11, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-04-25T00:44:57.043 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-04-25T00:44:57.044 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 2848, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 401487701, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 2848, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T00:44:57.045 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2848, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 401487701, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T00:44:57.046 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 2658, 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 400781824, 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2658, 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 400781824, 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:57.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 2658, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 2658, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2658, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 2658, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.048 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T00:44:57.049 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x55657e7a86e0": { 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-04-25T00:44:57.050 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:57.051 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:57.052 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-25T00:44:57.764 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-25T00:44:57.764 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-25T00:44:57.765 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-25T00:44:57.765 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-04-25T00:44:57.765 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T00:44:57.765 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551 shell --fsid 9d95543e-0297-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T00:44:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:57 smithi136 ceph-mon[32923]: pgmap v1176: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-25T00:44:58.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:57 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:44:58.021 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T00:44:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:57 smithi106 ceph-mon[32510]: pgmap v1176: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-25T00:44:58.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:57 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:44:58.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:57 smithi042 ceph-mon[28130]: pgmap v1176: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-25T00:44:58.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:57 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T00:44:59.518 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-25T00:44:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T00:44:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 4, 2024-04-25T00:44:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 4, 2024-04-25T00:44:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 3024, 2024-04-25T00:44:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 448, 2024-04-25T00:44:59.518 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 4, 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 0, 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.001305534, 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.000039491, 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.000494634, 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3, 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000669157, 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000223052 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:59.519 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3024, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 468, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 465, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 549504, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 288272, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 2, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 1, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.055404220, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.022207684, 2024-04-25T00:44:59.520 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.019690020, 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.009923734, 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 464, 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.016752997, 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000036105 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 549504, 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 288272 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.521 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2, 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 357, 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 320, 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 296384, 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 5, 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.016260452, 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.012354367, 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.000930143, 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T00:44:59.522 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 355, 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.018179141, 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000051208 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 296384 2024-04-25T00:44:59.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 0, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T00:44:59.524 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T00:44:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T00:44:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-04-25T00:44:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T00:44:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T00:44:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-04-25T00:44:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-04-25T00:44:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-04-25T00:44:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-04-25T00:44:59.525 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-04-25T00:44:59.526 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 148682, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 29, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 0, 2024-04-25T00:44:59.527 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 0, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 928, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 2, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-04-25T00:44:59.528 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 470, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 483311, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 470, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 470, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 483311, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-25T00:44:59.529 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-25T00:45:00.038 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:44:59 smithi042 ceph-mon[28130]: pgmap v1177: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-25T00:45:00.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:44:59 smithi106 ceph-mon[32510]: pgmap v1177: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-25T00:45:00.147 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:44:59 smithi136 ceph-mon[32923]: pgmap v1177: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-04-25T00:45:00.314 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T00:45:00.315 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T00:45:00.315 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-25T00:45:00.315 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-25T00:45:00.315 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T00:45:00.325 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T00:45:00.326 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:45:00.326 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:45:00.398 DEBUG:teuthology.orchestra.run.smithi042:> set -ex 2024-04-25T00:45:00.398 DEBUG:teuthology.orchestra.run.smithi042:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T00:45:00.494 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:45:00.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.689+0000 7fdfe9a00640 1 -- 172.21.15.42:0/3538983924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe4101790 msgr2=0x7fdfe4101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:00.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.689+0000 7fdfe9a00640 1 --2- 172.21.15.42:0/3538983924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe4101790 0x7fdfe4101bb0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009ca0 tx=0x7fdfd402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:45:00.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.690+0000 7fdfe9a00640 1 -- 172.21.15.42:0/3538983924 shutdown_connections 2024-04-25T00:45:00.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.690+0000 7fdfe9a00640 1 --2- 172.21.15.42:0/3538983924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe4103340 0x7fdfe4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:00.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.690+0000 7fdfe9a00640 1 --2- 172.21.15.42:0/3538983924 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdfe4102980 0x7fdfe4102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:00.693 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.690+0000 7fdfe9a00640 1 --2- 172.21.15.42:0/3538983924 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe4101790 0x7fdfe4101bb0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.690+0000 7fdfe9a00640 1 -- 172.21.15.42:0/3538983924 >> 172.21.15.42:0/3538983924 conn(0x7fdfe40fcd20 msgr2=0x7fdfe40ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.690+0000 7fdfe9a00640 1 -- 172.21.15.42:0/3538983924 shutdown_connections 2024-04-25T00:45:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.690+0000 7fdfe9a00640 1 -- 172.21.15.42:0/3538983924 wait complete. 2024-04-25T00:45:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.691+0000 7fdfe9a00640 1 Processor -- start 2024-04-25T00:45:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.691+0000 7fdfe9a00640 1 -- start start 2024-04-25T00:45:00.694 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.691+0000 7fdfe9a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdfe4101790 0x7fdfe4199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:00.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.691+0000 7fdfe9a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe4102980 0x7fdfe419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:00.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.691+0000 7fdfe9a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe4103340 0x7fdfe41a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:00.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.691+0000 7fdfe9a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe410d460 con 0x7fdfe4101790 2024-04-25T00:45:00.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.691+0000 7fdfe9a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe410d2e0 con 0x7fdfe4103340 2024-04-25T00:45:00.695 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.691+0000 7fdfe9a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe410d5e0 con 0x7fdfe4102980 2024-04-25T00:45:00.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfda000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe4102980 0x7fdfe419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:00.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdfe4101790 0x7fdfe4199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:00.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe3400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe4103340 0x7fdfe41a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:00.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfda000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe4102980 0x7fdfe419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:55126/0 (socket says 172.21.15.42:55126) 2024-04-25T00:45:00.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe2a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdfe4101790 0x7fdfe4199cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60924/0 (socket says 172.21.15.42:60924) 2024-04-25T00:45:00.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe2a00640 1 -- 172.21.15.42:0/4244126398 learned_addr learned my addr 172.21.15.42:0/4244126398 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:00.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe3400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe4103340 0x7fdfe41a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59276/0 (socket says 172.21.15.42:59276) 2024-04-25T00:45:00.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe3400640 1 -- 172.21.15.42:0/4244126398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe4102980 msgr2=0x7fdfe419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:00.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe3400640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe4102980 0x7fdfe419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:00.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe3400640 1 -- 172.21.15.42:0/4244126398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdfe4101790 msgr2=0x7fdfe4199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:00.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe3400640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdfe4101790 0x7fdfe4199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:00.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe3400640 1 -- 172.21.15.42:0/4244126398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe41a1900 con 0x7fdfe4103340 2024-04-25T00:45:00.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.692+0000 7fdfe3400640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe4103340 0x7fdfe41a1260 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00e970 tx=0x7fdfdc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:00.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.693+0000 7fdfdbe00640 1 -- 172.21.15.42:0/4244126398 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc00ccc0 con 0x7fdfe4103340 2024-04-25T00:45:00.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.693+0000 7fdfdbe00640 1 -- 172.21.15.42:0/4244126398 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdfdc005020 con 0x7fdfe4103340 2024-04-25T00:45:00.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.693+0000 7fdfdbe00640 1 -- 172.21.15.42:0/4244126398 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc004a00 con 0x7fdfe4103340 2024-04-25T00:45:00.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.693+0000 7fdfe9a00640 1 -- 172.21.15.42:0/4244126398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe41a1bf0 con 0x7fdfe4103340 2024-04-25T00:45:00.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.693+0000 7fdfe9a00640 1 -- 172.21.15.42:0/4244126398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe41a2190 con 0x7fdfe4103340 2024-04-25T00:45:00.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.694+0000 7fdfe9a00640 1 -- 172.21.15.42:0/4244126398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe41a23c0 con 0x7fdfe4103340 2024-04-25T00:45:00.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.696+0000 7fdfdbe00640 1 -- 172.21.15.42:0/4244126398 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdfdc0051d0 con 0x7fdfe4103340 2024-04-25T00:45:00.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.697+0000 7fdfdbe00640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdfc80a2c10 0x7fdfc80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:00.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.697+0000 7fdfdbe00640 1 -- 172.21.15.42:0/4244126398 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fdfdc0cef20 con 0x7fdfe4103340 2024-04-25T00:45:00.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.697+0000 7fdfe2a00640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdfc80a2c10 0x7fdfc80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:00.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.698+0000 7fdfe2a00640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdfc80a2c10 0x7fdfc80a50d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4031c10 tx=0x7fdfd4009880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:00.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.701+0000 7fdfdbe00640 1 -- 172.21.15.42:0/4244126398 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fdfdc09be50 con 0x7fdfe4103340 2024-04-25T00:45:00.915 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.913+0000 7fdfe9a00640 1 -- 172.21.15.42:0/4244126398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdfe41a2590 con 0x7fdfe4103340 2024-04-25T00:45:00.917 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.915+0000 7fdfdbe00640 1 -- 172.21.15.42:0/4244126398 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22547 (secure 0 0 0) 0x7fdfdc09b750 con 0x7fdfe4103340 2024-04-25T00:45:00.917 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:45:00.917 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":91,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:34:20.134521+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:45:00.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.918+0000 7fdfe9a00640 1 -- 172.21.15.42:0/4244126398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdfc80a2c10 msgr2=0x7fdfc80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:00.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.918+0000 7fdfe9a00640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdfc80a2c10 0x7fdfc80a50d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4031c10 tx=0x7fdfd4009880 comp rx=0 tx=0).stop 2024-04-25T00:45:00.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.918+0000 7fdfe9a00640 1 -- 172.21.15.42:0/4244126398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe4103340 msgr2=0x7fdfe41a1260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:00.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.918+0000 7fdfe9a00640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe4103340 0x7fdfe41a1260 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00e970 tx=0x7fdfdc00ee30 comp rx=0 tx=0).stop 2024-04-25T00:45:00.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.919+0000 7fdfe9a00640 1 -- 172.21.15.42:0/4244126398 shutdown_connections 2024-04-25T00:45:00.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.919+0000 7fdfe9a00640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fdfc80a2c10 0x7fdfc80a50d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:00.920 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.919+0000 7fdfe9a00640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfe4103340 0x7fdfe41a1260 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.919+0000 7fdfe9a00640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdfe4102980 0x7fdfe419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.919+0000 7fdfe9a00640 1 --2- 172.21.15.42:0/4244126398 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fdfe4101790 0x7fdfe4199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.919+0000 7fdfe9a00640 1 -- 172.21.15.42:0/4244126398 >> 172.21.15.42:0/4244126398 conn(0x7fdfe40fcd20 msgr2=0x7fdfe4073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.919+0000 7fdfe9a00640 1 -- 172.21.15.42:0/4244126398 shutdown_connections 2024-04-25T00:45:00.921 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:00.919+0000 7fdfe9a00640 1 -- 172.21.15.42:0/4244126398 wait complete. 2024-04-25T00:45:00.939 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T00:11:14.472869+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T00:16:18.168243+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T00:16:19.289585+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T00:45:00.940 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T00:45:01.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.168+0000 7fe157200640 1 -- 172.21.15.42:0/3874152901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1581016d0 msgr2=0x7fe158101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:01.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.168+0000 7fe157200640 1 --2- 172.21.15.42:0/3874152901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1581016d0 0x7fe158101af0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe148009860 tx=0x7fe14802a340 comp rx=0 tx=0).stop 2024-04-25T00:45:01.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.168+0000 7fe157200640 1 -- 172.21.15.42:0/3874152901 shutdown_connections 2024-04-25T00:45:01.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.168+0000 7fe157200640 1 --2- 172.21.15.42:0/3874152901 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe158103280 0x7fe158109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.168+0000 7fe157200640 1 --2- 172.21.15.42:0/3874152901 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1581028c0 0x7fe158102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.168+0000 7fe157200640 1 --2- 172.21.15.42:0/3874152901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1581016d0 0x7fe158101af0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.168+0000 7fe157200640 1 -- 172.21.15.42:0/3874152901 >> 172.21.15.42:0/3874152901 conn(0x7fe1580fcc40 msgr2=0x7fe1580ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:01.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.168+0000 7fe157200640 1 -- 172.21.15.42:0/3874152901 shutdown_connections 2024-04-25T00:45:01.170 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.168+0000 7fe157200640 1 -- 172.21.15.42:0/3874152901 wait complete. 2024-04-25T00:45:01.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.169+0000 7fe157200640 1 Processor -- start 2024-04-25T00:45:01.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe157200640 1 -- start start 2024-04-25T00:45:01.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe157200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1581016d0 0x7fe158199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:01.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe157200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1581028c0 0x7fe15819a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:01.172 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe157200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe158103280 0x7fe1581a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe157200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fe15810d2e0 con 0x7fe158103280 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe157200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fe15810d160 con 0x7fe1581016d0 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe157200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe15810d460 con 0x7fe1581028c0 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe155e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1581016d0 0x7fe158199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe156800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe158103280 0x7fe1581a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe155400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1581028c0 0x7fe15819a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe155e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1581016d0 0x7fe158199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59290/0 (socket says 172.21.15.42:59290) 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.170+0000 7fe155e00640 1 -- 172.21.15.42:0/1111691102 learned_addr learned my addr 172.21.15.42:0/1111691102 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.171+0000 7fe155e00640 1 -- 172.21.15.42:0/1111691102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1581028c0 msgr2=0x7fe15819a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.171+0000 7fe155e00640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1581028c0 0x7fe15819a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.171+0000 7fe155e00640 1 -- 172.21.15.42:0/1111691102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe158103280 msgr2=0x7fe1581a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.171+0000 7fe155e00640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe158103280 0x7fe1581a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.171+0000 7fe155e00640 1 -- 172.21.15.42:0/1111691102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1581a17e0 con 0x7fe1581016d0 2024-04-25T00:45:01.173 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.171+0000 7fe155400640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1581028c0 0x7fe15819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:45:01.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.171+0000 7fe155e00640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1581016d0 0x7fe158199bc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe148035040 tx=0x7fe148033d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:01.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.172+0000 7fe14ea00640 1 -- 172.21.15.42:0/1111691102 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148037040 con 0x7fe1581016d0 2024-04-25T00:45:01.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.172+0000 7fe157200640 1 -- 172.21.15.42:0/1111691102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1581a1a70 con 0x7fe1581016d0 2024-04-25T00:45:01.174 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.172+0000 7fe14ea00640 1 -- 172.21.15.42:0/1111691102 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe148030850 con 0x7fe1581016d0 2024-04-25T00:45:01.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.172+0000 7fe14ea00640 1 -- 172.21.15.42:0/1111691102 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148004920 con 0x7fe1581016d0 2024-04-25T00:45:01.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.172+0000 7fe157200640 1 -- 172.21.15.42:0/1111691102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1581a1fb0 con 0x7fe1581016d0 2024-04-25T00:45:01.175 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.173+0000 7fe157200640 1 -- 172.21.15.42:0/1111691102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1581a21e0 con 0x7fe1581016d0 2024-04-25T00:45:01.176 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.175+0000 7fe14ea00640 1 -- 172.21.15.42:0/1111691102 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe148004050 con 0x7fe1581016d0 2024-04-25T00:45:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.176+0000 7fe14ea00640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe13c0a2b90 0x7fe13c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.176+0000 7fe14ea00640 1 -- 172.21.15.42:0/1111691102 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fe1480eeec0 con 0x7fe1581016d0 2024-04-25T00:45:01.177 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.176+0000 7fe155400640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe13c0a2b90 0x7fe13c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:01.178 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.177+0000 7fe155400640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe13c0a2b90 0x7fe13c0a5050 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe15819b0d0 tx=0x7fe144005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:01.181 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.180+0000 7fe14ea00640 1 -- 172.21.15.42:0/1111691102 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fe1480bbdf0 con 0x7fe1581016d0 2024-04-25T00:45:01.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.382+0000 7fe157200640 1 -- 172.21.15.42:0/1111691102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe1581a23b0 con 0x7fe1581016d0 2024-04-25T00:45:01.383 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.382+0000 7fe14ea00640 1 -- 172.21.15.42:0/1111691102 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fe1480bb6f0 con 0x7fe1581016d0 2024-04-25T00:45:01.384 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 1 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 -- 172.21.15.42:0/1111691102 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe13c0a2b90 msgr2=0x7fe13c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe13c0a2b90 0x7fe13c0a5050 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe15819b0d0 tx=0x7fe144005f90 comp rx=0 tx=0).stop 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 -- 172.21.15.42:0/1111691102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1581016d0 msgr2=0x7fe158199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1581016d0 0x7fe158199bc0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe148035040 tx=0x7fe148033d90 comp rx=0 tx=0).stop 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 -- 172.21.15.42:0/1111691102 shutdown_connections 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fe13c0a2b90 0x7fe13c0a5050 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fe158103280 0x7fe1581a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1581028c0 0x7fe15819a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 --2- 172.21.15.42:0/1111691102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1581016d0 0x7fe158199bc0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 -- 172.21.15.42:0/1111691102 >> 172.21.15.42:0/1111691102 conn(0x7fe1580fcc40 msgr2=0x7fe158073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 -- 172.21.15.42:0/1111691102 shutdown_connections 2024-04-25T00:45:01.387 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.385+0000 7fe157200640 1 -- 172.21.15.42:0/1111691102 wait complete. 2024-04-25T00:45:01.403 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T00:45:01.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.587+0000 7f555d800640 1 -- 172.21.15.42:0/363073210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55581016d0 msgr2=0x7f5558101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:01.589 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.587+0000 7f555d800640 1 --2- 172.21.15.42:0/363073210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55581016d0 0x7f5558101af0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5548009900 tx=0x7f554802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.588+0000 7f555d800640 1 -- 172.21.15.42:0/363073210 shutdown_connections 2024-04-25T00:45:01.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.588+0000 7f555d800640 1 --2- 172.21.15.42:0/363073210 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5558103280 0x7f5558109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.588+0000 7f555d800640 1 --2- 172.21.15.42:0/363073210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55581028c0 0x7f5558102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.588+0000 7f555d800640 1 --2- 172.21.15.42:0/363073210 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f55581016d0 0x7f5558101af0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.588+0000 7f555d800640 1 -- 172.21.15.42:0/363073210 >> 172.21.15.42:0/363073210 conn(0x7f55580fcc40 msgr2=0x7f55580ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:01.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.588+0000 7f555d800640 1 -- 172.21.15.42:0/363073210 shutdown_connections 2024-04-25T00:45:01.590 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.588+0000 7f555d800640 1 -- 172.21.15.42:0/363073210 wait complete. 2024-04-25T00:45:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.589+0000 7f555d800640 1 Processor -- start 2024-04-25T00:45:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.589+0000 7f555d800640 1 -- start start 2024-04-25T00:45:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.589+0000 7f555d800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55581016d0 0x7f5558199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.589+0000 7f555d800640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55581028c0 0x7f555819a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.589+0000 7f555d800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5558103280 0x7f55581a1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.589+0000 7f555d800640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f555810d330 con 0x7f5558103280 2024-04-25T00:45:01.592 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.589+0000 7f555d800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f555810d1b0 con 0x7f55581016d0 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.589+0000 7f555d800640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f555810d4b0 con 0x7f55581028c0 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f5557e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55581016d0 0x7f5558199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f555ce00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5558103280 0x7f55581a1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f5557e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55581016d0 0x7f5558199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59314/0 (socket says 172.21.15.42:59314) 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f5557e00640 1 -- 172.21.15.42:0/2815832782 learned_addr learned my addr 172.21.15.42:0/2815832782 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f554fe00640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55581028c0 0x7f555819a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f5557e00640 1 -- 172.21.15.42:0/2815832782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55581028c0 msgr2=0x7f555819a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f5557e00640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55581028c0 0x7f555819a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f5557e00640 1 -- 172.21.15.42:0/2815832782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5558103280 msgr2=0x7f55581a1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f5557e00640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5558103280 0x7f55581a1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f5557e00640 1 -- 172.21.15.42:0/2815832782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55581a1780 con 0x7f55581016d0 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.590+0000 7f555ce00640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5558103280 0x7f55581a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.591+0000 7f5557e00640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55581016d0 0x7f5558199b60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5548031440 tx=0x7f5548032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:01.593 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.591+0000 7f5555600640 1 -- 172.21.15.42:0/2815832782 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f554803f030 con 0x7f55581016d0 2024-04-25T00:45:01.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.591+0000 7f5555600640 1 -- 172.21.15.42:0/2815832782 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5548003ce0 con 0x7f55581016d0 2024-04-25T00:45:01.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.591+0000 7f5555600640 1 -- 172.21.15.42:0/2815832782 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5548035070 con 0x7f55581016d0 2024-04-25T00:45:01.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.591+0000 7f555d800640 1 -- 172.21.15.42:0/2815832782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55581a1a10 con 0x7f55581016d0 2024-04-25T00:45:01.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.591+0000 7f555d800640 1 -- 172.21.15.42:0/2815832782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55581a1f50 con 0x7f55581016d0 2024-04-25T00:45:01.594 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.592+0000 7f555d800640 1 -- 172.21.15.42:0/2815832782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55581a2180 con 0x7f55581016d0 2024-04-25T00:45:01.595 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.594+0000 7f5555600640 1 -- 172.21.15.42:0/2815832782 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5548042030 con 0x7f55581016d0 2024-04-25T00:45:01.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.595+0000 7f5555600640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f553c0a2c10 0x7f553c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:01.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.596+0000 7f5555600640 1 -- 172.21.15.42:0/2815832782 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f55480ef420 con 0x7f55581016d0 2024-04-25T00:45:01.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.596+0000 7f554fe00640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f553c0a2c10 0x7f553c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:01.597 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.596+0000 7f554fe00640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f553c0a2c10 0x7f553c0a50d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f555819b070 tx=0x7f5540005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:01.601 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.599+0000 7f5555600640 1 -- 172.21.15.42:0/2815832782 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f55480bbd00 con 0x7f55581016d0 2024-04-25T00:45:01.804 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.803+0000 7f555d800640 1 -- 172.21.15.42:0/2815832782 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f55581a2350 con 0x7f55581016d0 2024-04-25T00:45:01.805 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.804+0000 7f5555600640 1 -- 172.21.15.42:0/2815832782 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f55480c0000 con 0x7f55581016d0 2024-04-25T00:45:01.805 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 32 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.806+0000 7f555d800640 1 -- 172.21.15.42:0/2815832782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f553c0a2c10 msgr2=0x7f553c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.806+0000 7f555d800640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f553c0a2c10 0x7f553c0a50d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f555819b070 tx=0x7f5540005f90 comp rx=0 tx=0).stop 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.806+0000 7f555d800640 1 -- 172.21.15.42:0/2815832782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55581016d0 msgr2=0x7f5558199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.806+0000 7f555d800640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55581016d0 0x7f5558199b60 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5548031440 tx=0x7f5548032ce0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.807+0000 7f555d800640 1 -- 172.21.15.42:0/2815832782 shutdown_connections 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.807+0000 7f555d800640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f553c0a2c10 0x7f553c0a50d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.807+0000 7f555d800640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f5558103280 0x7f55581a1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.807+0000 7f555d800640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55581028c0 0x7f555819a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.807+0000 7f555d800640 1 --2- 172.21.15.42:0/2815832782 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55581016d0 0x7f5558199b60 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.807+0000 7f555d800640 1 -- 172.21.15.42:0/2815832782 >> 172.21.15.42:0/2815832782 conn(0x7f55580fcc40 msgr2=0x7f5558073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:01.808 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.807+0000 7f555d800640 1 -- 172.21.15.42:0/2815832782 shutdown_connections 2024-04-25T00:45:01.809 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:01.807+0000 7f555d800640 1 -- 172.21.15.42:0/2815832782 wait complete. 2024-04-25T00:45:01.827 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T00:45:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:01 smithi042 ceph-mon[28130]: pgmap v1178: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T00:45:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:01 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/4244126398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:45:01.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:01 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1111691102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:45:02.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.021+0000 7f18e1200640 1 -- 172.21.15.42:0/2034897366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18dc1016d0 msgr2=0x7f18dc101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:02.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.021+0000 7f18e1200640 1 --2- 172.21.15.42:0/2034897366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18dc1016d0 0x7f18dc101af0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f18cc009ca0 tx=0x7f18cc02a8c0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.021+0000 7f18e1200640 1 -- 172.21.15.42:0/2034897366 shutdown_connections 2024-04-25T00:45:02.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.021+0000 7f18e1200640 1 --2- 172.21.15.42:0/2034897366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18dc103280 0x7f18dc109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.021+0000 7f18e1200640 1 --2- 172.21.15.42:0/2034897366 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18dc1028c0 0x7f18dc102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.021+0000 7f18e1200640 1 --2- 172.21.15.42:0/2034897366 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18dc1016d0 0x7f18dc101af0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.021+0000 7f18e1200640 1 -- 172.21.15.42:0/2034897366 >> 172.21.15.42:0/2034897366 conn(0x7f18dc0fcc40 msgr2=0x7f18dc0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:02.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.021+0000 7f18e1200640 1 -- 172.21.15.42:0/2034897366 shutdown_connections 2024-04-25T00:45:02.023 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.021+0000 7f18e1200640 1 -- 172.21.15.42:0/2034897366 wait complete. 2024-04-25T00:45:02.025 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.022+0000 7f18e1200640 1 Processor -- start 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.022+0000 7f18e1200640 1 -- start start 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18e1200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18dc1016d0 0x7f18dc0754f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18e1200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18dc1028c0 0x7f18dc073b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18e1200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18dc103280 0x7f18dc074070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18e1200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc10d330 con 0x7f18dc1028c0 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18e1200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc10d1b0 con 0x7f18dc1016d0 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18e1200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18dc10d4b0 con 0x7f18dc103280 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18daa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18dc1028c0 0x7f18dc073b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18daa00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18dc1028c0 0x7f18dc073b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:60982/0 (socket says 172.21.15.42:60982) 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18daa00640 1 -- 172.21.15.42:0/2918524368 learned_addr learned my addr 172.21.15.42:0/2918524368 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18db400640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18dc1016d0 0x7f18dc0754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18dbe00640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18dc103280 0x7f18dc074070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:02.026 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18daa00640 1 -- 172.21.15.42:0/2918524368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18dc103280 msgr2=0x7f18dc074070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.023+0000 7f18daa00640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18dc103280 0x7f18dc074070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.024+0000 7f18daa00640 1 -- 172.21.15.42:0/2918524368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18dc1016d0 msgr2=0x7f18dc0754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.024+0000 7f18daa00640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18dc1016d0 0x7f18dc0754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.024+0000 7f18daa00640 1 -- 172.21.15.42:0/2918524368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18dc074900 con 0x7f18dc1028c0 2024-04-25T00:45:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.024+0000 7f18daa00640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18dc1028c0 0x7f18dc073b30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9f0 tx=0x7f18c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.024+0000 7f18d3e00640 1 -- 172.21.15.42:0/2918524368 <== mon.0 v2:172.21.15.42:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c800cd80 con 0x7f18dc1028c0 2024-04-25T00:45:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.024+0000 7f18d3e00640 1 -- 172.21.15.42:0/2918524368 <== mon.0 v2:172.21.15.42:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c8010910 con 0x7f18dc1028c0 2024-04-25T00:45:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.024+0000 7f18e1200640 1 -- 172.21.15.42:0/2918524368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18dc1a1a60 con 0x7f18dc1028c0 2024-04-25T00:45:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.024+0000 7f18d3e00640 1 -- 172.21.15.42:0/2918524368 <== mon.0 v2:172.21.15.42:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c8018990 con 0x7f18dc1028c0 2024-04-25T00:45:02.027 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.024+0000 7f18e1200640 1 -- 172.21.15.42:0/2918524368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18dc1a1fb0 con 0x7f18dc1028c0 2024-04-25T00:45:02.028 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.026+0000 7f18e1200640 1 -- 172.21.15.42:0/2918524368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18dc1089f0 con 0x7f18dc1028c0 2024-04-25T00:45:02.029 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.028+0000 7f18d3e00640 1 -- 172.21.15.42:0/2918524368 <== mon.0 v2:172.21.15.42:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f18c8010ac0 con 0x7f18dc1028c0 2024-04-25T00:45:02.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.029+0000 7f18d3e00640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18c00a2b90 0x7f18c00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:02.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.030+0000 7f18d3e00640 1 -- 172.21.15.42:0/2918524368 <== mon.0 v2:172.21.15.42:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f18c80cf420 con 0x7f18dc1028c0 2024-04-25T00:45:02.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.030+0000 7f18db400640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18c00a2b90 0x7f18c00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:02.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.031+0000 7f18db400640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18c00a2b90 0x7f18c00a5050 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f18cc000f80 tx=0x7f18cc005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:02.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.034+0000 7f18d3e00640 1 -- 172.21.15.42:0/2918524368 <== mon.0 v2:172.21.15.42:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f18c809c350 con 0x7f18dc1028c0 2024-04-25T00:45:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:01 smithi106 ceph-mon[32510]: pgmap v1178: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T00:45:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:01 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/4244126398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:45:02.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:01 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1111691102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:45:02.261 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.260+0000 7f18e1200640 1 -- 172.21.15.42:0/2918524368 --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f18dc062340 con 0x7f18dc1028c0 2024-04-25T00:45:02.262 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.261+0000 7f18d3e00640 1 -- 172.21.15.42:0/2918524368 <== mon.0 v2:172.21.15.42:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f18c809bc50 con 0x7f18dc1028c0 2024-04-25T00:45:02.262 INFO:teuthology.orchestra.run.smithi042.stdout:pg_num: 64 2024-04-25T00:45:02.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:01 smithi136 ceph-mon[32923]: pgmap v1178: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-04-25T00:45:02.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:01 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/4244126398' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:45:02.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:01 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1111691102' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T00:45:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.263+0000 7f18e1200640 1 -- 172.21.15.42:0/2918524368 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18c00a2b90 msgr2=0x7f18c00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.263+0000 7f18e1200640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18c00a2b90 0x7f18c00a5050 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f18cc000f80 tx=0x7f18cc005f30 comp rx=0 tx=0).stop 2024-04-25T00:45:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.263+0000 7f18e1200640 1 -- 172.21.15.42:0/2918524368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18dc1028c0 msgr2=0x7f18dc073b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.263+0000 7f18e1200640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18dc1028c0 0x7f18dc073b30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f18c800e9f0 tx=0x7f18c800eeb0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.263+0000 7f18e1200640 1 -- 172.21.15.42:0/2918524368 shutdown_connections 2024-04-25T00:45:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.263+0000 7f18e1200640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f18c00a2b90 0x7f18c00a5050 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.263+0000 7f18e1200640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f18dc103280 0x7f18dc074070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.263+0000 7f18e1200640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f18dc1028c0 0x7f18dc073b30 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.263+0000 7f18e1200640 1 --2- 172.21.15.42:0/2918524368 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18dc1016d0 0x7f18dc0754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.265 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.264+0000 7f18e1200640 1 -- 172.21.15.42:0/2918524368 >> 172.21.15.42:0/2918524368 conn(0x7f18dc0fcc40 msgr2=0x7f18dc105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.264+0000 7f18e1200640 1 -- 172.21.15.42:0/2918524368 shutdown_connections 2024-04-25T00:45:02.266 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.264+0000 7f18e1200640 1 -- 172.21.15.42:0/2918524368 wait complete. 2024-04-25T00:45:02.285 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T00:45:02.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.472+0000 7ff4ae200640 1 -- 172.21.15.42:0/3380164733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a8101740 msgr2=0x7ff4a8101bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:02.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.472+0000 7ff4ae200640 1 --2- 172.21.15.42:0/3380164733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a8101740 0x7ff4a8101bc0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7ff494009900 tx=0x7ff49402a410 comp rx=0 tx=0).stop 2024-04-25T00:45:02.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.472+0000 7ff4ae200640 1 -- 172.21.15.42:0/3380164733 shutdown_connections 2024-04-25T00:45:02.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.472+0000 7ff4ae200640 1 --2- 172.21.15.42:0/3380164733 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4a8102100 0x7ff4a8108990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.472+0000 7ff4ae200640 1 --2- 172.21.15.42:0/3380164733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a8101740 0x7ff4a8101bc0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.472+0000 7ff4ae200640 1 --2- 172.21.15.42:0/3380164733 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8100550 0x7ff4a8100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.472+0000 7ff4ae200640 1 -- 172.21.15.42:0/3380164733 >> 172.21.15.42:0/3380164733 conn(0x7ff4a80fbae0 msgr2=0x7ff4a80fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:02.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.472+0000 7ff4ae200640 1 -- 172.21.15.42:0/3380164733 shutdown_connections 2024-04-25T00:45:02.474 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.472+0000 7ff4ae200640 1 -- 172.21.15.42:0/3380164733 wait complete. 2024-04-25T00:45:02.475 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.473+0000 7ff4ae200640 1 Processor -- start 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.473+0000 7ff4ae200640 1 -- start start 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4ae200640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8100550 0x7ff4a8199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4ae200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a8101740 0x7ff4a819a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4ae200640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4a8102100 0x7ff4a81a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4ae200640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a810c230 con 0x7ff4a8102100 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4ae200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a810c0b0 con 0x7ff4a8101740 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4ae200640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a810c3b0 con 0x7ff4a8100550 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a8101740 0x7ff4a819a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4ace00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8100550 0x7ff4a8199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4ad800640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4a8102100 0x7ff4a81a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4a7e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a8101740 0x7ff4a819a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59350/0 (socket says 172.21.15.42:59350) 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4a7e00640 1 -- 172.21.15.42:0/2374925555 learned_addr learned my addr 172.21.15.42:0/2374925555 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:02.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.474+0000 7ff4ace00640 1 -- 172.21.15.42:0/2374925555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a8101740 msgr2=0x7ff4a819a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:02.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4ace00640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a8101740 0x7ff4a819a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4ace00640 1 -- 172.21.15.42:0/2374925555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4a8102100 msgr2=0x7ff4a81a10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:02.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4ace00640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4a8102100 0x7ff4a81a10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4ace00640 1 -- 172.21.15.42:0/2374925555 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a81a1760 con 0x7ff4a8100550 2024-04-25T00:45:02.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4a7e00640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a8101740 0x7ff4a819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:45:02.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4ace00640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8100550 0x7ff4a8199b40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00e9c0 tx=0x7ff49c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:02.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4a5600640 1 -- 172.21.15.42:0/2374925555 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00ccc0 con 0x7ff4a8100550 2024-04-25T00:45:02.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4a5600640 1 -- 172.21.15.42:0/2374925555 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff49c0035f0 con 0x7ff4a8100550 2024-04-25T00:45:02.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4ae200640 1 -- 172.21.15.42:0/2374925555 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a81a1a50 con 0x7ff4a8100550 2024-04-25T00:45:02.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4a5600640 1 -- 172.21.15.42:0/2374925555 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff49c00ce60 con 0x7ff4a8100550 2024-04-25T00:45:02.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.475+0000 7ff4ae200640 1 -- 172.21.15.42:0/2374925555 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a81a1fa0 con 0x7ff4a8100550 2024-04-25T00:45:02.478 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.476+0000 7ff4ae200640 1 -- 172.21.15.42:0/2374925555 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a81a2230 con 0x7ff4a8100550 2024-04-25T00:45:02.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.479+0000 7ff4a5600640 1 -- 172.21.15.42:0/2374925555 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff49c0037a0 con 0x7ff4a8100550 2024-04-25T00:45:02.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.480+0000 7ff4a5600640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4900a2b90 0x7ff4900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:02.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.480+0000 7ff4a5600640 1 -- 172.21.15.42:0/2374925555 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7ff49c0cf480 con 0x7ff4a8100550 2024-04-25T00:45:02.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.480+0000 7ff4a7e00640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4900a2b90 0x7ff4900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:02.482 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.481+0000 7ff4a7e00640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4900a2b90 0x7ff4900a5050 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff4a819b050 tx=0x7ff4940315b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:02.486 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.485+0000 7ff4a5600640 1 -- 172.21.15.42:0/2374925555 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7ff49c09c3b0 con 0x7ff4a8100550 2024-04-25T00:45:02.707 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.706+0000 7ff4ae200640 1 -- 172.21.15.42:0/2374925555 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff4a81a2400 con 0x7ff4a8100550 2024-04-25T00:45:02.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.707+0000 7ff4a5600640 1 -- 172.21.15.42:0/2374925555 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v91) ==== 81+0+588 (secure 0 0 0) 0x7ff49c09bcb0 con 0x7ff4a8100550 2024-04-25T00:45:02.708 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1811796714 2024-04-26T00:08:25.763656+0000 2024-04-25T00:45:02.708 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/3681869279 2024-04-26T00:08:25.763656+0000 2024-04-25T00:45:02.708 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/3681869279 2024-04-26T00:08:25.763656+0000 2024-04-25T00:45:02.708 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/3081484237 2024-04-26T00:08:25.763656+0000 2024-04-25T00:45:02.709 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/4222574490 2024-04-26T00:07:50.514990+0000 2024-04-25T00:45:02.709 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/15970351 2024-04-26T00:07:50.514990+0000 2024-04-25T00:45:02.709 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1603265964 2024-04-26T00:07:50.514990+0000 2024-04-25T00:45:02.709 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6800/174637222 2024-04-26T00:07:50.514990+0000 2024-04-25T00:45:02.709 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:6801/174637222 2024-04-26T00:07:50.514990+0000 2024-04-25T00:45:02.709 INFO:teuthology.orchestra.run.smithi042.stdout:172.21.15.42:0/1839487361 2024-04-26T00:08:25.763656+0000 2024-04-25T00:45:02.709 INFO:teuthology.orchestra.run.smithi042.stderr:listed 10 entries 2024-04-25T00:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.709+0000 7ff4ae200640 1 -- 172.21.15.42:0/2374925555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4900a2b90 msgr2=0x7ff4900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.709+0000 7ff4ae200640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4900a2b90 0x7ff4900a5050 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7ff4a819b050 tx=0x7ff4940315b0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.709+0000 7ff4ae200640 1 -- 172.21.15.42:0/2374925555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8100550 msgr2=0x7ff4a8199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.709+0000 7ff4ae200640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8100550 0x7ff4a8199b40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff49c00e9c0 tx=0x7ff49c00ee80 comp rx=0 tx=0).stop 2024-04-25T00:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.709+0000 7ff4ae200640 1 -- 172.21.15.42:0/2374925555 shutdown_connections 2024-04-25T00:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.709+0000 7ff4ae200640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7ff4900a2b90 0x7ff4900a5050 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.709+0000 7ff4ae200640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7ff4a8102100 0x7ff4a81a10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.709+0000 7ff4ae200640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4a8101740 0x7ff4a819a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.711 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.709+0000 7ff4ae200640 1 --2- 172.21.15.42:0/2374925555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8100550 0x7ff4a8199b40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.709+0000 7ff4ae200640 1 -- 172.21.15.42:0/2374925555 >> 172.21.15.42:0/2374925555 conn(0x7ff4a80fbae0 msgr2=0x7ff4a80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.710+0000 7ff4ae200640 1 -- 172.21.15.42:0/2374925555 shutdown_connections 2024-04-25T00:45:02.712 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:02.710+0000 7ff4ae200640 1 -- 172.21.15.42:0/2374925555 wait complete. 2024-04-25T00:45:02.730 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T00:45:02.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:45:02.730 DEBUG:teuthology.orchestra.run.smithi042:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T00:45:02.807 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi042.front.sepia.ceph.com 2024-04-25T00:45:02.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:45:02.807 DEBUG:teuthology.orchestra.run.smithi042:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T00:45:03.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:45:03.011 DEBUG:teuthology.orchestra.run.smithi042:> ip netns list 2024-04-25T00:45:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:02 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2815832782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:45:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:02 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2918524368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:45:03.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:02 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2374925555' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:45:03.096 INFO:teuthology.orchestra.run.smithi042.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T00:45:03.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:45:03.097 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T00:45:03.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:02 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2815832782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:45:03.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:02 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2918524368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:45:03.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:02 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2374925555' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:45:03.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T00:45:03.232 DEBUG:teuthology.orchestra.run.smithi042:> sudo ip link delete ceph-brx 2024-04-25T00:45:03.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:02 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2815832782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T00:45:03.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:02 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2918524368' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T00:45:03.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:02 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2374925555' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T00:45:03.439 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T00:45:03.452 INFO:tasks.cephadm:Teardown begin 2024-04-25T00:45:03.453 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T00:45:03.453 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:45:03.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.750+0000 7f82afe00640 1 -- 172.21.15.42:0/4220732072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82a81039c0 msgr2=0x7f82a8103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:03.751 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.750+0000 7f82afe00640 1 --2- 172.21.15.42:0/4220732072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82a81039c0 0x7f82a8103de0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8294009900 tx=0x7f829402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:45:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.752+0000 7f82afe00640 1 -- 172.21.15.42:0/4220732072 shutdown_connections 2024-04-25T00:45:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.752+0000 7f82afe00640 1 --2- 172.21.15.42:0/4220732072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a8105570 0x7f82a810be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.752+0000 7f82afe00640 1 --2- 172.21.15.42:0/4220732072 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82a8104bb0 0x7f82a8105030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.752+0000 7f82afe00640 1 --2- 172.21.15.42:0/4220732072 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82a81039c0 0x7f82a8103de0 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8294009900 tx=0x7f829402a3f0 comp rx=0 tx=0).stop 2024-04-25T00:45:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.752+0000 7f82afe00640 1 -- 172.21.15.42:0/4220732072 >> 172.21.15.42:0/4220732072 conn(0x7f82a80fef30 msgr2=0x7f82a8101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.752+0000 7f82afe00640 1 -- 172.21.15.42:0/4220732072 shutdown_connections 2024-04-25T00:45:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.752+0000 7f82afe00640 1 -- 172.21.15.42:0/4220732072 wait complete. 2024-04-25T00:45:03.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.754+0000 7f82afe00640 1 Processor -- start 2024-04-25T00:45:03.755 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.754+0000 7f82afe00640 1 -- start start 2024-04-25T00:45:03.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.754+0000 7f82afe00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a8104bb0 0x7f82a8199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:03.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.755+0000 7f82afe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82a8105570 0x7f82a819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:03.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.755+0000 7f82afe00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82a819f420 0x7f82a81a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:03.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.755+0000 7f82afe00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f82a810f660 con 0x7f82a8105570 2024-04-25T00:45:03.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.755+0000 7f82afe00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f82a810f4e0 con 0x7f82a8104bb0 2024-04-25T00:45:03.756 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.755+0000 7f82afe00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f82a810f7e0 con 0x7f82a819f420 2024-04-25T00:45:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.755+0000 7f82ad400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a8104bb0 0x7f82a8199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.755+0000 7f82ad400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a8104bb0 0x7f82a8199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59370/0 (socket says 172.21.15.42:59370) 2024-04-25T00:45:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.755+0000 7f82ad400640 1 -- 172.21.15.42:0/300752708 learned_addr learned my addr 172.21.15.42:0/300752708 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.756+0000 7f82ad400640 1 -- 172.21.15.42:0/300752708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82a819f420 msgr2=0x7f82a81a1810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T00:45:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.756+0000 7f82ad400640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82a819f420 0x7f82a81a1810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.756+0000 7f82ad400640 1 -- 172.21.15.42:0/300752708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82a8105570 msgr2=0x7f82a819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.756+0000 7f82ad400640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82a8105570 0x7f82a819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.756+0000 7f82ad400640 1 -- 172.21.15.42:0/300752708 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82a81a1e80 con 0x7f82a8104bb0 2024-04-25T00:45:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.756+0000 7f82ad400640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a8104bb0 0x7f82a8199e80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8294009850 tx=0x7f8294032bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.756+0000 7f82a6000640 1 -- 172.21.15.42:0/300752708 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f829403f030 con 0x7f82a8104bb0 2024-04-25T00:45:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.757+0000 7f82a6000640 1 -- 172.21.15.42:0/300752708 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8294003ad0 con 0x7f82a8104bb0 2024-04-25T00:45:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.757+0000 7f82afe00640 1 -- 172.21.15.42:0/300752708 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82a81a2110 con 0x7f82a8104bb0 2024-04-25T00:45:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.757+0000 7f82a6000640 1 -- 172.21.15.42:0/300752708 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8294035070 con 0x7f82a8104bb0 2024-04-25T00:45:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.757+0000 7f82afe00640 1 -- 172.21.15.42:0/300752708 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82a81a2620 con 0x7f82a8104bb0 2024-04-25T00:45:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.759+0000 7f82afe00640 1 -- 172.21.15.42:0/300752708 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82a810ad30 con 0x7f82a8104bb0 2024-04-25T00:45:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.760+0000 7f82a6000640 1 -- 172.21.15.42:0/300752708 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8294042030 con 0x7f82a8104bb0 2024-04-25T00:45:03.764 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.762+0000 7f82a6000640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82900a2c60 0x7f82900a5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:03.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.763+0000 7f82a6000640 1 -- 172.21.15.42:0/300752708 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f82940ef600 con 0x7f82a8104bb0 2024-04-25T00:45:03.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.763+0000 7f82aca00640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82900a2c60 0x7f82900a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:03.765 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.764+0000 7f82aca00640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82900a2c60 0x7f82900a5120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f829c0058c0 tx=0x7f829c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:03.767 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:03.766+0000 7f82a6000640 1 -- 172.21.15.42:0/300752708 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f82940bc590 con 0x7f82a8104bb0 2024-04-25T00:45:03.876 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:03 smithi042 ceph-mon[28130]: pgmap v1179: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-04-25T00:45:04.030 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.029+0000 7f82afe00640 1 -- 172.21.15.42:0/300752708 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82a8062340 con 0x7f82a8104bb0 2024-04-25T00:45:04.031 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.030+0000 7f82a6000640 1 -- 172.21.15.42:0/300752708 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11876 (secure 0 0 0) 0x7f82940bbe90 con 0x7f82a8104bb0 2024-04-25T00:45:04.031 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:45:04.031 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:16:19.771762+0000","modified":"2024-04-25T00:44:17.357094+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24568,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24568":{"gid":24568,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6841/4068466248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4068466248},{"type":"v1","addr":"172.21.15.42:6841","nonce":4068466248}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3349564106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3349564106},{"type":"v1","addr":"172.21.15.106:6839","nonce":3349564106}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26428":{"gid":26428,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6839/2386071853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2386071853},{"type":"v1","addr":"172.21.15.136:6839","nonce":2386071853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24568,24569,24517]},"id":1}]} 2024-04-25T00:45:04.032 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 49 2024-04-25T00:45:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 -- 172.21.15.42:0/300752708 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82900a2c60 msgr2=0x7f82900a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82900a2c60 0x7f82900a5120 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f829c0058c0 tx=0x7f829c005850 comp rx=0 tx=0).stop 2024-04-25T00:45:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 -- 172.21.15.42:0/300752708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a8104bb0 msgr2=0x7f82a8199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a8104bb0 0x7f82a8199e80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8294009850 tx=0x7f8294032bf0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 -- 172.21.15.42:0/300752708 shutdown_connections 2024-04-25T00:45:04.034 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f82900a2c60 0x7f82900a5120 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f82a819f420 0x7f82a81a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f82a8105570 0x7f82a819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 --2- 172.21.15.42:0/300752708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a8104bb0 0x7f82a8199e80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 -- 172.21.15.42:0/300752708 >> 172.21.15.42:0/300752708 conn(0x7f82a80fef30 msgr2=0x7f82a8107f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 -- 172.21.15.42:0/300752708 shutdown_connections 2024-04-25T00:45:04.035 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.033+0000 7f82afe00640 1 -- 172.21.15.42:0/300752708 wait complete. 2024-04-25T00:45:04.052 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T00:45:04.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:03 smithi106 ceph-mon[32510]: pgmap v1179: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-04-25T00:45:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.223+0000 7fab4b400640 1 -- 172.21.15.42:0/892851041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44102200 msgr2=0x7fab441045f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.223+0000 7fab4b400640 1 --2- 172.21.15.42:0/892851041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44102200 0x7fab441045f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fab38009860 tx=0x7fab3802a340 comp rx=0 tx=0).stop 2024-04-25T00:45:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.223+0000 7fab4b400640 1 -- 172.21.15.42:0/892851041 shutdown_connections 2024-04-25T00:45:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.223+0000 7fab4b400640 1 --2- 172.21.15.42:0/892851041 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab44107460 0x7fab441098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.225 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.223+0000 7fab4b400640 1 --2- 172.21.15.42:0/892851041 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab44104b30 0x7fab44106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.223+0000 7fab4b400640 1 --2- 172.21.15.42:0/892851041 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44102200 0x7fab441045f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.223+0000 7fab4b400640 1 -- 172.21.15.42:0/892851041 >> 172.21.15.42:0/892851041 conn(0x7fab440fbc30 msgr2=0x7fab440fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:04.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.224+0000 7fab4b400640 1 -- 172.21.15.42:0/892851041 shutdown_connections 2024-04-25T00:45:04.226 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.224+0000 7fab4b400640 1 -- 172.21.15.42:0/892851041 wait complete. 2024-04-25T00:45:04.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.225+0000 7fab4b400640 1 Processor -- start 2024-04-25T00:45:04.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.225+0000 7fab4b400640 1 -- start start 2024-04-25T00:45:04.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.225+0000 7fab4b400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab44102200 0x7fab44197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:04.227 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.225+0000 7fab4b400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44104b30 0x7fab44198000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.225+0000 7fab4b400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab44107460 0x7fab4419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.225+0000 7fab4b400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7fab4410d420 con 0x7fab44107460 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.225+0000 7fab4b400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7fab4410d2a0 con 0x7fab44102200 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.225+0000 7fab4b400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab4410d5a0 con 0x7fab44104b30 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab43e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44104b30 0x7fab44198000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab43e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44104b30 0x7fab44198000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:55186/0 (socket says 172.21.15.42:55186) 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab43e00640 1 -- 172.21.15.42:0/4189574387 learned_addr learned my addr 172.21.15.42:0/4189574387 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab49400640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab44107460 0x7fab4419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab48a00640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab44102200 0x7fab44197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab43e00640 1 -- 172.21.15.42:0/4189574387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab44102200 msgr2=0x7fab44197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab43e00640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab44102200 0x7fab44197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab43e00640 1 -- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab44107460 msgr2=0x7fab4419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab43e00640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab44107460 0x7fab4419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.228 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab43e00640 1 -- 172.21.15.42:0/4189574387 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4419f6e0 con 0x7fab44104b30 2024-04-25T00:45:04.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab48a00640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab44102200 0x7fab44197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:45:04.229 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.226+0000 7fab49400640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab44107460 0x7fab4419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:45:04.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.227+0000 7fab43e00640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44104b30 0x7fab44198000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fab3000e9a0 tx=0x7fab3000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:04.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.227+0000 7fab41600640 1 -- 172.21.15.42:0/4189574387 <== mon.1 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab3000cd80 con 0x7fab44104b30 2024-04-25T00:45:04.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.227+0000 7fab41600640 1 -- 172.21.15.42:0/4189574387 <== mon.1 v2:172.21.15.136:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab300035f0 con 0x7fab44104b30 2024-04-25T00:45:04.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.227+0000 7fab41600640 1 -- 172.21.15.42:0/4189574387 <== mon.1 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30010670 con 0x7fab44104b30 2024-04-25T00:45:04.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.227+0000 7fab4b400640 1 -- 172.21.15.42:0/4189574387 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab4419f9d0 con 0x7fab44104b30 2024-04-25T00:45:04.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.227+0000 7fab4b400640 1 -- 172.21.15.42:0/4189574387 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab4419fed0 con 0x7fab44104b30 2024-04-25T00:45:04.230 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.229+0000 7fab4b400640 1 -- 172.21.15.42:0/4189574387 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab440fd720 con 0x7fab44104b30 2024-04-25T00:45:04.231 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.230+0000 7fab41600640 1 -- 172.21.15.42:0/4189574387 <== mon.1 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fab300037a0 con 0x7fab44104b30 2024-04-25T00:45:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.232+0000 7fab41600640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab2c0a2c10 0x7fab2c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.232+0000 7fab41600640 1 -- 172.21.15.42:0/4189574387 <== mon.1 v2:172.21.15.136:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7fab300cf260 con 0x7fab44104b30 2024-04-25T00:45:04.233 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.232+0000 7fab48a00640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab2c0a2c10 0x7fab2c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:04.234 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.233+0000 7fab48a00640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab2c0a2c10 0x7fab2c0a50d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fab38009830 tx=0x7fab38030550 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:04.236 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.235+0000 7fab41600640 1 -- 172.21.15.42:0/4189574387 <== mon.1 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7fab3009c190 con 0x7fab44104b30 2024-04-25T00:45:04.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:03 smithi136 ceph-mon[32923]: pgmap v1179: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 7 op/s 2024-04-25T00:45:04.476 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.475+0000 7fab4b400640 1 -- 172.21.15.42:0/4189574387 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab4402d170 con 0x7fab44104b30 2024-04-25T00:45:04.477 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.476+0000 7fab41600640 1 -- 172.21.15.42:0/4189574387 <== mon.1 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+11876 (secure 0 0 0) 0x7fab3009ba90 con 0x7fab44104b30 2024-04-25T00:45:04.478 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:45:04.478 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":49,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14385,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6835/1318501714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6834","nonce":1318501714},{"type":"v1","addr":"172.21.15.42:6835","nonce":1318501714}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14403,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6837/3589701125","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6836","nonce":3589701125},{"type":"v1","addr":"172.21.15.42:6837","nonce":3589701125}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14451,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.42:6839/3053633590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6838","nonce":3053633590},{"type":"v1","addr":"172.21.15.42:6839","nonce":3053633590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24442,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/756794270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":756794270},{"type":"v1","addr":"172.21.15.136:6833","nonce":756794270}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24449,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6833/1706914889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":1706914889},{"type":"v1","addr":"172.21.15.106:6833","nonce":1706914889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24494,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1507277236","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1507277236},{"type":"v1","addr":"172.21.15.106:6835","nonce":1507277236}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[{"mdsmap":{"epoch":49,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T00:16:19.771762+0000","modified":"2024-04-25T00:44:17.357094+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24569,"mds_1":24568,"mds_2":24517},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24505":{"gid":24505,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/12913503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":12913503},{"type":"v1","addr":"172.21.15.136:6835","nonce":12913503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24517":{"gid":24517,"name":"c","rank":2,"incarnation":21,"state":"up:active","state_seq":8,"addr":"172.21.15.136:6837/222488442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":222488442},{"type":"v1","addr":"172.21.15.136:6837","nonce":222488442}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24521":{"gid":24521,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.106:6837/1331229112","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1331229112},{"type":"v1","addr":"172.21.15.106:6837","nonce":1331229112}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24568":{"gid":24568,"name":"a","rank":1,"incarnation":19,"state":"up:active","state_seq":4,"addr":"172.21.15.42:6841/4068466248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6840","nonce":4068466248},{"type":"v1","addr":"172.21.15.42:6841","nonce":4068466248}]},"join_fscid":-1,"export_targets":[0,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24569":{"gid":24569,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6839/3349564106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6838","nonce":3349564106},{"type":"v1","addr":"172.21.15.106:6839","nonce":3349564106}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26428":{"gid":26428,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6839/2386071853","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2386071853},{"type":"v1","addr":"172.21.15.136:6839","nonce":2386071853}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24569,"qdb_cluster":[24568,24569,24517]},"id":1}]} 2024-04-25T00:45:04.478 INFO:teuthology.orchestra.run.smithi042.stderr:dumped fsmap epoch 49 2024-04-25T00:45:04.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 -- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab2c0a2c10 msgr2=0x7fab2c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab2c0a2c10 0x7fab2c0a50d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fab38009830 tx=0x7fab38030550 comp rx=0 tx=0).stop 2024-04-25T00:45:04.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 -- 172.21.15.42:0/4189574387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44104b30 msgr2=0x7fab44198000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44104b30 0x7fab44198000 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fab3000e9a0 tx=0x7fab3000ee60 comp rx=0 tx=0).stop 2024-04-25T00:45:04.480 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 -- 172.21.15.42:0/4189574387 shutdown_connections 2024-04-25T00:45:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7fab2c0a2c10 0x7fab2c0a50d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7fab44107460 0x7fab4419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab44104b30 0x7fab44198000 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 --2- 172.21.15.42:0/4189574387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fab44102200 0x7fab44197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 -- 172.21.15.42:0/4189574387 >> 172.21.15.42:0/4189574387 conn(0x7fab440fbc30 msgr2=0x7fab44102cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 -- 172.21.15.42:0/4189574387 shutdown_connections 2024-04-25T00:45:04.481 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.479+0000 7fab4b400640 1 -- 172.21.15.42:0/4189574387 wait complete. 2024-04-25T00:45:04.500 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T00:45:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 -- 172.21.15.42:0/3739892428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1464102270 msgr2=0x7f1464104660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 --2- 172.21.15.42:0/3739892428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1464102270 0x7f1464104660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f1454009ca0 tx=0x7f145402a8c0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 -- 172.21.15.42:0/3739892428 shutdown_connections 2024-04-25T00:45:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 --2- 172.21.15.42:0/3739892428 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14641074d0 0x7f14641098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 --2- 172.21.15.42:0/3739892428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1464104ba0 0x7f1464106f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.697 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 --2- 172.21.15.42:0/3739892428 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1464102270 0x7f1464104660 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 -- 172.21.15.42:0/3739892428 >> 172.21.15.42:0/3739892428 conn(0x7f14640fbc20 msgr2=0x7f14640fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 -- 172.21.15.42:0/3739892428 shutdown_connections 2024-04-25T00:45:04.698 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 -- 172.21.15.42:0/3739892428 wait complete. 2024-04-25T00:45:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 Processor -- start 2024-04-25T00:45:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 -- start start 2024-04-25T00:45:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1464102270 0x7f1464199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1464104ba0 0x7f146419a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14641074d0 0x7f14641a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f146410d480 con 0x7f1464104ba0 2024-04-25T00:45:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f146410d300 con 0x7f1464102270 2024-04-25T00:45:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.693+0000 7f1469600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f146410d600 con 0x7f14641074d0 2024-04-25T00:45:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f145a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1464104ba0 0x7f146419a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:04.699 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1463400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14641074d0 0x7f14641a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1462a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1464102270 0x7f1464199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1463400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14641074d0 0x7f14641a1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:55196/0 (socket says 172.21.15.42:55196) 2024-04-25T00:45:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f145a000640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1464104ba0 0x7f146419a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32846/0 (socket says 172.21.15.42:32846) 2024-04-25T00:45:04.700 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1463400640 1 -- 172.21.15.42:0/3698066846 learned_addr learned my addr 172.21.15.42:0/3698066846 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1462a00640 1 -- 172.21.15.42:0/3698066846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14641074d0 msgr2=0x7f14641a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1462a00640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14641074d0 0x7f14641a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1462a00640 1 -- 172.21.15.42:0/3698066846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1464104ba0 msgr2=0x7f146419a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1462a00640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1464104ba0 0x7f146419a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1462a00640 1 -- 172.21.15.42:0/3698066846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14641a1900 con 0x7f1464102270 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1462a00640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1464102270 0x7f1464199cf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f145402aeb0 tx=0x7f14540319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f145be00640 1 -- 172.21.15.42:0/3698066846 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1454003b70 con 0x7f1464102270 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f145be00640 1 -- 172.21.15.42:0/3698066846 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1454003d10 con 0x7f1464102270 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1469600640 1 -- 172.21.15.42:0/3698066846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14641a1b90 con 0x7f1464102270 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f145be00640 1 -- 172.21.15.42:0/3698066846 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145402fc00 con 0x7f1464102270 2024-04-25T00:45:04.701 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.698+0000 7f1469600640 1 -- 172.21.15.42:0/3698066846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14641a20b0 con 0x7f1464102270 2024-04-25T00:45:04.702 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.700+0000 7f1469600640 1 -- 172.21.15.42:0/3698066846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1464108fc0 con 0x7f1464102270 2024-04-25T00:45:04.703 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.700+0000 7f145be00640 1 -- 172.21.15.42:0/3698066846 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f14540046b0 con 0x7f1464102270 2024-04-25T00:45:04.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.700+0000 7f145be00640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14480a2b90 0x7f14480a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:04.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.700+0000 7f145be00640 1 -- 172.21.15.42:0/3698066846 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f14540f03d0 con 0x7f1464102270 2024-04-25T00:45:04.704 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.700+0000 7f145a000640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14480a2b90 0x7f14480a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:04.705 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.700+0000 7f145a000640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14480a2b90 0x7f14480a5050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f146406a8a0 tx=0x7f144c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:04.708 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.707+0000 7f145be00640 1 -- 172.21.15.42:0/3698066846 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f14540bd300 con 0x7f1464102270 2024-04-25T00:45:04.908 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.907+0000 7f1469600640 1 -- 172.21.15.42:0/3698066846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1464062340 con 0x7f1464102270 2024-04-25T00:45:04.910 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.909+0000 7f145be00640 1 -- 172.21.15.42:0/3698066846 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22547 (secure 0 0 0) 0x7f14540bcc00 con 0x7f1464102270 2024-04-25T00:45:04.911 INFO:teuthology.orchestra.run.smithi042.stdout: 2024-04-25T00:45:04.911 INFO:teuthology.orchestra.run.smithi042.stdout:{"epoch":91,"fsid":"9d95543e-0297-11ef-bc93-c7b262605968","created":"2024-04-25T00:07:16.359664+0000","modified":"2024-04-25T00:34:20.134521+0000","last_up_change":"2024-04-25T00:15:24.836408+0000","last_in_change":"2024-04-25T00:15:05.195411+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T00:11:14.472869+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T00:16:18.168243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T00:16:19.289585+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"9e126226-45cb-411a-ac44-64c6f80f8eaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6802","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6803","nonce":2959872296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6804","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6805","nonce":2959872296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6808","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6809","nonce":2959872296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6806","nonce":2959872296},{"type":"v1","addr":"172.21.15.42:6807","nonce":2959872296}]},"public_addr":"172.21.15.42:6803/2959872296","cluster_addr":"172.21.15.42:6805/2959872296","heartbeat_back_addr":"172.21.15.42:6809/2959872296","heartbeat_front_addr":"172.21.15.42:6807/2959872296","state":["exists","up"]},{"osd":1,"uuid":"df2777b5-5078-4e63-b3f9-6f20ea649a3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6801","nonce":2488433173}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6803","nonce":2488433173}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6807","nonce":2488433173}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":2488433173},{"type":"v1","addr":"172.21.15.106:6805","nonce":2488433173}]},"public_addr":"172.21.15.106:6801/2488433173","cluster_addr":"172.21.15.106:6803/2488433173","heartbeat_back_addr":"172.21.15.106:6807/2488433173","heartbeat_front_addr":"172.21.15.106:6805/2488433173","state":["exists","up"]},{"osd":2,"uuid":"0f53ac73-5268-4b0f-9256-bf92ba350ce5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6801","nonce":930826162}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6803","nonce":930826162}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6807","nonce":930826162}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":930826162},{"type":"v1","addr":"172.21.15.136:6805","nonce":930826162}]},"public_addr":"172.21.15.136:6801/930826162","cluster_addr":"172.21.15.136:6803/930826162","heartbeat_back_addr":"172.21.15.136:6807/930826162","heartbeat_front_addr":"172.21.15.136:6805/930826162","state":["exists","up"]},{"osd":3,"uuid":"a8044c10-17a2-4a89-926c-2fe500f36943","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6810","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6811","nonce":2791262023}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6812","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6813","nonce":2791262023}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6816","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6817","nonce":2791262023}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6814","nonce":2791262023},{"type":"v1","addr":"172.21.15.42:6815","nonce":2791262023}]},"public_addr":"172.21.15.42:6811/2791262023","cluster_addr":"172.21.15.42:6813/2791262023","heartbeat_back_addr":"172.21.15.42:6817/2791262023","heartbeat_front_addr":"172.21.15.42:6815/2791262023","state":["exists","up"]},{"osd":4,"uuid":"61b5a912-f971-4678-98fb-5f95190f8c46","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6809","nonce":2825344351}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6811","nonce":2825344351}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6815","nonce":2825344351}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6812","nonce":2825344351},{"type":"v1","addr":"172.21.15.106:6813","nonce":2825344351}]},"public_addr":"172.21.15.106:6809/2825344351","cluster_addr":"172.21.15.106:6811/2825344351","heartbeat_back_addr":"172.21.15.106:6815/2825344351","heartbeat_front_addr":"172.21.15.106:6813/2825344351","state":["exists","up"]},{"osd":5,"uuid":"978af6c3-4b15-4a19-bc5b-587c384520ef","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6809","nonce":3211315252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6811","nonce":3211315252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6815","nonce":3211315252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":3211315252},{"type":"v1","addr":"172.21.15.136:6813","nonce":3211315252}]},"public_addr":"172.21.15.136:6809/3211315252","cluster_addr":"172.21.15.136:6811/3211315252","heartbeat_back_addr":"172.21.15.136:6815/3211315252","heartbeat_front_addr":"172.21.15.136:6813/3211315252","state":["exists","up"]},{"osd":6,"uuid":"0ac3ab28-29e1-4316-af39-4ca7e9785cf3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6818","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6819","nonce":2564442522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6820","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6821","nonce":2564442522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6824","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6825","nonce":2564442522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6822","nonce":2564442522},{"type":"v1","addr":"172.21.15.42:6823","nonce":2564442522}]},"public_addr":"172.21.15.42:6819/2564442522","cluster_addr":"172.21.15.42:6821/2564442522","heartbeat_back_addr":"172.21.15.42:6825/2564442522","heartbeat_front_addr":"172.21.15.42:6823/2564442522","state":["exists","up"]},{"osd":7,"uuid":"c6bbcd7b-8a26-4ee2-b795-6e5c3a9ab235","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6817","nonce":178560796}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6819","nonce":178560796}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6823","nonce":178560796}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6820","nonce":178560796},{"type":"v1","addr":"172.21.15.106:6821","nonce":178560796}]},"public_addr":"172.21.15.106:6817/178560796","cluster_addr":"172.21.15.106:6819/178560796","heartbeat_back_addr":"172.21.15.106:6823/178560796","heartbeat_front_addr":"172.21.15.106:6821/178560796","state":["exists","up"]},{"osd":8,"uuid":"ff7307f4-b84a-4297-941f-544e5e06b1d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6817","nonce":3281651567}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6819","nonce":3281651567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6823","nonce":3281651567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":3281651567},{"type":"v1","addr":"172.21.15.136:6821","nonce":3281651567}]},"public_addr":"172.21.15.136:6817/3281651567","cluster_addr":"172.21.15.136:6819/3281651567","heartbeat_back_addr":"172.21.15.136:6823/3281651567","heartbeat_front_addr":"172.21.15.136:6821/3281651567","state":["exists","up"]},{"osd":9,"uuid":"179b1772-e33e-4970-9a88-6af3a1eb19dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6826","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6827","nonce":876648842}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6828","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6829","nonce":876648842}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6832","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6833","nonce":876648842}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.42:6830","nonce":876648842},{"type":"v1","addr":"172.21.15.42:6831","nonce":876648842}]},"public_addr":"172.21.15.42:6827/876648842","cluster_addr":"172.21.15.42:6829/876648842","heartbeat_back_addr":"172.21.15.42:6833/876648842","heartbeat_front_addr":"172.21.15.42:6831/876648842","state":["exists","up"]},{"osd":10,"uuid":"fc443ab7-2aea-427e-9bfd-d25f4f4dc239","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6825","nonce":3446161379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6826","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6827","nonce":3446161379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6831","nonce":3446161379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":3446161379},{"type":"v1","addr":"172.21.15.106:6829","nonce":3446161379}]},"public_addr":"172.21.15.106:6825/3446161379","cluster_addr":"172.21.15.106:6827/3446161379","heartbeat_back_addr":"172.21.15.106:6831/3446161379","heartbeat_front_addr":"172.21.15.106:6829/3446161379","state":["exists","up"]},{"osd":11,"uuid":"bbdadbeb-aa71-49d8-bffb-4977f2dbaa44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6825","nonce":3705953158}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6827","nonce":3705953158}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6831","nonce":3705953158}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":3705953158},{"type":"v1","addr":"172.21.15.136:6829","nonce":3705953158}]},"public_addr":"172.21.15.136:6825/3705953158","cluster_addr":"172.21.15.136:6827/3705953158","heartbeat_back_addr":"172.21.15.136:6831/3705953158","heartbeat_front_addr":"172.21.15.136:6829/3705953158","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:22.371270+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:10:47.588620+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:11.486308+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:11:43.907020+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:09.751455+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:12:33.819117+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:08.117244+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:34.015792+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:13:58.353062+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:31.894834+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:14:58.220075+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T00:15:22.995427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.12","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":6}]},{"pgid":"3.3c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.42:0/1811796714":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6801/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:6800/3681869279":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/3081484237":"2024-04-26T00:08:25.763656+0000","172.21.15.42:0/4222574490":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/15970351":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1603265964":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6800/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:6801/174637222":"2024-04-26T00:07:50.514990+0000","172.21.15.42:0/1839487361":"2024-04-26T00:08:25.763656+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 -- 172.21.15.42:0/3698066846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14480a2b90 msgr2=0x7f14480a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14480a2b90 0x7f14480a5050 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f146406a8a0 tx=0x7f144c005f90 comp rx=0 tx=0).stop 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 -- 172.21.15.42:0/3698066846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1464102270 msgr2=0x7f1464199cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1464102270 0x7f1464199cf0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f145402aeb0 tx=0x7f14540319f0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 -- 172.21.15.42:0/3698066846 shutdown_connections 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f14480a2b90 0x7f14480a5050 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14641074d0 0x7f14641a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f1464104ba0 0x7f146419a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 --2- 172.21.15.42:0/3698066846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1464102270 0x7f1464199cf0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 -- 172.21.15.42:0/3698066846 >> 172.21.15.42:0/3698066846 conn(0x7f14640fbc20 msgr2=0x7f14641019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 -- 172.21.15.42:0/3698066846 shutdown_connections 2024-04-25T00:45:04.913 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:04.911+0000 7f1469600640 1 -- 172.21.15.42:0/3698066846 wait complete. 2024-04-25T00:45:04.929 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T00:45:05.085 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:04 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/300752708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:45:05.085 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:04 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/4189574387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:45:05.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:04 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/300752708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:45:05.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:04 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/4189574387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:45:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.127+0000 7f62d3a00640 1 -- 172.21.15.42:0/1763086965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62cc076100 msgr2=0x7f62cc076520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.127+0000 7f62d3a00640 1 --2- 172.21.15.42:0/1763086965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62cc076100 0x7f62cc076520 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f62c0009900 tx=0x7f62c002a3f0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.127+0000 7f62d3a00640 1 -- 172.21.15.42:0/1763086965 shutdown_connections 2024-04-25T00:45:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.127+0000 7f62d3a00640 1 --2- 172.21.15.42:0/1763086965 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62cc077ce0 0x7f62cc078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.127+0000 7f62d3a00640 1 --2- 172.21.15.42:0/1763086965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62cc077340 0x7f62cc0777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.127+0000 7f62d3a00640 1 --2- 172.21.15.42:0/1763086965 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62cc076100 0x7f62cc076520 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.127+0000 7f62d3a00640 1 -- 172.21.15.42:0/1763086965 >> 172.21.15.42:0/1763086965 conn(0x7f62cc0fef30 msgr2=0x7f62cc101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.127+0000 7f62d3a00640 1 -- 172.21.15.42:0/1763086965 shutdown_connections 2024-04-25T00:45:05.129 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.127+0000 7f62d3a00640 1 -- 172.21.15.42:0/1763086965 wait complete. 2024-04-25T00:45:05.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.128+0000 7f62d3a00640 1 Processor -- start 2024-04-25T00:45:05.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.128+0000 7f62d3a00640 1 -- start start 2024-04-25T00:45:05.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62d3a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62cc076100 0x7f62cc19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:05.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62d3a00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62cc077340 0x7f62cc19e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:05.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62d3a00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62cc077ce0 0x7f62cc1a56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:05.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62d3a00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc113a90 con 0x7f62cc077340 2024-04-25T00:45:05.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62d3a00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc113910 con 0x7f62cc076100 2024-04-25T00:45:05.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62d3a00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc113c10 con 0x7f62cc077ce0 2024-04-25T00:45:05.132 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62cbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62cc077340 0x7f62cc19e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62d1000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62cc076100 0x7f62cc19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62cbe00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62cc077340 0x7f62cc19e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:32860/0 (socket says 172.21.15.42:32860) 2024-04-25T00:45:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62cbe00640 1 -- 172.21.15.42:0/1483127480 learned_addr learned my addr 172.21.15.42:0/1483127480 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.129+0000 7f62d1a00640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62cc077ce0 0x7f62cc1a56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.130+0000 7f62d1000640 1 -- 172.21.15.42:0/1483127480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62cc077ce0 msgr2=0x7f62cc1a56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.130+0000 7f62d1000640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62cc077ce0 0x7f62cc1a56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.130+0000 7f62d1000640 1 -- 172.21.15.42:0/1483127480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62cc077340 msgr2=0x7f62cc19e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.130+0000 7f62d1000640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62cc077340 0x7f62cc19e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.130+0000 7f62d1000640 1 -- 172.21.15.42:0/1483127480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc1a5d30 con 0x7f62cc076100 2024-04-25T00:45:05.133 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.130+0000 7f62d1000640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62cc076100 0x7f62cc19e110 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f62c002a910 tx=0x7f62c0032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.130+0000 7f62c9600640 1 -- 172.21.15.42:0/1483127480 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c003f030 con 0x7f62cc076100 2024-04-25T00:45:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.130+0000 7f62d3a00640 1 -- 172.21.15.42:0/1483127480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62cc1a5fc0 con 0x7f62cc076100 2024-04-25T00:45:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.130+0000 7f62c9600640 1 -- 172.21.15.42:0/1483127480 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62c0003ce0 con 0x7f62cc076100 2024-04-25T00:45:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.130+0000 7f62c9600640 1 -- 172.21.15.42:0/1483127480 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c0035070 con 0x7f62cc076100 2024-04-25T00:45:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.131+0000 7f62d3a00640 1 -- 172.21.15.42:0/1483127480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62cc1a6420 con 0x7f62cc076100 2024-04-25T00:45:05.134 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.132+0000 7f62d3a00640 1 -- 172.21.15.42:0/1483127480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62cc110330 con 0x7f62cc076100 2024-04-25T00:45:05.135 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.134+0000 7f62c9600640 1 -- 172.21.15.42:0/1483127480 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f62c0042030 con 0x7f62cc076100 2024-04-25T00:45:05.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.135+0000 7f62c9600640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62b40a2b90 0x7f62b40a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:05.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.135+0000 7f62c9600640 1 -- 172.21.15.42:0/1483127480 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8182+0+0 (secure 0 0 0) 0x7f62c00ef9f0 con 0x7f62cc076100 2024-04-25T00:45:05.136 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.135+0000 7f62cbe00640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62b40a2b90 0x7f62b40a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:05.137 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.136+0000 7f62cbe00640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62b40a2b90 0x7f62b40a5050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f62cc19f620 tx=0x7f62b8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:05.138 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.137+0000 7f62c9600640 1 -- 172.21.15.42:0/1483127480 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f62c00bc2d0 con 0x7f62cc076100 2024-04-25T00:45:05.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:04 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/300752708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:45:05.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:04 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/4189574387' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T00:45:05.385 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.383+0000 7f62d3a00640 1 -- 172.21.15.42:0/1483127480 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f62cc02d340 con 0x7f62cc076100 2024-04-25T00:45:05.395 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.394+0000 7f62c9600640 1 -- 172.21.15.42:0/1483127480 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 161+0+0 (secure 0 0 0) 0x7f62c00c05d0 con 0x7f62cc076100 2024-04-25T00:45:05.395 INFO:teuthology.orchestra.run.smithi042.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T00:45:05.398 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 -- 172.21.15.42:0/1483127480 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62b40a2b90 msgr2=0x7f62b40a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:05.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62b40a2b90 0x7f62b40a5050 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f62cc19f620 tx=0x7f62b8005f90 comp rx=0 tx=0).stop 2024-04-25T00:45:05.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 -- 172.21.15.42:0/1483127480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62cc076100 msgr2=0x7f62cc19e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:05.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62cc076100 0x7f62cc19e110 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f62c002a910 tx=0x7f62c0032ce0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 -- 172.21.15.42:0/1483127480 shutdown_connections 2024-04-25T00:45:05.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f62b40a2b90 0x7f62b40a5050 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f62cc077ce0 0x7f62cc1a56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f62cc077340 0x7f62cc19e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 --2- 172.21.15.42:0/1483127480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f62cc076100 0x7f62cc19e110 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 -- 172.21.15.42:0/1483127480 >> 172.21.15.42:0/1483127480 conn(0x7f62cc0fef30 msgr2=0x7f62cc100a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:05.399 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 -- 172.21.15.42:0/1483127480 shutdown_connections 2024-04-25T00:45:05.400 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.397+0000 7f62d3a00640 1 -- 172.21.15.42:0/1483127480 wait complete. 2024-04-25T00:45:05.417 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T00:45:05.606 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.605+0000 7f10a9400640 1 -- 172.21.15.42:0/3348204062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a41006a0 msgr2=0x7f10a4100ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.605+0000 7f10a9400640 1 --2- 172.21.15.42:0/3348204062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a41006a0 0x7f10a4100ac0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f1094009be0 tx=0x7f109402a6c0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.605+0000 7f10a9400640 1 -- 172.21.15.42:0/3348204062 shutdown_connections 2024-04-25T00:45:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.605+0000 7f10a9400640 1 --2- 172.21.15.42:0/3348204062 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a4102250 0x7f10a4108ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.605+0000 7f10a9400640 1 --2- 172.21.15.42:0/3348204062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10a4101890 0x7f10a4101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.605+0000 7f10a9400640 1 --2- 172.21.15.42:0/3348204062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a41006a0 0x7f10a4100ac0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.605+0000 7f10a9400640 1 -- 172.21.15.42:0/3348204062 >> 172.21.15.42:0/3348204062 conn(0x7f10a40fbc10 msgr2=0x7f10a40fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.605+0000 7f10a9400640 1 -- 172.21.15.42:0/3348204062 shutdown_connections 2024-04-25T00:45:05.607 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.605+0000 7f10a9400640 1 -- 172.21.15.42:0/3348204062 wait complete. 2024-04-25T00:45:05.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.606+0000 7f10a9400640 1 Processor -- start 2024-04-25T00:45:05.609 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.606+0000 7f10a9400640 1 -- start start 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.606+0000 7f10a9400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a41006a0 0x7f10a4199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.606+0000 7f10a9400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a4101890 0x7f10a419a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.606+0000 7f10a9400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10a4102250 0x7f10a41a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.606+0000 7f10a9400640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f10a410d440 con 0x7f10a4101890 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.606+0000 7f10a9400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f10a410d2c0 con 0x7f10a41006a0 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.606+0000 7f10a9400640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f10a410d5c0 con 0x7f10a4102250 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f10a2800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a41006a0 0x7f10a4199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f10a2800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a41006a0 0x7f10a4199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59466/0 (socket says 172.21.15.42:59466) 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f10a2800640 1 -- 172.21.15.42:0/3716371781 learned_addr learned my addr 172.21.15.42:0/3716371781 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f10a3200640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10a4102250 0x7f10a41a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f1099e00640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a4101890 0x7f10a419a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:05.610 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f10a2800640 1 -- 172.21.15.42:0/3716371781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10a4102250 msgr2=0x7f10a41a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f10a2800640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10a4102250 0x7f10a41a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f10a2800640 1 -- 172.21.15.42:0/3716371781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a4101890 msgr2=0x7f10a419a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f10a2800640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a4101890 0x7f10a419a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f10a2800640 1 -- 172.21.15.42:0/3716371781 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a41a1990 con 0x7f10a41006a0 2024-04-25T00:45:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.607+0000 7f10a2800640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a41006a0 0x7f10a4199d70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f109402acb0 tx=0x7f10940305d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.608+0000 7f109be00640 1 -- 172.21.15.42:0/3716371781 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094041020 con 0x7f10a41006a0 2024-04-25T00:45:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.608+0000 7f10a9400640 1 -- 172.21.15.42:0/3716371781 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a41a1c20 con 0x7f10a41006a0 2024-04-25T00:45:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.608+0000 7f109be00640 1 -- 172.21.15.42:0/3716371781 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1094031920 con 0x7f10a41006a0 2024-04-25T00:45:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.608+0000 7f109be00640 1 -- 172.21.15.42:0/3716371781 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094036040 con 0x7f10a41006a0 2024-04-25T00:45:05.611 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.608+0000 7f10a9400640 1 -- 172.21.15.42:0/3716371781 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a41a2160 con 0x7f10a41006a0 2024-04-25T00:45:05.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.609+0000 7f10a9400640 1 -- 172.21.15.42:0/3716371781 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a41a23f0 con 0x7f10a41006a0 2024-04-25T00:45:05.612 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.611+0000 7f109be00640 1 -- 172.21.15.42:0/3716371781 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1094044020 con 0x7f10a41006a0 2024-04-25T00:45:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.613+0000 7f109be00640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f10880a2c90 0x7f10880a5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.613+0000 7f109be00640 1 -- 172.21.15.42:0/3716371781 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8440+0+0 (secure 0 0 0) 0x7f10940f01e0 con 0x7f10a41006a0 2024-04-25T00:45:05.614 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.613+0000 7f1099e00640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f10880a2c90 0x7f10880a5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:05.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.614+0000 7f1099e00640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f10880a2c90 0x7f10880a5150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f10a419b280 tx=0x7f108c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:05.615 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.614+0000 7f109be00640 1 -- 172.21.15.42:0/3716371781 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f109404b020 con 0x7f10a41006a0 2024-04-25T00:45:05.763 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f[62996]: ignoring --setuser ceph since I am not root 2024-04-25T00:45:05.763 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f[62996]: ignoring --setgroup ceph since I am not root 2024-04-25T00:45:05.763 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mds[63000]: -- 172.21.15.136:0/2316376941 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56472bcdfa40 con 0x56472bcb3000 2024-04-25T00:45:05.763 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f[62996]: starting mds.f at 2024-04-25T00:45:05.764 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: ignoring --setuser ceph since I am not root 2024-04-25T00:45:05.764 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: ignoring --setgroup ceph since I am not root 2024-04-25T00:45:05.764 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mds[63607]: -- 172.21.15.136:0/844316528 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af56bd9a40 con 0x55af56bac800 2024-04-25T00:45:05.764 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: starting mds.i at 2024-04-25T00:45:05.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c[63296]: ignoring --setuser ceph since I am not root 2024-04-25T00:45:05.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c[63296]: ignoring --setgroup ceph since I am not root 2024-04-25T00:45:05.764 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mds[63300]: -- 172.21.15.136:0/2754449546 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5623878a9a40 con 0x56238787c800 2024-04-25T00:45:05.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c[63296]: starting mds.c at 2024-04-25T00:45:05.765 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c[63296]: 2024-04-25T00:45:05.478+0000 7f2e6e7b5700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6835/0: (98) Address already in use 2024-04-25T00:45:05.805 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a[28126]: 2024-04-25T00:45:05.386+0000 7fe632e00700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:45:05.805 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a[69399]: ignoring --setuser ceph since I am not root 2024-04-25T00:45:05.805 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a[69399]: ignoring --setgroup ceph since I am not root 2024-04-25T00:45:05.805 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mds[69403]: -- 172.21.15.42:0/557130448 <== mon.1 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9907cfa40 con 0x55a9907a2800 2024-04-25T00:45:05.805 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a[69399]: starting mds.a at 2024-04-25T00:45:05.838 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b[63388]: ignoring --setuser ceph since I am not root 2024-04-25T00:45:05.838 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b[63388]: ignoring --setgroup ceph since I am not root 2024-04-25T00:45:05.839 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mds[63392]: -- 172.21.15.106:0/2418571296 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56315eefda40 con 0x56315eed1000 2024-04-25T00:45:05.839 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b[63388]: starting mds.b at 2024-04-25T00:45:05.839 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h[63078]: ignoring --setuser ceph since I am not root 2024-04-25T00:45:05.839 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h[63078]: ignoring --setgroup ceph since I am not root 2024-04-25T00:45:05.839 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mds[63082]: -- 172.21.15.106:0/2372641529 <== mon.2 v2:172.21.15.106:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5588516afa40 con 0x558851682800 2024-04-25T00:45:05.839 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h[63078]: starting mds.h at 2024-04-25T00:45:05.839 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: pgmap v1180: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3698066846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/1483127480' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:45:05.840 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:05 smithi106 ceph-mon[32510]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T00:45:05.855 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:05.853+0000 7f10a9400640 1 -- 172.21.15.42:0/3716371781 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f10a41a25c0 con 0x7f10a41006a0 2024-04-25T00:45:06.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: pgmap v1180: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-25T00:45:06.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3698066846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:45:06.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:45:06.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:45:06.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:45:06.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:45:06.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:45:06.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/1483127480' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:45:06.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:45:06.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:45:06.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:45:06.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:45:06.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:45:06.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:05 smithi042 ceph-mon[28130]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T00:45:06.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: pgmap v1180: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 6 op/s 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3698066846' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/1483127480' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T00:45:06.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:05 smithi136 ceph-mon[32923]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T00:45:06.397 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.396+0000 7f109be00640 1 -- 172.21.15.42:0/3716371781 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) ==== 104+0+0 (secure 0 0 0) 0x7f10940bd010 con 0x7f10a41006a0 2024-04-25T00:45:06.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.402+0000 7f10a9400640 1 -- 172.21.15.42:0/3716371781 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f10880a2c90 msgr2=0x7f10880a5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:06.404 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.402+0000 7f10a9400640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f10880a2c90 0x7f10880a5150 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f10a419b280 tx=0x7f108c005f90 comp rx=0 tx=0).stop 2024-04-25T00:45:06.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.402+0000 7f10a9400640 1 -- 172.21.15.42:0/3716371781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a41006a0 msgr2=0x7f10a4199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:06.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.402+0000 7f10a9400640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a41006a0 0x7f10a4199d70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f109402acb0 tx=0x7f10940305d0 comp rx=0 tx=0).stop 2024-04-25T00:45:06.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.402+0000 7f10a9400640 1 -- 172.21.15.42:0/3716371781 shutdown_connections 2024-04-25T00:45:06.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.402+0000 7f10a9400640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f10880a2c90 0x7f10880a5150 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:06.405 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.402+0000 7f10a9400640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f10a4102250 0x7f10a41a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:06.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.402+0000 7f10a9400640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f10a4101890 0x7f10a419a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:06.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.402+0000 7f10a9400640 1 --2- 172.21.15.42:0/3716371781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10a41006a0 0x7f10a4199d70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:06.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.402+0000 7f10a9400640 1 -- 172.21.15.42:0/3716371781 >> 172.21.15.42:0/3716371781 conn(0x7f10a40fbc10 msgr2=0x7f10a41094a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:06.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.403+0000 7f10a9400640 1 -- 172.21.15.42:0/3716371781 shutdown_connections 2024-04-25T00:45:06.406 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.403+0000 7f10a9400640 1 -- 172.21.15.42:0/3716371781 wait complete. 2024-04-25T00:45:06.426 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T00:45:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.638+0000 7f049be00640 1 -- 172.21.15.42:0/305785121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0494077340 msgr2=0x7f04940777a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.638+0000 7f049be00640 1 --2- 172.21.15.42:0/305785121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0494077340 0x7f04940777a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0480009900 tx=0x7f048002a410 comp rx=0 tx=0).stop 2024-04-25T00:45:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.638+0000 7f049be00640 1 -- 172.21.15.42:0/305785121 shutdown_connections 2024-04-25T00:45:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.638+0000 7f049be00640 1 --2- 172.21.15.42:0/305785121 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0494077ce0 0x7f0494078580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.638+0000 7f049be00640 1 --2- 172.21.15.42:0/305785121 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0494077340 0x7f04940777a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.638+0000 7f049be00640 1 --2- 172.21.15.42:0/305785121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0494076100 0x7f0494076520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.638+0000 7f049be00640 1 -- 172.21.15.42:0/305785121 >> 172.21.15.42:0/305785121 conn(0x7f04940feef0 msgr2=0x7f0494101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.638+0000 7f049be00640 1 -- 172.21.15.42:0/305785121 shutdown_connections 2024-04-25T00:45:06.640 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.638+0000 7f049be00640 1 -- 172.21.15.42:0/305785121 wait complete. 2024-04-25T00:45:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.640+0000 7f049be00640 1 Processor -- start 2024-04-25T00:45:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.640+0000 7f049be00640 1 -- start start 2024-04-25T00:45:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.640+0000 7f049be00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0494076100 0x7f049419e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.640+0000 7f049be00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0494077340 0x7f049419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:06.642 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.640+0000 7f049be00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0494077ce0 0x7f04941a5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.640+0000 7f049be00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f0494113a30 con 0x7f0494077340 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.640+0000 7f049be00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f04941138b0 con 0x7f0494077ce0 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.640+0000 7f049be00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0494113bb0 con 0x7f0494076100 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.640+0000 7f0499e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0494077ce0 0x7f04941a5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.640+0000 7f0499400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0494076100 0x7f049419e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0499e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0494077ce0 0x7f04941a5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.42:59480/0 (socket says 172.21.15.42:59480) 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0499400640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0494076100 0x7f049419e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.42:55276/0 (socket says 172.21.15.42:55276) 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0499e00640 1 -- 172.21.15.42:0/2833817273 learned_addr learned my addr 172.21.15.42:0/2833817273 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0498a00640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0494077340 0x7f049419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0499e00640 1 -- 172.21.15.42:0/2833817273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0494076100 msgr2=0x7f049419e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0499e00640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0494076100 0x7f049419e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0499e00640 1 -- 172.21.15.42:0/2833817273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0494077340 msgr2=0x7f049419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0499e00640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0494077340 0x7f049419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:06.643 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0499e00640 1 -- 172.21.15.42:0/2833817273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04941a5d00 con 0x7f0494077ce0 2024-04-25T00:45:06.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0499400640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0494076100 0x7f049419e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:45:06.644 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.641+0000 7f0499e00640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0494077ce0 0x7f04941a5690 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f048c009fd0 tx=0x7f048c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:06.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.642+0000 7f0492000640 1 -- 172.21.15.42:0/2833817273 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c017070 con 0x7f0494077ce0 2024-04-25T00:45:06.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.642+0000 7f0492000640 1 -- 172.21.15.42:0/2833817273 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f048c003e50 con 0x7f0494077ce0 2024-04-25T00:45:06.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.642+0000 7f0492000640 1 -- 172.21.15.42:0/2833817273 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0106c0 con 0x7f0494077ce0 2024-04-25T00:45:06.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.642+0000 7f049be00640 1 -- 172.21.15.42:0/2833817273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04941a5ff0 con 0x7f0494077ce0 2024-04-25T00:45:06.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.642+0000 7f049be00640 1 -- 172.21.15.42:0/2833817273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04941a64b0 con 0x7f0494077ce0 2024-04-25T00:45:06.645 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.643+0000 7f049be00640 1 -- 172.21.15.42:0/2833817273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0494110500 con 0x7f0494077ce0 2024-04-25T00:45:06.646 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.645+0000 7f0492000640 1 -- 172.21.15.42:0/2833817273 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f048c003970 con 0x7f0494077ce0 2024-04-25T00:45:06.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.647+0000 7f0492000640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f047c0a2c10 0x7f047c0a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:06.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.647+0000 7f0492000640 1 -- 172.21.15.42:0/2833817273 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8440+0+0 (secure 0 0 0) 0x7f048c0cedd0 con 0x7f0494077ce0 2024-04-25T00:45:06.648 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.647+0000 7f0499400640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f047c0a2c10 0x7f047c0a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:06.649 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.648+0000 7f0499400640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f047c0a2c10 0x7f047c0a50d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0488009fc0 tx=0x7f0488005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:06.652 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.652+0000 7f0492000640 1 -- 172.21.15.42:0/2833817273 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f048c09b5b0 con 0x7f0494077ce0 2024-04-25T00:45:06.857 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:06.856+0000 7f049be00640 1 -- 172.21.15.42:0/2833817273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0494062340 con 0x7f0494077ce0 2024-04-25T00:45:07.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/3716371781' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: Cluster is now healthy 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6836/3405066916,v1:172.21.15.136:6839/3405066916] up:boot 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6835/3355732057,v1:172.21.15.136:6838/3355732057] up:boot 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.136:6834/3046047667,v1:172.21.15.136:6837/3046047667] up:boot 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.42:6840/1906270980,v1:172.21.15.42:6841/1906270980] up:boot 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.106:6836/3568479438,v1:172.21.15.106:6838/3568479438] up:boot 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: mds.? [v2:172.21.15.106:6837/1130902313,v1:172.21.15.106:6839/1130902313] up:boot 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: fsmap 12 up:standby 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:45:07.089 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:45:07.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:45:07.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:45:07.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:45:07.090 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:06 smithi106 ceph-mon[32510]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:45:07.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/3716371781' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: Cluster is now healthy 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6836/3405066916,v1:172.21.15.136:6839/3405066916] up:boot 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6835/3355732057,v1:172.21.15.136:6838/3355732057] up:boot 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.136:6834/3046047667,v1:172.21.15.136:6837/3046047667] up:boot 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.42:6840/1906270980,v1:172.21.15.42:6841/1906270980] up:boot 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.106:6836/3568479438,v1:172.21.15.106:6838/3568479438] up:boot 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: mds.? [v2:172.21.15.106:6837/1130902313,v1:172.21.15.106:6839/1130902313] up:boot 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: fsmap 12 up:standby 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:45:07.128 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:45:07.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:45:07.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:45:07.129 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:06 smithi042 ceph-mon[28130]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:45:07.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/3716371781' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:45:07.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: Cluster is now healthy 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6836/3405066916,v1:172.21.15.136:6839/3405066916] up:boot 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6835/3355732057,v1:172.21.15.136:6838/3355732057] up:boot 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.136:6834/3046047667,v1:172.21.15.136:6837/3046047667] up:boot 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.42:6840/1906270980,v1:172.21.15.42:6841/1906270980] up:boot 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.106:6836/3568479438,v1:172.21.15.106:6838/3568479438] up:boot 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: mds.? [v2:172.21.15.106:6837/1130902313,v1:172.21.15.106:6839/1130902313] up:boot 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: fsmap 12 up:standby 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T00:45:07.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T00:45:07.265 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:06 smithi136 ceph-mon[32923]: from='mgr.14152 172.21.15.42:0/3327828986' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T00:45:07.824 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.822+0000 7f0492000640 1 -- 172.21.15.42:0/2833817273 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) ==== 181+0+0 (secure 0 0 0) 0x7f048c09f8b0 con 0x7f0494077ce0 2024-04-25T00:45:07.825 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_metadata' removed 2024-04-25T00:45:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.830+0000 7f049be00640 1 -- 172.21.15.42:0/2833817273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f047c0a2c10 msgr2=0x7f047c0a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.830+0000 7f049be00640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f047c0a2c10 0x7f047c0a50d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f0488009fc0 tx=0x7f0488005f90 comp rx=0 tx=0).stop 2024-04-25T00:45:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.830+0000 7f049be00640 1 -- 172.21.15.42:0/2833817273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0494077ce0 msgr2=0x7f04941a5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:07.831 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.830+0000 7f049be00640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0494077ce0 0x7f04941a5690 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f048c009fd0 tx=0x7f048c00b260 comp rx=0 tx=0).stop 2024-04-25T00:45:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.830+0000 7f049be00640 1 -- 172.21.15.42:0/2833817273 shutdown_connections 2024-04-25T00:45:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.830+0000 7f049be00640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f047c0a2c10 0x7f047c0a50d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.830+0000 7f049be00640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0494077ce0 0x7f04941a5690 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.830+0000 7f049be00640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f0494077340 0x7f049419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:07.832 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.830+0000 7f049be00640 1 --2- 172.21.15.42:0/2833817273 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0494076100 0x7f049419e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:07.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.830+0000 7f049be00640 1 -- 172.21.15.42:0/2833817273 >> 172.21.15.42:0/2833817273 conn(0x7f04940feef0 msgr2=0x7f0494100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:07.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.831+0000 7f049be00640 1 -- 172.21.15.42:0/2833817273 shutdown_connections 2024-04-25T00:45:07.833 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:07.831+0000 7f049be00640 1 -- 172.21.15.42:0/2833817273 wait complete. 2024-04-25T00:45:07.869 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T00:45:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.081+0000 7f7f22e00640 1 -- 172.21.15.42:0/2023258922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c104a00 msgr2=0x7f7f1c106df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.081+0000 7f7f22e00640 1 --2- 172.21.15.42:0/2023258922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c104a00 0x7f7f1c106df0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009900 tx=0x7f7f0802a3f0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.081+0000 7f7f22e00640 1 -- 172.21.15.42:0/2023258922 shutdown_connections 2024-04-25T00:45:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.081+0000 7f7f22e00640 1 --2- 172.21.15.42:0/2023258922 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f1c107330 0x7f7f1c109780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.081+0000 7f7f22e00640 1 --2- 172.21.15.42:0/2023258922 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c104a00 0x7f7f1c106df0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.081+0000 7f7f22e00640 1 --2- 172.21.15.42:0/2023258922 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f1c1020d0 0x7f7f1c1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.083 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.081+0000 7f7f22e00640 1 -- 172.21.15.42:0/2023258922 >> 172.21.15.42:0/2023258922 conn(0x7f7f1c0fbae0 msgr2=0x7f7f1c0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.081+0000 7f7f22e00640 1 -- 172.21.15.42:0/2023258922 shutdown_connections 2024-04-25T00:45:08.084 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.081+0000 7f7f22e00640 1 -- 172.21.15.42:0/2023258922 wait complete. 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.083+0000 7f7f22e00640 1 Processor -- start 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.083+0000 7f7f22e00640 1 -- start start 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.083+0000 7f7f22e00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c1020d0 0x7f7f1c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.083+0000 7f7f22e00640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f1c104a00 0x7f7f1c19a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.083+0000 7f7f22e00640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f1c107330 0x7f7f1c1a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.083+0000 7f7f22e00640 1 -- --> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c10d2c0 con 0x7f7f1c107330 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.083+0000 7f7f22e00640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c10d140 con 0x7f7f1c1020d0 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.083+0000 7f7f22e00640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c10d440 con 0x7f7f1c104a00 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f22400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f1c107330 0x7f7f1c1a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f21a00640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c1020d0 0x7f7f1c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:08.085 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f21000640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f1c104a00 0x7f7f1c19a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f22400640 1 --2- >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f1c107330 0x7f7f1c1a1170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.42:3300/0 says I am v2:172.21.15.42:53512/0 (socket says 172.21.15.42:53512) 2024-04-25T00:45:08.086 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f22400640 1 -- 172.21.15.42:0/2904575457 learned_addr learned my addr 172.21.15.42:0/2904575457 (peer_addr_for_me v2:172.21.15.42:0/0) 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f21a00640 1 -- 172.21.15.42:0/2904575457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f1c104a00 msgr2=0x7f7f1c19a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f21a00640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f1c104a00 0x7f7f1c19a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f21a00640 1 -- 172.21.15.42:0/2904575457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f1c107330 msgr2=0x7f7f1c1a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f21a00640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f1c107330 0x7f7f1c1a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f21a00640 1 -- 172.21.15.42:0/2904575457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c1a17e0 con 0x7f7f1c1020d0 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.084+0000 7f7f22400640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f1c107330 0x7f7f1c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.085+0000 7f7f21a00640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c1020d0 0x7f7f1c199bc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000eb00 tx=0x7f7f1000c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.085+0000 7f7f1a000640 1 -- 172.21.15.42:0/2904575457 <== mon.2 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10010430 con 0x7f7f1c1020d0 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.085+0000 7f7f1a000640 1 -- 172.21.15.42:0/2904575457 <== mon.2 v2:172.21.15.106:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f10010ab0 con 0x7f7f1c1020d0 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.085+0000 7f7f1a000640 1 -- 172.21.15.42:0/2904575457 <== mon.2 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f10017bb0 con 0x7f7f1c1020d0 2024-04-25T00:45:08.087 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.085+0000 7f7f22e00640 1 -- 172.21.15.42:0/2904575457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c1a1ad0 con 0x7f7f1c1020d0 2024-04-25T00:45:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.085+0000 7f7f22e00640 1 -- 172.21.15.42:0/2904575457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c1a1fa0 con 0x7f7f1c1020d0 2024-04-25T00:45:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:07 smithi106 ceph-mon[32510]: pgmap v1182: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-25T00:45:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:07 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2833817273' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:08.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:07 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:08.088 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.086+0000 7f7f22e00640 1 -- 172.21.15.42:0/2904575457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c068480 con 0x7f7f1c1020d0 2024-04-25T00:45:08.089 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.088+0000 7f7f1a000640 1 -- 172.21.15.42:0/2904575457 <== mon.2 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f10010c60 con 0x7f7f1c1020d0 2024-04-25T00:45:08.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.089+0000 7f7f1a000640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7f040a2c10 0x7f7f040a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T00:45:08.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.090+0000 7f7f1a000640 1 -- 172.21.15.42:0/2904575457 <== mon.2 v2:172.21.15.106:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 7944+0+0 (secure 0 0 0) 0x7f7f100cf860 con 0x7f7f1c1020d0 2024-04-25T00:45:08.091 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.090+0000 7f7f21000640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7f040a2c10 0x7f7f040a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T00:45:08.092 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.091+0000 7f7f21000640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7f040a2c10 0x7f7f040a50d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7f08005ab0 tx=0x7f7f08000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T00:45:08.094 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.093+0000 7f7f1a000640 1 -- 172.21.15.42:0/2904575457 <== mon.2 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186147 (secure 0 0 0) 0x7f7f1009c890 con 0x7f7f1c1020d0 2024-04-25T00:45:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:07 smithi042 ceph-mon[28130]: pgmap v1182: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-25T00:45:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:07 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2833817273' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:08.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:07 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:08.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:07 smithi136 ceph-mon[32923]: pgmap v1182: 97 pgs: 97 active+clean; 461 MiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 716 B/s wr, 7 op/s 2024-04-25T00:45:08.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:07 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2833817273' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:08.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:07 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:08.314 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.312+0000 7f7f22e00640 1 -- 172.21.15.42:0/2904575457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7f1c062340 con 0x7f7f1c1020d0 2024-04-25T00:45:08.846 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.844+0000 7f7f1a000640 1 -- 172.21.15.42:0/2904575457 <== mon.2 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) ==== 169+0+0 (secure 0 0 0) 0x7f7f1009c190 con 0x7f7f1c1020d0 2024-04-25T00:45:08.846 INFO:teuthology.orchestra.run.smithi042.stderr:pool 'cephfs_data' removed 2024-04-25T00:45:08.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.850+0000 7f7f22e00640 1 -- 172.21.15.42:0/2904575457 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7f040a2c10 msgr2=0x7f7f040a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:08.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.850+0000 7f7f22e00640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7f040a2c10 0x7f7f040a50d0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7f08005ab0 tx=0x7f7f08000bc0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.850+0000 7f7f22e00640 1 -- 172.21.15.42:0/2904575457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c1020d0 msgr2=0x7f7f1c199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T00:45:08.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.850+0000 7f7f22e00640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c1020d0 0x7f7f1c199bc0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7f1000eb00 tx=0x7f7f1000c2d0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.850+0000 7f7f22e00640 1 -- 172.21.15.42:0/2904575457 shutdown_connections 2024-04-25T00:45:08.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.850+0000 7f7f22e00640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.42:6800/384891702,v1:172.21.15.42:6801/384891702] conn(0x7f7f040a2c10 0x7f7f040a50d0 secure :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f7f08005ab0 tx=0x7f7f08000bc0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.850+0000 7f7f22e00640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.42:3300/0,v1:172.21.15.42:6789/0] conn(0x7f7f1c107330 0x7f7f1c1a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.853 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.850+0000 7f7f22e00640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f1c104a00 0x7f7f1c19a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.850+0000 7f7f22e00640 1 --2- 172.21.15.42:0/2904575457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7f1c1020d0 0x7f7f1c199bc0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T00:45:08.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.850+0000 7f7f22e00640 1 -- 172.21.15.42:0/2904575457 >> 172.21.15.42:0/2904575457 conn(0x7f7f1c0fbae0 msgr2=0x7f7f1c101870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T00:45:08.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.851+0000 7f7f22e00640 1 -- 172.21.15.42:0/2904575457 shutdown_connections 2024-04-25T00:45:08.854 INFO:teuthology.orchestra.run.smithi042.stderr:2024-04-25T00:45:08.851+0000 7f7f22e00640 1 -- 172.21.15.42:0/2904575457 wait complete. 2024-04-25T00:45:08.893 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:45:08.994 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:45:09.020 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:45:09.046 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T00:45:09.047 DEBUG:teuthology.orchestra.run.smithi042:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:45:09.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:08 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:45:09.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:08 smithi106 ceph-mon[32510]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:45:09.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:08 smithi106 ceph-mon[32510]: from='client.? 172.21.15.42:0/2904575457' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:09.088 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:08 smithi106 ceph-mon[32510]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:09.090 DEBUG:teuthology.orchestra.run.smithi106:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:45:09.105 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T00:45:09.123 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T00:45:09.124 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T00:45:09.124 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.a 2024-04-25T00:45:09.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:08 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:45:09.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:08 smithi042 ceph-mon[28130]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:45:09.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:08 smithi042 ceph-mon[28130]: from='client.? 172.21.15.42:0/2904575457' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:09.126 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:08 smithi042 ceph-mon[28130]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:09.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:08 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T00:45:09.263 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:08 smithi136 ceph-mon[32923]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T00:45:09.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:08 smithi136 ceph-mon[32923]: from='client.? 172.21.15.42:0/2904575457' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:09.264 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:08 smithi136 ceph-mon[32923]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T00:45:09.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:09 smithi042 systemd[1]: Stopping Ceph mon.a for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:10.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:09 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a[28126]: 2024-04-25T00:45:09.718+0000 7fe639c00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:10.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:09 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a[28126]: 2024-04-25T00:45:09.718+0000 7fe639c00700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T00:45:10.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:09 smithi042 podman[94205]: 2024-04-25 00:45:09.955731614 +0000 UTC m=+0.381033451 container died 7bf7a651ead32d2297e340c4997b25c820961bb8e304cfa2febca2431bbe4dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:45:10.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:10 smithi042 podman[94205]: 2024-04-25 00:45:10.480259384 +0000 UTC m=+0.905561248 container cleanup 7bf7a651ead32d2297e340c4997b25c820961bb8e304cfa2febca2431bbe4dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T00:45:10.627 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:10 smithi042 bash[94205]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a 2024-04-25T00:45:11.127 INFO:journalctl@ceph.mon.a.smithi042.stdout:Apr 25 00:45:10 smithi042 podman[94220]: 2024-04-25 00:45:10.815542832 +0000 UTC m=+0.854323766 container remove 7bf7a651ead32d2297e340c4997b25c820961bb8e304cfa2febca2431bbe4dad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T00:45:11.439 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T00:45:11.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:11.659 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T00:45:11.659 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T00:45:11.659 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.b 2024-04-25T00:45:11.835 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:11 smithi106 systemd[1]: Stopping Ceph mon.b for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:12.624 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:12 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-b[32505]: 2024-04-25T00:45:12.451+0000 7f73d5c8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:12.625 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:12 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-b[32505]: 2024-04-25T00:45:12.451+0000 7f73d5c8f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T00:45:12.625 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:12 smithi106 podman[71556]: 2024-04-25 00:45:12.62433188 +0000 UTC m=+0.345217533 container died 959ee9e9d9b3b070ccbc8f3508ed51d6d3918fba3e65f54503dfd767c57b331f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T00:45:13.338 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:13 smithi106 podman[71556]: 2024-04-25 00:45:13.190919561 +0000 UTC m=+0.911805220 container cleanup 959ee9e9d9b3b070ccbc8f3508ed51d6d3918fba3e65f54503dfd767c57b331f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-b, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T00:45:13.339 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:13 smithi106 bash[71556]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-b 2024-04-25T00:45:13.756 INFO:journalctl@ceph.mon.b.smithi106.stdout:Apr 25 00:45:13 smithi106 podman[71581]: 2024-04-25 00:45:13.375487707 +0000 UTC m=+0.751628820 container remove 959ee9e9d9b3b070ccbc8f3508ed51d6d3918fba3e65f54503dfd767c57b331f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:45:14.234 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T00:45:14.281 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:14.281 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T00:45:14.281 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T00:45:14.282 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.c 2024-04-25T00:45:14.630 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:14 smithi136 systemd[1]: Stopping Ceph mon.c for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:15.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:14 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-c[32905]: 2024-04-25T00:45:14.629+0000 7fae377dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:15.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:14 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-c[32905]: 2024-04-25T00:45:14.629+0000 7fae377dd700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T00:45:15.013 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:14 smithi136 podman[73139]: 2024-04-25 00:45:14.762638687 +0000 UTC m=+0.224682759 container died 02072d7cff8cdca76d1d990160dbbe3585c1c00f1b3ce7430aeed2712fab773b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T00:45:15.364 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:15 smithi136 podman[73139]: 2024-04-25 00:45:15.104647931 +0000 UTC m=+0.566692009 container cleanup 02072d7cff8cdca76d1d990160dbbe3585c1c00f1b3ce7430aeed2712fab773b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-c, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T00:45:15.364 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:15 smithi136 bash[73139]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-c 2024-04-25T00:45:15.615 INFO:journalctl@ceph.mon.c.smithi136.stdout:Apr 25 00:45:15 smithi136 podman[73153]: 2024-04-25 00:45:15.364683492 +0000 UTC m=+0.600512373 container remove 02072d7cff8cdca76d1d990160dbbe3585c1c00f1b3ce7430aeed2712fab773b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mon-c, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-25T00:45:15.619 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T00:45:15.669 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:15.669 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T00:45:15.669 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T00:45:15.669 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.x 2024-04-25T00:45:16.126 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:45:15 smithi042 systemd[1]: Stopping Ceph mgr.x for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:16.626 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:45:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:45:16.258+0000 7f825e800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:16.627 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:45:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x[28365]: 2024-04-25T00:45:16.258+0000 7f825e800700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T00:45:16.627 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:45:16 smithi042 podman[94325]: 2024-04-25 00:45:16.3529878 +0000 UTC m=+0.252299581 container died c05d3001fc487063fa5bc2e14f9a0099d22349fcd71664cecbbfd7c8f9a6b037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-25T00:45:17.126 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:45:17 smithi042 podman[94325]: 2024-04-25 00:45:17.020225506 +0000 UTC m=+0.919537247 container cleanup c05d3001fc487063fa5bc2e14f9a0099d22349fcd71664cecbbfd7c8f9a6b037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:45:17.127 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:45:17 smithi042 bash[94325]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x 2024-04-25T00:45:17.626 INFO:journalctl@ceph.mgr.x.smithi042.stdout:Apr 25 00:45:17 smithi042 podman[94340]: 2024-04-25 00:45:17.281822244 +0000 UTC m=+0.904169798 container remove c05d3001fc487063fa5bc2e14f9a0099d22349fcd71664cecbbfd7c8f9a6b037 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d2c45d977142e024a59b68a8f1dce8f0b9840551, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T00:45:17.827 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T00:45:17.951 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:17.951 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T00:45:17.952 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T00:45:17.952 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.y 2024-04-25T00:45:18.265 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:45:18 smithi106 systemd[1]: Stopping Ceph mgr.y for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:18.569 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:45:18 smithi106 podman[74075]: 2024-04-25 00:45:18.265080245 +0000 UTC m=+0.118396647 container died 5077203a74ec9fa8b616b5221bd42d31147727051055918ebd78cdf0c959dbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:45:18.838 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:45:18 smithi106 podman[74075]: 2024-04-25 00:45:18.660004465 +0000 UTC m=+0.513320856 container cleanup 5077203a74ec9fa8b616b5221bd42d31147727051055918ebd78cdf0c959dbc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T00:45:18.838 INFO:journalctl@ceph.mgr.y.smithi106.stdout:Apr 25 00:45:18 smithi106 bash[74075]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-y 2024-04-25T00:45:19.000 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T00:45:19.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:19.034 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T00:45:19.034 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T00:45:19.034 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.z 2024-04-25T00:45:19.513 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:45:19 smithi136 systemd[1]: Stopping Ceph mgr.z for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:19.978 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:45:19 smithi136 podman[73261]: 2024-04-25 00:45:19.664040815 +0000 UTC m=+0.440046230 container died d8b0fe708cd71aa4937b2273b7d05ba6b7785e64ea5c458349a4ebe956e390d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T00:45:20.264 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:45:20 smithi136 podman[73261]: 2024-04-25 00:45:20.076527483 +0000 UTC m=+0.852532885 container cleanup d8b0fe708cd71aa4937b2273b7d05ba6b7785e64ea5c458349a4ebe956e390d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T00:45:20.264 INFO:journalctl@ceph.mgr.z.smithi136.stdout:Apr 25 00:45:20 smithi136 bash[73261]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mgr-z 2024-04-25T00:45:20.450 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T00:45:20.483 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:20.484 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T00:45:20.484 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T00:45:20.484 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.0 2024-04-25T00:45:20.876 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:20 smithi042 systemd[1]: Stopping Ceph osd.0 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:21.377 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:21 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0[42810]: 2024-04-25T00:45:21.115+0000 7fbf6ae00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:21.377 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:21 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0[42810]: 2024-04-25T00:45:21.115+0000 7fbf6ae00700 -1 osd.0 94 *** Got signal Terminated *** 2024-04-25T00:45:21.377 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:21 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0[42810]: 2024-04-25T00:45:21.115+0000 7fbf6ae00700 -1 osd.0 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:45:27.132 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:26 smithi042 podman[94445]: 2024-04-25 00:45:26.874323358 +0000 UTC m=+5.919121355 container died 2c8d9311c283820ffde8d73d1e5a781d40e00a94ea0e70e3e1468908c5e92c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T00:45:27.519 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:27 smithi042 podman[94445]: 2024-04-25 00:45:27.299931552 +0000 UTC m=+6.344729552 container cleanup 2c8d9311c283820ffde8d73d1e5a781d40e00a94ea0e70e3e1468908c5e92c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T00:45:27.519 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:27 smithi042 bash[94445]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0 2024-04-25T00:45:27.877 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:27 smithi042 podman[94468]: 2024-04-25 00:45:27.517893312 +0000 UTC m=+0.624945053 container remove 2c8d9311c283820ffde8d73d1e5a781d40e00a94ea0e70e3e1468908c5e92c63 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-25T00:45:28.877 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:28 smithi042 podman[94553]: 2024-04-25 00:45:28.313520216 +0000 UTC m=+0.034478091 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:45:28.877 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:28 smithi042 podman[94553]: 2024-04-25 00:45:28.468905687 +0000 UTC m=+0.189863604 container create ed5d1ed4a398bcc04a8b698848739dc5390189bda050965a86dcb10c0efc7378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True) 2024-04-25T00:45:29.377 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:28 smithi042 podman[94553]: 2024-04-25 00:45:28.89223357 +0000 UTC m=+0.613191504 container init ed5d1ed4a398bcc04a8b698848739dc5390189bda050965a86dcb10c0efc7378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T00:45:29.377 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:28 smithi042 podman[94553]: 2024-04-25 00:45:28.904132355 +0000 UTC m=+0.625090260 container start ed5d1ed4a398bcc04a8b698848739dc5390189bda050965a86dcb10c0efc7378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T00:45:29.378 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:29 smithi042 podman[94553]: 2024-04-25 00:45:29.016145895 +0000 UTC m=+0.737103800 container attach ed5d1ed4a398bcc04a8b698848739dc5390189bda050965a86dcb10c0efc7378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:45:29.876 INFO:journalctl@ceph.osd.0.smithi042.stdout:Apr 25 00:45:29 smithi042 podman[94553]: 2024-04-25 00:45:29.387686518 +0000 UTC m=+1.108644465 container died ed5d1ed4a398bcc04a8b698848739dc5390189bda050965a86dcb10c0efc7378 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-25T00:45:30.342 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T00:45:30.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:30.471 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T00:45:30.471 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T00:45:30.472 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.1 2024-04-25T00:45:30.790 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:30 smithi106 systemd[1]: Stopping Ceph osd.1 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:31.088 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:30 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1[37115]: 2024-04-25T00:45:30.787+0000 7ffa91964700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:31.088 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:30 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1[37115]: 2024-04-25T00:45:30.787+0000 7ffa91964700 -1 osd.1 94 *** Got signal Terminated *** 2024-04-25T00:45:31.088 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:30 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1[37115]: 2024-04-25T00:45:30.787+0000 7ffa91964700 -1 osd.1 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:45:36.658 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:36 smithi106 podman[74197]: 2024-04-25 00:45:36.350934895 +0000 UTC m=+5.689749150 container stop e1d5e5fe39db71295d0573d871c1d27b55b680cc5e57b689b1487d85625f7f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424) 2024-04-25T00:45:36.658 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:36 smithi106 podman[74197]: 2024-04-25 00:45:36.376750945 +0000 UTC m=+5.715565210 container died e1d5e5fe39db71295d0573d871c1d27b55b680cc5e57b689b1487d85625f7f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:45:36.949 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:36 smithi106 podman[74197]: 2024-04-25 00:45:36.756058676 +0000 UTC m=+6.094872931 container cleanup e1d5e5fe39db71295d0573d871c1d27b55b680cc5e57b689b1487d85625f7f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-25T00:45:36.949 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:36 smithi106 bash[74197]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1 2024-04-25T00:45:37.338 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:36 smithi106 podman[74227]: 2024-04-25 00:45:36.949135855 +0000 UTC m=+0.591587274 container remove e1d5e5fe39db71295d0573d871c1d27b55b680cc5e57b689b1487d85625f7f46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:45:37.695 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:37 smithi106 podman[74318]: 2024-04-25 00:45:37.273260077 +0000 UTC m=+0.018738286 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:45:37.695 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:37 smithi106 podman[74318]: 2024-04-25 00:45:37.400668814 +0000 UTC m=+0.146147009 container create 5cb28f8a92ad8438ac029c1bae0408240267865f6d44d5a933473d378eaabb9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T00:45:38.002 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:37 smithi106 podman[74318]: 2024-04-25 00:45:37.694901731 +0000 UTC m=+0.440379930 container init 5cb28f8a92ad8438ac029c1bae0408240267865f6d44d5a933473d378eaabb9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:45:38.002 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:37 smithi106 podman[74318]: 2024-04-25 00:45:37.698608306 +0000 UTC m=+0.444086503 container start 5cb28f8a92ad8438ac029c1bae0408240267865f6d44d5a933473d378eaabb9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T00:45:38.002 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:37 smithi106 podman[74318]: 2024-04-25 00:45:37.778095258 +0000 UTC m=+0.523573456 container attach 5cb28f8a92ad8438ac029c1bae0408240267865f6d44d5a933473d378eaabb9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T00:45:38.298 INFO:journalctl@ceph.osd.1.smithi106.stdout:Apr 25 00:45:38 smithi106 podman[74318]: 2024-04-25 00:45:37.999639956 +0000 UTC m=+0.745118161 container died 5cb28f8a92ad8438ac029c1bae0408240267865f6d44d5a933473d378eaabb9b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T00:45:38.655 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T00:45:38.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:38.696 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T00:45:38.696 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T00:45:38.696 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.2 2024-04-25T00:45:39.013 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:38 smithi136 systemd[1]: Stopping Ceph osd.2 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:39.263 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2[38101]: 2024-04-25T00:45:39.011+0000 7f5eecf99700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:39.263 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2[38101]: 2024-04-25T00:45:39.011+0000 7f5eecf99700 -1 osd.2 94 *** Got signal Terminated *** 2024-04-25T00:45:39.263 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2[38101]: 2024-04-25T00:45:39.011+0000 7f5eecf99700 -1 osd.2 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:45:44.905 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:44 smithi136 podman[73387]: 2024-04-25 00:45:44.620434915 +0000 UTC m=+5.737818254 container died 444626d169799d7f66f792f8f6a9adae4abcfe5df08e87ec61ff108c94e76a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T00:45:45.238 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:45 smithi136 podman[73387]: 2024-04-25 00:45:45.004046809 +0000 UTC m=+6.121430169 container cleanup 444626d169799d7f66f792f8f6a9adae4abcfe5df08e87ec61ff108c94e76a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:45:45.238 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:45 smithi136 bash[73387]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2 2024-04-25T00:45:45.513 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:45 smithi136 podman[73417]: 2024-04-25 00:45:45.238858006 +0000 UTC m=+0.614559749 container remove 444626d169799d7f66f792f8f6a9adae4abcfe5df08e87ec61ff108c94e76a35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-25T00:45:45.825 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:45 smithi136 podman[73509]: 2024-04-25 00:45:45.539990945 +0000 UTC m=+0.107881642 container create 94742fe7b82beb1b38decfc0e8972c6fd6fc664609eeeb637f5a1ff74df81c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T00:45:45.825 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:45 smithi136 podman[73509]: 2024-04-25 00:45:45.44979774 +0000 UTC m=+0.017688454 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:45:45.826 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:45 smithi136 podman[73509]: 2024-04-25 00:45:45.734518871 +0000 UTC m=+0.302409579 container init 94742fe7b82beb1b38decfc0e8972c6fd6fc664609eeeb637f5a1ff74df81c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T00:45:45.826 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:45 smithi136 podman[73509]: 2024-04-25 00:45:45.737975456 +0000 UTC m=+0.305866156 container start 94742fe7b82beb1b38decfc0e8972c6fd6fc664609eeeb637f5a1ff74df81c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-25T00:45:45.826 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:45 smithi136 podman[73509]: 2024-04-25 00:45:45.82577359 +0000 UTC m=+0.393664286 container attach 94742fe7b82beb1b38decfc0e8972c6fd6fc664609eeeb637f5a1ff74df81c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:45:46.587 INFO:journalctl@ceph.osd.2.smithi136.stdout:Apr 25 00:45:46 smithi136 podman[73509]: 2024-04-25 00:45:46.29799444 +0000 UTC m=+0.865885130 container died 94742fe7b82beb1b38decfc0e8972c6fd6fc664609eeeb637f5a1ff74df81c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:45:46.853 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T00:45:46.891 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:46.891 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T00:45:46.891 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T00:45:46.891 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.3 2024-04-25T00:45:47.377 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:47 smithi042 systemd[1]: Stopping Ceph osd.3 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:47.877 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:47 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3[49500]: 2024-04-25T00:45:47.572+0000 7f107b000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:47.877 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:47 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3[49500]: 2024-04-25T00:45:47.572+0000 7f107b000700 -1 osd.3 94 *** Got signal Terminated *** 2024-04-25T00:45:47.877 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:47 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3[49500]: 2024-04-25T00:45:47.572+0000 7f107b000700 -1 osd.3 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:45:48.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:47.630+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:48.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:47 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:47.867+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:49.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:48 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:48.659+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:49.123 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:48 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3[49500]: 2024-04-25T00:45:48.786+0000 7f1077200700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.473462+0000 front 2024-04-25T00:45:23.474460+0000 (oldest deadline 2024-04-25T00:45:48.172680+0000) 2024-04-25T00:45:49.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:48 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:48.913+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:49.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:49 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:49.135+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:49.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:49 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:49.122+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:50.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:49 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:49.702+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:50.106 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:49 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3[49500]: 2024-04-25T00:45:49.788+0000 7f1077200700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.473462+0000 front 2024-04-25T00:45:23.474460+0000 (oldest deadline 2024-04-25T00:45:48.172680+0000) 2024-04-25T00:45:50.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:49 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:49.880+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:50.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:50.127+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:50.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:50 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:50.112+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:50.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:50 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:50.105+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:45:51.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:50 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:50.658+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:51.108 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:50 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3[49500]: 2024-04-25T00:45:50.739+0000 7f1077200700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.473462+0000 front 2024-04-25T00:45:23.474460+0000 (oldest deadline 2024-04-25T00:45:48.172680+0000) 2024-04-25T00:45:51.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:50.893+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:51.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:51 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:51.094+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:51.376 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:51 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:51.107+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:51.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:51 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:51.152+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:45:52.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:51 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:51.633+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:52.114 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:51 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3[49500]: 2024-04-25T00:45:51.748+0000 7f1077200700 -1 osd.3 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.473462+0000 front 2024-04-25T00:45:23.474460+0000 (oldest deadline 2024-04-25T00:45:48.172680+0000) 2024-04-25T00:45:52.127 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:51 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:51.857+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:52.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:52 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:52.112+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:52.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:52 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:52.118+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:45:52.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:52.127+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:53.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:52.619+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:53.155 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:52.841+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:53.477 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:53 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:53.155+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:45:53.477 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:53 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:53.159+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:53.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:53.154+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:53.767 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:53 smithi042 podman[94669]: 2024-04-25 00:45:53.476941388 +0000 UTC m=+6.124999652 container died d715bea2b3138ecd291e764989c5dd406c2ce8d4257372f58bd942fe6df1f708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T00:45:54.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:53 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:53.602+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:54.124 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:53 smithi042 podman[94669]: 2024-04-25 00:45:53.931104181 +0000 UTC m=+6.579162489 container cleanup d715bea2b3138ecd291e764989c5dd406c2ce8d4257372f58bd942fe6df1f708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T00:45:54.124 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:53 smithi042 bash[94669]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3 2024-04-25T00:45:54.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:53.873+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:54.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:54.115+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:54.376 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:54 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:54.123+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:45:54.377 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:54 smithi042 podman[94687]: 2024-04-25 00:45:54.219467524 +0000 UTC m=+0.722489549 container remove d715bea2b3138ecd291e764989c5dd406c2ce8d4257372f58bd942fe6df1f708 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:45:54.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:54 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:54.124+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:55.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:54 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:54.598+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:55.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:54.906+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:55.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:55.114+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:55.376 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:55 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:55.120+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:45:55.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:55 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:55.144+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:55.377 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:55 smithi042 podman[94774]: 2024-04-25 00:45:54.997151262 +0000 UTC m=+0.039982223 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:45:55.377 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:55 smithi042 podman[94774]: 2024-04-25 00:45:55.12595342 +0000 UTC m=+0.168784406 container create fb1c259a562cbd0a04d92be4c9abc18dda15f0605579b01fbde810300263dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-04-25T00:45:55.877 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:55 smithi042 podman[94774]: 2024-04-25 00:45:55.466426533 +0000 UTC m=+0.509257582 container init fb1c259a562cbd0a04d92be4c9abc18dda15f0605579b01fbde810300263dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-25T00:45:55.877 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:55 smithi042 podman[94774]: 2024-04-25 00:45:55.478918949 +0000 UTC m=+0.521749939 container start fb1c259a562cbd0a04d92be4c9abc18dda15f0605579b01fbde810300263dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T00:45:55.877 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:55 smithi042 podman[94774]: 2024-04-25 00:45:55.554056417 +0000 UTC m=+0.596887470 container attach fb1c259a562cbd0a04d92be4c9abc18dda15f0605579b01fbde810300263dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T00:45:56.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:55 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:55.634+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:56.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:55.897+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:56.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:56.097+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:56.303 INFO:journalctl@ceph.osd.3.smithi042.stdout:Apr 25 00:45:55 smithi042 podman[94774]: 2024-04-25 00:45:55.943135806 +0000 UTC m=+0.985966762 container died fb1c259a562cbd0a04d92be4c9abc18dda15f0605579b01fbde810300263dde0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-25T00:45:56.304 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:56.091+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:45:56.304 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:56 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:56.168+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:56.744 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T00:45:56.869 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:45:56.869 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T00:45:56.869 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T00:45:56.870 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.4 2024-04-25T00:45:56.896 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:56 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:56.608+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:56.896 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:45:56 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:45:56.643+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:45:57.160 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:45:56 smithi106 systemd[1]: Stopping Ceph osd.4 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:45:57.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:56.880+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:57.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:57.123+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:57.376 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:57 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:57.203+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:57.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:57 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:57.068+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:45:57.560 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:45:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-25T00:45:57.159+0000 7f597f4a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:45:57.560 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:45:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-25T00:45:57.159+0000 7f597f4a4700 -1 osd.4 94 *** Got signal Terminated *** 2024-04-25T00:45:57.560 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:45:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-25T00:45:57.159+0000 7f597f4a4700 -1 osd.4 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:45:57.560 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:45:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-25T00:45:57.404+0000 7f597c299700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:33.547353+0000 front 2024-04-25T00:45:33.547327+0000 (oldest deadline 2024-04-25T00:45:56.447473+0000) 2024-04-25T00:45:57.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:45:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:45:57.443+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:45:57.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:45:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:45:57.683+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:45:57.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:57 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:57.560+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:58.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:58.147+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:58.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:57.927+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:58.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:57.927+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:45:58.376 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:58 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:58.184+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:58.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:58 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:58.118+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:45:58.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:58 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:58.118+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:45:58.692 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:45:58 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-25T00:45:58.386+0000 7f597c299700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:33.547353+0000 front 2024-04-25T00:45:33.547327+0000 (oldest deadline 2024-04-25T00:45:56.447473+0000) 2024-04-25T00:45:58.693 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:58 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:58.532+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:58.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:45:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:45:58.406+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:45:59.088 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:45:58 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:45:58.692+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:45:59.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:58.949+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:45:59.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:58.949+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:45:59.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:59.168+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:45:59.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:45:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:45:59.168+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:45:59.376 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:59 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:59.153+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:45:59.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:45:59 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:45:59.153+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:45:59.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:59 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:59.069+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:45:59.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:45:59 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:45:59.069+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:45:59.691 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:45:59 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:45:59.563+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:45:59.692 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:45:59 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-25T00:45:59.343+0000 7f597c299700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:33.547353+0000 front 2024-04-25T00:45:33.547327+0000 (oldest deadline 2024-04-25T00:45:56.447473+0000) 2024-04-25T00:45:59.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:45:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:45:59.400+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:00.088 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:45:59 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:45:59.691+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:00.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:59.915+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:00.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:45:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:45:59.915+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:00.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:00 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:00.137+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:00.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:00 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:00.137+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:00.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:00 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:00.148+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:00.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:00 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:00.148+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:00.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:00 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:00.091+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:00.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:00 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:00.091+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:00.701 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:00 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-25T00:46:00.383+0000 7f597c299700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:33.547353+0000 front 2024-04-25T00:45:33.547327+0000 (oldest deadline 2024-04-25T00:45:56.447473+0000) 2024-04-25T00:46:00.701 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:00 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:00.591+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:00.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:00 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:00.432+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:00.972 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:00 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:00.701+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:01.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:00 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:00.941+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:01.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:00 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:00.941+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:01.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:01 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:01.143+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:01.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:01 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:01.143+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:01.309 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:01 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:01.172+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:01.309 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:01 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:01.172+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:01.309 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:01 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:01.042+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:01.310 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:01 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:01.042+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:01.732 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:01 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-25T00:46:01.369+0000 7f597c299700 -1 osd.4 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:33.547353+0000 front 2024-04-25T00:45:33.547327+0000 (oldest deadline 2024-04-25T00:45:56.447473+0000) 2024-04-25T00:46:01.733 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:01 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:01.601+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:01.733 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:01 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:01.601+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:01.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:01 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:01.408+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:02.088 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:01 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:01.731+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:02.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:01 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:01.939+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:02.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:01 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:01.939+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:02.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:02 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:02.187+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:02.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:02 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:02.187+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:02.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:02 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:02.138+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:02.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:02 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:02.138+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:02.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:02 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:02.091+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:02.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:02 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:02.091+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:02.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:02 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:02.407+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:03.002 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:02 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:02.706+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:03.003 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:02 smithi106 podman[74426]: 2024-04-25 00:46:02.743734693 +0000 UTC m=+5.675153473 container died 9a00358cb52626eee26cdd362e24453cdcc6dd84da005df400fcf9bcdebf87f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-25T00:46:03.003 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:02 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:02.611+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:03.003 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:02 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:02.611+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:03.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:03 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:03.214+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:03.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:03 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:03.214+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:03.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:02 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:02.983+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:03.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:02 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:02.983+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:03.338 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:03 smithi106 podman[74426]: 2024-04-25 00:46:03.085092875 +0000 UTC m=+6.016511653 container cleanup 9a00358cb52626eee26cdd362e24453cdcc6dd84da005df400fcf9bcdebf87f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:46:03.338 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:03 smithi106 bash[74426]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4 2024-04-25T00:46:03.338 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:03 smithi106 podman[74456]: 2024-04-25 00:46:03.244294086 +0000 UTC m=+0.501440449 container remove 9a00358cb52626eee26cdd362e24453cdcc6dd84da005df400fcf9bcdebf87f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-04-25T00:46:03.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:03 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:03.132+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:03.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:03 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:03.132+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:03.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:03 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:03.095+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:03.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:03 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:03.095+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:03.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:03 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:03.433+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:03.922 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:03 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:03.659+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:03.923 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:03 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:03.605+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:03.923 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:03 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:03.605+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:03.923 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:03 smithi106 podman[74549]: 2024-04-25 00:46:03.553801412 +0000 UTC m=+0.017878385 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:46:03.923 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:03 smithi106 podman[74549]: 2024-04-25 00:46:03.678578069 +0000 UTC m=+0.142655016 container create 04dcbb74a99dfff78d13fb4412c9e0e7bdc6e9c23007386cf0eb63f70a7f7036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T00:46:03.923 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:03 smithi106 podman[74549]: 2024-04-25 00:46:03.922879316 +0000 UTC m=+0.386956272 container init 04dcbb74a99dfff78d13fb4412c9e0e7bdc6e9c23007386cf0eb63f70a7f7036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T00:46:04.216 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:03 smithi106 podman[74549]: 2024-04-25 00:46:03.926582982 +0000 UTC m=+0.390659940 container start 04dcbb74a99dfff78d13fb4412c9e0e7bdc6e9c23007386cf0eb63f70a7f7036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True) 2024-04-25T00:46:04.216 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:04 smithi106 podman[74549]: 2024-04-25 00:46:04.006433339 +0000 UTC m=+0.470510300 container attach 04dcbb74a99dfff78d13fb4412c9e0e7bdc6e9c23007386cf0eb63f70a7f7036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T00:46:04.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:03 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:03.984+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:04.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:03 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:03.984+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:04.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:04 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:04.208+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:04.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:04 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:04.208+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:04.376 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:04 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:04.084+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:04.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:04 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:04.084+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:04.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:04 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:04.062+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:04.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:04 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:04.062+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:04.500 INFO:journalctl@ceph.osd.4.smithi106.stdout:Apr 25 00:46:04 smithi106 podman[74549]: 2024-04-25 00:46:04.216551576 +0000 UTC m=+0.680628532 container died 04dcbb74a99dfff78d13fb4412c9e0e7bdc6e9c23007386cf0eb63f70a7f7036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T00:46:04.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:04 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:04.463+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:04.768 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:04 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:04.612+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:04.768 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:04 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:04.573+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:04.768 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:04 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:04.573+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:04.788 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T00:46:04.867 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:46:04.867 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T00:46:04.868 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T00:46:04.868 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.5 2024-04-25T00:46:05.165 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:04 smithi136 systemd[1]: Stopping Ceph osd.5 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:46:05.165 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:05.008+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:05.165 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:05.008+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:05.165 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:05.164+0000 7f631880c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:46:05.165 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:05.164+0000 7f631880c700 -1 osd.5 94 *** Got signal Terminated *** 2024-04-25T00:46:05.376 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:05 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:05.017+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:05.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:05 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:05.017+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:05.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:05 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:05.069+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:05.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:05 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:05.069+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:05.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:05 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:05.069+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:05.506 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:05.164+0000 7f631880c700 -1 osd.5 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:46:05.506 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:05.168+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:05.506 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:05.168+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:05.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:05.506+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:05.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:05 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:05.506+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:05.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:05 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:05.564+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:05.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:05 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:05.547+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:05.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:05 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:05.547+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:06.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:06 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:06.142+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:06.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:06 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:06.142+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:06.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:06 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:06.008+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:06.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:06 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:06.008+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:06.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:06 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:06.008+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:39.823965+0000 front 2024-04-25T00:45:39.823874+0000 (oldest deadline 2024-04-25T00:46:05.724016+0000) 2024-04-25T00:46:06.376 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:06 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:06.038+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:06.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:06 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:06.038+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:06.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:06 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:06.099+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:06.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:06 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:06.099+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:06.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:06 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:06.099+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:06.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:06 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:06.498+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:06.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:06 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:06.498+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:06.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:06 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:06.574+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:06.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:06 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:06.541+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:06.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:06 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:06.541+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:06.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:06 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:06.541+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:07.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:07 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:07.189+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:07.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:07 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:07.189+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:07.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:07 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:07.005+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:07.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:07 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:07.005+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:07.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:07 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:07.005+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:39.823965+0000 front 2024-04-25T00:45:39.823874+0000 (oldest deadline 2024-04-25T00:46:05.724016+0000) 2024-04-25T00:46:07.376 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:07 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:07.042+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:07.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:07 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:07.042+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:07.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:07 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:07.112+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:07.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:07 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:07.112+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:07.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:07 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:07.112+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:07.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:07 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:07.457+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:07.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:07 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:07.457+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:07.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:07 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:07.572+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:07.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:07 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:07.570+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:07.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:07 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:07.570+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:07.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:07 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:07.570+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:08.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:08 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:08.139+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:08.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:08 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:08.139+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:08.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:07 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:07.965+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:08.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:07 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:07.965+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:08.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:07 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:07.965+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:39.823965+0000 front 2024-04-25T00:45:39.823874+0000 (oldest deadline 2024-04-25T00:46:05.724016+0000) 2024-04-25T00:46:08.376 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:08 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:08.067+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:08.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:08 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:08.067+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:08.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:08 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:08.132+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:08.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:08 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:08.132+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:08.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:08 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:08.132+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:08.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:08 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:08.433+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:08.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:08 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:08.433+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:08.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:08 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:08.561+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:08.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:08 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:08.561+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:08.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:08 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:08.582+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:08.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:08 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:08.582+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:08.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:08 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:08.582+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:09.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:08 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:08.940+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:09.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:08 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:08.940+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:09.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:08 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:08.940+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:39.823965+0000 front 2024-04-25T00:45:39.823874+0000 (oldest deadline 2024-04-25T00:46:05.724016+0000) 2024-04-25T00:46:09.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:09 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:09.108+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:09.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:09 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:09.108+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:09.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:09 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:09.108+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:09.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:09 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:09.105+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:09.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:09 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:09.105+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:09.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:09 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:09.105+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:09.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:09 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:09.048+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:09.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:09 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:09.048+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:09.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:09 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:09.048+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:09.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:09 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:09.415+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:09.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:09 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:09.415+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:09.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:09 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:09.553+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:09.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:09 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:09.553+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:09.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:09 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:09.623+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:09.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:09 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:09.623+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:09.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:09 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:09.623+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:10.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:09 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:09.929+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.620101+0000 front 2024-04-25T00:45:23.620321+0000 (oldest deadline 2024-04-25T00:45:47.620691+0000) 2024-04-25T00:46:10.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:09 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:09.929+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.321941+0000 front 2024-04-25T00:45:32.322080+0000 (oldest deadline 2024-04-25T00:45:57.622032+0000) 2024-04-25T00:46:10.264 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:09 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5[45086]: 2024-04-25T00:46:09.929+0000 7f63115f9700 -1 osd.5 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:39.823965+0000 front 2024-04-25T00:45:39.823874+0000 (oldest deadline 2024-04-25T00:46:05.724016+0000) 2024-04-25T00:46:10.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:10 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:10.096+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:10.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:10 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:10.096+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:10.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:10 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:10.096+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:10.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:10 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:10.111+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:10.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:10 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:10.111+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:10.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:10 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:10.111+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:10.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:10 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:10.019+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:10.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:10 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:10.019+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:10.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:10 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:10.019+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:10.763 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:10 smithi136 podman[73613]: 2024-04-25 00:46:10.570447409 +0000 UTC m=+5.506909521 container died adc0a9c524c1bd3d0a0d2c398f8bf00bf628597404f94056ebe13765c134bced (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:46:10.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:10 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:10.365+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:10.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:10 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:10.365+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:10.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:10 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:10.548+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:10.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:10 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:10.548+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:10.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:10 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:10.593+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:10.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:10 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:10.593+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:10.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:10 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:10.593+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:11.233 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:10 smithi136 podman[73613]: 2024-04-25 00:46:10.948087956 +0000 UTC m=+5.884550064 container cleanup adc0a9c524c1bd3d0a0d2c398f8bf00bf628597404f94056ebe13765c134bced (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T00:46:11.233 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:10 smithi136 bash[73613]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5 2024-04-25T00:46:11.234 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:11 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:11.083+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:11.234 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:11 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:11.083+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:11.234 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:11 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:11.083+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:11.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:11 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:11.154+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:11.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:11 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:11.154+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:11.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:11 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:11.154+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:11.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:11 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:11.046+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:11.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:11 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:11.046+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:11.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:11 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:11.046+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:11.514 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:11 smithi136 podman[73632]: 2024-04-25 00:46:11.233059861 +0000 UTC m=+0.662065431 container remove adc0a9c524c1bd3d0a0d2c398f8bf00bf628597404f94056ebe13765c134bced (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-25T00:46:11.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:11 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:11.361+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:11.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:11 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:11.361+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:11.820 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:11 smithi136 podman[73720]: 2024-04-25 00:46:11.443786257 +0000 UTC m=+0.018193568 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:46:11.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:11 smithi136 podman[73720]: 2024-04-25 00:46:11.551003468 +0000 UTC m=+0.125410765 container create c6bb147da1f99bdd19da67d22024f825d25ef67f82e71cad0bc366e5953e0e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5) 2024-04-25T00:46:11.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:11 smithi136 podman[73720]: 2024-04-25 00:46:11.761970384 +0000 UTC m=+0.336377684 container init c6bb147da1f99bdd19da67d22024f825d25ef67f82e71cad0bc366e5953e0e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:46:11.848 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:11 smithi136 podman[73720]: 2024-04-25 00:46:11.765208008 +0000 UTC m=+0.339615308 container start c6bb147da1f99bdd19da67d22024f825d25ef67f82e71cad0bc366e5953e0e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T00:46:11.849 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:11 smithi136 podman[73720]: 2024-04-25 00:46:11.820440683 +0000 UTC m=+0.394847993 container attach c6bb147da1f99bdd19da67d22024f825d25ef67f82e71cad0bc366e5953e0e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:46:11.849 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:11 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:11.563+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:11.849 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:11 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:11.563+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:11.849 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:11 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:11.579+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:11.850 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:11 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:11.579+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:11.850 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:11 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:11.579+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:12.263 INFO:journalctl@ceph.osd.5.smithi136.stdout:Apr 25 00:46:12 smithi136 podman[73720]: 2024-04-25 00:46:12.049837505 +0000 UTC m=+0.624244814 container died c6bb147da1f99bdd19da67d22024f825d25ef67f82e71cad0bc366e5953e0e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9) 2024-04-25T00:46:12.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:12 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:12.049+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:12.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:12 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:12.049+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:12.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:12 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:12.049+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:12.376 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:12 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:12.197+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:12.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:12 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:12.197+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:12.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:12 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:12.197+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:12.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:12 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:12.046+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:12.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:12 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:12.046+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:12.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:12 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:12.046+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:12.624 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:12 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:12.358+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:12.624 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:12 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:12.358+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:12.643 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T00:46:12.680 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:46:12.681 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T00:46:12.681 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T00:46:12.681 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.6 2024-04-25T00:46:12.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:12 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:12.609+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:12.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:12 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:12.609+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:12.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:12 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:12.564+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:12.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:12 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:12.564+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:12.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:12 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:12.564+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:13.058 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:12 smithi042 systemd[1]: Stopping Ceph osd.6 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:46:13.368 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:13 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:13.075+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:13.368 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:13 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:13.075+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:13.368 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:13 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:13.075+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:13.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:13 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:13.062+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:13.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:13 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:13.062+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:13.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:13 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:13.062+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:13.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:13 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:13.202+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:13.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:13 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:13.202+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:13.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:13 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:13.202+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:13.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:13 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:13.251+0000 7f8dd3800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:46:13.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:13 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:13.251+0000 7f8dd3800700 -1 osd.6 94 *** Got signal Terminated *** 2024-04-25T00:46:13.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:13 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:13.251+0000 7f8dd3800700 -1 osd.6 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:46:13.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:13 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:13.367+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:13.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:13 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:13.367+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:13.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:13 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:13.581+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:13.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:13 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:13.581+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:13.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:13 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:13.580+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:13.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:13 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:13.580+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:13.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:13 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:13.580+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:14.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:14 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:14.195+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:14.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:14 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:14.195+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:14.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:14 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:14.195+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:14.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:14 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:14.059+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:14.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:14 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:14.059+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:14.378 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:14 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:14.059+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:14.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:14 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:14.325+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:14.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:14 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:14.325+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:14.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:14 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:14.084+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:14.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:14 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:14.084+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:14.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:14 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:14.084+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:14.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:14 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:14.540+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:14.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:14 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:14.540+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:14.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:14 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:14.540+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:14.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:14 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:14.560+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:14.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:14 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:14.560+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:14.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:14 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:14.560+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:15.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:15 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:15.042+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:15.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:15 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:15.042+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:15.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:15 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:15.042+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:15.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:15 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:15.150+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:15.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:15 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:15.150+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:15.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:15 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:15.150+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:15.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:15 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:15.150+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:51.028090+0000 front 2024-04-25T00:45:51.028467+0000 (oldest deadline 2024-04-25T00:46:15.127641+0000) 2024-04-25T00:46:15.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:15 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:15.316+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:15.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:15 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:15.316+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:15.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:15 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:15.106+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:15.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:15 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:15.106+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:15.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:15 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:15.106+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:15.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:15 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:15.540+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:15.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:15 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:15.540+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:15.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:15 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:15.540+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:15.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:15 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:15.511+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:15.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:15 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:15.511+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:15.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:15 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:15.511+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:16.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:16.166+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:16.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:16.166+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:16.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:16.166+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:16.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:16.166+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:51.028090+0000 front 2024-04-25T00:45:51.028467+0000 (oldest deadline 2024-04-25T00:46:15.127641+0000) 2024-04-25T00:46:16.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:16.010+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:16.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:16.010+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:16.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:16.010+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:16.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:16 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:16.284+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:16.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:16 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:16.284+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:16.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:16 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:16.139+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:16.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:16 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:16.139+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:16.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:16 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:16.139+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:16.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:16 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:16.516+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:16.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:16 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:16.516+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:16.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:16 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:16.516+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:16.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:16 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:16.545+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:16.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:16 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:16.545+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:16.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:16 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:16.545+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:16.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:16 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:16.545+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:17.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:16.971+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:17.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:16.971+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:17.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:16 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:16.971+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:17.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:17.202+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:17.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:17.202+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:17.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:17.202+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:17.377 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:17.202+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:51.028090+0000 front 2024-04-25T00:45:51.028467+0000 (oldest deadline 2024-04-25T00:46:15.127641+0000) 2024-04-25T00:46:17.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:17 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:17.308+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:17.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:17 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:17.308+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:17.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:17 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:17.308+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:17.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:17 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:17.172+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:17.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:17 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:17.172+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:17.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:17 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:17.172+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:17.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:17 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:17.505+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:17.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:17 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:17.505+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:17.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:17 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:17.505+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:17.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:17 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:17.529+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:17.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:17 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:17.529+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:17.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:17 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:17.529+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:17.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:17 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:17.529+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:18.195 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:17.942+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:18.195 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:17.942+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:18.195 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:17 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:17.942+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:18.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:18 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:18.324+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:18.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:18 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:18.324+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:18.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:18 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:18.324+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:18.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:18 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:18.157+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:18.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:18 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:18.158+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:18.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:18 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:18.158+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:18.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:18 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:18.158+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:18.627 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:18 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:18.194+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:23.616037+0000 front 2024-04-25T00:45:23.616109+0000 (oldest deadline 2024-04-25T00:45:48.316182+0000) 2024-04-25T00:46:18.627 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:18 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:18.194+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.920680+0000 front 2024-04-25T00:45:32.920618+0000 (oldest deadline 2024-04-25T00:45:58.820878+0000) 2024-04-25T00:46:18.627 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:18 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:18.194+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:38.822297+0000 front 2024-04-25T00:45:38.822205+0000 (oldest deadline 2024-04-25T00:46:04.623900+0000) 2024-04-25T00:46:18.627 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:18 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6[56031]: 2024-04-25T00:46:18.194+0000 7f8dcfa00700 -1 osd.6 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:51.028090+0000 front 2024-04-25T00:45:51.028467+0000 (oldest deadline 2024-04-25T00:46:15.127641+0000) 2024-04-25T00:46:18.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:18 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:18.503+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:18.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:18 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:18.503+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:18.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:18 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:18.503+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:18.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:18 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:18.537+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:18.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:18 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:18.537+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:18.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:18 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:18.537+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:18.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:18 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:18.537+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:19.214 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:18 smithi042 podman[94891]: 2024-04-25 00:46:18.897713536 +0000 UTC m=+5.779319758 container died dd0437b5cb7c0ef38670581c1b5bdde2cfb2badd214fcfdd5872ac858ca91179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, RELEASE=squid-d2c45d9) 2024-04-25T00:46:19.215 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:18 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:18.985+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:19.215 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:18 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:18.985+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:19.215 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:18 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:18.985+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:19.215 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:18 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:18.985+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:19.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:19 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:19.310+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:19.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:19 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:19.310+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:19.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:19 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:19.310+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:19.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:19 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:19.125+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:19.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:19 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:19.125+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:19.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:19 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:19.125+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:19.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:19 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:19.125+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:19.571 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:19 smithi042 podman[94891]: 2024-04-25 00:46:19.336472503 +0000 UTC m=+6.218078714 container cleanup dd0437b5cb7c0ef38670581c1b5bdde2cfb2badd214fcfdd5872ac858ca91179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:46:19.571 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:19 smithi042 bash[94891]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6 2024-04-25T00:46:19.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:19 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:19.535+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:19.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:19 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:19.535+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:19.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:19 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:19.535+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:19.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:19 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:19.496+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:19.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:19 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:19.496+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:19.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:19 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:19.496+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:19.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:19 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:19.496+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:19.876 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:19 smithi042 podman[94912]: 2024-04-25 00:46:19.570957757 +0000 UTC m=+0.663929218 container remove dd0437b5cb7c0ef38670581c1b5bdde2cfb2badd214fcfdd5872ac858ca91179 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-25T00:46:20.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:19 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:19.945+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:20.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:19 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:19.945+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:20.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:19 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:19.945+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:20.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:19 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:19.945+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:20.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:20 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:20.285+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:20.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:20 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:20.285+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:20.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:20 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:20.285+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:20.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:20 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:20.143+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:20.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:20 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:20.143+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:20.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:20 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:20.143+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:20.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:20 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:20.143+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:20.793 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:20 smithi042 podman[95000]: 2024-04-25 00:46:20.328405043 +0000 UTC m=+0.041269154 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:46:20.794 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:20 smithi042 podman[95000]: 2024-04-25 00:46:20.472067724 +0000 UTC m=+0.184931873 container create 393ce2e5ee45d49e37c15885d477bf9a2a3baf08c7091b090943f4611b78d097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T00:46:20.794 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:20 smithi042 podman[95000]: 2024-04-25 00:46:20.792554923 +0000 UTC m=+0.505419068 container init 393ce2e5ee45d49e37c15885d477bf9a2a3baf08c7091b090943f4611b78d097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9) 2024-04-25T00:46:20.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:20 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:20.520+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:20.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:20 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:20.520+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:20.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:20 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:20.520+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:20.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:20 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:20.532+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:20.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:20 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:20.532+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:20.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:20 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:20.532+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:20.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:20 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:20.532+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:21.127 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:20 smithi042 podman[95000]: 2024-04-25 00:46:20.804827232 +0000 UTC m=+0.517691349 container start 393ce2e5ee45d49e37c15885d477bf9a2a3baf08c7091b090943f4611b78d097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T00:46:21.127 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:20 smithi042 podman[95000]: 2024-04-25 00:46:20.908281303 +0000 UTC m=+0.621145488 container attach 393ce2e5ee45d49e37c15885d477bf9a2a3baf08c7091b090943f4611b78d097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9) 2024-04-25T00:46:21.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:20 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:20.942+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:21.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:20 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:20.942+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:21.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:20 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:20.942+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:21.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:20 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:20.942+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:21.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:21 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:21.335+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:21.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:21 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:21.335+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:21.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:21 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:21.335+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:21.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:21 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:21.109+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:21.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:21 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:21.109+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:21.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:21 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:21.109+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:21.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:21 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:21.109+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:21.626 INFO:journalctl@ceph.osd.6.smithi042.stdout:Apr 25 00:46:21 smithi042 podman[95000]: 2024-04-25 00:46:21.278484076 +0000 UTC m=+0.991348222 container died 393ce2e5ee45d49e37c15885d477bf9a2a3baf08c7091b090943f4611b78d097 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:46:21.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:21 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:21.543+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:21.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:21 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:21.543+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:21.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:21 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:21.543+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:21.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:21 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:21.536+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:21.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:21 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:21.536+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:21.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:21 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:21.536+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:21.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:21 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:21.536+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:22.020 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T00:46:22.142 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:46:22.142 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T00:46:22.142 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T00:46:22.143 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.7 2024-04-25T00:46:22.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:21 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:21.927+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:22.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:21 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:21.927+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:22.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:21 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:21.927+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:22.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:21 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:21.927+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:22.437 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:22 smithi106 systemd[1]: Stopping Ceph osd.7 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:46:22.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:22 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:22.289+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:22.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:22 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:22.289+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:22.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:22 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:22.289+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:22.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:22 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:22.077+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:22.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:22 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:22.077+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:22.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:22 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:22.077+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:22.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:22 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:22.077+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:22.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:22 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:22.515+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:22.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:22 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:22.515+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:22.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:22 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:22.515+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:22.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:22 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:22.515+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:22.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:22 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:22.437+0000 7fdfd53ed700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:46:22.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:22 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:22.437+0000 7fdfd53ed700 -1 osd.7 94 *** Got signal Terminated *** 2024-04-25T00:46:22.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:22 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:22.437+0000 7fdfd53ed700 -1 osd.7 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:46:22.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:22 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:22.520+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:22.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:22 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:22.520+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:22.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:22 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:22.520+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:23.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:22 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:22.941+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:23.378 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:22 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:22.941+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:23.378 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:22 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:22.941+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:23.378 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:22 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:22.941+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:23.378 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:22 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:22.941+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:23.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:23.296+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:23.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:23.296+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:23.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:23.296+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:23.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:23.296+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:23.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:23.056+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:23.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:23.056+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:23.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:23.057+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:23.514 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:23.057+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:23.515 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:23 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:23.057+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:23.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:23 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:23.496+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:23.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:23 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:23.496+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:23.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:23 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:23.496+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:23.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:23 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:23.496+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:59.123961+0000 front 2024-04-25T00:45:59.123942+0000 (oldest deadline 2024-04-25T00:46:23.224134+0000) 2024-04-25T00:46:23.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:23 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:23.539+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:23.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:23 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:23.539+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:23.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:23 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:23.539+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:23.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:23 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:23.539+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:24.300 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:24 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:24.045+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:24.300 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:24 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:24.045+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:24.300 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:24 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:24.045+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:24.300 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:24 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:24.045+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:24.300 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:24 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:24.045+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:24.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:23 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:23.970+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:24.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:23 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:23.970+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:24.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:23 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:23.970+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:24.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:23 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:23.970+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:24.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:23 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:23.970+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:24.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:24 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:24.299+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:24.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:24 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:24.299+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:24.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:24 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:24.299+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:24.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:24 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:24.299+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:24.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:24 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:24.515+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:24.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:24 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:24.515+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:24.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:24 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:24.515+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:24.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:24 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:24.515+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:59.123961+0000 front 2024-04-25T00:45:59.123942+0000 (oldest deadline 2024-04-25T00:46:23.224134+0000) 2024-04-25T00:46:24.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:24 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:24.563+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:24.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:24 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:24.563+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:24.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:24 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:24.563+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:24.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:24 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:24.563+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:25.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:25.009+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:25.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:25.009+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:25.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:25.009+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:25.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:25.009+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:25.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:25.009+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:25.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:24 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:24.957+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:25.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:24 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:24.957+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:25.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:24 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:24.957+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:25.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:24 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:24.957+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:25.378 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:24 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:24.957+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:25.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:25.294+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:25.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:25.294+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:25.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:25.294+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:25.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:25.294+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:25.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:25 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:25.540+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:25.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:25 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:25.540+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:25.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:25 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:25.540+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:25.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:25 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:25.540+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:59.123961+0000 front 2024-04-25T00:45:59.123942+0000 (oldest deadline 2024-04-25T00:46:23.224134+0000) 2024-04-25T00:46:25.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:25 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:25.598+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:25.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:25 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:25.598+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:25.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:25 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:25.598+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:25.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:25 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:25.598+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:26.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:25.968+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:26.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:25.968+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:26.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:25.968+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:26.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:25.968+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:26.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:25 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:25.968+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:26.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:25.952+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:26.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:25.952+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:26.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:25.952+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:26.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:25.952+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:26.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:25 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:25.952+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:26.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:26 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:26.272+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:26.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:26 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:26.272+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:26.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:26 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:26.272+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:26.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:26 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:26.272+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:26.838 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:26.527+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:34.718906+0000 front 2024-04-25T00:45:34.719013+0000 (oldest deadline 2024-04-25T00:45:56.419046+0000) 2024-04-25T00:46:26.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:26.527+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:41.621014+0000 front 2024-04-25T00:45:41.620902+0000 (oldest deadline 2024-04-25T00:46:08.021787+0000) 2024-04-25T00:46:26.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:26.527+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.022892+0000 front 2024-04-25T00:45:48.022532+0000 (oldest deadline 2024-04-25T00:46:13.922440+0000) 2024-04-25T00:46:26.839 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7[50805]: 2024-04-25T00:46:26.527+0000 7fdfd21e2700 -1 osd.7 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:59.123961+0000 front 2024-04-25T00:45:59.123942+0000 (oldest deadline 2024-04-25T00:46:23.224134+0000) 2024-04-25T00:46:26.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:26.638+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:26.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:26.638+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:26.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:26.638+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:26.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:26.638+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:27.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:26 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:26.957+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:27.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:26 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:26.957+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:27.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:26 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:26.957+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:27.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:26 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:26.957+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:27.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:26 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:26.957+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:27.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:26 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:26.910+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:27.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:26 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:26.910+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:27.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:26 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:26.910+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:27.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:26 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:26.910+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:27.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:26 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:26.910+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:27.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:27 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:27.285+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:27.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:27 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:27.285+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:27.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:27 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:27.285+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:27.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:27 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:27.285+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:27.984 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:27 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:27.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:27.984 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:27 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:27.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:27.984 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:27 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:27.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:27.985 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:27 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:27.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:27.985 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:27 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:27.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:28.260 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:27 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:27.934+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:28.260 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:27 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:27.934+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:28.260 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:27 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:27.934+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:28.260 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:27 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:27.934+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:28.260 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:27 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:27.934+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:28.271 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:27 smithi106 podman[74655]: 2024-04-25 00:46:27.983818729 +0000 UTC m=+5.655161574 container died eef0f07bd7a3a4243ec0f1f045ba5f5460901c1a3f2e7b824f7f615d70ab50de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T00:46:28.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:27 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:27.906+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:28.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:27 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:27.906+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:28.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:27 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:27.906+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:28.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:27 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:27.906+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:28.378 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:27 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:27.906+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:28.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:28 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:28.259+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:28.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:28 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:28.259+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:28.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:28 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:28.259+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:28.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:28 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:28.259+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:28.588 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:28 smithi106 podman[74655]: 2024-04-25 00:46:28.362936586 +0000 UTC m=+6.034279435 container cleanup eef0f07bd7a3a4243ec0f1f045ba5f5460901c1a3f2e7b824f7f615d70ab50de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:46:28.588 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:28 smithi106 bash[74655]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7 2024-04-25T00:46:28.904 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:28 smithi106 podman[74671]: 2024-04-25 00:46:28.589188089 +0000 UTC m=+0.607266075 container remove eef0f07bd7a3a4243ec0f1f045ba5f5460901c1a3f2e7b824f7f615d70ab50de (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T00:46:28.904 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:28 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:28.681+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:28.904 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:28 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:28.681+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:28.904 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:28 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:28.681+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:28.904 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:28 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:28.681+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:28.904 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:28 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:28.681+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:29.159 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:28 smithi106 podman[74762]: 2024-04-25 00:46:28.806959502 +0000 UTC m=+0.017890921 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:46:29.161 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:28 smithi106 podman[74762]: 2024-04-25 00:46:28.931802959 +0000 UTC m=+0.142734366 container create c422f3cd5cd8c48b12327d17a6435b2a5dce432e749832aece4e86a14b77b40c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T00:46:29.232 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:28 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:28.968+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:29.232 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:28 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:28.968+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:29.232 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:28 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:28.968+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:29.232 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:28 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:28.968+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:29.233 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:28 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:28.968+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:29.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:28 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:28.951+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:29.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:28 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:28.951+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:29.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:28 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:28.951+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:29.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:28 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:28.951+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:29.377 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:28 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:28.951+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:29.455 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:29 smithi106 podman[74762]: 2024-04-25 00:46:29.159497047 +0000 UTC m=+0.370428463 container init c422f3cd5cd8c48b12327d17a6435b2a5dce432e749832aece4e86a14b77b40c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7-deactivate, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T00:46:29.455 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:29 smithi106 podman[74762]: 2024-04-25 00:46:29.162920559 +0000 UTC m=+0.373851978 container start c422f3cd5cd8c48b12327d17a6435b2a5dce432e749832aece4e86a14b77b40c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True) 2024-04-25T00:46:29.455 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:29 smithi106 podman[74762]: 2024-04-25 00:46:29.242321882 +0000 UTC m=+0.453253290 container attach c422f3cd5cd8c48b12327d17a6435b2a5dce432e749832aece4e86a14b77b40c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-d2c45d9) 2024-04-25T00:46:29.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:29 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:29.231+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:29.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:29 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:29.231+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:29.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:29 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:29.231+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:29.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:29 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:29.231+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:29.745 INFO:journalctl@ceph.osd.7.smithi106.stdout:Apr 25 00:46:29 smithi106 podman[74762]: 2024-04-25 00:46:29.45571308 +0000 UTC m=+0.666644488 container died c422f3cd5cd8c48b12327d17a6435b2a5dce432e749832aece4e86a14b77b40c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424) 2024-04-25T00:46:29.745 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:29 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:29.696+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:29.745 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:29 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:29.696+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:29.745 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:29 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:29.696+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:29.746 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:29 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:29.696+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:29.746 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:29 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:29.696+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:30.046 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T00:46:30.084 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:46:30.084 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T00:46:30.084 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T00:46:30.084 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.8 2024-04-25T00:46:30.230 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:30 smithi136 systemd[1]: Stopping Ceph osd.8 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:46:30.230 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:29 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:29.977+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:30.230 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:29 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:29.977+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:30.230 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:29 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:29.977+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:30.230 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:29 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:29.977+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:30.230 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:29 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:29.977+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:30.279 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:29 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:29.904+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:30.279 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:29 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:29.904+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:30.279 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:29 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:29.904+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:30.279 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:29 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:29.904+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:30.279 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:29 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:29.904+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:30.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:30.228+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:30.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:30.228+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:30.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:30.228+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:30.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:30.228+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:30.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:30.375+0000 7f451eb5a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:46:30.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:30.375+0000 7f451eb5a700 -1 osd.8 94 *** Got signal Terminated *** 2024-04-25T00:46:30.514 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:30.375+0000 7f451eb5a700 -1 osd.8 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:46:31.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:30 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:30.669+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:31.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:30 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:30.669+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:31.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:30 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:30.669+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:31.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:30 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:30.669+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:31.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:30 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:30.669+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:31.245 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:30.957+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:31.245 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:30.957+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:31.245 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:30.957+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:31.245 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:30.957+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:31.245 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:30 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:30.957+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:31.313 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:30 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:30.907+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:31.313 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:30 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:30.907+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:31.313 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:30 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:30.907+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:31.313 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:30 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:30.907+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:31.313 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:30 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:30.907+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:31.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:31 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:31.244+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:31.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:31 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:31.244+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:31.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:31 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:31.244+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:31.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:31 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:31.244+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:32.009 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:31 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:31.894+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:32.009 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:31 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:31.894+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:32.009 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:31 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:31.894+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:32.009 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:31 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:31.894+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:32.009 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:31 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:31.894+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:32.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:31 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:31.681+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:32.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:31 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:31.681+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:32.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:31 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:31.681+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:32.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:31 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:31.681+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:32.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:31 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:31.681+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:32.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:31 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:31.961+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:32.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:31 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:31.961+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:32.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:31 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:31.961+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:32.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:31 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:31.961+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:32.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:31 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:31.961+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:32.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:31 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:31.961+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:32.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:32 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:32.283+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:32.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:32 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:32.283+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:32.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:32 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:32.283+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:32.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:32 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:32.283+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:32.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:32 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:32.632+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:32.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:32 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:32.632+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:32.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:32 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:32.632+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:32.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:32 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:32.632+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:32.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:32 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:32.632+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:32.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:32 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:32.632+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:33.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:32 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:32.846+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:33.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:32 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:32.846+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:33.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:32 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:32.846+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:33.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:32 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:32.846+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:33.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:32 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:32.846+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:33.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:32 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:32.846+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:33.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:32 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:32.964+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:33.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:32 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:32.964+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:33.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:32 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:32.964+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:33.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:32 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:32.964+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:33.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:32 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:32.964+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:33.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:32 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:32.964+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:33.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:33.295+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:33.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:33.295+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:33.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:33.295+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:33.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:33.295+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:33.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:33.295+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:06.916055+0000 front 2024-04-25T00:46:06.916085+0000 (oldest deadline 2024-04-25T00:46:32.816275+0000) 2024-04-25T00:46:34.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:33 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:33.646+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:34.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:33 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:33.646+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:34.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:33 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:33.646+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:34.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:33 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:33.646+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:34.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:33 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:33.646+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:34.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:33 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:33.646+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:34.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:33.812+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:34.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:33.812+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:34.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:33.812+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:34.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:33.812+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:34.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:33.812+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:34.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:33 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:33.812+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:34.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:33.967+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:34.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:33.967+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:34.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:33.967+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:34.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:33.967+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:34.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:33.967+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:34.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:33 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:33.967+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:34.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:34.323+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:34.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:34.323+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:34.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:34.323+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:34.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:34.323+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:34.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:34.323+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:06.916055+0000 front 2024-04-25T00:46:06.916085+0000 (oldest deadline 2024-04-25T00:46:32.816275+0000) 2024-04-25T00:46:35.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:34 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:34.688+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:35.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:34 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:34.688+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:35.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:34 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:34.688+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:35.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:34 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:34.688+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:35.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:34 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:34.688+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:35.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:34 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:34.688+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:35.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:34 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:34.821+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:35.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:34 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:34.821+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:35.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:34 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:34.821+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:35.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:34 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:34.821+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:35.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:34 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:34.821+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:35.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:34 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:34.821+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:35.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:34.998+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:35.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:34.998+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:35.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:34.998+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:35.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:34.998+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:35.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:34.998+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:35.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:34 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:34.998+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:35.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:35.282+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.509778+0000 front 2024-04-25T00:45:32.509758+0000 (oldest deadline 2024-04-25T00:45:56.609880+0000) 2024-04-25T00:46:35.763 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:35.282+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.411724+0000 front 2024-04-25T00:45:42.411771+0000 (oldest deadline 2024-04-25T00:46:05.311925+0000) 2024-04-25T00:46:35.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:35.282+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:52.313735+0000 front 2024-04-25T00:45:52.313828+0000 (oldest deadline 2024-04-25T00:46:17.013604+0000) 2024-04-25T00:46:35.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:35.282+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.014017+0000 front 2024-04-25T00:45:57.014045+0000 (oldest deadline 2024-04-25T00:46:22.814885+0000) 2024-04-25T00:46:35.764 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8[51661]: 2024-04-25T00:46:35.282+0000 7f4517947700 -1 osd.8 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:06.916055+0000 front 2024-04-25T00:46:06.916085+0000 (oldest deadline 2024-04-25T00:46:32.816275+0000) 2024-04-25T00:46:36.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:35 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:35.694+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:36.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:35 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:35.694+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:36.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:35 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:35.694+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:36.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:35 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:35.694+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:36.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:35 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:35.694+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:36.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:35 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:35.694+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:36.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:35 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:35.816+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:36.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:35 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:35.816+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:36.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:35 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:35.816+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:36.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:35 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:35.816+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:36.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:35 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:35.816+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:36.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:35 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:35.816+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:36.175 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:35 smithi136 podman[73825]: 2024-04-25 00:46:35.863264959 +0000 UTC m=+5.590925582 container died 3129ebf7f49df8ffaf4ec7a1767abcc14858b4f70700289572ab69b9ce4ed6b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:46:36.175 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:35.951+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:36.175 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:35.951+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:36.175 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:35.951+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:36.175 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:35.951+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:36.175 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:35.951+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:36.175 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:35 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:35.951+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:36.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:36 smithi136 podman[73825]: 2024-04-25 00:46:36.175235803 +0000 UTC m=+5.902896432 container cleanup 3129ebf7f49df8ffaf4ec7a1767abcc14858b4f70700289572ab69b9ce4ed6b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T00:46:36.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:36 smithi136 bash[73825]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8 2024-04-25T00:46:36.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:36 smithi136 podman[73844]: 2024-04-25 00:46:36.384877843 +0000 UTC m=+0.517649667 container remove 3129ebf7f49df8ffaf4ec7a1767abcc14858b4f70700289572ab69b9ce4ed6b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:46:37.013 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:36 smithi136 podman[73930]: 2024-04-25 00:46:36.607227974 +0000 UTC m=+0.018216535 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:46:37.014 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:36 smithi136 podman[73930]: 2024-04-25 00:46:36.711314546 +0000 UTC m=+0.122303081 container create 3303e5e9f4001af2f899f26371ee6290cbc83d07a849bbf05cb6281be45e8138 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T00:46:37.014 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:36 smithi136 podman[73930]: 2024-04-25 00:46:36.922871083 +0000 UTC m=+0.333859625 container init 3303e5e9f4001af2f899f26371ee6290cbc83d07a849bbf05cb6281be45e8138 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T00:46:37.014 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:36 smithi136 podman[73930]: 2024-04-25 00:46:36.926445399 +0000 UTC m=+0.337433933 container start 3303e5e9f4001af2f899f26371ee6290cbc83d07a849bbf05cb6281be45e8138 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T00:46:37.014 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:36 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:36.927+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:37.014 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:36 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:36.927+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:37.014 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:36 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:36.927+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:37.014 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:36 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:36.927+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:37.015 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:36 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:36.927+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:37.015 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:36 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:36.927+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:37.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:36 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:36.673+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:37.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:36 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:36.673+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:37.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:36 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:36.673+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:37.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:36 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:36.673+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:37.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:36 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:36.673+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:37.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:36 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:36.673+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:37.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:36 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:36.776+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:37.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:36 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:36.776+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:37.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:36 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:36.776+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:37.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:36 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:36.776+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:37.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:36 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:36.776+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:37.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:36 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:36.776+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:37.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:37 smithi136 podman[73930]: 2024-04-25 00:46:37.022477491 +0000 UTC m=+0.433466025 container attach 3303e5e9f4001af2f899f26371ee6290cbc83d07a849bbf05cb6281be45e8138 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T00:46:37.513 INFO:journalctl@ceph.osd.8.smithi136.stdout:Apr 25 00:46:37 smithi136 podman[73930]: 2024-04-25 00:46:37.212610622 +0000 UTC m=+0.623599158 container died 3303e5e9f4001af2f899f26371ee6290cbc83d07a849bbf05cb6281be45e8138 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T00:46:37.828 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T00:46:37.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:46:37.866 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T00:46:37.866 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T00:46:37.866 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.9 2024-04-25T00:46:38.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:37 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:37.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:38.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:37 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:37.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:38.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:37 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:37.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:38.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:37 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:37.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:38.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:37 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:37.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:38.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:37 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:37.675+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:38.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:37 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:37.764+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:38.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:37 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:37.764+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:38.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:37 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:37.764+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:38.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:37 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:37.764+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:38.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:37 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:37.764+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:38.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:37 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:37.764+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:38.128 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:37 smithi042 systemd[1]: Stopping Ceph osd.9 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:46:38.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:37 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:37.878+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:38.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:37 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:37.878+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:38.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:37 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:37.878+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:38.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:37 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:37.878+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:38.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:37 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:37.878+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:38.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:37 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:37.878+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:38.766 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:38 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:38.439+0000 7f166d800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:46:38.766 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:38 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:38.439+0000 7f166d800700 -1 osd.9 94 *** Got signal Terminated *** 2024-04-25T00:46:38.766 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:38 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:38.439+0000 7f166d800700 -1 osd.9 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:46:38.767 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:38 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:38.765+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:38.767 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:38 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:38.765+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:38.767 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:38 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:38.765+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:39.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:38 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:38.629+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:39.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:38 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:38.629+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:39.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:38 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:38.629+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:39.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:38 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:38.629+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:39.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:38 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:38.629+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:39.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:38 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:38.629+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:39.126 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:38 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:38.765+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:39.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:38 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:38.765+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:39.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:38 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:38.765+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:39.180 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:38 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:38.847+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:39.180 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:38 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:38.847+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:39.180 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:38 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:38.847+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:39.180 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:38 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:38.847+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:39.180 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:38 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:38.847+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:39.180 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:38 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:38.847+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:39.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:39 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:39.582+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:39.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:39 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:39.582+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:39.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:39 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:39.582+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:39.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:39 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:39.582+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:39.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:39 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:39.582+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:39.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:39 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:39.582+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:40.118 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:39.864+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:40.118 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:39.864+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:40.118 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:39.864+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:40.118 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:39.864+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:40.118 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:39.864+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:40.119 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:39.864+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:40.126 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:39 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:39.717+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:40.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:39 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:39.717+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:40.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:39 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:39.717+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:40.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:39 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:39.717+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:40.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:39 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:39.717+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:40.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:39 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:39.717+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:40.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:40 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:40.572+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:40.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:40 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:40.572+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:40.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:40 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:40.572+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:40.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:40 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:40.572+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:40.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:40 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:40.572+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:40.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:40 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:40.572+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:41.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:40 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:40.700+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:41.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:40 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:40.700+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:41.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:40 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:40.700+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:41.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:40 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:40.700+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:41.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:40 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:40.700+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:41.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:40 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:40.700+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:41.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:40 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:40.836+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:41.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:40 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:40.836+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:41.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:40 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:40.836+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:41.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:40 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:40.836+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:41.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:40 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:40.836+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:41.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:40 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:40.836+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:42.047 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:41 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:41.595+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:42.047 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:41 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:41.595+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:42.047 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:41 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:41.595+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:42.047 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:41 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:41.595+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:42.047 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:41 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:41.595+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:42.047 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:41 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:41.595+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:42.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:41 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:41.660+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:42.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:41 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:41.660+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:42.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:41 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:41.660+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:42.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:41 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:41.660+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:42.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:41 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:41.660+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:42.127 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:41 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:41.660+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:42.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:41 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:41.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:42.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:41 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:41.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:42.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:41 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:41.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:42.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:41 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:41.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:42.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:41 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:41.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:42.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:41 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:41.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:43.046 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:42 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:42.698+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:25.017691+0000 front 2024-04-25T00:45:25.017832+0000 (oldest deadline 2024-04-25T00:45:49.717871+0000) 2024-04-25T00:46:43.047 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:42 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:42.698+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.523093+0000 front 2024-04-25T00:45:35.523190+0000 (oldest deadline 2024-04-25T00:45:57.223326+0000) 2024-04-25T00:46:43.047 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:42 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:42.698+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.930505+0000 front 2024-04-25T00:45:42.930418+0000 (oldest deadline 2024-04-25T00:46:08.230687+0000) 2024-04-25T00:46:43.047 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:42 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:42.698+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.232112+0000 front 2024-04-25T00:45:48.232182+0000 (oldest deadline 2024-04-25T00:46:18.234191+0000) 2024-04-25T00:46:43.047 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:42 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:42.698+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:58.737530+0000 front 2024-04-25T00:45:58.737620+0000 (oldest deadline 2024-04-25T00:46:22.837769+0000) 2024-04-25T00:46:43.047 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:42 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9[62631]: 2024-04-25T00:46:42.698+0000 7f1664a00700 -1 osd.9 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.740616+0000 front 2024-04-25T00:46:08.740677+0000 (oldest deadline 2024-04-25T00:46:32.840748+0000) 2024-04-25T00:46:43.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:42 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:42.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:43.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:42 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:42.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:43.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:42 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:42.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:43.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:42 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:42.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:43.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:42 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:42.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:43.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:42 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:42.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:43.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:42 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:42.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:43.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:42 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:42.859+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:43.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:42 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:42.859+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:43.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:42 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:42.859+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:43.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:42 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:42.859+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:43.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:42 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:42.859+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:43.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:42 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:42.859+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:44.090 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:43 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:43.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:44.090 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:43 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:43.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:44.090 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:43 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:43.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:44.090 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:43 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:43.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:44.090 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:43 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:43.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:44.090 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:43 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:43.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:44.090 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:43 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:43.636+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:44.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:43 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:43.898+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:44.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:43 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:43.898+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:44.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:43 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:43.898+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:44.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:43 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:43.898+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:44.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:43 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:43.898+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:44.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:43 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:43.898+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:44.563 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:44 smithi042 podman[95117]: 2024-04-25 00:46:44.217731649 +0000 UTC m=+5.921426756 container died e2fe7e09bc242bd9b2e557cc37bced8ca85e16f973dc06298808290a78796f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True) 2024-04-25T00:46:44.866 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:44 smithi042 podman[95117]: 2024-04-25 00:46:44.673269092 +0000 UTC m=+6.376964196 container cleanup e2fe7e09bc242bd9b2e557cc37bced8ca85e16f973dc06298808290a78796f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:46:44.867 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:44 smithi042 bash[95117]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9 2024-04-25T00:46:45.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:44 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:44.610+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:45.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:44 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:44.610+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:45.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:44 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:44.610+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:45.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:44 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:44.610+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:45.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:44 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:44.610+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:45.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:44 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:44.610+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:45.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:44 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:44.610+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:45.126 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:44 smithi042 podman[95135]: 2024-04-25 00:46:44.866090116 +0000 UTC m=+0.639432999 container remove e2fe7e09bc242bd9b2e557cc37bced8ca85e16f973dc06298808290a78796f8a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-25T00:46:45.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:44 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:44.887+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:45.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:44 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:44.887+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:45.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:44 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:44.887+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:45.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:44 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:44.887+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:45.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:44 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:44.887+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:45.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:44 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:44.887+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:46.005 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:45 smithi042 podman[95222]: 2024-04-25 00:46:45.608025961 +0000 UTC m=+0.037666081 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:46:46.005 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:45 smithi042 podman[95222]: 2024-04-25 00:46:45.766938888 +0000 UTC m=+0.196579034 container create efc39b71ef84afaa304976c359b59320a6d6bb6366b763d8438c87166c06ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T00:46:46.005 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:46 smithi042 podman[95222]: 2024-04-25 00:46:46.004249736 +0000 UTC m=+0.433889901 container init efc39b71ef84afaa304976c359b59320a6d6bb6366b763d8438c87166c06ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:46:46.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:45 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:45.607+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:46.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:45 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:45.607+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:46.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:45 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:45.607+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:46.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:45 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:45.607+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:46.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:45 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:45.607+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:46.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:45 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:45.607+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:46.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:45 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:45.607+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:46.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:45 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:45.850+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:46.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:45 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:45.850+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:46.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:45 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:45.850+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:46.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:45 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:45.850+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:46.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:45 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:45.850+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:46.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:45 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:45.850+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:46.376 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:46 smithi042 podman[95222]: 2024-04-25 00:46:46.015839629 +0000 UTC m=+0.445479760 container start efc39b71ef84afaa304976c359b59320a6d6bb6366b763d8438c87166c06ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:46:46.376 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:46 smithi042 podman[95222]: 2024-04-25 00:46:46.1279359 +0000 UTC m=+0.557576128 container attach efc39b71ef84afaa304976c359b59320a6d6bb6366b763d8438c87166c06ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T00:46:46.862 INFO:journalctl@ceph.osd.9.smithi042.stdout:Apr 25 00:46:46 smithi042 podman[95222]: 2024-04-25 00:46:46.495459927 +0000 UTC m=+0.925100092 container died efc39b71ef84afaa304976c359b59320a6d6bb6366b763d8438c87166c06ea6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T00:46:47.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:46 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:46.620+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:47.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:46 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:46.620+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:47.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:46 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:46.620+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:47.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:46 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:46.620+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:47.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:46 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:46.620+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:47.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:46 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:46.620+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:47.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:46 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:46.620+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:47.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:46 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:46.834+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:47.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:46 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:46.834+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:47.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:46 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:46.834+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:47.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:46 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:46.834+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:47.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:46 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:46.834+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:47.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:46 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:46.834+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:47.274 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T00:46:47.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:46:47.388 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T00:46:47.389 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T00:46:47.389 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.10 2024-04-25T00:46:47.698 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 systemd[1]: Stopping Ceph osd.10 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:46:47.699 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:47.645+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:47.699 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:47.645+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:47.699 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:47.645+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:47.699 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:47.645+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:47.699 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:47.645+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:47.699 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:47.645+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:47.699 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:47.645+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:47.699 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:47.697+0000 7ff6c22f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:46:47.699 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:47.697+0000 7ff6c22f3700 -1 osd.10 94 *** Got signal Terminated *** 2024-04-25T00:46:47.699 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:47 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:47.697+0000 7ff6c22f3700 -1 osd.10 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:46:48.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:47 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:47.809+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:48.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:47 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:47.809+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:48.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:47 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:47.809+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:48.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:47 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:47.809+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:48.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:47 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:47.809+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:48.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:47 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:47.809+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:49.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:48 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:48.603+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:49.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:48 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:48.603+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:49.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:48 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:48.603+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:49.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:48 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:48.603+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:49.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:48 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:48.603+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:49.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:48 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:48.603+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:49.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:48 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:48.603+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:49.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:48 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:48.830+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:49.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:48 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:48.830+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:49.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:48 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:48.830+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:49.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:48 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:48.830+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:49.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:48 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:48.830+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:49.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:48 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:48.830+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:49.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:48 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:48.830+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:46:50.013 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:49 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:49.835+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:50.013 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:49 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:49.835+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:50.013 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:49 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:49.835+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:50.014 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:49 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:49.835+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:50.014 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:49 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:49.835+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:50.014 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:49 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:49.835+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:50.014 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:49 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:49.835+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:46:50.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:49 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:49.626+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:50.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:49 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:49.626+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:50.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:49 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:49.626+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:50.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:49 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:49.626+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:50.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:49 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:49.626+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:50.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:49 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:49.626+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:50.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:49 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:49.626+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:51.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:50 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:50.590+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:51.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:50 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:50.590+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:51.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:50 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:50.590+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:51.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:50 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:50.590+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:51.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:50 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:50.590+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:51.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:50 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:50.590+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:51.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:50 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:50.590+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:51.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:50.874+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:51.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:50.874+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:51.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:50.874+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:51.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:50.874+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:51.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:50.874+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:51.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:50.874+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:51.266 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:50.874+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:46:52.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:51 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:51.637+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:52.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:51 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:51.637+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:52.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:51 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:51.637+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:52.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:51 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:51.637+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:52.088 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:51 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:51.637+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:52.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:51 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:51.637+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:52.089 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:51 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:51.637+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:52.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:51 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:51.913+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:52.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:51 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:51.913+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:52.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:51 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:51.913+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:52.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:51 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:51.913+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:52.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:51 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:51.913+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:52.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:51 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:51.913+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:52.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:51 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:51.913+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:46:53.085 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:52.593+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.924884+0000 front 2024-04-25T00:45:22.924956+0000 (oldest deadline 2024-04-25T00:45:47.024431+0000) 2024-04-25T00:46:53.085 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:52.593+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:35.627406+0000 front 2024-04-25T00:45:35.627385+0000 (oldest deadline 2024-04-25T00:46:00.927550+0000) 2024-04-25T00:46:53.085 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:52.593+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:40.928023+0000 front 2024-04-25T00:45:40.927978+0000 (oldest deadline 2024-04-25T00:46:05.628180+0000) 2024-04-25T00:46:53.085 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:52.593+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:49.729627+0000 front 2024-04-25T00:45:49.729396+0000 (oldest deadline 2024-04-25T00:46:15.629163+0000) 2024-04-25T00:46:53.085 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:52.593+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:46:00.830633+0000 front 2024-04-25T00:46:00.830675+0000 (oldest deadline 2024-04-25T00:46:26.730842+0000) 2024-04-25T00:46:53.085 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:52.593+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:07.831864+0000 front 2024-04-25T00:46:07.831798+0000 (oldest deadline 2024-04-25T00:46:32.531767+0000) 2024-04-25T00:46:53.086 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:52 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10[57877]: 2024-04-25T00:46:52.593+0000 7ff6bf0e8700 -1 osd.10 94 heartbeat_check: no reply from 172.21.15.42:6822 osd.6 since back 2024-04-25T00:46:12.532434+0000 front 2024-04-25T00:46:12.532536+0000 (oldest deadline 2024-04-25T00:46:42.532551+0000) 2024-04-25T00:46:53.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:52.918+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:53.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:52.918+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:53.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:52.918+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:53.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:52.918+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:53.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:52.918+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:53.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:52.918+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:53.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:52 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:52.918+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:46:53.338 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:53 smithi106 podman[74878]: 2024-04-25 00:46:53.08505957 +0000 UTC m=+5.495685665 container died b2f9b28729297f98d1b0cff78e44e661a359e540c21dd7d2dee57af5ca69fec1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-25T00:46:53.607 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:53 smithi106 podman[74878]: 2024-04-25 00:46:53.423071065 +0000 UTC m=+5.833697168 container cleanup b2f9b28729297f98d1b0cff78e44e661a359e540c21dd7d2dee57af5ca69fec1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-25T00:46:53.607 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:53 smithi106 bash[74878]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10 2024-04-25T00:46:54.036 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:53 smithi106 podman[74897]: 2024-04-25 00:46:53.607677774 +0000 UTC m=+0.518821424 container remove b2f9b28729297f98d1b0cff78e44e661a359e540c21dd7d2dee57af5ca69fec1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T00:46:54.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:53.879+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:54.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:53.879+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:54.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:53.879+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:54.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:53.879+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:54.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:53.879+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:54.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:53.879+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:54.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:53 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:53.879+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:46:54.338 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:54 smithi106 podman[74982]: 2024-04-25 00:46:53.940229324 +0000 UTC m=+0.018108141 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:46:54.338 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:54 smithi106 podman[74982]: 2024-04-25 00:46:54.041953873 +0000 UTC m=+0.119832736 container create 99b3866d3b295d76978bfda6e4176d0dcc9f6a4a1f954549ebba0997b621e599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T00:46:54.338 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:54 smithi106 podman[74982]: 2024-04-25 00:46:54.269734974 +0000 UTC m=+0.347613775 container init 99b3866d3b295d76978bfda6e4176d0dcc9f6a4a1f954549ebba0997b621e599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T00:46:54.338 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:54 smithi106 podman[74982]: 2024-04-25 00:46:54.272841469 +0000 UTC m=+0.350720280 container start 99b3866d3b295d76978bfda6e4176d0dcc9f6a4a1f954549ebba0997b621e599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:46:54.838 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:54 smithi106 podman[74982]: 2024-04-25 00:46:54.352520326 +0000 UTC m=+0.430399134 container attach 99b3866d3b295d76978bfda6e4176d0dcc9f6a4a1f954549ebba0997b621e599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T00:46:54.839 INFO:journalctl@ceph.osd.10.smithi106.stdout:Apr 25 00:46:54 smithi106 podman[74982]: 2024-04-25 00:46:54.565077085 +0000 UTC m=+0.642955899 container died 99b3866d3b295d76978bfda6e4176d0dcc9f6a4a1f954549ebba0997b621e599 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T00:46:55.138 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T00:46:55.175 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:46:55.175 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T00:46:55.176 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T00:46:55.176 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.11 2024-04-25T00:46:55.202 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:54.868+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:55.202 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:54.868+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:55.202 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:54.868+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:55.202 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:54.868+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:55.202 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:54.868+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:55.202 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:54.868+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:55.202 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:54 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:54.868+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:46:55.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 systemd[1]: Stopping Ceph osd.11 for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:46:55.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:55.445+0000 7f3e159f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:46:55.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:55.445+0000 7f3e159f9700 -1 osd.11 94 *** Got signal Terminated *** 2024-04-25T00:46:55.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:55.445+0000 7f3e159f9700 -1 osd.11 94 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T00:46:56.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:55.892+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:56.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:55.892+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:56.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:55.892+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:56.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:55.892+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:56.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:55.892+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:56.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:55.892+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:56.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:55 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:55.892+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:46:57.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:56.866+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:57.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:56.866+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:57.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:56.866+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:57.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:56.866+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:57.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:56.866+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:57.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:56.866+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:57.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:56 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:56.866+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:46:58.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:57.816+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:58.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:57.816+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:58.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:57.816+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:58.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:57.816+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:58.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:57.816+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:58.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:57.816+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:58.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:57 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:57.816+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:46:59.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:58.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:46:59.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:58.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:46:59.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:58.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:46:59.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:58.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:46:59.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:58.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:46:59.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:58.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:46:59.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:58 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:58.853+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:47:00.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:59.883+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6806 osd.0 since back 2024-04-25T00:45:22.967039+0000 front 2024-04-25T00:45:22.967027+0000 (oldest deadline 2024-04-25T00:45:48.266955+0000) 2024-04-25T00:47:00.263 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:59.883+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6804 osd.1 since back 2024-04-25T00:45:32.968415+0000 front 2024-04-25T00:45:32.968177+0000 (oldest deadline 2024-04-25T00:45:58.868235+0000) 2024-04-25T00:47:00.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:59.883+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6804 osd.2 since back 2024-04-25T00:45:42.969160+0000 front 2024-04-25T00:45:42.969132+0000 (oldest deadline 2024-04-25T00:46:08.269394+0000) 2024-04-25T00:47:00.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:59.883+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.42:6814 osd.3 since back 2024-04-25T00:45:48.270215+0000 front 2024-04-25T00:45:48.270051+0000 (oldest deadline 2024-04-25T00:46:17.670643+0000) 2024-04-25T00:47:00.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:59.883+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6812 osd.4 since back 2024-04-25T00:45:57.671294+0000 front 2024-04-25T00:45:57.671188+0000 (oldest deadline 2024-04-25T00:46:22.971187+0000) 2024-04-25T00:47:00.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:59.883+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.136:6812 osd.5 since back 2024-04-25T00:46:08.872386+0000 front 2024-04-25T00:46:08.872354+0000 (oldest deadline 2024-04-25T00:46:31.172479+0000) 2024-04-25T00:47:00.264 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:46:59 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11[58305]: 2024-04-25T00:46:59.883+0000 7f3e127ee700 -1 osd.11 94 heartbeat_check: no reply from 172.21.15.106:6820 osd.7 since back 2024-04-25T00:46:26.975289+0000 front 2024-04-25T00:46:26.975322+0000 (oldest deadline 2024-04-25T00:46:48.075167+0000) 2024-04-25T00:47:01.105 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:00 smithi136 podman[74050]: 2024-04-25 00:47:00.836393056 +0000 UTC m=+5.472779677 container died 963cb7636e71432051a41fe5f691ec1edc4a2245cf073928c7973bb40c9b57a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T00:47:01.481 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:01 smithi136 podman[74050]: 2024-04-25 00:47:01.204218724 +0000 UTC m=+5.840605340 container cleanup 963cb7636e71432051a41fe5f691ec1edc4a2245cf073928c7973bb40c9b57a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9) 2024-04-25T00:47:01.481 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:01 smithi136 bash[74050]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11 2024-04-25T00:47:01.763 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:01 smithi136 podman[74067]: 2024-04-25 00:47:01.481200359 +0000 UTC m=+0.633703030 container remove 963cb7636e71432051a41fe5f691ec1edc4a2245cf073928c7973bb40c9b57a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-25T00:47:02.093 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:01 smithi136 podman[74158]: 2024-04-25 00:47:01.815437963 +0000 UTC m=+0.113973467 container create 301590b60e930147aeadcac99807f9570426f03a5976cc97984da71d5c09f508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T00:47:02.093 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:01 smithi136 podman[74158]: 2024-04-25 00:47:01.719383756 +0000 UTC m=+0.017919273 image pull f68e37b807ef3dfd57708ff7362bfa0edaabe0111536b8e8a4c26de22a72557e quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901 2024-04-25T00:47:02.093 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:02 smithi136 podman[74158]: 2024-04-25 00:47:02.009775777 +0000 UTC m=+0.308311281 container init 301590b60e930147aeadcac99807f9570426f03a5976cc97984da71d5c09f508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-25T00:47:02.094 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:02 smithi136 podman[74158]: 2024-04-25 00:47:02.01281552 +0000 UTC m=+0.311351027 container start 301590b60e930147aeadcac99807f9570426f03a5976cc97984da71d5c09f508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:47:02.094 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:02 smithi136 podman[74158]: 2024-04-25 00:47:02.09307406 +0000 UTC m=+0.391609590 container attach 301590b60e930147aeadcac99807f9570426f03a5976cc97984da71d5c09f508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T00:47:02.513 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:02 smithi136 podman[74158]: 2024-04-25 00:47:02.295614804 +0000 UTC m=+0.594150303 container died 301590b60e930147aeadcac99807f9570426f03a5976cc97984da71d5c09f508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-25T00:47:02.795 INFO:journalctl@ceph.osd.11.smithi136.stdout:Apr 25 00:47:02 smithi136 podman[74185]: 2024-04-25 00:47:02.779958044 +0000 UTC m=+0.478314382 container remove 301590b60e930147aeadcac99807f9570426f03a5976cc97984da71d5c09f508 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=squid-d2c45d9, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T00:47:02.800 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T00:47:02.878 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:02.878 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T00:47:02.878 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T00:47:02.878 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.a 2024-04-25T00:47:03.376 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:47:02 smithi042 systemd[1]: Stopping Ceph mds.a for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:03.755 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:47:03 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a[69399]: 2024-04-25T00:47:03.473+0000 7febbde00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:03.756 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:47:03 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a[69399]: 2024-04-25T00:47:03.473+0000 7febbde00700 -1 mds.a *** got signal Terminated *** 2024-04-25T00:47:05.877 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:47:05 smithi042 podman[95327]: 2024-04-25 00:47:05.570890333 +0000 UTC m=+2.222415925 container died 640f6360ec465e117b397853204e118b227683875cd118cba1ad282a43080275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T00:47:06.277 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:47:06 smithi042 podman[95327]: 2024-04-25 00:47:06.041819728 +0000 UTC m=+2.693345341 container cleanup 640f6360ec465e117b397853204e118b227683875cd118cba1ad282a43080275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240424, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T00:47:06.277 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:47:06 smithi042 bash[95327]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a 2024-04-25T00:47:06.626 INFO:journalctl@ceph.mds.a.smithi042.stdout:Apr 25 00:47:06 smithi042 podman[95352]: 2024-04-25 00:47:06.2762534 +0000 UTC m=+0.685432049 container remove 640f6360ec465e117b397853204e118b227683875cd118cba1ad282a43080275 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T00:47:06.805 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T00:47:06.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:06.910 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T00:47:06.911 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T00:47:06.911 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.d 2024-04-25T00:47:07.376 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:47:07 smithi042 systemd[1]: Stopping Ceph mds.d for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:07.877 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:47:07 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-d[68415]: 2024-04-25T00:47:07.528+0000 7f6b18400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:07.877 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:47:07 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-d[68415]: 2024-04-25T00:47:07.528+0000 7f6b18400700 -1 mds.d *** got signal Terminated *** 2024-04-25T00:47:10.876 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:47:10 smithi042 podman[95459]: 2024-04-25 00:47:10.591543929 +0000 UTC m=+3.196649053 container died 2e34cbe51038388e38e7f8777db54d55fa138be2505fc0b61a1523d5ef8e44bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-25T00:47:11.290 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:47:11 smithi042 podman[95459]: 2024-04-25 00:47:11.029789869 +0000 UTC m=+3.634894993 container cleanup 2e34cbe51038388e38e7f8777db54d55fa138be2505fc0b61a1523d5ef8e44bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-25T00:47:11.290 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:47:11 smithi042 bash[95459]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-d 2024-04-25T00:47:11.626 INFO:journalctl@ceph.mds.d.smithi042.stdout:Apr 25 00:47:11 smithi042 podman[95473]: 2024-04-25 00:47:11.289113359 +0000 UTC m=+0.679757056 container remove 2e34cbe51038388e38e7f8777db54d55fa138be2505fc0b61a1523d5ef8e44bf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-25T00:47:11.824 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T00:47:11.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:11.942 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T00:47:11.942 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T00:47:11.942 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.g 2024-04-25T00:47:12.376 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:47:12 smithi042 systemd[1]: Stopping Ceph mds.g for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:12.876 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:47:12 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-g[69022]: 2024-04-25T00:47:12.555+0000 7f59f5800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:12.876 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:47:12 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-g[69022]: 2024-04-25T00:47:12.555+0000 7f59f5800700 -1 mds.g *** got signal Terminated *** 2024-04-25T00:47:16.590 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:47:16 smithi042 podman[95577]: 2024-04-25 00:47:16.310054191 +0000 UTC m=+3.876461733 container died 2960e809835a209e6df18b3d9b1f603db91e7ed8a77cd7396fd5abc6f35fee96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-g, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T00:47:16.876 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:47:16 smithi042 podman[95577]: 2024-04-25 00:47:16.749838252 +0000 UTC m=+4.316245797 container cleanup 2960e809835a209e6df18b3d9b1f603db91e7ed8a77cd7396fd5abc6f35fee96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T00:47:16.876 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:47:16 smithi042 bash[95577]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-g 2024-04-25T00:47:17.376 INFO:journalctl@ceph.mds.g.smithi042.stdout:Apr 25 00:47:16 smithi042 podman[95591]: 2024-04-25 00:47:16.942666332 +0000 UTC m=+0.613137554 container remove 2960e809835a209e6df18b3d9b1f603db91e7ed8a77cd7396fd5abc6f35fee96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-25T00:47:17.449 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T00:47:17.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:17.561 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T00:47:17.561 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T00:47:17.561 DEBUG:teuthology.orchestra.run.smithi042:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.j 2024-04-25T00:47:18.126 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:47:17 smithi042 systemd[1]: Stopping Ceph mds.j for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:18.626 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:47:18 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-j[68055]: 2024-04-25T00:47:18.168+0000 7f40e1400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:18.627 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:47:18 smithi042 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-j[68055]: 2024-04-25T00:47:18.168+0000 7f40e1400700 -1 mds.j *** got signal Terminated *** 2024-04-25T00:47:22.513 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:47:22 smithi042 podman[95699]: 2024-04-25 00:47:22.222353471 +0000 UTC m=+4.184289777 container died cc36f7531251f245b5801273ceb2c04256c317476261b1b4057caa52e92de253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T00:47:22.877 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:47:22 smithi042 podman[95699]: 2024-04-25 00:47:22.666503217 +0000 UTC m=+4.628439553 container cleanup cc36f7531251f245b5801273ceb2c04256c317476261b1b4057caa52e92de253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-j, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9) 2024-04-25T00:47:22.877 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:47:22 smithi042 bash[95699]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-j 2024-04-25T00:47:23.376 INFO:journalctl@ceph.mds.j.smithi042.stdout:Apr 25 00:47:22 smithi042 podman[95714]: 2024-04-25 00:47:22.899756506 +0000 UTC m=+0.657165243 container remove cc36f7531251f245b5801273ceb2c04256c317476261b1b4057caa52e92de253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-d2c45d9) 2024-04-25T00:47:23.475 DEBUG:teuthology.orchestra.run.smithi042:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T00:47:23.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:23.594 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T00:47:23.594 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T00:47:23.594 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.b 2024-04-25T00:47:23.891 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:47:23 smithi106 systemd[1]: Stopping Ceph mds.b for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:24.338 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:47:23 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b[63388]: 2024-04-25T00:47:23.890+0000 7fb4d8fa5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:24.338 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:47:23 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b[63388]: 2024-04-25T00:47:23.890+0000 7fb4d8fa5700 -1 mds.b *** got signal Terminated *** 2024-04-25T00:47:25.856 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:47:25 smithi106 podman[75101]: 2024-04-25 00:47:25.591543571 +0000 UTC m=+1.800858251 container died 132e3dfb6c2687f815f677db695f2825215fb011dcc2311f283344be59f4f295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T00:47:25.856 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:47:25 smithi106 podman[75101]: 2024-04-25 00:47:25.856481859 +0000 UTC m=+2.065796590 container cleanup 132e3dfb6c2687f815f677db695f2825215fb011dcc2311f283344be59f4f295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-25T00:47:26.148 INFO:journalctl@ceph.mds.b.smithi106.stdout:Apr 25 00:47:25 smithi106 bash[75101]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-b 2024-04-25T00:47:26.262 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T00:47:26.296 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:26.296 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T00:47:26.296 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T00:47:26.296 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.e 2024-04-25T00:47:26.588 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:47:26 smithi106 systemd[1]: Stopping Ceph mds.e for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:27.088 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:47:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-e[62765]: 2024-04-25T00:47:26.626+0000 7fbef5018700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:27.088 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:47:26 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-e[62765]: 2024-04-25T00:47:26.626+0000 7fbef5018700 -1 mds.e *** got signal Terminated *** 2024-04-25T00:47:29.564 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:47:29 smithi106 podman[75223]: 2024-04-25 00:47:29.31132505 +0000 UTC m=+2.787282402 container died 6b97b95500167e021bbf1385626bff6db9f458ede69957448ef7069a09af4ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-d2c45d9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T00:47:29.830 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:47:29 smithi106 podman[75223]: 2024-04-25 00:47:29.659947731 +0000 UTC m=+3.135905082 container cleanup 6b97b95500167e021bbf1385626bff6db9f458ede69957448ef7069a09af4ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=squid-d2c45d9, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T00:47:29.830 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:47:29 smithi106 bash[75223]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-e 2024-04-25T00:47:30.080 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T00:47:30.088 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:47:29 smithi106 podman[75237]: 2024-04-25 00:47:29.829927232 +0000 UTC m=+0.513599340 container remove 6b97b95500167e021bbf1385626bff6db9f458ede69957448ef7069a09af4ecb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-04-25T00:47:30.088 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:47:30 smithi106 systemd[1]: ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-25T00:47:30.088 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:47:30 smithi106 systemd[1]: Stopped Ceph mds.e for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:47:30.088 INFO:journalctl@ceph.mds.e.smithi106.stdout:Apr 25 00:47:30 smithi106 systemd[1]: ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.e.service: Consumed 1.155s CPU time. 2024-04-25T00:47:30.114 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:30.115 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T00:47:30.115 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T00:47:30.115 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.h 2024-04-25T00:47:30.424 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:47:30 smithi106 systemd[1]: Stopping Ceph mds.h for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:30.838 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:47:30 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h[63078]: 2024-04-25T00:47:30.422+0000 7f07a8f98700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:30.838 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:47:30 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h[63078]: 2024-04-25T00:47:30.422+0000 7f07a8f98700 -1 mds.h *** got signal Terminated *** 2024-04-25T00:47:33.866 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:47:33 smithi106 podman[75345]: 2024-04-25 00:47:33.588615104 +0000 UTC m=+3.263124835 container died b047dea94f74c5c6c64d447916d94bd1cf97d3b7d4915181808ae654a8e99187 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-d2c45d9, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T00:47:34.183 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:47:33 smithi106 podman[75345]: 2024-04-25 00:47:33.973954284 +0000 UTC m=+3.648464015 container cleanup b047dea94f74c5c6c64d447916d94bd1cf97d3b7d4915181808ae654a8e99187 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T00:47:34.183 INFO:journalctl@ceph.mds.h.smithi106.stdout:Apr 25 00:47:33 smithi106 bash[75345]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-h 2024-04-25T00:47:34.315 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T00:47:34.345 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:34.346 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T00:47:34.346 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T00:47:34.346 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.k 2024-04-25T00:47:34.588 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:47:34 smithi106 systemd[1]: Stopping Ceph mds.k for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:35.088 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:47:34 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-k[62447]: 2024-04-25T00:47:34.692+0000 7f32a3c65700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:35.088 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:47:34 smithi106 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-k[62447]: 2024-04-25T00:47:34.692+0000 7f32a3c65700 -1 mds.k *** got signal Terminated *** 2024-04-25T00:47:38.692 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:47:38 smithi106 podman[75465]: 2024-04-25 00:47:38.434459297 +0000 UTC m=+3.872411888 container died 9af6b9f04f64e1956c88d9281b468db6d238ca97912d3124a13ded533bce4f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T00:47:39.078 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:47:38 smithi106 podman[75465]: 2024-04-25 00:47:38.802403539 +0000 UTC m=+4.240356182 container cleanup 9af6b9f04f64e1956c88d9281b468db6d238ca97912d3124a13ded533bce4f6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-25T00:47:39.078 INFO:journalctl@ceph.mds.k.smithi106.stdout:Apr 25 00:47:38 smithi106 bash[75465]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-k 2024-04-25T00:47:39.236 DEBUG:teuthology.orchestra.run.smithi106:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T00:47:39.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:39.266 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T00:47:39.266 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T00:47:39.266 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.c 2024-04-25T00:47:39.585 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:47:39 smithi136 systemd[1]: Stopping Ceph mds.c for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:40.013 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:47:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c[63296]: 2024-04-25T00:47:39.585+0000 7f2e6c7b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:40.013 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:47:39 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c[63296]: 2024-04-25T00:47:39.585+0000 7f2e6c7b1700 -1 mds.c *** got signal Terminated *** 2024-04-25T00:47:41.763 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:47:41 smithi136 podman[74264]: 2024-04-25 00:47:41.498967826 +0000 UTC m=+2.036602566 container died dc40c9ab2d5391f155a497e3b460fe7972a396e6bbbc5e21574aa76bf428d74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-d2c45d9, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:47:42.094 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:47:41 smithi136 podman[74264]: 2024-04-25 00:47:41.843444825 +0000 UTC m=+2.381079550 container cleanup dc40c9ab2d5391f155a497e3b460fe7972a396e6bbbc5e21574aa76bf428d74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-d2c45d9, maintainer=Guillaume Abrioux , ceph=True) 2024-04-25T00:47:42.094 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:47:41 smithi136 bash[74264]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c 2024-04-25T00:47:42.345 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T00:47:42.366 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:47:42 smithi136 podman[74290]: 2024-04-25 00:47:42.094811936 +0000 UTC m=+0.587408220 container remove dc40c9ab2d5391f155a497e3b460fe7972a396e6bbbc5e21574aa76bf428d74a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-c, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-d2c45d9, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T00:47:42.366 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:47:42 smithi136 systemd[1]: ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.c.service: Deactivated successfully. 2024-04-25T00:47:42.366 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:47:42 smithi136 systemd[1]: Stopped Ceph mds.c for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:47:42.366 INFO:journalctl@ceph.mds.c.smithi136.stdout:Apr 25 00:47:42 smithi136 systemd[1]: ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.c.service: Consumed 18min 54.870s CPU time. 2024-04-25T00:47:42.379 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:42.379 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T00:47:42.379 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T00:47:42.379 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.f 2024-04-25T00:47:42.690 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:47:42 smithi136 systemd[1]: Stopping Ceph mds.f for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:43.013 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:47:42 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f[62996]: 2024-04-25T00:47:42.688+0000 7fb40d4fb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:43.013 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:47:42 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f[62996]: 2024-04-25T00:47:42.688+0000 7fb40d4fb700 -1 mds.f *** got signal Terminated *** 2024-04-25T00:47:45.748 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:47:45 smithi136 podman[74399]: 2024-04-25 00:47:45.493442129 +0000 UTC m=+2.899321828 container died 2c248d79ecf85a2ae0cb57a5cbb975f950fcf2517792ccf7792b02e14856c577 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0) 2024-04-25T00:47:46.013 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:47:45 smithi136 podman[74399]: 2024-04-25 00:47:45.821974416 +0000 UTC m=+3.227854102 container cleanup 2c248d79ecf85a2ae0cb57a5cbb975f950fcf2517792ccf7792b02e14856c577 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T00:47:46.013 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:47:45 smithi136 bash[74399]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f 2024-04-25T00:47:46.302 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:47:46 smithi136 podman[74413]: 2024-04-25 00:47:46.052809776 +0000 UTC m=+0.554062011 container remove 2c248d79ecf85a2ae0cb57a5cbb975f950fcf2517792ccf7792b02e14856c577 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-d2c45d9, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-25T00:47:46.303 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:47:46 smithi136 systemd[1]: ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.f.service: Deactivated successfully. 2024-04-25T00:47:46.303 INFO:journalctl@ceph.mds.f.smithi136.stdout:Apr 25 00:47:46 smithi136 systemd[1]: Stopped Ceph mds.f for 9d95543e-0297-11ef-bc93-c7b262605968. 2024-04-25T00:47:46.305 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T00:47:46.338 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:46.338 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T00:47:46.338 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T00:47:46.338 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.i 2024-04-25T00:47:46.667 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:47:46 smithi136 systemd[1]: Stopping Ceph mds.i for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:47.013 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:47:46 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: 2024-04-25T00:47:46.666+0000 7f0c1fedb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:47.013 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:47:46 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i[63603]: 2024-04-25T00:47:46.666+0000 7f0c1fedb700 -1 mds.i *** got signal Terminated *** 2024-04-25T00:47:49.763 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:47:49 smithi136 podman[74524]: 2024-04-25 00:47:49.495750599 +0000 UTC m=+2.943065988 container died 29203519efc76980b0f39345291f22fcc65b564f664204eea6106cbd21d4d6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-d2c45d9) 2024-04-25T00:47:50.184 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:47:49 smithi136 podman[74524]: 2024-04-25 00:47:49.834442601 +0000 UTC m=+3.281757980 container cleanup 29203519efc76980b0f39345291f22fcc65b564f664204eea6106cbd21d4d6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9) 2024-04-25T00:47:50.184 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:47:49 smithi136 bash[74524]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i 2024-04-25T00:47:50.184 INFO:journalctl@ceph.mds.i.smithi136.stdout:Apr 25 00:47:49 smithi136 podman[74538]: 2024-04-25 00:47:49.993850346 +0000 UTC m=+0.494741194 container remove 29203519efc76980b0f39345291f22fcc65b564f664204eea6106cbd21d4d6ca (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-i, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=squid-d2c45d9, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T00:47:50.199 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T00:47:50.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:50.230 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T00:47:50.230 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T00:47:50.231 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.l 2024-04-25T00:47:50.513 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:47:50 smithi136 systemd[1]: Stopping Ceph mds.l for 9d95543e-0297-11ef-bc93-c7b262605968... 2024-04-25T00:47:50.989 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:47:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-l[62692]: 2024-04-25T00:47:50.554+0000 7fd5e3b55700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T00:47:50.989 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:47:50 smithi136 ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-l[62692]: 2024-04-25T00:47:50.554+0000 7fd5e3b55700 -1 mds.l *** got signal Terminated *** 2024-04-25T00:47:51.855 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:47:51 smithi136 podman[74644]: 2024-04-25 00:47:51.526992473 +0000 UTC m=+1.088487104 container died 0b8ea76fe11339e92d4ab09553bfebe3a6b3f75fb644465d73d2a8f4a8b4301c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-d2c45d9, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T00:47:51.855 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:47:51 smithi136 podman[74644]: 2024-04-25 00:47:51.855055079 +0000 UTC m=+1.416549738 container cleanup 0b8ea76fe11339e92d4ab09553bfebe3a6b3f75fb644465d73d2a8f4a8b4301c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-l, RELEASE=squid-d2c45d9, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-25T00:47:52.261 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:47:51 smithi136 bash[74644]: ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-l 2024-04-25T00:47:52.261 INFO:journalctl@ceph.mds.l.smithi136.stdout:Apr 25 00:47:52 smithi136 podman[74658]: 2024-04-25 00:47:52.064721429 +0000 UTC m=+0.537521636 container remove 0b8ea76fe11339e92d4ab09553bfebe3a6b3f75fb644465d73d2a8f4a8b4301c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:a7481a6d0dca93e16a29c8acf1c964b318bd33c455aa5b4e4025b7a00d008901, name=ceph-9d95543e-0297-11ef-bc93-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-d2c45d9, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T00:47:52.276 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d95543e-0297-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T00:47:52.306 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T00:47:52.307 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T00:47:52.307 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d95543e-0297-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:47:52.638 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:47:57.891 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d95543e-0297-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:47:58.083 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:48:00.544 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d95543e-0297-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T00:48:00.735 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T00:48:03.116 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:48:03.215 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:48:03.245 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T00:48:03.271 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T00:48:03.273 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230/remote/smithi042/crash 2024-04-25T00:48:03.274 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:48:03.381 INFO:teuthology.orchestra.run.smithi042.stderr:tar: /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:48:03.381 INFO:teuthology.orchestra.run.smithi042.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:48:03.385 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230/remote/smithi106/crash 2024-04-25T00:48:03.386 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:48:03.435 INFO:teuthology.orchestra.run.smithi106.stderr:tar: /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:48:03.435 INFO:teuthology.orchestra.run.smithi106.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:48:03.437 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230/remote/smithi136/crash 2024-04-25T00:48:03.438 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/crash -- . 2024-04-25T00:48:03.471 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/9d95543e-0297-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T00:48:03.471 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-04-25T00:48:03.472 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T00:48:03.472 DEBUG:teuthology.orchestra.run.smithi042:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T00:48:03.641 INFO:tasks.cephadm:Compressing logs... 2024-04-25T00:48:03.641 DEBUG:teuthology.orchestra.run.smithi042:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:48:03.646 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:48:03.649 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T00:48:03.676 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T00:48:03.677 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:48:03.677 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:gzip/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.audit.log: -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:gzip/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mon.c.log: -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T00:48:03.680 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.cephadm.log: /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:48:03.681 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T00:48:03.681 INFO:teuthology.orchestra.run.smithi136.stderr:gzip/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-volume.log: -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:48:03.730 INFO:teuthology.orchestra.run.smithi106.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.log: No such file or directory 2024-04-25T00:48:03.730 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-25T00:48:03.730 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:48:03.732 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T00:48:03.732 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T00:48:03.733 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T00:48:03.733 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T00:48:03.733 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T00:48:03.733 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T00:48:03.733 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T00:48:03.733 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:48:03.733 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T00:48:03.734 INFO:teuthology.orchestra.run.smithi106.stderr:gzip/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.cephadm.log: -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:48:03.734 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T00:48:03.734 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T00:48:03.735 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:48:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-04-25T00:48:03.745 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.log 2024-04-25T00:48:03.746 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T00:48:03.747 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T00:48:03.750 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T00:48:03.751 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T00:48:03.752 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.j.log: 86.6% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:48:03.753 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T00:48:03.754 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T00:48:03.755 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T00:48:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mgr.x.log: 94.5% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T00:48:03.757 INFO:teuthology.orchestra.run.smithi042.stderr:gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T00:48:03.758 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T00:48:03.759 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T00:48:03.759 INFO:teuthology.orchestra.run.smithi042.stderr: 94.4% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T00:48:03.760 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T00:48:03.761 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T00:48:03.762 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T00:48:03.763 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T00:48:03.769 INFO:teuthology.orchestra.run.smithi042.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.9.log: /var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T00:48:03.790 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 83.1% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:48:03.804 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 83.2% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:48:03.820 INFO:teuthology.orchestra.run.smithi136.stderr: 93.6% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T00:48:03.877 INFO:teuthology.orchestra.run.smithi106.stderr: 90.4% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:48:03.889 INFO:teuthology.orchestra.run.smithi106.stderr: 86.6% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:48:03.910 INFO:teuthology.orchestra.run.smithi106.stderr: 94.5% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T00:48:03.910 INFO:teuthology.orchestra.run.smithi106.stderr: 94.5% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T00:48:03.924 INFO:teuthology.orchestra.run.smithi106.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:48:03.934 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T00:48:03.948 INFO:teuthology.orchestra.run.smithi106.stderr: 91.1% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:48:03.954 INFO:teuthology.orchestra.run.smithi042.stderr: 83.1% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T00:48:03.975 INFO:teuthology.orchestra.run.smithi136.stderr: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:48:03.981 INFO:teuthology.orchestra.run.smithi136.stderr: 86.6% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T00:48:04.005 INFO:teuthology.orchestra.run.smithi042.stderr: 93.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T00:48:04.037 INFO:teuthology.orchestra.run.smithi136.stderr: 94.5% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T00:48:04.038 INFO:teuthology.orchestra.run.smithi136.stderr: 91.0% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:48:04.058 INFO:teuthology.orchestra.run.smithi136.stderr: 90.4% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:48:04.059 INFO:teuthology.orchestra.run.smithi042.stderr: 90.2% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T00:48:04.111 INFO:teuthology.orchestra.run.smithi042.stderr: 91.0% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T00:48:05.123 INFO:teuthology.orchestra.run.smithi042.stderr: 90.4% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T00:48:12.723 INFO:teuthology.orchestra.run.smithi042.stderr: 90.1% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T00:48:17.607 INFO:teuthology.orchestra.run.smithi106.stderr: 91.8% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T00:49:16.130 INFO:teuthology.orchestra.run.smithi106.stderr: 93.8% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T00:49:47.030 INFO:teuthology.orchestra.run.smithi106.stderr: 94.1% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T00:49:48.555 INFO:teuthology.orchestra.run.smithi042.stderr: 93.7% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T00:49:51.433 INFO:teuthology.orchestra.run.smithi042.stderr: 93.8% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T00:50:05.785 INFO:teuthology.orchestra.run.smithi136.stderr: 93.3% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T00:50:14.588 INFO:teuthology.orchestra.run.smithi042.stderr: 93.7% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T00:50:17.809 INFO:teuthology.orchestra.run.smithi106.stderr: 94.9% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T00:50:26.568 INFO:teuthology.orchestra.run.smithi042.stderr: 93.7% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T00:52:23.784 INFO:teuthology.orchestra.run.smithi106.stderr: 93.9% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T00:52:30.606 INFO:teuthology.orchestra.run.smithi136.stderr: 94.9% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T00:52:31.317 INFO:teuthology.orchestra.run.smithi136.stderr: 93.8% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T00:52:55.859 INFO:teuthology.orchestra.run.smithi106.stderr: 93.8% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T00:53:23.776 INFO:teuthology.orchestra.run.smithi136.stderr: 94.9% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T00:54:10.334 INFO:teuthology.orchestra.run.smithi136.stderr: 93.9% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T00:55:02.179 INFO:teuthology.orchestra.run.smithi042.stderr: 93.1% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T00:55:02.184 INFO:teuthology.orchestra.run.smithi042.stderr: 2024-04-25T00:55:02.184 INFO:teuthology.orchestra.run.smithi042.stderr:real 6m58.496s 2024-04-25T00:55:02.184 INFO:teuthology.orchestra.run.smithi042.stderr:user 7m21.465s 2024-04-25T00:55:02.184 INFO:teuthology.orchestra.run.smithi042.stderr:sys 0m51.265s 2024-04-25T00:55:12.090 INFO:teuthology.orchestra.run.smithi136.stderr: 93.9% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T00:55:22.427 INFO:teuthology.orchestra.run.smithi136.stderr: 94.0% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T00:58:40.831 INFO:teuthology.orchestra.run.smithi106.stderr: 93.2% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T00:58:40.833 INFO:teuthology.orchestra.run.smithi106.stderr: 2024-04-25T00:58:40.833 INFO:teuthology.orchestra.run.smithi106.stderr:real 10m37.169s 2024-04-25T00:58:40.833 INFO:teuthology.orchestra.run.smithi106.stderr:user 6m56.932s 2024-04-25T00:58:40.833 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m18.523s 2024-04-25T01:00:58.902 INFO:teuthology.orchestra.run.smithi136.stderr: 93.1% -- replaced with /var/log/ceph/9d95543e-0297-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T01:00:58.936 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-25T01:00:58.936 INFO:teuthology.orchestra.run.smithi136.stderr:real 12m55.273s 2024-04-25T01:00:58.936 INFO:teuthology.orchestra.run.smithi136.stderr:user 7m17.362s 2024-04-25T01:00:58.936 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m20.275s 2024-04-25T01:00:58.937 INFO:tasks.cephadm:Archiving logs... 2024-04-25T01:00:58.938 DEBUG:teuthology.misc:Transferring archived files from smithi042:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230/remote/smithi042/log 2024-04-25T01:00:58.939 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:01:51.041 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230/remote/smithi106/log 2024-04-25T01:01:51.042 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:02:56.200 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230/remote/smithi136/log 2024-04-25T01:02:56.230 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T01:04:07.777 INFO:tasks.cephadm:Removing cluster... 2024-04-25T01:04:07.778 DEBUG:teuthology.orchestra.run.smithi042:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d95543e-0297-11ef-bc93-c7b262605968 --force 2024-04-25T01:04:08.297 INFO:teuthology.orchestra.run.smithi042.stdout:Deleting cluster with fsid: 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T01:04:09.855 DEBUG:teuthology.orchestra.run.smithi106:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d95543e-0297-11ef-bc93-c7b262605968 --force 2024-04-25T01:04:10.270 INFO:teuthology.orchestra.run.smithi106.stdout:Deleting cluster with fsid: 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T01:04:10.897 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d95543e-0297-11ef-bc93-c7b262605968 --force 2024-04-25T01:04:11.764 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: 9d95543e-0297-11ef-bc93-c7b262605968 2024-04-25T01:04:12.431 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T01:04:12.431 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T01:04:12.491 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T01:04:12.510 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T01:04:12.529 INFO:tasks.cephadm:Teardown complete 2024-04-25T01:04:12.530 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T01:04:12.540 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T01:04:12.540 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:04:12.548 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:04:12.552 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T01:04:12.705 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T01:04:12.715 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T01:04:12.715 DEBUG:teuthology.orchestra.run.smithi042:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:04:12.723 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:04:12.725 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T01:04:12.739 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:04:12.739 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:04:12.772 INFO:teuthology.orchestra.run.smithi042.stderr:bash: line 1: ntpq: command not found 2024-04-25T01:04:12.811 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:04:12.811 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-04-25T01:04:12.811 INFO:teuthology.orchestra.run.smithi136.stdout:^- hv01.front.sepia.ceph.com 2 6 377 48 +1025us[+1019us] +/- 574ms 2024-04-25T01:04:12.811 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv02.front.sepia.ceph.com 3 6 377 48 -134us[ -140us] +/- 76ms 2024-04-25T01:04:12.811 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 +138us[ +138us] +/- 84ms 2024-04-25T01:04:12.811 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:04:12.812 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:04:12.812 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2024-04-25T01:04:12.812 INFO:teuthology.orchestra.run.smithi106.stdout:^- hv01.front.sepia.ceph.com 2 6 377 48 +1046us[+1046us] +/- 574ms 2024-04-25T01:04:12.812 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -100us[ -82us] +/- 76ms 2024-04-25T01:04:12.812 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +176us[ +176us] +/- 84ms 2024-04-25T01:04:12.812 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:04:12.857 INFO:teuthology.orchestra.run.smithi042.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T01:04:12.857 INFO:teuthology.orchestra.run.smithi042.stdout:=============================================================================== 2024-04-25T01:04:12.858 INFO:teuthology.orchestra.run.smithi042.stdout:^- hv01.front.sepia.ceph.com 2 6 377 44 +1048us[+1048us] +/- 574ms 2024-04-25T01:04:12.858 INFO:teuthology.orchestra.run.smithi042.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 -107us[ -79us] +/- 76ms 2024-04-25T01:04:12.858 INFO:teuthology.orchestra.run.smithi042.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 44 +177us[ +177us] +/- 84ms 2024-04-25T01:04:12.858 INFO:teuthology.orchestra.run.smithi042.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T01:04:12.860 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T01:04:12.871 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T01:04:12.871 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T01:04:12.881 DEBUG:teuthology.orchestra.run.smithi042:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:04:13.149 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:04:13.318 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T01:04:13.432 DEBUG:teuthology.orchestra.run.smithi042:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:04:13.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:04:13.531 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:04:13.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:04:13.562 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T01:04:13.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:04:13.589 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T01:04:13.599 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T01:04:13.615 INFO:teuthology.task.internal:Duration was 3930.558244 seconds 2024-04-25T01:04:13.615 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T01:04:13.660 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T01:04:13.660 DEBUG:teuthology.orchestra.run.smithi042:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:04:13.668 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:04:13.670 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T01:04:13.747 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:04:13.780 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:04:13.818 INFO:teuthology.orchestra.run.smithi042.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T01:04:14.241 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T01:04:14.241 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi042.front.sepia.ceph.com 2024-04-25T01:04:14.242 DEBUG:teuthology.orchestra.run.smithi042:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:04:14.312 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2024-04-25T01:04:14.312 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:04:14.346 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-04-25T01:04:14.346 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T01:04:14.377 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T01:04:14.377 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:04:14.384 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:04:14.389 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T01:04:14.469 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T01:04:14.470 DEBUG:teuthology.orchestra.run.smithi042:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:04:14.475 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:04:14.477 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T01:04:14.982 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T01:04:14.994 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T01:04:14.995 DEBUG:teuthology.orchestra.run.smithi042:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:04:15.101 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:04:15.127 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T01:04:15.153 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T01:04:15.165 DEBUG:teuthology.orchestra.run.smithi042:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:04:15.172 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:04:15.175 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:04:15.209 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2024-04-25T01:04:15.246 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-04-25T01:04:15.277 INFO:teuthology.orchestra.run.smithi042.stdout:kernel.core_pattern = core 2024-04-25T01:04:15.331 DEBUG:teuthology.orchestra.run.smithi042:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:04:15.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:04:15.384 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:04:15.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:04:15.399 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T01:04:15.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T01:04:15.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T01:04:15.424 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T01:04:15.426 DEBUG:teuthology.misc:Transferring archived files from smithi042:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230/remote/smithi042 2024-04-25T01:04:15.426 DEBUG:teuthology.orchestra.run.smithi042:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:04:15.528 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230/remote/smithi106 2024-04-25T01:04:15.528 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:04:15.565 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-24_21:24:02-fs-squid-distro-default-smithi/7672230/remote/smithi136 2024-04-25T01:04:15.566 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T01:04:15.602 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T01:04:15.602 DEBUG:teuthology.orchestra.run.smithi042:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:04:15.610 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:04:15.613 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T01:04:15.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T01:04:15.678 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T01:04:15.678 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T01:04:15.689 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T01:04:15.689 DEBUG:teuthology.orchestra.run.smithi042:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:04:15.696 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:04:15.698 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T01:04:15.711 INFO:teuthology.orchestra.run.smithi106.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:04 /home/ubuntu/cephtest 2024-04-25T01:04:15.712 INFO:teuthology.orchestra.run.smithi136.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:04 /home/ubuntu/cephtest 2024-04-25T01:04:15.747 INFO:teuthology.orchestra.run.smithi042.stdout: 393251 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 01:04 /home/ubuntu/cephtest 2024-04-25T01:04:15.750 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T01:04:15.763 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T01:04:15.792 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/yes 5-workunit/suites/dbench}} duration: 3930.5582439899445 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-25T01:04:15.792 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T01:04:15.885 INFO:teuthology.run:pass